Documentation listing supported layers for Keras, Caffe and TensorFlow by Azamlynny · Pull Request #409 · Cloud-CV/Fabrik

Azamlynny · 2018-10-30T05:26:03Z

No description provided.

… TensorFlow

thatbrguy · 2018-10-30T09:43:24Z

@Azamlynny That's a very neat documentation, good job! I have a few suggestions before we can approve this:

Place the documentation in docs/source. That's where most of our documentation is.
Link this documentation to our README.md file.
Please open a new PR with the graph reset functionality that you added for your previous task. Looks like you closed the old one and did not create another PR for graph reset.

Ram81

@Azamlynny good work, few comments

Ram81 · 2018-10-30T14:49:36Z

+### Custom Layers
+| Caffe         | Keras        | Tensorflow  |
+| :-----------: | :----------: | :---------: |
+| √             | √            | Use Keras API for custom layers |


You can add LRN as an example in custom layers supported.

Ram81 · 2018-10-30T14:50:29Z

+## Additional Notes:
+* Keras does not support the LRN layer used in Alexnet & many other models. To use the LRN layer refer to here: https://github.com/Cloud-CV/Fabrik/blob/master/tutorials/keras_custom_layer_usage.md.
+* Documentation for writing your own Keras layers is found here: https://keras.io/layers/writing-your-own-keras-layers/ 
+* Caffe 3D layers can be used in Caffe 2.0


We haven't tested any features for caffe 2.0 yet, have you tested this if not we can drop this line.

Ram81 · 2018-10-30T14:51:15Z

+* Caffe 3D layers can be used in Caffe 2.0
+
+## Documentation for Caffe, Keras, and Tensorflow layers
+* Documentation for all Keras Layers is found here: https://keras.io/layers/about-keras-layers/


You can add a markdown link on here.

Ram81 · 2018-10-30T14:52:20Z

+
+## Documentation for Caffe, Keras, and Tensorflow layers
+* Documentation for all Keras Layers is found here: https://keras.io/layers/about-keras-layers/
+* Documentation for all Caffe Layers is found here: http://caffe.berkeleyvision.org/tutorial/layers.html


Same as above

Ram81 · 2018-10-30T14:52:45Z

+## Documentation for Caffe, Keras, and Tensorflow layers
+* Documentation for all Keras Layers is found here: https://keras.io/layers/about-keras-layers/
+* Documentation for all Caffe Layers is found here: http://caffe.berkeleyvision.org/tutorial/layers.html
+* Documentation for all Tensorflow Layers is found here: https://www.tensorflow.org/api_docs/python/tf/layers


Same as above

Ram81 · 2018-10-30T14:53:03Z

+
+## Additional Notes:
+* Keras does not support the LRN layer used in Alexnet & many other models. To use the LRN layer refer to here: https://github.com/Cloud-CV/Fabrik/blob/master/tutorials/keras_custom_layer_usage.md.
+* Documentation for writing your own Keras layers is found here: https://keras.io/layers/writing-your-own-keras-layers/ 


You can add a markdown link on here.

Ram81 · 2018-10-30T14:53:07Z

+| :-----------: | :----------: | :---------: |
+| √             | √            | Use Keras API for custom layers |
+
+## Additional Notes:


You can add a markdown link on here.

Ram81 · 2018-10-30T14:53:50Z

@@ -0,0 +1,176 @@
+# Layers supported in Caffe, Keras, and Tensorflow
+Below is are table showing which layers are supported by Caffe, Keras, and Tensorflow:


Grammatical mistake.

Ram81 · 2018-10-31T14:49:33Z

+| ZeroPadding 3D            | ×             | √            | √           |
+| Im2Col                    | √             | ×            | ×           |
+| Spatial Pyramid Pooling   | √             | ×            | ×           |
+* Upsampling in Caffe can be done by using methods shown here: https://gist.github.com/tnarihi/54744612d35776f53278


Add as a markdown link on here

Ram81 · 2018-10-31T14:50:37Z

+ ### Custom Layers
+| Layer         | Caffe        | Keras       | Tensorflow  |
+| :-----------: | :----------: | :---------: | :---------: |
+| Custom Layers | √            | √           | Use Keras API for custom layers |


No need to mention custom layers explicitly

Ram81 · 2018-10-31T14:50:59Z

@Azamlynny minor changes

yashdusing · 2018-10-31T14:54:34Z

Also, could you please specify how you've tested the layers and the procedure used. We will need to cross verify it once on our end as well before approving the PR. A code template would be helpful as well .

thatbrguy · 2018-11-02T14:40:14Z

@Azamlynny Please add a section for Data layers. I've just pasted the layers here for your reference:

thatbrguy

Hey, impressive work so far. Made some minor changes.

thatbrguy · 2018-11-05T06:13:14Z

+| Data                      | √             | √            | √           | √             | √               | √                    |
+| HDF5 Data                 | √             | √            | √           | √             | √               | √                    |
+| HDF5 Output Data          | √             | √            | √           | √             | √               | √                    |
+| Input                     | √             | √            | √           | √             | √               | √                    |


If i remember correctly, for Keras and Tensorflow (in Fabrik) only Input is available. All other data layers are not available.

thatbrguy · 2018-11-05T06:15:00Z

+| Log                       | √             | √            | ×           | √             | ×                | ×                   |
+| BNLL                      | √             | ×            | ×           | √             | ×                | ×                   |
+| Bias                      | √             | ×            | ×           | √             | √                | √                   |
+| Scale                     | √             | ×            | ×           | √             | √                | √                   |


I think Scale is Caffe only (in Fabrik)

thatbrguy · 2018-11-05T06:15:52Z

+| Eltwise                   | √             | ×            | ×           | √             | √                | √                   |
+| Parameter                 | √             | ×            | ×           | √             | ×                | √                   |
+| Reduction                 | √             | ×            | ×           | √             | ×                | ×                  |
+| Silence                   | √             | ×            | ×           | ×             | ×                | ×                   |


Fabrik Caffe does has silence.

thatbrguy · 2018-11-05T06:16:29Z

+ ### Loss Layers
+| Layer                     | Caffe         | Keras        | Tensorflow  | Fabrik Caffe  | Fabrik Keras    | Fabrik Tensorflow  |
+| :-----------------------: | :-----------: | :----------: | :---------: | :-----------: | :--------------: | :-----------------: | 
+| Multinomial Logistic Loss | √             | ×            | ×           | ×             | ×                | ×                   |


Fabrik caffe has this layer

thatbrguy · 2018-11-05T06:17:17Z

+| Multinomial Logistic Loss | √             | ×            | ×           | ×             | ×                | ×                   |
+| Infogain Loss             | √             | ×            | ×           | √             | ×                | ×                   |
+| Softmax with Loss         | √             | ×            | √           | √             | ×                | ×                   |
+| Sum-of-Squares/Euclidean  | √             | ×            | ×           | ×             | ×                | ×                   |


Fabrik caffe has this layer

thatbrguy · 2018-11-05T06:17:48Z

+| Softmax with Loss         | √             | ×            | √           | √             | ×                | ×                   |
+| Sum-of-Squares/Euclidean  | √             | ×            | ×           | ×             | ×                | ×                   |
+| Hinge / Margin            | √             | √            | √           | √             | ×                | ×                   |
+| Sigmoid Cross-Entropy Loss| √             | ×            | √           | ×             | ×                | ×                   |


Fabrik caffe has this layer

thatbrguy · 2018-11-05T06:21:22Z

@Azamlynny Excellent work so far! That's a very neat documentation.

I've posted a review with some changes. @yashdusing Can you just verify if I have not missed something?

yashdusing · 2018-11-05T07:12:30Z

Hello @Azamlynny . Pretty good work so far. Few changes though (I've attached a screenshot completing the core layers) :

follow the format as show in picture i.e. if layer is keras only or caffe only then mention it in bracket. If layer is keras only, then even tensorflow would support it through keras API so mention keras in tensorflow column. If layer is supported in all of the frameworks (for eg. dropout), keep it as blank (i.e. don't add in brackets). Also, please refer the documentations for the frameworks before marking it as done. While caffe does have activation layers, it does not have a separate activation layer like keras does and has a separate layer for each of them which are mentioned layer (relu, sigmoid, tanH, etc) so activation layer should be crossed in caffe column.
Reference :

For caffe : http://caffe.berkeleyvision.org/tutorial/layers.html
For keras : https://keras.io/
For tensorflow 1.4 : https://github.com/tensorflow/docs/blob/r1.4/site/en/api_docs/api_docs/python/index.md
For Fabrik caffe : data.js
For Fabrik Keras : Fabrik's keras app's import, export files
For Fabrik Tensorflow : import_graphdef.py

(EDIT:

Also for layers having different names in caffe and keras, like inner product, mention Keras's layer name in Keras section (dense in this case)
ActivityRegularization is a keras only layer so caffe column should be crossed as well contrary to what is shown in image.
)

yashdusing · 2018-11-05T07:19:13Z

@Ram81 @thatbrguy Please look into the format of the screenshot I posted and see if I missed something out or any changes are needed.

thatbrguy · 2018-11-05T09:43:18Z

@yashdusing I think we'll just mention the different names across platforms in brackets. (Inner-Product vs Dense). Let's not mention keras in the TensorFlow section for now. We'll have a separate task to deal with supported tensorflow APIs.

@Azamlynny Also, looks like inner-product is crossed out for Caffe. It's available on caffe so please modify that

Azamlynny · 2018-11-05T14:10:15Z

@yashdusing I think we'll just mention the different names across platforms in brackets. (Inner-Product vs Dense). Let's not mention keras in the TensorFlow section for now. We'll have a separate task to deal with supported tensorflow APIs.

I am not sure by what you mean by mention the different names across platforms in brackets. Where would you want this? In the Layer column, or in the actual Caffe and Keras columns and if in these locations what would the layer column be called?

Not complete

thatbrguy · 2018-11-05T15:50:25Z

@Azamlynny Actually the table looks neater with just ticks. So, if you notice any layer available in Caffe as well as Keras or Tensorflow, with just different names (but mapped to the same Fabrik layer name) you can add it to the notes section after the table

Ram81 · 2018-11-13T14:56:34Z

+### Locally-connected Layers
+| Layer                     | Caffe         | Keras        | Tensorflow  | Fabrik Caffe  | Fabrik Keras    | Fabrik Tensorflow  |
+| :-----------------------: | :-----------: | :----------: | :---------: | :-----------: | :--------------: | :-----------------: | 
+| LocallyConnected1D        | ×             | √            | √           | ×             | √                | √×                  |


Why are there both √× in the Fabrik Tensorflow section?

Ram81 · 2018-11-13T14:58:34Z

+| Stochastic Pooling        | √             | ×            | ×           | √             | ×               | ×                    |
+
+ ### Data Layers
+| Layer                     | Caffe         | Keras        | Tensorflow  | Fabrik Caffe  | Fabrik Keras    | Fabrik Tensorflow  |


Can you also tell me why we have 6 columns two for each layer?
I think only mentioning which layers for a framework does Fabrik support is good enough.

Documentation listing currently supported layers for Keras, Caffe and…

5df9531

… TensorFlow

Moving supported_layers to docs/source

5a1fdbf

Ram81 suggested changes Oct 30, 2018

View reviewed changes

Azamlynny added 2 commits October 30, 2018 10:54

Moved to proper location

35791e4

Fixed errors

66b9b6a

Ram81 reviewed Oct 31, 2018

View reviewed changes

Ram81 suggested changes Oct 31, 2018

View reviewed changes

Update supported_layers.md

6a9e4e3

ZeroZeroJedenJeden mentioned this pull request Nov 3, 2018

GCI "Filter layers that are exclusive to selected frameworks" #418

Merged

Azamlynny added 5 commits November 4, 2018 17:04

Update supported_layers.md

438f104

Keras Updated

dc14f86

Updated Tensorflow

f929575

Updated Caffe

16a1705

Added Data Layers

fc82298

thatbrguy suggested changes Nov 5, 2018

View reviewed changes

Azamlynny added 2 commits November 5, 2018 09:12

Fixed Errors

9a6267a

Not complete

Typo

a67bbe3

Ram81 suggested changes Nov 13, 2018

View reviewed changes

		@@ -0,0 +1,176 @@
		# Layers supported in Caffe, Keras, and Tensorflow
		Below is are table showing which layers are supported by Caffe, Keras, and Tensorflow:

Uh oh!

Conversation

Azamlynny commented Oct 30, 2018

Uh oh!

thatbrguy commented Oct 30, 2018

Uh oh!

Ram81 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ram81 commented Oct 31, 2018

Uh oh!

yashdusing commented Oct 31, 2018

Uh oh!

thatbrguy commented Nov 2, 2018

Uh oh!

thatbrguy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thatbrguy commented Nov 5, 2018

Uh oh!

yashdusing commented Nov 5, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yashdusing commented Nov 5, 2018

Uh oh!

thatbrguy commented Nov 5, 2018

Uh oh!

Azamlynny commented Nov 5, 2018

Uh oh!

thatbrguy commented Nov 5, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yashdusing commented Nov 5, 2018 •

edited

Loading