We need to support such composition in Python as well. To do so, we need a higher level Python wrapping of operator creation than paddle.cpp.create_operator. This higher level operator API should be compatible with the layer API.

Let's explain using an example. Suppose that we are going to compose the FC using mul and add in Python, we'd like to have Python functions mul and add defined in module operator:

We'd like to have Python bindings to operators in package paddle.operator, and Python compositions of operators in package paddle.layer. So we have the following concepts in above illustrative example:

C++ functions/functors

mul

add

C++ operator class

mulOp

addOp

FCOp

Python binding

operator.mul

operator.add

operator.fc

Python function

layer.fc

This is how we differentiate layer and operators in PaddlePaddle:

those defined in C++ and have a lightweighted Python wrapper in module operators are operators; whereas

those who don't have C++ implementations but a Python implementation that compose C++ operators are known as layers.