Then how could we delivery gradients from first part to second part? Here is the equation to answer:

The means ‘intermediate vector’, which is the interface vector between first-part and second-part and it is belong to both first-part and second-part. The is the weights of second part of model. Therefore we could use tf.gradients() to connect gradients of two parts: