Here, you actually have three functions, so you have to apply the chain rule twice.

We have [tex] f(g(h(x))), [/tex] where

[tex] h(x) = x^5 [/tex]

[tex] g(x) = cot(x) [/tex]

[tex] f(x) = x^7 [/tex]

When applying the chain rule, the approach is from "outside to inside," in that we start from the outer most function and systematically go towards the innermost function. Remember, when doing the chain rule, each step is distinct.

You have: [tex] 7(-csc^2)^6..... [/tex]

You were correct in taking the derivative of f(x) = x^7 first; however, you need to let everything inside f(x) remain untouched during this step. Meaning, for f(g(h(x)), we leave g(h(x)) untouched.

In other words, to differentiate [itex]cot^7(x^5)[/itex] you think: "The 'outer' function is a 7th power so first I have [itex]7 cot^6(x^5)[/itex]. Now the 'next' inner function is a cotangent, "cot(x^5)", so I have to multiply that by its derivative: [itex]-csc^2(x^5)[/itex]. Finally, the last function is [itex]x^5[/itex] so that has to be multiplied by its derivative, [itex]5x^4[/itex].