The purpose of this tutorial series is to introduce the mathematics behind a lot of programming concentrations, including game and scientific programming. This tutorial will introduce the concept of a derivative, how to calculate one, and a tangent line approximation. I am assuming some familiarity with precalculus and trig concepts, including the trig functions, trig identities, and limits.

Let's start out with the definition of a derivative. We can use either form, but they both say the same thing:

lim(h-->0)[f(x+h)-f(x)]/h
lim(x-->a)[f(x)-f(a)]/(x-a)

The second limit looks a lot like our slope formula: m = (y2 - y1)/(x2 - x1). So what exactly does this tell us? The derivative is the slope of a line tangent to a point. What happens is we calculate the slope between two points. As h-->0 (or x-->a), we grow infinitesimally closer to calculating [f(x)-f(x)]/(x-x).

Before we proceed to our shortcut rules for derivatives, I want to introduce notations first. We can either use the Newtonian notation or Leibnitzian notation. The Newtonian form is the f-prime form. For each derivative, we add an apostraphe to the fucntion. So the first derivative of f(x) is f'(x) (pronounced f-prime of x). The second derivative is f"(x), or f-double prime of x. Leibnitzian notation treats a derivative as a fraction, which will come in handy further down the road. For example, if we are differentiating y with respect to a variable x, we would have dy/dx as our first derivative. For an nth derivative, we would have dny/dxn.

Power Rule
This is one of the first rules listed above to memorize: d/dx xn = n * xn-1. So for example, if we have 3x^2, it's derivative is 6x. This is fairly straight-forward. We can also use this rule to quickly calculate a higher-order derivative, like say the 100th derivative, using factorials. As an example, let's say we have a function f(x) = 3x^150 + x^100 + 2x^20, and we ant to take the 100th derivative of f(x). To start, we know that the derivative of a constant is 0. Now b/c 20 < 100, we can simply treat 2x^20 as a constant. By the 20th derivative, only a constant will remain for that term. And on the 21st derivative, it will cancel out. For the term x^100, we can simply calculate 100! as the remaining term.

For our last term, 3x^150, we will have a power of 50 leftover after we take the 100th derivative. So we can calculate d100/dx100 3x^150 by cancelling factorials. It comes out to 3 * 150!/(150-100)! * x^50.

Now simply add them together to get d100/dx100 f(x) = 100! + 3(150!)/(50!) * x^50.

Or in a more general form, the rule is dn/dxn of x^r = r!/(r-n)! * x^(r-n).

Product and Quotient Rules
We use the product rule when calculating the derivative of a product of functions. With a constant, we can simply pull it out front and calculate the derivative of f(x). But when we try to d/dx f(x)g(x), we have to use a sepcial rule: d/dx f(x)g(x) = f(x)g'(x) + f'(x)g(x).

For example, if we have 3x^2 * ln(x), the derivative would be 3x^2/x + 6xln(x), which would simplify to 3x(1+2ln(x)). Notice how we go back to our derivative rule for ln(x).

When evaluating quotients, many people prefer to simply use the quotient rule: d/dx f(x)/g(x) = [g(x)f'(x) - f(x)g'(x)]/g^2(x). This is simply a special case of the product rule. Writing g(x) as g(x)^-1 will give us the same answer, and is one less rule to memorize.

For practice, let's find the derivative of sin(x)/x^3. By our quotient rule, that is [cos(x)x^3 - 3x^2sin(x)]/x^6. Rewriting 1/x^3 as x^-3, we still get the same answer: sin(x) * -3x^-4 + x^-3 *cos(x), just written a little bit differently.

Chain Rule
All of our derivative rules come together with the chain rule. Basically, the chain rule allows us to take derivatives of more complex functions like sin(cos^2(x)) or e^(3x^3). The rule is as follows: d/dx f(g(x)) = f'(g(x)) * g'(x). Or in short, we take the derivatives of the "inside functions" and multiple them by the derivative of the "outside function".

Let's look at our sin(cos^2(x)) function. To differentiate this, we need to take the derivative of the inside-most function: cos^2(x). Its derivative comes out to -2cos(x)sin(x). We use the power rule to get 2cos(x), then we multiple by the derivative of cos(x). Next, we take the derivative of sin(x), which is cos(x), and use the same parameter: cos^2(x). This leaves us with -2cos(x)sin(x)sin(cos^2(x)). Based on our trig identity, we can rewrite this as -sin(2x)sin(cos^2(x)).

Let's try our hand at the second example: d/dx e^(3x^3). We know that the derivative of e^x is e^x, so in the derivative function we will still have e^(3x^3). Now let's differentiate the 3x^3 term to get 9x^2. So our derivative is 9x^2 * e^(3x^3).

Tangent Line Approximation
Earlier in the tutorial, we said that the derivative of a function represents the slope of a tangent line at a point. So let's try our hand at calculating a tangent line for a function f(x) = 3x^3 + e^2x + ln(5x) at point x = 3.

Next, let's find the derivative of f(x). We can determine that f'(x) = 9x^2 + 2e^2x + 5/x by our chain rule, as well as the other derivative rules listed above. From this, we get f'(3) = 81 + 2e^6 + 5/6. Since f'(3) is our slope, we simply plug the numbers into the formula:

y - (81 + e^6 + ln(30)) = (81 + 2e^6 + 5/6)(x-(81+e^6+ln(30))

Conclusion
This concludes my first tutorial. The next tutorial will cover applications of the derivative, interpreting what a derivative means in context, the first and second derivative tests, and L'Hospital's rule.

Replies To: A Calculus Primer Part I- Introduction to Derivatives

I did calculus 2 years ago when i was in school and the things you mentioned above were the things that we covered. I really wanted to deepen my knowledge in the subject so i covered on my own partial derivatives. Then i stopped. I hope you will expand the tutorial to more parts so that i can learn even more!
Good stuff.

Great intro! I'm relearning calculus right now and it's a good reminder.

A great tool to learning math in general is the Khan Academy. There are 1000s of videos and their practice section in the website is well thought out.
If anyone is having trouble with Algebra I or II, Calculus, Geometry... I suggest you guys check it out.

@peace_fixation: Sorry for the late reply. I just saw this post! DIC doesn't support LaTeX. However, for my future math-related tutorials, I'm planning on including a PDF typeset version where it would otherwise be difficult to follow the math.

I attempted calculus I -- twice -- and because of lousy professors -- at least that is what I think it was -- did not complete. I still enjoy the hell out of math and am trying to build on math fundamentals in programming despite earlier fallacies.