This course offers a brief introduction to the multivariate calculus required to build many common machine learning techniques. We start at the very beginning with a refresher on the “rise over run” formulation of a slope, before converting this to the formal definition of the gradient of a function. We then start to build up a set of tools for making calculus easier and faster. Next, we learn how to calculate vectors that point up hill on multidimensional surfaces and even put this into action using an interactive game. We take a look at how we can use calculus to build approximations to functions, as well as helping us to quantify how accurate we should expect those approximations to be. We also spend some time talking about where calculus comes up in the training of neural networks, before finally showing you how it is applied in linear regression models. This course is intended to offer an intuitive understanding of calculus, as well as the language necessary to look concepts up yourselves when you get stuck. Hopefully, without going into too much detail, you’ll still come away with the confidence to dive into some more focused machine learning courses in future.

DP

Great course to develop some understanding and intuition about the basic concepts used in optimization. Last 2 weeks were a bit on a lower level of quality then the rest in my opinion but still great.

AS

Apr 15, 2018

Filled StarFilled StarFilled StarFilled StarFilled Star

Excellent course!\n\nI studied multivariate calculus during engineering. I hardly understood the concepts at that time, this course helped me understand and visualize what is going behind formulas.

从本节课中

Taylor series and linearisation

The Taylor series is a method for re-expressing functions as polynomial series. This approach is the rational behind the use of simple linear approximations to complicated functions. In this module, we will derive the formal expression for the univariate Taylor series and discuss some important consequences of this result relevant to machine learning. Finally, we will discuss the multivariate case and see how the Jacobian and the Hessian come in to play.

教学方

Samuel J. Cooper

Lecturer

David Dye

Professor of Metallurgy

A. Freddie Page

Strategic Teaching Fellow

脚本

In this short video, I'm just going to give you one more teaser about what the Taylor series is doing, before we try and write down anything like a formal definition. This will allow you to have a go at some graphical questions first, which is much like how we approach learning differentiation at the start of this course. Taylor series are also referred to as power series. And this is because they are composed of coefficients in front of increasing powers of x. So we can write a simple generalised expression for a power series as g of x, equals a, plus bx, plus cx squared, plus dx cubed et cetera. Potentially, going off for infinitely many times depending on what function we're considering. When we calculate a Taylor series in the next video, we will build up coefficient by coefficient, where each term that we add improves the approximation. In many cases, we will then be able to see a pattern emerge in the coefficients, which thankfully saves us the trouble of calculating infinitely many terms. However, many of the applications of Taylor series, involve making use of just the first few terms of the series, in the hope that this will be a good enough approximation for a certain application. Starting from just a single term, we call these expressions the zeroth, first, second, and third order approximations etc. Collectively, these short sections of the series are called Truncated series. So let's begin by looking at some arbitrary, but fairly complicated functions. If we start with something like this, where we have a function, let me just make it up. Perhaps, shaped like this. All we're going to do is focus on one particular point on this curve. So let's say, this point here. And then we're going to start building our function by trying to make it more, and more like the point that we've chosen. So, as the first term of our generalised power series is just a number a, and we are ignoring all the other terms for now. We know that our opening approximation must just be a number that goes through the same point. So we can just dive straight in, and add our zeroth order approximation function to our plot. It's going to be something like that. Clearly, this hasn't done a great job with approximating the red curve. So let's now go to our first order approximation. This thing can also have a gradient. And if we'd like to match our function at this point, it should have the same gradient. So we can have something a bit more like this. Which is supposed to be a straight line. Clearly, our approximation has improved a little in the region around our point. Although, there is still plenty of room for improvement. We can of course move onto the second order function, which as we can see is a parabola. Although at this point, things get a little tough to draw. And matching second derivatives by eye is also not easy. But it might look something like this. So we can say, okay let's go up and we're going to come down like that. Hopefully, without having gone into any of the details about the maths, you'll now be able to match up some mystery functions to their corresponding truncated taylor series approximations in the following exercises. In the next video, we're going to work through the detailed derivation of the terms. But I hope this activity will help you not to lose sight of what we're trying to achieve in the end. See you then.