Machine Learning

Machine Learning

Machine Learning

Stanford University

About this course: Machine learning is the science of getting computers to act without being explicitly programmed. In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it. Many researchers also think it is the best way to make progress towards human-level AI. In this class, you will learn about the most effective machine learning techniques, and gain practice implementing them and getting them to work for yourself. More importantly, you'll learn about not only the theoretical underpinnings of learning, but also gain the practical know-how needed to quickly and powerfully apply these techniques to new problems. Finally, you'll learn about some of Silicon Valley's best practices in innovation as it pertains to machine learning and AI.
This course provides a broad introduction to machine learning, datamining, and statistical pattern recognition. Topics include: (i) Supervised learning (parametric/non-parametric algorithms, support vector machines, kernels, neural networks). (ii) Unsupervised learning (clustering, dimensionality reduction, recommender systems, deep learning). (iii) Best practices in machine learning (bias/variance theory; innovation process in machine learning and AI). The course will also draw from numerous case studies and applications, so that you'll also learn how to apply learning algorithms to building smart robots (perception, control), text understanding (web search, anti-spam), computer vision, medical informatics, audio, database mining, and other areas.

Welcome to Machine Learning! In this module, we introduce the core idea of teaching a computer to learn concepts using data—without being explicitly programmed. The Course Wiki is under construction. Please visit the resources tab for the most complete and up-to-date information.

5 videos, 9 readings

Video: Welcome to Machine Learning!

Reading: Machine Learning Honor Code

Video: Welcome

Video: What is Machine Learning?

Reading: What is Machine Learning?

Reading: How to Use Discussion Forums

Video: Supervised Learning

Reading: Supervised Learning

Video: Unsupervised Learning

Reading: Unsupervised Learning

Reading: Who are Mentors?

Reading: Get to Know Your Classmates

Reading: Frequently Asked Questions

Reading: Lecture Slides

Graded: Introduction

Linear Regression with One Variable

Linear regression predicts a real-valued output based on an input value. We discuss the application of linear regression to housing price prediction, present the notion of a cost function, and introduce the gradient descent method for learning.

7 videos, 8 readings

Video: Model Representation

Reading: Model Representation

Video: Cost Function

Reading: Cost Function

Video: Cost Function - Intuition I

Reading: Cost Function - Intuition I

Video: Cost Function - Intuition II

Reading: Cost Function - Intuition II

Video: Gradient Descent

Reading: Gradient Descent

Video: Gradient Descent Intuition

Reading: Gradient Descent Intuition

Video: Gradient Descent For Linear Regression

Reading: Gradient Descent For Linear Regression

Reading: Lecture Slides

Graded: Linear Regression with One Variable

Linear Algebra Review

This optional module provides a refresher on linear algebra concepts. Basic understanding of linear algebra is necessary for the rest of the course, especially as we begin to cover models with multiple variables.

6 videos, 7 readings, 1 practice quiz

Video: Matrices and Vectors

Reading: Matrices and Vectors

Video: Addition and Scalar Multiplication

Reading: Addition and Scalar Multiplication

Video: Matrix Vector Multiplication

Reading: Matrix Vector Multiplication

Video: Matrix Matrix Multiplication

Reading: Matrix Matrix Multiplication

Video: Matrix Multiplication Properties

Reading: Matrix Multiplication Properties

Video: Inverse and Transpose

Reading: Inverse and Transpose

Reading: Lecture Slides

Practice Quiz: Linear Algebra

WEEK 2

Linear Regression with Multiple Variables

What if your input has more than one value? In this module, we show how linear regression can be extended to accommodate multiple input features. We also discuss best practices for implementing linear regression.

This course includes programming assignments designed to help you understand how to implement the learning algorithms in practice. To complete the programming assignments, you will need to use Octave or MATLAB. This module introduces Octave/Matlab and shows you how to submit an assignment.

6 videos, 1 reading

Video: Basic Operations

Video: Moving Data Around

Video: Computing on Data

Video: Plotting Data

Video: Control Statements: for, while, if statement

Video: Vectorization

Reading: Lecture Slides

Programming: Linear Regression

Graded: Octave/Matlab Tutorial

WEEK 3

Logistic Regression

Logistic regression is a method for classifying data into discrete outcomes. For example, we might use logistic regression to classify an email as spam or not spam. In this module, we introduce the notion of classification, the cost function for logistic regression, and the application of logistic regression to multi-class classification.

7 videos, 8 readings

Video: Classification

Reading: Classification

Video: Hypothesis Representation

Reading: Hypothesis Representation

Video: Decision Boundary

Reading: Decision Boundary

Video: Cost Function

Reading: Cost Function

Video: Simplified Cost Function and Gradient Descent

Reading: Simplified Cost Function and Gradient Descent

Video: Advanced Optimization

Reading: Advanced Optimization

Video: Multiclass Classification: One-vs-all

Reading: Multiclass Classification: One-vs-all

Reading: Lecture Slides

Graded: Logistic Regression

Regularization

Machine learning models need to generalize well to new examples that the model has not seen in practice. In this module, we introduce regularization, which helps prevent models from overfitting the training data.

4 videos, 5 readings

Video: The Problem of Overfitting

Reading: The Problem of Overfitting

Video: Cost Function

Reading: Cost Function

Video: Regularized Linear Regression

Reading: Regularized Linear Regression

Video: Regularized Logistic Regression

Reading: Regularized Logistic Regression

Reading: Lecture Slides

Programming: Logistic Regression

Graded: Regularization

WEEK 4

Neural Networks: Representation

Neural networks is a model inspired by how the brain works. It is widely used today in many applications: when your phone interprets and understand your voice commands, it is likely that a neural network is helping to understand your speech; when you cash a check, the machines that automatically read the digits also use neural networks.

7 videos, 6 readings

Video: Non-linear Hypotheses

Video: Neurons and the Brain

Video: Model Representation I

Reading: Model Representation I

Video: Model Representation II

Reading: Model Representation II

Video: Examples and Intuitions I

Reading: Examples and Intuitions I

Video: Examples and Intuitions II

Reading: Examples and Intuitions II

Video: Multiclass Classification

Reading: Multiclass Classification

Reading: Lecture Slides

Programming: Multi-class Classification and Neural Networks

Graded: Neural Networks: Representation

WEEK 5

Neural Networks: Learning

In this module, we introduce the backpropagation algorithm that is used to help learn parameters for a neural network. At the end of this module, you will be implementing your own neural network for digit recognition.

8 videos, 8 readings

Video: Cost Function

Reading: Cost Function

Video: Backpropagation Algorithm

Reading: Backpropagation Algorithm

Video: Backpropagation Intuition

Reading: Backpropagation Intuition

Video: Implementation Note: Unrolling Parameters

Reading: Implementation Note: Unrolling Parameters

Video: Gradient Checking

Reading: Gradient Checking

Video: Random Initialization

Reading: Random Initialization

Video: Putting It Together

Reading: Putting It Together

Video: Autonomous Driving

Reading: Lecture Slides

Programming: Neural Network Learning

Graded: Neural Networks: Learning

WEEK 6

Advice for Applying Machine Learning

Applying machine learning in practice is not always straightforward. In this module, we share best practices for applying machine learning in practice, and discuss the best ways to evaluate performance of the learned models.

7 videos, 7 readings

Video: Deciding What to Try Next

Video: Evaluating a Hypothesis

Reading: Evaluating a Hypothesis

Video: Model Selection and Train/Validation/Test Sets

Reading: Model Selection and Train/Validation/Test Sets

Video: Diagnosing Bias vs. Variance

Reading: Diagnosing Bias vs. Variance

Video: Regularization and Bias/Variance

Reading: Regularization and Bias/Variance

Video: Learning Curves

Reading: Learning Curves

Video: Deciding What to Do Next Revisited

Reading: Deciding What to do Next Revisited

Reading: Lecture Slides

Programming: Regularized Linear Regression and Bias/Variance

Graded: Advice for Applying Machine Learning

Machine Learning System Design

To optimize a machine learning algorithm, you’ll need to first understand where the biggest improvements can be made. In this module, we discuss how to understand the performance of a machine learning system with multiple parts, and also how to deal with skewed data.

5 videos, 3 readings

Video: Prioritizing What to Work On

Reading: Prioritizing What to Work On

Video: Error Analysis

Reading: Error Analysis

Video: Error Metrics for Skewed Classes

Video: Trading Off Precision and Recall

Video: Data For Machine Learning

Reading: Lecture Slides

Graded: Machine Learning System Design

WEEK 7

Support Vector Machines

Support vector machines, or SVMs, is a machine learning algorithm for classification. We introduce the idea and intuitions behind SVMs and discuss how to use it in practice.

6 videos, 1 reading

Video: Optimization Objective

Video: Large Margin Intuition

Video: Mathematics Behind Large Margin Classification

Video: Kernels I

Video: Kernels II

Video: Using An SVM

Reading: Lecture Slides

Programming: Support Vector Machines

Graded: Support Vector Machines

WEEK 8

Unsupervised Learning

We use unsupervised learning to build models that help us understand our data better. We discuss the k-Means algorithm for clustering that enable us to learn groupings of unlabeled data points.

5 videos, 1 reading

Video: Unsupervised Learning: Introduction

Video: K-Means Algorithm

Video: Optimization Objective

Video: Random Initialization

Video: Choosing the Number of Clusters

Reading: Lecture Slides

Graded: Unsupervised Learning

Dimensionality Reduction

In this module, we introduce Principal Components Analysis, and show how it can be used for data compression to speed up learning algorithms as well as for visualizations of complex datasets.

7 videos, 1 reading

Video: Motivation I: Data Compression

Video: Motivation II: Visualization

Video: Principal Component Analysis Problem Formulation

Video: Principal Component Analysis Algorithm

Video: Reconstruction from Compressed Representation

Video: Choosing the Number of Principal Components

Video: Advice for Applying PCA

Reading: Lecture Slides

Programming: K-Means Clustering and PCA

Graded: Principal Component Analysis

WEEK 9

Anomaly Detection

Given a large number of data points, we may sometimes want to figure out which ones vary significantly from the average. For example, in manufacturing, we may want to detect defects or anomalies. We show how a dataset can be modeled using a Gaussian distribution, and how the model can be used for anomaly detection.

8 videos, 1 reading

Video: Problem Motivation

Video: Gaussian Distribution

Video: Algorithm

Video: Developing and Evaluating an Anomaly Detection System

Video: Anomaly Detection vs. Supervised Learning

Video: Choosing What Features to Use

Video: Multivariate Gaussian Distribution

Video: Anomaly Detection using the Multivariate Gaussian Distribution

Reading: Lecture Slides

Graded: Anomaly Detection

Recommender Systems

When you buy a product online, most websites automatically recommend other products that you may like. Recommender systems look at patterns of activities between different users and different products to produce these recommendations. In this module, we introduce recommender algorithms such as the collaborative filtering algorithm and low-rank matrix factorization.

6 videos, 1 reading

Video: Problem Formulation

Video: Content Based Recommendations

Video: Collaborative Filtering

Video: Collaborative Filtering Algorithm

Video: Vectorization: Low Rank Matrix Factorization

Video: Implementational Detail: Mean Normalization

Reading: Lecture Slides

Programming: Anomaly Detection and Recommender Systems

Graded: Recommender Systems

WEEK 10

Large Scale Machine Learning

Machine learning works best when there is an abundance of data to leverage for training. In this module, we discuss how to apply the machine learning algorithms with large datasets.

6 videos, 1 reading

Video: Learning With Large Datasets

Video: Stochastic Gradient Descent

Video: Mini-Batch Gradient Descent

Video: Stochastic Gradient Descent Convergence

Video: Online Learning

Video: Map Reduce and Data Parallelism

Reading: Lecture Slides

Graded: Large Scale Machine Learning

WEEK 11

Application Example: Photo OCR

Identifying and recognizing objects, words, and digits in an image is a challenging task. We discuss how a pipeline can be built to tackle this problem and how to analyze and improve the performance of such a system.

5 videos, 1 reading

Video: Problem Description and Pipeline

Video: Sliding Windows

Video: Getting Lots of Data and Artificial Data

Video: Ceiling Analysis: What Part of the Pipeline to Work on Next

Reading: Lecture Slides

Video: Summary and Thank You

Graded: Application: Photo OCR

FAQs

How It Works

Coursework

Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.

Help from Your Peers

Connect with thousands of other learners and debate ideas, discuss course material,
and get help mastering concepts.

Certificates

Earn official recognition for your work, and share your success with friends,
colleagues, and employers.

Creators

Stanford University

The Leland Stanford Junior University, commonly referred to as Stanford University or Stanford, is an American private research university located in Stanford, California on an 8,180-acre (3,310 ha) campus near Palo Alto, California, United States.

Ratings and Reviews

Rated 4.9 out of 5 of 49,542 ratings

m

I think this is a best entry point into machine learning world!

Easy to understand, a lot of examples and a lot of inspiring ideas!

I'm very happy and proud to have studied with Andrew Ng!

Thank you so much, Andrew!

And thanks to the whole Coursera Team!

jm

best ML class ever

MT

Andrew is a true master at machine learning, and has designed this course extremely well, covering a lot of material succinctly and also covering all the fundamental concepts very well, so that even if students like me cannot call them experts in calculus, they can still complete the course. The course and exercises are challenging at times, but well worth the effort. Thanks very much to Andrew and his team for putting together an excellent course on a complex subject.