What is this course about?
Machine learning algorithms aim to "make sense of data", often
developing some generalizations that are useful for
future tasks, for example in classification, prediction, and control.
Computational Learning Theory investigates such tasks and algorithms
in a formal framework. The focus is to identify performance guarantees
for algorithms in terms of computational complexity (how long it takes
to run), data complexity (how much data is required), and convergence
properties (how well does the result/output of the learning algorithm
perform).
Alternatively, for some machine learning
problems we seek lower bounds on the amount of resources for any
potential algorithm.
Models vary from adversarial worst case scenarios,
to statistical settings where a random process generates the data, and
to interactive settings where the learner can control the flow of
data.
The course will mix taught classes with seminar type reading of
research papers of recent topics.
This semester: In addition to a review of core topics,
this semester we will explore online convex optimization, bandit
problems, PAC-Bayes theorems, learning theory for pattern recognition
models (GMM, HMM), matrix completion problems, and maybe other topics.
For reference
see
details from 2008 offering

Activities & Grading

The course will mix taught classes with seminar type reading of
research papers.
Course work will include
writing up official lecture notes, reading and
presenting papers, a small number of homework assignments,
and an in-class exam on Thursday 4/25. Details are as follows:

Lecture notes: students will write formal lecture
notes that will be handed back to the class. Notes will be written
using the Latex package. I will provide a latex template for the notes.
I expect each student to cover 1-2 lectures at some point during the semester.

Paper presentations: students will read papers from recent literature
and present these to the class.
A report (1-2 pages) on the paper is due with the
presentation. The presentation and report should explain the main
results, and highlights from proofs or constructions in the papers.
I expect each student to take 2-3 presentations during the semester.

Lecture notes and presentations together contribute 30% of the
final grade.

There will be 3-4 assignments involving problem solving and
proofs. Assignments contribute 40% of the
final grade.

An in-class exam will take place on Thursday 4/25.
The exam contributes 30% of the
final grade.

Policy on collaboration on homework assignments: You may
discuss the problems and general ideas about their solutions with
other students, and similarly you may consult other textbooks or the
web. However, you must work out the details on your own and write the
solution on your own. Every such collaboration (either getting help of
giving help) and every use of text or electronic sources must be
clearly cited and acknowledged in the submitted homework.
Failure to
follow these guidelines will result in disciplinary action for all
parties involved.
Any questions? for this and other issues concerning
academic integrity please consult the
booklet available from the office of the Dean
of Student Affairs.

Texts/Reference

We will sample from different sources including some of the texts
below (on reserve in library) and research and survey articles.