Browse Events

Splash is Vator’s once-a-quarter evening event, celebrating entrepreneurship with seasoned entrepreneurs sharing lessons and advice, and 10 promising startups pitching onstage.
The top 10 are chosen through an online competition.
The event draws some 350 to 400 attendees in the entrepreneur community, from seasoned to emerging entrepreneurs, venture capitalists and media.

The purpose of the tutorial is to promote an appreciation of the need for rigorous and objective evaluation and an understanding of the available alternatives along with their assumptions, constraints and context of application. Machine learning researchers and practitioners alike will all benefit from the contents of the tutorial, which discusses the need for sound evaluation strategies, practical approaches and tools for evaluation, going well beyond those described in existing machine learning and data mining textbooks, so far.

Examples of popular latent variable models include latent tree
graphical models and dynamical system models, both of which occupy a
fundamental place in engineering, control theory, economics as well as
the physical, biological, and social sciences. Unfortunately, to
discover the right latent state representation and model parameters,
we must solve difficult structural and temporal credit assignment
problems. Work on learning latent variable structure has
predominantly relied on likelihood maximization and local search
heuristics such as expectation maximization (EM); these heuristics
often lead to a search space with a host of bad local optima, and may
therefore require impractically many restarts to reach a prescribed
training precision.

This tutorial will focus on a recently-discovered class of spectral learning algorithms. These algorithms hold the promise of
overcoming these problems and enabling learning of latent structure in
tree and dynamical system models. Unlike the EM algorithm, spectral
methods are computationally efficient, statistically consistent, and
have no local optima; in addition, they can be simple to implement,
and have state-of-the-art practical performance for many interesting
learning problems.

We will describe the main theoretical, algorithmic, and empirical
results related to spectral learning algorithms, starting with an
overview of linear system identification results obtained in the last
two decades, and then focusing on the remarkable recent progress in
learning nonlinear dynamical systems, latent tree graphical models,
and kernel-based nonparametric models.

PAC-Bayesian analysis is a basic and very general tool for
data-dependent analysis in machine learning. By now, it has been
applied in such diverse areas as supervised learning, unsupervised
learning, and reinforcement learning, leading to state-of-the-art
algorithms and accompanying generalization bounds. PAC-Bayesian
analysis, in a sense, takes the best out of Bayesian methods and
PAC learning and puts it together: (1) it provides an easy way to
exploit prior knowledge (like Bayesian methods); (2) it provides
strict and explicit generalization guarantees (like VC theory);
and (3) it is data-dependent and provides an easy and strict way
of exploiting benign conditions (like Rademacher complexities). In
addition, PAC-Bayesian bounds directly lead to efficient learning
algorithms.

We will start with a general introduction to PAC-Bayesian
analysis, which should be accessible to an average student, who is
familiar with machine learning at the basic level. Then, we will
survey multiple forms of PAC-Bayesian bounds and their numerous
applications in different fields (including supervised and
unsupervised learning, finite and continuous domains, and the very
recent extension to martingales and reinforcement learning). Some
of these applications will be explained in more details, while
others will be surveyed at a high level. We will also describe the
relations and distinctions between PAC-Bayesian analysis, Bayesian
learning, VC theory, and Rademacher complexities. We will discuss
the role, value, and shortcomings of frequentist bounds that are
inspired by Bayesian analysis.