Regularization Methods for High Dimensional Learning 2011

A 20 hours course designed to provide a self contained introduction to state of the art statistical learning techniques for the analysis of complex high dimensional data. It is designed to be accessible to graduate students in computational sciences.

Understanding how intelligence work and how it can be emulated in machines has been an elusive problem for decades and it is arguably one of the biggest challenges in modern science. Learning, its principles, and computational implementations are at the very core of this endeavor. Only recently we have been able, for the first time, to develop artificial intelligence systems able to solve complex tasks that were considered out of reach for several decades. Modern camera can recognize faces, and smart phones recognize people voice, car provided with cameras can detect pedestriansat human level performance and ATM machines can automatically read checks. In most cases at the root of these success stories there are machine learning algorithms, that is softwares that are trained rather than programmed to solve a task.

Among the variety of approaches and ideas in modern computational learning , we focus on a core class of methods, namely regularization methods, which represents a fundamental set of concepts and techniques that allow to treat in a unified way a huge class of diverse approaches, while providing the tools to design new ones. Staring from classical notions of smoothness, shrinkage and margin, we will cover state of the art techniques based on the concepts of geometry (e.g. manifold learning) sparsity, low rank, allowing to design algorithm for tasks such as supervised learning, feature selection, structured prediction, multitask learning and model selection.
Practical applications will be discussed, primarily from the field of computational vision.

The classes will focus on algorithmic and methodological aspects, while trying to give an idea of the underlying theoretical underpinnings. Practical laboratory sessions will give the opportunity to have hands on experience.