Machine Learning, a Bayesian Perspective

This course provides a solid mathematical foundation for machine learning algorithms, presented from a statistical (Bayesian) and optimization point of view. It is considered to be a second (more advanced) course, after an introductory course that would have presented the general concepts.

Machine Learning is an umbrella for a range of methods coming from different scientific communities, such as (EE) statistical learning, statistical signal processing, adaptive signal processing, image processing and analysis, system identification and control, and (CS) pattern recognition, data mining and information retrieval, computer vision, and computational learning. The name 'Machine Learning' indicates what these disciplines have in common: to learn from data, and then make predictions.

What one tries to learn from data is their underlying structure and regularities, via the development of a model, which can then be used to provide predictions. To this end, a number of diverse approaches have been developed, ranging from optimization of cost functions, whose goal is to optimize the deviation between what one observes from data and what the model predicts, to probabilistic models that attempt to model the statistical properties of the observed data.

The course covers 3 introductory chapters and 9 regular chapters of a recently published rather exhaustive book. The introductory chapters summarize the mathematical language used in the course: conditional probability, distributions, random processes (as also seen in EE4C03), and repeat basic tools in Estimation (regression) and Detection (classification), as seen in ET4386. The course itself then presents three groups of techniques:

- Bayesian learning: inference; conjugate priors; the EM algorithm; Gaussian mixture models; non-parametric models; variational bound approximation technique; sparse Bayesian learning; relevance vector machine (RVM).

- Graphical models: Bayesian networks, factor graphs, message passing algorithms. Hidden Markov models. Learning graphical models.

- Neural networks and deep learning: (probabilistic) CNN, variational autoencoder, GAN, learning deep networks.