Università degli studi di Modena e Reggio Emilia

You are here: Home » Study Plan » Subject

Contents download

Teachers

Luca ZANNI
Giorgia FRANCHINI

Overview

Introduction to basic mathematical concepts for using Machine Learning methodologies.
Introduction to the main numerical methods for deterministic and stochastic optimization; application of these methods to large-scale machine learning problems.

Admission requirements

Basic ideas in computer programming, mathematical analysis and linear algebra.

Course contents

Introduction to Machine Learning (2 CFU, 14 hours)

Machine Learning problems (supervised, semisupervised, online/incremental).
Supervised learning: Loss Function, empirical risk, hypotheses space.
Empirical risk minimization: overfitting and generalization property.
Regularization theory for machine learning: regularization functionals and complexity of the solution. Hypotheses spaces defined by Kernels. Linear and nonlinear Kernels. The Representer Theorem.


Numerical optimization (3 CFU, 21 hours)

Mathematical background: differentiable functions and convex analysis.
Unconstrained optimization:
Optimality conditions, gradient methods (convergence and convergence rate).
Acceleration techniques: adaptive steplength and quasi-Newton strategies. Conjugate direction methods and conjugate gradient method.
Constrained optimization: optimality conditions, KKT conditions and duality results. Gradient projection methods: descent direction, convergence properties and acceleration techniques.
Optimization problems in training Support Vector Machines: the SVM learning methodology for binary classification, the SVM dual problem, decomposition techniques and software for SVM training.

Stochastic Optimization (1 CFU, 7 hours)

Stochastic gradient methods and their convergence properties. Acceleration strategies: dynamic sample size methods, gradient aggregation, inexact Newton methods, diagonal scaling methods. Stochastic optimization in machine learning: empirical risk minimization, regularized models, stochastic gradient methods for large-scale problems.

Teaching methods

Lectures and practical exercises in computer laboratory. Working students who can not attend classes regularly, shall contact the teacher in order to verify the teaching material necessary for the final examination. Student Consultation hours: Friday, 15.00-18.00 at the Department of Physics, Informatics and Mathematics or by appointment requested by an e-mail.

Assessment methods

The evaluation consists in an oral assessment at the end of the course. The test will last about 30 minutes. In the oral assessment, the machine learning methodologies and the numerical methods presented in the course will be discussed. The questions posed during the examination are aimed at evaluating: - knowledge and understanding skills; - the application of knowledge and understanding; - communication skills; - autonomy of judgment. The grade reported in the exam is given by the overall evaluation of the acquired skills and the expository rigor shown during the discussion. The result will be communicated to the individual student at the end of the oral exam.

Learning outcomes

Knowledge and understanding:
At the end of the course, the student will know the
mathematical notions at the basis of the machine learning methodologies and the numerical optimization methods useful for training these methodologies.

Applying knowledge and understanding:
At the end of the course, the student will have adequate knowledge for using machine learning methodologies and numerical optimization methods

Making judgements:
At the end of the course, the student must be able to choose the appropriate approaches for solving machine learning problems and the numerical optimization methods suitable for training a machine learning methodology.

Communicating skills:
At the end of the course, the student must be able to explain the machine learning concepts and the numerical optimization methods presented in the course.

Learning skills:
At the end of the course, it is hoped that the students will be able to:
a) acquire computational knowledge as one's own heritage, which can be used at any other moment of one's cultural path;
b) have developed an aptitude for a methodological approach that leads to an improvement of the study method with consequent deepening of the ability to learn.

Readings

Lecture notes provided during the course

Scholkopf B., Smola A.J., Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press Cambridge, MA, USA, 2001

Bishop C.M., Pattern Recognition and Machine Learning, Springer, 2006

Bertsekas D.P., Nonlinear Programming, Athena Scientific,1999.

Nocedal J., Wright S.J., Numerical Optimization, Springer-Verlag, 2000.

Bottou, L., Curtis, F.E., Nocedal, J., Optimization methods for large-scale machine learning, SIAM Review, 2018.