Sciences
Subject: FONDAMENTI DI MACHINE LEARNING (A.A. 2023/2024)
master degree course in COMPUTER SCIENCE
Course year  1 

CFU  6 
Teaching units 
Unit Fondamenti di Machine Learning
Related or Additional Studies (lesson)

Moodle portal  
Exam type  oral 
Evaluation  final vote 
Teaching language  Italiano 
Teachers
Overview
Introduction to basic mathematical concepts for using Machine Learning methodologies.
Introduction to the main numerical methods for deterministic and stochastic optimization; application of these methods to largescale machine learning problems.
Admission requirements
Basic ideas in computer programming, mathematical analysis and linear algebra.
Course contents
Introduction to Machine Learning (2 CFU, 14 hours)
Machine Learning problems (supervised, semisupervised, online/incremental).
Supervised learning: Loss Function, empirical risk, hypotheses space.
Empirical risk minimization: overfitting and generalization property.
Regularization theory for machine learning: regularization functionals and complexity of the solution. Hypotheses spaces defined by Kernels. Linear and nonlinear Kernels. The Representer Theorem.
Numerical optimization (3 CFU, 21 hours)
Mathematical background: differentiable functions and convex analysis.
Unconstrained optimization:
Optimality conditions, gradient methods (convergence and convergence rate).
Acceleration techniques: adaptive steplength and quasiNewton strategies. Conjugate direction methods and conjugate gradient method.
Constrained optimization: optimality conditions, KKT conditions and duality results. Gradient projection methods: descent direction, convergence properties and acceleration techniques.
Optimization problems in training Support Vector Machines: the SVM learning methodology for binary classification, the SVM dual problem, decomposition techniques and software for SVM training.
Stochastic Optimization (1 CFU, 7 hours)
Stochastic gradient methods and their convergence properties. Acceleration strategies: dynamic sample size methods, gradient aggregation, inexact Newton methods, diagonal scaling methods. Stochastic optimization in machine learning: empirical risk minimization, regularized models, stochastic gradient methods for largescale problems.
Teaching methods
Lectures and practical exercises in computer laboratory. The course is delivered in Italian in presence. Attendance is not mandatory, but strongly recommended, especially for the laboratory part. Working students who can not attend classes regularly, shall contact the teacher in order to verify the teaching material necessary for the final examination. Student Consultation hours: Friday, 15.0018.00 at the Department of Physics, Informatics and Mathematics or by appointment requested by an email.
Assessment methods
The evaluation consists in an oral assessment at the end of the course. The test will consist of 4 questions and will last about 30 minutes. In the oral assessment, the machine learning methodologies and the numerical methods presented in the course will be discussed. The questions posed during the examination are aimed at evaluating:  knowledge and understanding skills;  the application of knowledge and understanding;  communication skills;  autonomy of judgment. The grade reported in the exam is given by the overall evaluation of the acquired skills and the expository rigor shown during the discussion. In particular, a minimum mark (18/30) will be obtained in case of basic knowledge of the topics and partial ability to apply the knowledge, while a maximum mark (30/30 cum laude) will be reserved for exams characterized by full knowledge of the topics and excellent ability to apply the knowledge, with a graduation of intermediate marks based on the achievement of the expected learning outcomes demonstrated during the exam. The result will be communicated to the individual student at the end of the oral exam.
Learning outcomes
Knowledge and understanding:
At the end of the course, the student will know the
mathematical notions at the basis of the machine learning methodologies and the numerical optimization methods useful for training these methodologies.
Applying knowledge and understanding:
At the end of the course, the student will have adequate knowledge for using machine learning methodologies and numerical optimization methods
Making judgements:
At the end of the course, the student must be able to choose the appropriate approaches for solving machine learning problems and the numerical optimization methods suitable for training a machine learning methodology.
Communicating skills:
At the end of the course, the student must be able to explain the machine learning concepts and the numerical optimization methods presented in the course.
Learning skills:
At the end of the course, it is hoped that the students will be able to:
a) acquire computational knowledge as one's own heritage, which can be used at any other moment of one's cultural path;
b) have developed an aptitude for a methodological approach that leads to an improvement of the study method with consequent deepening of the ability to learn.
Readings
Lecture notes provided during the course, in the form of slides and videos of lectures on Moodle.
Scholkopf B., Smola A.J., Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press Cambridge, MA, USA, 2001
Bishop C.M., Pattern Recognition and Machine Learning, Springer, 2006
Bertsekas D.P., Nonlinear Programming, Athena Scientific,1999.
Nocedal J., Wright S.J., Numerical Optimization, SpringerVerlag, 2000.
Bottou, L., Curtis, F.E., Nocedal, J., Optimization methods for largescale machine learning, SIAM Review, 2018.