Theoretical Foundations of Machine Learning
This module introduces the theoretical foundations of machine learning, with a focus on learning theory and its role in understanding the performance of learning algorithms. Students will learn how to formally analyze the generalization ability of models, understand the role of hypothesis spaces, and derive guarantees on learning performance. The course covers both classical learning theory and modern theoretical approaches that have emerged in response to the success of deep learning, where many classical assumptions break down.
Code : INF-MSc-530
Programs: Master Computer Science, Master Applied Computer Science, Master Data Science
Language: English
We will critically examine where traditional tools such as uniform convergence and VCdimension fall short, and explore recent perspectives that aim to explain the generalization behavior of highly overparameterized models.
• Probably Approximately Correct (PAC)
• Hypothesis classes, loss functions, and empirical risk minimizat
• Generalization bounds via VC-dimension, Rademacher complexity, and uniform convergence
• Algorithmic stability and regularization
• The failure of classical theory in deep learning: memorization and double descent
• Modern approaches to deep learning theory