252-0526-00L  Statistical Learning Theory

SemesterSpring Semester 2020
LecturersJ. M. Buhmann, C. Cotrini Jimenez
Periodicityyearly recurring course
Language of instructionEnglish


252-0526-00 VStatistical Learning Theory3 hrs
Mon14:00-16:00ER SA TZ »
14:15-16:00HG G 3 »
Tue17:00-18:00ER SA TZ »
17:15-18:00HG G 3 »
J. M. Buhmann, C. Cotrini Jimenez
252-0526-00 UStatistical Learning Theory2 hrs
Mon16:00-18:00ER SA TZ »
16:15-18:00HG G 3 »
J. M. Buhmann, C. Cotrini Jimenez
252-0526-00 AStatistical Learning Theory1 hrsJ. M. Buhmann, C. Cotrini Jimenez

Catalogue data

AbstractThe course covers advanced methods of statistical learning:

- Variational methods and optimization.
- Deterministic annealing.
- Clustering for diverse types of data.
- Model validation by information theory.
ObjectiveThe course surveys recent methods of statistical learning. The fundamentals of machine learning, as presented in the courses "Introduction to Machine Learning" and "Advanced Machine Learning", are expanded from the perspective of statistical learning.
Content- Variational methods and optimization. We consider optimization approaches for problems where the optimizer is a probability distribution. We will discuss concepts like maximum entropy, information bottleneck, and deterministic annealing.

- Clustering. This is the problem of sorting data into groups without using training samples. We discuss alternative notions of "similarity" between data points and adequate optimization procedures.

- Model selection and validation. This refers to the question of how complex the chosen model should be. In particular, we present an information theoretic approach for model validation.

- Statistical physics models. We discuss approaches for approximately optimizing large systems, which originate in statistical physics (free energy minimization applied to spin glasses and other models). We also study sampling methods based on these models.
Lecture notesA draft of a script will be provided. Lecture slides will be made available.
LiteratureHastie, Tibshirani, Friedman: The Elements of Statistical Learning, Springer, 2001.

L. Devroye, L. Gyorfi, and G. Lugosi: A probabilistic theory of pattern recognition. Springer, New York, 1996
Prerequisites / NoticeKnowledge of machine learning (introduction to machine learning and/or advanced machine learning)
Basic knowledge of statistics.

Performance assessment

Performance assessment information (valid until the course unit is held again)
Performance assessment as a semester course
ECTS credits7 credits
ExaminersJ. M. Buhmann, C. Cotrini Jimenez
Typesession examination
Language of examinationEnglish
RepetitionThe performance assessment is only offered in the session after the course unit. Repetition only possible after re-enrolling.
Mode of examinationwritten 180 minutes
Additional information on mode of examination70% session examination, 30% project; the final grade will be calculated as weighted average of both these elements. As a compulsory continuous performance assessment task, the project must be passed on its own and has a bonus/penalty function.

The practical projects are an integral part (60 hours of work, 2 credits) of the course. Participation is mandatory. Failing the project results in a failing grade for the overall course examination.

Students who fail to fulfil the project requirement must de-register from the exam. Otherwise, they are not admitted to the exam and they will be treated as a no show.
Written aids4 A4 handwritten or fontsize 12 pages (2 sheets with notes on its two sides); course script.
This information can be updated until the beginning of the semester; information on the examination timetable is binding.

Learning materials

Main linkInformation
RecordingStatistical Learning Theory recorings
Only public learning materials are listed.


No information on groups available.


There are no additional restrictions for the registration.

Offered in

CAS in Computer ScienceFocus Courses and ElectivesWInformation
Computational Biology and Bioinformatics MasterTheoryWInformation
Cyber Security MasterElective CoursesWInformation
Cyber Security MasterElectivesWInformation
DAS in Data ScienceMachine Learning and Artificial IntelligenceWInformation
Data Science MasterCore ElectivesWInformation
Electrical Engineering and Information Technology MasterRecommended SubjectsWInformation
Electrical Engineering and Information Technology MasterSpecialization CoursesWInformation
Electrical Engineering and Information Technology MasterSpecialization CoursesWInformation
Electrical Engineering and Information Technology MasterSpecialization CoursesWInformation
Electrical Engineering and Information Technology MasterRecommended SubjectsWInformation
Electrical Engineering and Information Technology MasterRecommended SubjectsWInformation
Computer Science MasterFocus Elective Courses Information SystemsWInformation
Computer Science MasterFocus Elective Courses Computational ScienceWInformation
Computer Science MasterFocus Elective Courses Visual ComputingWInformation
Computer Science MasterElective Focus Courses General StudiesWInformation
Mechanical Engineering MasterRobotics, Systems and ControlWInformation
Computational Science and Engineering MasterElectivesWInformation
Robotics, Systems and Control MasterCore CoursesWInformation
Statistics MasterStatistical and Mathematical CoursesWInformation