261-5110-00L  Optimization for Data Science

SemesterSpring Semester 2021
LecturersB. Gärtner, D. Steurer, N. He
Periodicityyearly recurring course
Language of instructionEnglish


261-5110-00 VOptimization for Data Science3 hrs
Mon13:15-14:00NO C 60 »
Tue10:15-12:00ETF C 1 »
B. Gärtner, D. Steurer, N. He
261-5110-00 UOptimization for Data Science2 hrs
Tue14:15-16:00HG D 5.2 »
14:15-16:00ML H 44 »
B. Gärtner, D. Steurer, N. He
261-5110-00 AOptimization for Data Science4 hrsB. Gärtner, D. Steurer, N. He

Catalogue data

AbstractThis course provides an in-depth theoretical treatment of optimization methods that are particularly relevant in data science.
ObjectiveUnderstanding the theoretical guarantees (and their limits) of relevant optimization methods used in data science. Learning general paradigms to deal with optimization problems arising in data science.
ContentThis course provides an in-depth theoretical treatment of optimization methods that are particularly relevant in machine learning and data science.

In the first part of the course, we will first give a brief introduction to convex optimization, with some basic motivating examples from machine learning. Then we will analyse classical and more recent first and second order methods for convex optimization: gradient descent, Nesterov's accelerated method, proximal and splitting algorithms, subgradient descent, stochastic gradient descent, variance-reduced methods, Newton's method, and Quasi-Newton methods. The emphasis will be on analysis techniques that occur repeatedly in convergence analyses for various classes of convex functions. We will also discuss some classical and recent theoretical results for nonconvex optimization.

In the second part, we discuss convex programming relaxations as a powerful and versatile paradigm for designing efficient algorithms to solve computational problems arising in data science. We will learn about this paradigm and develop a unified perspective on it through the lens of the sum-of-squares semidefinite programming hierarchy. As applications, we are discussing non-negative matrix factorization, compressed sensing and sparse linear regression, matrix completion and phase retrieval, as well as robust estimation.
Prerequisites / NoticeAs background, we require material taught in the course "252-0209-00L Algorithms, Probability, and Computing". It is not necessary that participants have actually taken the course, but they should be prepared to catch up if necessary.

Performance assessment

Performance assessment information (valid until the course unit is held again)
Performance assessment as a semester course
ECTS credits10 credits
ExaminersB. Gärtner, N. He, D. Steurer
Typesession examination
Language of examinationEnglish
RepetitionThe performance assessment is offered every session. Repetition possible without re-enrolling for the course unit.
Mode of examinationwritten 180 minutes
Additional information on mode of examinationAt two times in the course of the semester, we will hand out specially marked exercises or term projects (compulsory continuous performance assessments) - the written part of the solutions are expected to be typeset in LaTeX or similar. Solutions will be graded, and the grades will account for 20% of the final grade. Assignments can be discussed with colleagues, but we expect an independent writeup.
Written aidsNone
This information can be updated until the beginning of the semester; information on the examination timetable is binding.

Learning materials

Main linkInformation
Only public learning materials are listed.


No information on groups available.


There are no additional restrictions for the registration.

Offered in

CAS in Computer ScienceFocus Courses and ElectivesWInformation
Cyber Security MasterCore CoursesWInformation
DAS in Data ScienceMachine Learning and Artificial IntelligenceWInformation
Data Science MasterData ManagementWInformation
Computer Science MasterCore Focus Courses General StudiesWInformation
Computer Science MasterCore CoursesWInformation
Computer Science MasterFocus Core Courses Theoretical Computer ScienceWInformation
Computer Science MasterCore CoursesWInformation
Computer Science MasterMinor in Theoretical Computer ScienceWInformation
Computational Science and Engineering MasterCore CoursesWInformation
Statistics MasterStatistical and Mathematical CoursesWInformation
Statistics MasterSubject Specific ElectivesWInformation