227-0423-00L  Neural Network Theory

SemesterAutumn Semester 2021
LecturersH. Bölcskei
Periodicityyearly recurring course
Language of instructionEnglish


227-0423-00 VNeural Network Theory2 hrs
Tue10:15-12:00HG F 5 »
H. Bölcskei
227-0423-00 UNeural Network Theory
The exercise will take place online on: Link.
The reserved room is meant for those students who want to follow the course from the campus.
1 hrs
Tue12:15-13:00HG F 5 »
H. Bölcskei

Catalogue data

AbstractThe class focuses on fundamental mathematical aspects of neural networks with an emphasis on deep networks: Universal approximation theorems, capacity of separating surfaces, generalization, fundamental limits of deep neural network learning, VC dimension.
ObjectiveAfter attending this lecture, participating in the exercise sessions, and working on the homework problem sets, students will have acquired a working knowledge of the mathematical foundations of neural networks.
Content1. Universal approximation with single- and multi-layer networks

2. Introduction to approximation theory: Fundamental limits on compressibility of signal classes, Kolmogorov epsilon-entropy of signal classes, non-linear approximation theory

3. Fundamental limits of deep neural network learning

4. Geometry of decision surfaces

5. Separating capacity of nonlinear decision surfaces

6. Vapnik-Chervonenkis (VC) dimension

7. VC dimension of neural networks

8. Generalization error in neural network learning
Lecture notesDetailed lecture notes are available on the course web page
Prerequisites / NoticeThis course is aimed at students with a strong mathematical background in general, and in linear algebra, analysis, and probability theory in particular.

Performance assessment

Performance assessment information (valid until the course unit is held again)
Performance assessment as a semester course
ECTS credits4 credits
ExaminersH. Bölcskei
Typesession examination
Language of examinationEnglish
RepetitionThe performance assessment is offered every session. Repetition possible without re-enrolling for the course unit.
Mode of examinationwritten 180 minutes
Written aidsNone
This information can be updated until the beginning of the semester; information on the examination timetable is binding.

Learning materials

Main linkLecture Website
Only public learning materials are listed.


No information on groups available.


There are no additional restrictions for the registration.

Offered in

Data Science MasterInformation and LearningWInformation
Doctoral Department of MathematicsGraduate SchoolWInformation
Electrical Engineering and Information Technology MasterCore SubjectsWInformation
Electrical Engineering and Information Technology MasterAdvanced Core CoursesWInformation
Electrical Engineering and Information Technology MasterSpecialisation CoursesWInformation
Electrical Engineering and Information Technology MasterRecommended SubjectsWInformation
Computer Science MasterComputer Science Elective CoursesWInformation
Mathematics MasterSelection: Further RealmsWInformation
Physics MasterGeneral ElectivesWInformation
Computational Science and Engineering MasterElectivesWInformation
Statistics MasterStatistical and Mathematical CoursesWInformation
Statistics MasterSubject Specific ElectivesWInformation