401-3620-70L  Student Seminar in Statistics: Multiple Testing for Modern Data Science

SemesterAutumn Semester 2020
LecturersM. Löffler, A. Taeb
Periodicityevery semester recurring course
Language of instructionEnglish
CommentNumber of participants limited to 24

Mainly for students from the Mathematics Bachelor and Master Programmes who, in addition to the introductory course unit 401-2604-00L Probability and Statistics, have heard at least one core or elective course in statistics. Also offered in the Master Programmes Statistics resp. Data Science.


401-3620-00 SStudent Seminar in Statistics: Multiple Testing for Modern Data Science2 hrs
Mon16:00-18:00ON LI NE »
M. Löffler, A. Taeb

Catalogue data

AbstractThe course encompasses a review of approaches to multiple testing.
ObjectiveThe students understand the relevance of multiple testing in modern applications. Further, they learn about two commonly used measures -- namely family-wise-error-rate (FWER) and false discovery rate (FDR) -- and approaches to control for them.
ContentIn modern statistical applications it is often desired to perform thousands of statistical tests simultaneously. Performing a test at a desired level (e.g. 0.05) for each variable separately will result in many false positives. In science this is known as the ‘reproducibility crisis’.
In this seminar we will review and discuss approaches to deal with this issue. First, we will consider the strong notion of FWER and how to control it via Bonferroni correction, permutation tests, step-up and hierarchical procedures or Tukey’s higher criticism. In the second part of the seminar we will investigate the less conservative FDR, discussing the classical Benjamini-Hochberg procedure, as well as more modern methods such as Knockoffs and Bayesian approaches. Throughout, we highlight the utility of discussed methods for real world applications.
LiteratureLecture 1: Bonferroni and Simes
Link Link
Lecture 2: Permutation tests
Link Link
Lecture 3: Hierarchical testing
Lecture 4: Higher criticism
Methodology: Link and for theoretical reference Link
Application: Link and for more reference
Lecture 5: Benjamini-Hochberg (BH) with martingales
Link, Link
Lecture 6: FDR control under dependence
Lecture 7: Empirical null distribution
Lecture 8: Bayes FDR methods
Lecture 9: SLOPE
Lecture 10: Knockoffs
Lecture 11: Generalization of FWER and connections to FDR
Lecture 12: Exploratory testing
Prerequisites / NoticeEvery lecture will consist of an oral presentation highlighting key ideas of selected papers by a pair of students. Another two students will be responsible for asking questions during the presentation and providing a discussion of the pros+cons of the papers at the end. Finally, an additional two students are responsible for giving an evaluation on the quality of the presentations/discussions and provide constructive feedback for improvement.

Performance assessment

Performance assessment information (valid until the course unit is held again)
Performance assessment as a semester course
ECTS credits4 credits
ExaminersM. Löffler, A. Taeb
Typeungraded semester performance
Language of examinationEnglish
RepetitionRepetition only possible after re-enrolling for the course unit.

Learning materials

No public learning materials available.
Only public learning materials are listed.


No information on groups available.


PlacesLimited number of places. Special selection procedure.
Beginning of registration periodRegistration possible from 01.08.2020
PriorityRegistration for the course unit is only possible for the primary target group
Primary target groupData Science MSc (261000)
Mathematics BSc (404000) starting semester 05
Statistics MSc (436000)
Mathematics MSc (437000)
Applied Mathematics MSc (437100)
Mathematics (Mobility) (448000)
Waiting listuntil 23.09.2020
End of registration periodRegistration only possible until 11.09.2020

Offered in

Data Science MasterSeminarWInformation
Mathematics BachelorSeminarsWInformation
Mathematics MasterSeminarsWInformation
Statistics MasterSeminar or Semester PaperWInformation