# Search result: Catalogue data in Spring Semester 2019

DAS in Data Science | ||||||

Specialisation Track | ||||||

Machine Learning and Artificial Intelligence | ||||||

Number | Title | Type | ECTS | Hours | Lecturers | |
---|---|---|---|---|---|---|

252-0526-00L | Statistical Learning Theory | W | 7 credits | 3V + 2U + 1A | J. M. Buhmann | |

Abstract | The course covers advanced methods of statistical learning : Statistical learning theory;variational methods and optimization, e.g., maximum entropy techniques, information bottleneck, deterministic and simulated annealing; clustering for vectorial, histogram and relational data; model selection; graphical models. | |||||

Objective | The course surveys recent methods of statistical learning. The fundamentals of machine learning as presented in the course "Introduction to Machine Learning" are expanded and in particular, the theory of statistical learning is discussed. | |||||

Content | # Theory of estimators: How can we measure the quality of a statistical estimator? We already discussed bias and variance of estimators very briefly, but the interesting part is yet to come. # Variational methods and optimization: We consider optimization approaches for problems where the optimizer is a probability distribution. Concepts we will discuss in this context include: * Maximum Entropy * Information Bottleneck * Deterministic Annealing # Clustering: The problem of sorting data into groups without using training samples. This requires a definition of ``similarity'' between data points and adequate optimization procedures. # Model selection: We have already discussed how to fit a model to a data set in ML I, which usually involved adjusting model parameters for a given type of model. Model selection refers to the question of how complex the chosen model should be. As we already know, simple and complex models both have advantages and drawbacks alike. # Statistical physics models: approaches for large systems approximate optimization, which originate in the statistical physics (free energy minimization applied to spin glasses and other models); sampling methods based on these models | |||||

Lecture notes | A draft of a script will be provided; transparencies of the lectures will be made available. | |||||

Literature | Hastie, Tibshirani, Friedman: The Elements of Statistical Learning, Springer, 2001. L. Devroye, L. Gyorfi, and G. Lugosi: A probabilistic theory of pattern recognition. Springer, New York, 1996 | |||||

Prerequisites / Notice | Requirements: knowledge of the Machine Learning course basic knowledge of statistics, interest in statistical methods. It is recommended that Introduction to Machine Learning (ML I) is taken first; but with a little extra effort Statistical Learning Theory can be followed without the introductory course. | |||||

252-3005-00L | Natural Language Understanding Number of participants limited to 200. | W | 4 credits | 2V + 1U | M. Ciaramita, T. Hofmann | |

Abstract | This course presents topics in natural language processing with an emphasis on modern techniques, primarily focusing on statistical and deep learning approaches. The course provides an overview of the primary areas of research in language processing as well as a detailed exploration of the models and techniques used both in research and in commercial natural language systems. | |||||

Objective | The objective of the course is to learn the basic concepts in the statistical processing of natural languages. The course will be project-oriented so that the students can also gain hands-on experience with state-of-the-art tools and techniques. | |||||

Content | This course presents an introduction to general topics and techniques used in natural language processing today, primarily focusing on statistical approaches. The course provides an overview of the primary areas of research in language processing as well as a detailed exploration of the models and techniques used both in research and in commercial natural language systems. | |||||

Literature | Lectures will make use of textbooks such as the one by Jurafsky and Martin where appropriate, but will also make use of original research and survey papers. | |||||

261-5110-00L | Optimization for Data Science | W | 8 credits | 3V + 2U + 2A | B. Gärtner, D. Steurer | |

Abstract | This course teaches an overview of modern optimization methods, with applications in particular for machine learning and data science. | |||||

Objective | Understanding the theoretical and practical aspects of relevant optimization methods used in data science. Learning general paradigms to deal with optimization problems arising in data science. | |||||

Content | This course teaches an overview of modern optimization methods, with applications in particular for machine learning and data science. In the first part of the course, we will discuss how classical first and second order methods such as gradient descent and Newton's method can be adapated to scale to large datasets, in theory and in practice. We also cover some new algorithms and paradigms that have been developed specifically in the context of data science. The emphasis is not so much on the application of these methods (many of which are covered in other courses), but on understanding and analyzing the methods themselves. In the second part, we discuss convex programming relaxations as a powerful and versatile paradigm for designing efficient algorithms to solve computational problems arising in data science. We will learn about this paradigm and develop a unified perspective on it through the lens of the sum-of-squares semidefinite programming hierarchy. As applications, we are discussing non-negative matrix factorization, compressed sensing and sparse linear regression, matrix completion and phase retrieval, as well as robust estimation. | |||||

Prerequisites / Notice | As background, we require material taught in the course "252-0209-00L Algorithms, Probability, and Computing". It is not necessary that participants have actually taken the course, but they should be prepared to catch up if necessary. | |||||

263-3710-00L | Machine Perception Number of participants limited to 150. | W | 5 credits | 2V + 1U + 1A | O. Hilliges | |

Abstract | Recent developments in neural networks (aka “deep learning”) have drastically advanced the performance of machine perception systems in a variety of areas including computer vision, robotics, and intelligent UIs. This course is a deep dive into deep learning algorithms and architectures with applications to a variety of perceptual tasks. | |||||

Objective | Students will learn about fundamental aspects of modern deep learning approaches for perception. Students will learn to implement, train and debug their own neural networks and gain a detailed understanding of cutting-edge research in learning-based computer vision, robotics and HCI. The final project assignment will involve training a complex neural network architecture and applying it on a real-world dataset of human activity. The core competency acquired through this course is a solid foundation in deep-learning algorithms to process and interpret human input into computing systems. In particular, students should be able to develop systems that deal with the problem of recognizing people in images, detecting and describing body parts, inferring their spatial configuration, performing action/gesture recognition from still images or image sequences, also considering multi-modal data, among others. | |||||

Content | We will focus on teaching: how to set up the problem of machine perception, the learning algorithms, network architectures and advanced deep learning concepts in particular probabilistic deep learning models The course covers the following main areas: I) Foundations of deep-learning. II) Probabilistic deep-learning for generative modelling of data (latent variable models, generative adversarial networks and auto-regressive models). III) Deep learning in computer vision, human-computer interaction and robotics. Specific topics include: I) Deep learning basics: a) Neural Networks and training (i.e., backpropagation) b) Feedforward Networks c) Timeseries modelling (RNN, GRU, LSTM) d) Convolutional Neural Networks for classification II) Probabilistic Deep Learning: a) Latent variable models (VAEs) b) Generative adversarial networks (GANs) c) Autoregressive models (PixelCNN, PixelRNN, TCNs) III) Deep Learning techniques for machine perception: a) Fully Convolutional architectures for dense per-pixel tasks (i.e., instance segmentation) b) Pose estimation and other tasks involving human activity c) Deep reinforcement learning IV) Case studies from research in computer vision, HCI, robotics and signal processing | |||||

Literature | Deep Learning Book by Ian Goodfellow and Yoshua Bengio | |||||

Prerequisites / Notice | This is an advanced grad-level course that requires a background in machine learning. Students are expected to have a solid mathematical foundation, in particular in linear algebra, multivariate calculus, and probability. The course will focus on state-of-the-art research in deep-learning and will not repeat basics of machine learning Please take note of the following conditions: 1) The number of participants is limited to 150 students (MSc and PhDs). 2) Students must have taken the exam in Machine Learning (252-0535-00) or have acquired equivalent knowledge 3) All practical exercises will require basic knowledge of Python and will use libraries such as TensorFlow, scikit-learn and scikit-image. We will provide introductions to TensorFlow and other libraries that are needed but will not provide introductions to basic programming or Python. The following courses are strongly recommended as prerequisite: * "Visual Computing" or "Computer Vision" The course will be assessed by a final written examination in English. No course materials or electronic devices can be used during the examination. Note that the examination will be based on the contents of the lectures, the associated reading materials and the exercises. |

- Page 1 of 1