227-0421-00L Learning in Deep Artificial and Biological Neuronal Networks
| Semester | Autumn Semester 2023 |
| Lecturers | B. Grewe |
| Periodicity | yearly recurring course |
| Language of instruction | English |
Courses
| Number | Title | Hours | Lecturers | ||||
|---|---|---|---|---|---|---|---|
| 227-0421-00 G | Learning in Deep Artificial and Biological Neuronal Networks | 3 hrs |
| B. Grewe |
Catalogue data
| Abstract | Deep-Learning (DL) a brain-inspired weak for of AI allows training of large artificial neuronal networks (ANNs) that, like humans, can learn real-world tasks such as recognizing objects in images. However, DL is far from being understood and investigating learning in biological networks might serve again as a compelling inspiration to think differently about state-of-the-art ANN training methods. |
| Learning objective | The main goal of this lecture is to provide a comprehensive overview into the learning principles neuronal networks as well as to introduce a diverse skill set (e.g. simulating a spiking neuronal network) that is required to understand learning in large, hierarchical neuronal networks. To achieve this the lectures and exercises will merge ideas, concepts and methods from machine learning and neuroscience. These will include training basic ANNs, simulating spiking neuronal networks as well as being able to read and understand the main ideas presented in today’s neuroscience papers. After this course students will be able to: - read and understand the main ideas and methods that are presented in today’s neuroscience papers - explain the basic ideas and concepts of plasticity in the mammalian brain - implement alternative ANN learning algorithms to ‘error backpropagation’ in order to train deep neuronal networks. - use a diverse set of ANN regularization methods to improve learning - simulate spiking neuronal networks that learn simple (e.g. digit classification) tasks in a supervised manner. |
| Content | Deep-learning a brain-inspired weak form of AI allows training of large artificial neuronal networks (ANNs) that, like humans, can learn real-world tasks such as recognizing objects in images. The origins of deep hierarchical learning can be traced back to early neuroscience research by Hubel and Wiesel in the 1960s, who first described the neuronal processing of visual inputs in the mammalian neocortex. Similar to their neocortical counterparts ANNs seem to learn by interpreting and structuring the data provided by the external world. However, while on specific tasks such as playing (video) games deep ANNs outperform humans (Minh et al, 2015, Silver et al., 2018), ANNs are still not performing on par when it comes to recognizing actions in movie data and their ability to act as generalizable problem solvers is still far behind of what the human brain seems to achieve effortlessly. Moreover, biological neuronal networks can learn far more effectively with fewer training examples, they achieve a much higher performance in recognizing complex patterns in time series data (e.g. recognizing actions in movies), they dynamically adapt to new tasks without losing performance and they achieve unmatched performance to detect and integrate out-of-domain data examples (data they have not been trained with). In other words, many of the big challenges and unknowns that have emerged in the field of deep learning over the last years are already mastered exceptionally well by biological neuronal networks in our brain. On the other hand, many facets of typical ANN design and training algorithms seem biologically implausible, such as the non-local weight updates, discrete processing of time, and scalar communication between neurons. Recent evidence suggests that learning in biological systems is the result of the complex interplay of diverse error feedback signaling processes acting at multiple scales, ranging from single synapses to entire networks. |
| Lecture notes | The lecture slides will be provided as a PDF after each lecture. |
| Prerequisites / Notice | This advanced level lecture requires some basic background in machine/deep learning. Thus, students are expected to have a basic mathematical foundation, including linear algebra, multivariate calculus, and probability. The course is not to be meant as an extended tutorial of how to train deep networks in PyTorch or Tensorflow, although these tools used. The participation in the course is subject to the following conditions: 1) The number of participants is limited to 120 students (MSc and PhDs). 2) Students must have taken the exam in Deep Learning (263-3210-00L) or have acquired equivalent knowledge. |
Performance assessment
| Performance assessment information (valid until the course unit is held again) | |
Performance assessment as a semester course | |
| ECTS credits | 4 credits |
| Examiners | B. Grewe |
| Type | session examination |
| Language of examination | English |
| Repetition | The performance assessment is only offered in the session after the course unit. Repetition only possible after re-enrolling. |
| Mode of examination | written 120 minutes |
| Additional information on mode of examination | The lectures are complemented with project work that will span across 14 lectures and comprise of 7 projects that have to be handed in on a bi-weekly basis. At the end, the learning objectives will be evaluated through a written session exam which will determine the course grade. Similar to the course, the session exam will consist of 50% biology and 50% machine learning questions. To be admitted to the session exam at least 5 out of 7 projects need to be handed in and evaluated as passed. It is not necessary to re-take the project work for a re-sit of the exam. |
| Written aids | None |
| This information can be updated until the beginning of the semester; information on the examination timetable is binding. | |
Learning materials
| No public learning materials available. | |
| Only public learning materials are listed. |
Groups
| No information on groups available. |
Restrictions
| There are no additional restrictions for the registration. |
