Benjamin Grewe: Catalogue data in Autumn Semester 2022

Name	Prof. Dr. Benjamin Grewe
Field	Neuronal Learning and Intelligent Systems
Address	Neuronales L. u. Intelligente Syst ETH Zürich, Y55 G 28 Winterthurerstrasse 190 8057 Zürich SWITZERLAND
Telephone	+41 44 635 30 91
E-mail	bgrewe@ethz.ch
Department	Information Technology and Electrical Engineering
Relationship	Associate Professor

Number	Title	ECTS	Hours	Lecturers
227-0421-00L	Deep Learning in Artificial and Biological Neuronal Networks	4 credits	3G	B. Grewe
Abstract	Deep-Learning (DL) a brain-inspired weak for of AI allows training of large artificial neuronal networks (ANNs) that, like humans, can learn real-world tasks such as recognizing objects in images. However, DL is far from being understood and investigating learning in biological networks might serve again as a compelling inspiration to think differently about state-of-the-art ANN training methods.
Learning objective	The main goal of this lecture is to provide a comprehensive overview into the learning principles neuronal networks as well as to introduce a diverse skill set (e.g. simulating a spiking neuronal network) that is required to understand learning in large, hierarchical neuronal networks. To achieve this the lectures and exercises will merge ideas, concepts and methods from machine learning and neuroscience. These will include training basic ANNs, simulating spiking neuronal networks as well as being able to read and understand the main ideas presented in today’s neuroscience papers. After this course students will be able to: - read and understand the main ideas and methods that are presented in today’s neuroscience papers - explain the basic ideas and concepts of plasticity in the mammalian brain - implement alternative ANN learning algorithms to ‘error backpropagation’ in order to train deep neuronal networks. - use a diverse set of ANN regularization methods to improve learning - simulate spiking neuronal networks that learn simple (e.g. digit classification) tasks in a supervised manner.
Content	Deep-learning a brain-inspired weak form of AI allows training of large artificial neuronal networks (ANNs) that, like humans, can learn real-world tasks such as recognizing objects in images. The origins of deep hierarchical learning can be traced back to early neuroscience research by Hubel and Wiesel in the 1960s, who first described the neuronal processing of visual inputs in the mammalian neocortex. Similar to their neocortical counterparts ANNs seem to learn by interpreting and structuring the data provided by the external world. However, while on specific tasks such as playing (video) games deep ANNs outperform humans (Minh et al, 2015, Silver et al., 2018), ANNs are still not performing on par when it comes to recognizing actions in movie data and their ability to act as generalizable problem solvers is still far behind of what the human brain seems to achieve effortlessly. Moreover, biological neuronal networks can learn far more effectively with fewer training examples, they achieve a much higher performance in recognizing complex patterns in time series data (e.g. recognizing actions in movies), they dynamically adapt to new tasks without losing performance and they achieve unmatched performance to detect and integrate out-of-domain data examples (data they have not been trained with). In other words, many of the big challenges and unknowns that have emerged in the field of deep learning over the last years are already mastered exceptionally well by biological neuronal networks in our brain. On the other hand, many facets of typical ANN design and training algorithms seem biologically implausible, such as the non-local weight updates, discrete processing of time, and scalar communication between neurons. Recent evidence suggests that learning in biological systems is the result of the complex interplay of diverse error feedback signaling processes acting at multiple scales, ranging from single synapses to entire networks.
Lecture notes	The lecture slides will be provided as a PDF after each lecture.
Prerequisites / Notice	This advanced level lecture requires some basic background in machine/deep learning. Thus, students are expected to have a basic mathematical foundation, including linear algebra, multivariate calculus, and probability. The course is not to be meant as an extended tutorial of how to train deep networks in PyTorch or Tensorflow, although these tools used. The participation in the course is subject to the following conditions: 1) The number of participants is limited to 120 students (MSc and PhDs). 2) Students must have taken the exam in Deep Learning (263-3210-00L) or have acquired equivalent knowledge.
227-1037-00L	Introduction to Neuroinformatics	6 credits	2V + 1U + 1A	V. Mante, M. Cook, B. Grewe, G. Indiveri, D. Kiper, W. von der Behrens
Abstract	The course provides an introduction to the functional properties of neurons. Particularly the description of membrane electrical properties (action potentials, channels), neuronal anatomy, synaptic structures, and neuronal networks. Simple models of computation, learning, and behavior will be explained. Some artificial systems (robot, chip) are presented.
Learning objective	Understanding computation by neurons and neuronal circuits is one of the great challenges of science. Many different disciplines can contribute their tools and concepts to solving mysteries of neural computation. The goal of this introductory course is to introduce the monocultures of physics, maths, computer science, engineering, biology, psychology, and even philosophy and history, to discover the enchantments and challenges that we all face in taking on this major 21st century problem and how each discipline can contribute to discovering solutions.
Content	This course considers the structure and function of biological neural networks at different levels. The function of neural networks lies fundamentally in their wiring and in the electro-chemical properties of nerve cell membranes. Thus, the biological structure of the nerve cell needs to be understood if biologically-realistic models are to be constructed. These simpler models are used to estimate the electrical current flow through dendritic cables and explore how a more complex geometry of neurons influences this current flow. The active properties of nerves are studied to understand both sensory transduction and the generation and transmission of nerve impulses along axons. The concept of local neuronal circuits arises in the context of the rules governing the formation of nerve connections and topographic projections within the nervous system. Communication between neurons in the network can be thought of as information flow across synapses, which can be modified by experience. We need an understanding of the action of inhibitory and excitatory neurotransmitters and neuromodulators, so that the dynamics and logic of synapses can be interpreted. Finally, simple neural architectures of feedforward and recurrent networks are discussed in the context of co-ordination, control, and integration of sensory and motor information. Connections to computer science and artificial intelligence are discussed, but the main focus of the course is on establishing the biological basis of computations in neurons.