263-3700-00L  User Interface Engineering

SemesterSpring Semester 2017
LecturersO. Hilliges, F. Pece
Periodicityyearly recurring course
Language of instructionEnglish


AbstractAn in-depth introduction to the core concepts of intelligent user-interfaces. The course primarily deals with machine analysis of human non-verbal behavior and its applications to human-computer, human-robot, and computer-mediated human-human interaction. Methods involve machine learning, deep learning and model based optimization.
ObjectiveStudents will learn about fundamental aspects of modern intelligent user interfaces. After completing the course students will have acquired theoretical and practical knowledge about the most important problems in machine understanding of human behavior and how to leverage such understanding in the design of intelligent user-facing technologies.

The core competency acquired through this course is a solid foundation in machine learning and deep-learning algorithms to process and interpret human input into computing systems. In particular, students should be able to develop systems that deal with the problem of recognizing people in images, detecting and describing body parts, inferring their spatial configuration, performing action/gesture recognition from still images or image sequences, also considering multi-modal data, among others. Furthermore, students will be able to leverage models of human behavior in optimization based (algorithmic) design of user interfaces.
ContentThe course covers theoretical and practical aspects of state-of-the-art algorithms that are foundational for intelligent user interfaces. A particular area of interest are machine-learning based algorithms, in particular deep-learning techniques, for semantic interpretation and machine analysis of human activity, including gestures and multi-modal interaction amongst others. 

The course covers the following main areas:
I) Machine-learning algorithms for input recognition (gestures, speech, etc.)
II) Deep-learning models for the analysis of time-series data (temporal sequences of motion)
III) Model-based optimization of user interfaces

Specific topics include: 
* Data-driven algorithms for user input recognition:
+ SVMs for classification and regression
+ Randomized Decision Forests for gesture recognition and pose estimation
+ Markov chains and HMMs for gesture and speech recognition
* Deep Learning techniques user input recognition:
+ Convolutional Neural Networks
+ Recurrent Neural Networks
* Applications of the above in HCI research
Lecture notesSlides and other materials will be available online. Lecture slides on a particular topic will typically not be made available prior the completion of that lecture.
LiteratureA detailed reading list will be made available on the course website.
Prerequisites / NoticePrerequisites: proficiency in a programming language such as C, programming methodology, problem analysis, program structure, etc. Normally met through an introductory course in programming in C, C++, Java. All practical exercises will require basic knowledge of Python and will use libraries such as TensorFlow (via Keras) and scikit-learn. We will provide introductions to TensorFlow and other libraries that are needed but will not provide introductions to basic programming or Python.

The following courses are strongly recommended as prerequisite:
* "Machine Learning"
* "Visual Computing" or "Computer Vision"
* "Human Computer Interaction"


The course will be assessed by a final examination in English. No course materials or electronic devices can be used during the examination. Note that the examination will be based on the contents of the lectures, the associated reading materials and the exercises.