Ghislain Fourny: Catalogue data in Autumn Semester 2019

Name	Dr. Ghislain Fourny
Address	Dep. Informatik ETH Zürich, STF H 311 Stampfenbachstrasse 114 8092 Zürich SWITZERLAND
Telephone	+41 44 632 31 55
E-mail	gfourny@inf.ethz.ch
URL	http://people.inf.ethz.ch/gfourny
Department	Computer Science
Relationship	Lecturer

Number	Title	ECTS	Hours	Lecturers
252-0341-01L	Information Retrieval	4 credits	2V + 1U	G. Fourny
Abstract	This course gives an introduction to information retrieval with a focus on text documents and unstructured data. Main topics comprise document modelling, various retrieval techniques, indexing techniques, query frameworks, optimization, evaluation and feedback.
Learning objective	We keep accumulating data at an unprecedented pace, much faster than we can process it. While Big Data techniques contribute solutions accounting for structured or semi-structured shapes such as tables, trees, graphs and cubes, the study of unstructured data is a field of its own: Information Retrieval. After this course, you will have in-depth understanding of broadly established techniques in order to model, index and query unstructured data (aka, text), including the vector space model, boolean queries, terms, posting lists, dealing with errors and imprecision. You will know how to make queries faster and how to make queries work on very large datasets. You will be capable of evaluating the quality of an information retrieval engine. Finally, you will also have knowledge about alternate models (structured data, probabilistic retrieval, language models) as well as basic search algorithms on the web such as Google's PageRank.
Content	1. Introduction 2. Boolean retrieval: the basics of how to index and query unstructured data. 3. Term vocabulary: pre-processing the data prior to indexing: building the term vocabulary, posting lists. 4. Tolerant retrieval: dealing with spelling errors: tolerant retrieval. 5. Index construction: scaling up to large datasets. 6. Index compression: how to improve performance by compressing the index in various ways. 7. Ranked retrieval: how to ranking results with scores and the vector space model 8. Scoring in a bigger picture: taking ranked retrieval to the next level with various improvements, including inexact retrieval 9. Probabilistic information retrieval: how to leverage Bayesian techniques to build an alternate, probabilistic model for information retrieval 10. Language models: another alternate model based on languages, automata and document generation 11. Evaluation: precision, recall and various other measurements of quality 12. Web search: PageRank 13. Wrap-up. The lecture structure will follow the pedagogical approach of the book (see material). The field of information retrieval also encompasses machine learning aspects. However, we will make a conscious effort to limit overlaps, and be complementary with, the Introduction to Machine Learning lecture.
Literature	C. D. Manning, P. Raghavan, H. Schütze, Introduction to Information Retrieval, Cambridge University Press.
Prerequisites / Notice	Prior knowledge in elementary set theory, logics, linear algebra, data structures, abstract data types, algorithms, and probability theory (at the Bachelor's level) is required, as well as programming skills (we will use Python).
263-3010-00L	Big Data	8 credits	3V + 2U + 2A	G. Fourny
Abstract	The key challenge of the information society is to turn data into information, information into knowledge, knowledge into value. This has become increasingly complex. Data comes in larger volumes, diverse shapes, from different sources. Data is more heterogeneous and less structured than forty years ago. Nevertheless, it still needs to be processed fast, with support for complex operations.
Learning objective	This combination of requirements, together with the technologies that have emerged in order to address them, is typically referred to as "Big Data." This revolution has led to a completely new way to do business, e.g., develop new products and business models, but also to do science -- which is sometimes referred to as data-driven science or the "fourth paradigm". Unfortunately, the quantity of data produced and available -- now in the Zettabyte range (that's 21 zeros) per year -- keeps growing faster than our ability to process it. Hence, new architectures and approaches for processing it were and are still needed. Harnessing them must involve a deep understanding of data not only in the large, but also in the small. The field of databases evolves at a fast pace. In order to be prepared, to the extent possible, to the (r)evolutions that will take place in the next few decades, the emphasis of the lecture will be on the paradigms and core design ideas, while today's technologies will serve as supporting illustrations thereof. After visiting this lecture, you should have gained an overview and understanding of the Big Data landscape, which is the basis on which one can make informed decisions, i.e., pick and orchestrate the relevant technologies together for addressing each business use case efficiently and consistently.
Content	This course gives an overview of database technologies and of the most important database design principles that lay the foundations of the Big Data universe. We take the monolithic, one-machine relational stack from the 1970s, smash it down and rebuild it on top of large clusters: starting with distributed storage, and all the way up to syntax, models, validation, processing, indexing, and querying. A broad range of aspects is covered with a focus on how they fit all together in the big picture of the Big Data ecosystem. No data is harmed during this course, however, please be psychologically prepared that our data may not always be in third normal form. - physical storage: distributed file systems (HDFS), object storage(S3), key-value stores - logical storage: document stores (MongoDB), column stores (HBase), graph databases (neo4j), data warehouses (ROLAP) - data formats and syntaxes (XML, JSON, RDF, Turtle, CSV, XBRL, YAML, protocol buffers, Avro) - data shapes and models (tables, trees, graphs, cubes) - type systems and schemas: atomic types, structured types (arrays, maps), set-based type systems (?, *, +) - an overview of functional, declarative programming languages across data shapes (SQL, XQuery, JSONiq, Cypher, MDX) - the most important query paradigms (selection, projection, joining, grouping, ordering, windowing) - paradigms for parallel processing, two-stage (MapReduce) and DAG-based (Spark) - resource management (YARN) - what a data center is made of and why it matters (racks, nodes, ...) - underlying architectures (internal machinery of HDFS, HBase, Spark, neo4j) - optimization techniques (functional and declarative paradigms, query plans, rewrites, indexing) - applications. Large scale analytics and machine learning are outside of the scope of this course.
Literature	Papers from scientific conferences and journals. References will be given as part of the course material during the semester.
Prerequisites / Notice	This course, in the autumn semester, is only intended for: - Computer Science students - Data Science students - CBB students with a Computer Science background Mobility students in CS are also welcome and encouraged to attend. If you experience any issue while registering, please contact the study administration and you will be gladly added. For students of all other departements interested in this fascinating topic: I would love to have you visit my lectures as well! So there is a series of two courses specially designed for you and offered in Spring 2020: - "Information Systems for Engineers" (SQL, relational databases) - "Big Data for Engineers" (similar to Big Data, but adapted for non Computer Scientists). There is no hard dependency, so you can either attend both in the same semester, or one after the other.