This course builds on Information Theory I. It introduces additional topics in single-user communication, connections between Information Theory and Statistics, and Network Information Theory.
The course has two objectives: to introduce the students to the key information theoretic results that underlay the design of communication systems and to equip the students with the tools that are needed to conduct research in Information Theory.
Differential entropy, maximum entropy, the Gaussian channel and water filling, the entropy-power inequality, Sanov's Theorem, Fisher information, the broadcast channel, the multiple-access channel, Slepian-Wolf coding, and the Gelfand-Pinsker problem.
T.M. Cover and J.A. Thomas, Elements of Information Theory, second edition, Wiley 2006