This course covers the basic concepts of information theory and of communication theory. Topics covered include the entropy rate of a source, mutual information, typical sequences, the asymptotic equi-partition property, Huffman coding, channel capacity, the channel coding theorem, the source-channel separation theorem, and feedback capacity.
Learning objective
The fundamentals of Information Theory including Shannon's source coding and channel coding theorems
Content
The entropy rate of a source, Typical sequences, the asymptotic equi-partition property, the source coding theorem, Huffman coding, Arithmetic coding, channel capacity, the channel coding theorem, the source-channel separation theorem, feedback capacity
Literature
T.M. Cover and J. Thomas, Elements of Information Theory (second edition)
Performance assessment
Performance assessment information (valid until the course unit is held again)
Performance assessment as a semester course