EEL 6935 Information-Theoretic Methods in Machine Learning (Spring 2023)Department of Electrical and Computer Engineering, University of Florida
Contact InformationInstructor: Dr. Yuheng Bu OutlineThe interplay between information theory and machine learning is a constant theme in the development of both fields. This course will discuss how techniques rooted in information theory play a key role in understanding the fundamental limits of statistical inference and supervised learning problems regarding generalization error and sample complexity. In particular, we will start from traditional statistical inference, which provides us with the basic intuition about the essence of information-theoretic methods, i.e., connecting mathematically defined information measures with operational meaning. Towards the end, we will also discuss the recent trend of using information-theoretic methods in characterizing the generalization error of learning algorithms. Further descriptions of these areas are given in the syllabus. General ReferencesThere is no required textbook – we will provide a set of hand-written notes on canvas. The notes contain pointers to relevant references. Some general references are given below.
|