CMPSC 292F: Information-theoretic Methods for Trustworthy Learning (Winter 2026)

Department of Computer Science, UC Santa Barbara
Lectures: TBA
Location: TBA
Class Website: Canvas (TBA)
Discussion Site: Piazza (TBA)

Contact Information

Instructor: Dr. Yuheng Bu
Office: HFH 1117
Email: buyuheng [at] ucsb [dot] edu
Office Hours: TBA (HFH 1117)

Course Description

This course explores fundamental tools for analyzing and designing trustworthy machine learning systems through the lens of information theory, statistics, and learning theory. We study reliability guarantees for modern models by quantifying generalization, fairness and privacy. We will also introduce practical aspects of watermarking and IP protection for large generative models.

Topics will include:

• Information-theoretic foundations for generalization and sample complexity

• Privacy (e.g., differential privacy) and leakage measures

• Fairness constraints in learning

• Watermarking for generative AI

This course builds conceptual connections between theoretical principles and real-world trustworthy AI challenges.

Prerequisites

Graduate standing in CS/ECE/Statistics or instructor consent.
Comfort with probability, linear algebra, and basic machine learning concepts is expected.

Materials

No required textbook.
Lecture notes and curated readings will be posted on Canvas.
Some general references on information theory are given below.

• Joy A. Thomas and Thomas M. Cover. “Elements of Information Theory,” Wiley-Interscience, 2006

• Y. Polyanskiy and Y. Wu. “Information Theory: From Coding to Learning,” Cambridge University Press, 2023 (book draft)

Tentative Topics and Schedule

Week 1 — Course overview and foundations of statistical learning
Week 2 — Information measures: entropy, KL, mutual information
Week 3 — Concentration inequalities and information bounds
Week 4 — Information-theoretic generalization bounds
Week 5 — Privacy in machine learning (DP and accounting)
Week 6 — Distribution shift and OOD detection
Week 7 — Domain adaptation and domain generalization
Week 8 — Fairness and constrained learning objectives
Week 9 — Watermarking for LLMs and generative models
Week 10 — Student presentations

Notes

Logistics (lecture time, TAs, office hours, schedule) will be updated once Winter 2026 assignments are confirmed.