CMPSC 190A/292F: Introduction to Optimization (Fall 2025)

Department of Computer Science, UC Santa Barbara
Lectures: T/R 3:30–4:45 PM, BUCHN 1920
Class Website: Canvas
Discussion Site: Piazza

Contact Information

Instructor: Dr. Yuheng Bu
Office: HFH 1117
Email: buyuheng [at] ucsb [dot] edu
Office Hours: by appointment (HFH 1117)

TAs:

• Yujian Liu — yujianliu [at] ucsb [dot] edu; Office Hours: Tue 5–6 PM (2118 Henley Hall)
  

• Yepeng Liu — yepengliu [at] ucsb [dot] edu; Office Hours: Thu 5–6 PM

Course Description

This senior undergraduate / first-year graduate course introduces the algorithmic foundations of optimization with emphasis on both theory and applications in CS and engineering. Topics include convex sets and functions, necessary and sufficient conditions for optimality, unconstrained and constrained optimization (gradient descent, Newton’s method, projection methods, Lagrange multipliers, KKT conditions), duality theory, and augmented Lagrangian methods. If time permits, we will also cover stochastic, subgradient, and proximal techniques for large-scale or nondifferentiable problems.

Prerequisites

Multivariable Calculus (directional derivatives, Taylor expansions) and Linear Algebra (vector spaces, inner products, eigenvalues, positive-definite matrices). Instructor approval is required to remain enrolled if prerequisites are not met.

Materials

We will mostly follow lecture notes (posted on Canvas).
Additional references:

• D. Bertsekas, Nonlinear Programming, Athena Scientific, 2016.
  

• S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004.

Logistics and Resources

• Canvas: Course site
  

• Piazza: Q&A forum
  

• Lab Sections (Fri): 12:00–12:50 (ILP 3205); 1:00–1:50 (ILP 3207); 2:00–2:50 (ILP 3107)

Grading

Graduate students (CMPSC 292F):

• Homework (H): 20%
  

• Lab Assignments (L): 20%
  

• Paper Review Project (P): 20%
  

• Midterm Exam (M): 40%

Undergraduates (CMPSC 190A):

• Homework (H): 25%
  

• Lab Assignments (L): 25%
  

• Midterm Exam (M): 50%

Calendar (Planned Topics)

Week 1 (Thu 09/25): Review of Calculus
Week 2 (Tue 09/30): Review of Linear Algebra
Week 2 (Thu 10/02): Unconstrained Optimization, Optimality
Week 3 (Tue 10/07): Convexity
Week 3 (Thu 10/09): Gradient Methods for Unconstrained Optimization
Week 4 (Tue 10/14): Convergence of GD on smooth functions
Week 4 (Thu 10/16): Convergence of GD on convex functions
Week 5 (Tue 10/21): Newton’s method and extensions
Week 5 (Thu 10/23): Constrained optimization
Week 6 (Tue 10/28): Gradient projection method
Week 6 (Thu 10/30): Optimization with equality constraints
Week 7 (Tue 11/04): Lagrange multipliers
Week 7 (Thu 11/06): Examples for constrained optimization
Week 8 (Tue 11/11): Veterans Day — No class
Week 8 (Thu 11/13): Optimization with inequality constraints
Week 9 (Tue 11/18): KKT Conditions
Week 9 (Thu 11/20): Midterm review
Week 10 (Tue 11/25): Midterm Exam
Week 10 (Thu 11/27): Thanksgiving — No class
Week 11 (Tue 12/02): Duality
Week 11 (Thu 12/04): SGD and Accelerated Gradient Descent

Notes

All updates and official announcements will be posted on Canvas and Piazza. Please include “CS190A” in email subject lines.