Department of Electrical and Computer
Engineering
University of Toronto
Fall 2023
Weekly Schedule
Week 
Topics 
Recording 
Text Reference 
Assignments 
9/11 
Introduction; Motivating Example: Data Fitting. Basic Concepts: Vector Space, Norm, Vector Calculus, Linear Algebra. 
Lecture 1 (1:08) Lecture 2.1 (0:48) 
Chapter 1 Appendix A 

9/18 
Gradient and Hessian of functions. Functions of matrices. Convex Sets. 
Lecture 2.2 (1:11) Lecture 3 (1:19) 
Chapter 2 

9/25 
Convex functions. Firstorder and Secondorder conditions for differentiable convex functions. Properties of Convex Functions. 
Lecture 4.1 (1:32) Lecture 4.2 (0:47) 
Chapter 3 
HW #1 Due Oct 2 
10/02 
Convex optimization problems. Local and global optimal solutions of convex optimization problems. 
Lecture 4.3 (0:53) Lecture 5 (1:27) 
Chapter 4 

10/09 
Linear Programming. Quadratic programming. Quadratically constrained quadratic programming. Secondorder cone programming. Robust linear programming. 
Lecture 6 (1:41) Lecture 7.1 (0:53) 
Chapter 4 

10/16 
Leastsquares problems. Optimal control problem. Geometric programming. Semidefinite programming. SDP relaxation. 
Lecture 7.2 (1:00) Lecture 8.1 (0:41) Lecture 8.2 (1:11) 
Chapter 4 
HW #2 Due Nov 3 
10/23 
Theory: Lagrangian. Dual optimization problem. Duality gap. Slater’s condition. Sensitivity analysis. 
Lecture 9.1 (1:09) Lecture 9.2 (1:29) 
Chapter 5, Rockafellar: Sec. 33 

10/30 
Dual of LP. Economics and Pricing Interpretation. Saddle points. Game theory. Duality theory for minimax optimization. 
Lecture 10.1 (0:55) Lecture 10.2 (0:46) 
Chapter 5, Rockafellar: Sec. 33 
Project Proposal Due Nov 13 
11/06 
Study Break 



11/13 
Dual of QP. Controllabilityobservability duality. Dual of $l_p$ optimization. Duality theory for optimization problem with generalized inequality. SDP relaxation. 
Lecture 11.1 (1:00) Lecture 11.2 (0:46) Lecture 12 (1:31) 
Chapter 5 
HW #3 Due Nov 26 
11/20 
Complementary slackness condition. KarushKuhnTucker (KKT) Conditions. Interpretation of the KKT condition. Regularity condition for local optimality. Generalized inequalities. 
Lecture 13.1 (0:41) Lecture 13.2 (1:11) Lecture 13.3 (1:07) 
Bertsekas Ch. 3 

11/27 
Algorithms: Descent methods. Newton's method. Equality Constrained Minimization. Infeasiblestart Newton's Method. Interiorpoint method for constrained optimization. 
Lecture 14 (1:07) Lecture 15 (0:45) 
Chapter 9 Chapter 10 Chapter 11 

12/04 
Interpretation of Interiorpoint method via KKT condition. PrimalDual InteriorPoint Method. Generalized Inequality. Analytic Centering. Ellipsoid method. Subgradient method for nondifferentiable functions. Dual Decomposition. Coordinate descent. 
Lecture 16 (1:20) Lecture 17 (1:31) 
Chapter 11 
Project Presentation Dec 6 
Extra Topics 
Sequential quadratic programming. Integer programming problems. Sparse optimization. Stochastic gradient descent. [Not covered this year.] 


HW #4 Due Dec 6 





Final Exam: December 9, 9am12noon @ HS106 (Health Sciences Building, 155 College Street) 


Project Report Due Dec 17 
The above schedule is subject to change.
Last Update: 11/26/23