ECE367H1: Matrix Algebra and Optimization (Fall 2025)

 

Instructor:

·      Prof. Wei Yu < weiyu@ece.utoronto.ca >  

·      Office hour: By appointment, usually available after class, e.g., Monday 2-3pm (except Sept 29)

 

Teaching Assistants:

·      Adnan Hamida < adnan.hamida@mail.utoronto.ca >

·      Mustafa Ammous < mustafa.ammous@mail.utoronto.ca >

·      Yuanxin Guo < yuanxin.guo@mail.utoronto.ca >

·      Anthony Ho < anth.ho@mail.utoronto.ca >

 

Lectures: (Starting Sept 2)

·      Monday 12:00-2:00pm MC254

·      Tuesday 2:00-3:00pm GB244

 

Tutorials: (Starting Sept 11)

·      Thursdays 9-11am (BA2135 & BA2175)

 

Important Dates:

·      First day of the lecture: Sept 2. (First tutorial: Sept 11)

·      No lectures/tutorials during the study break: Oct 27-31.

·      Midterm: Nov 6, 9-11am, EX310

·      Last day for dropping the course without academic penalty: Nov 11.

·      Last day of the lecture: Dec 2.

·      Final Exam: Dec 17, 6:30pm-9pm

 

Calendar Description:

This course will provide students with a grounding in optimization methods and the matrix algebra upon which they are based. The first part of the course focuses on fundamental building blocks in linear algebra and their geometric interpretation: matrices, their use to represent data and as linear operators, and the matrix decompositions (such as eigen-, spectral-, and singular-vector decompositions) that reveal structural and geometric insight. The second part of the course focuses on optimization, both unconstrained and constrained, linear and non-linear, as well as convex and nonconvex; conditions for local and global optimality, as well as basic classes of optimization problems are discussed. Applications from machine learning, signal processing, and engineering are used to illustrate the techniques developed.

 

Textbooks:

[1]  Giuseppe Calafiore and Laurent El Ghaoui, Optimization Models, Cambridge University Press, 2014. (Main textbook)

[2]  Stephen Boyd and Lieven Vandenberghe, Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares, Cambridge University Press, 2018. (PDF available at authors’ website. Some homework problems are taken from this textbook.)

 

Course Schedule:

 

Week	Topics	Text References	Assessment	Tutorial
Sept 2	Introduction	Ch. 1
Sept 8-9	Vectors, Norms, Inner Products, Orthogonal Decomposition	Ch. 2.1-2.2	Homework #1: Due Sept 24, 11:59pm (Word Vector, Fourier Series)	Homework #1: Theory
Sept 15-16	Projection onto Subspaces, Fourier Series. Gram-Schmidt and QR decomposition. Hyperplanes and Half-Spaces. Non-Euclidean Projection.	Ch. 2.3		Homework #1: Applications
Sept 22-23	Projection onto Affine Sets. Functions, Gradients and Hessians.	Ch. 2.3-2.4	Homework #2: Due Oct 11, 11:59pm (Function Approximation, PageRank)	Homework #2: Theory
Sept 29-30	Matrices, Range, Null Space, Eigenvalues and Eigenvectors Matrices Diagonalization.	Ch. 3.1-3.5		Homework #2: Application
Oct 6-7	PageRank Algorithm, Symmetric matrices. Function Approximation.	Ch. 4.1-4.4		Homework #2
Oct 14	Orthogonal Matrices. Spectral Decomposition. Positive Semidefinite Matrices. Ellipsoids.	Ch. 5.1, 5.3.2	Homework #3: Due Oct 24, 11:59pm (Latent Semantic Indexing, EigenFace)	Homework #3: Theory
Oct 20-21	Singular Value Decomposition. Principal Component Analysis.	Ch. 5.2-5.3.1		Homework #3: Applications
	Study Break
Nov 3-4	Interpretation of SVD. Low-Rank Approximation. Midterm review		Midterm Nov 6 Thursday 9-11am EX310
Nov 10-11	Least Squares. Overdetermined and Underdetermined Linear Equations.	Ch. 6.1-6.4	Homework #4: Due Nov 19, 11:59pm (Optimal Control, CAT Scan)	Homework #4: Theory and Applications
Nov 17-18	Regularized Least-Squares. Convex Sets and Convex Functions.	Ch. 6.7.3 Ch. 8.1-8.4		Homework #5: Theory
Nov 24-25	Lagrangian Method for Constrained Optimization. Linear Programming and Quadratic Programming.	Ch. 8.5 Ch. 9.1-9.6	Homework #5: Due Dec 3, 11:59pm (Portfolio Design, Sparse Coding of Image)	Homework #5: Applications
Dec 1-2	Numerical Algorithms for Unconstrained and Constrained Optimization Final review	Ch. 12.1-12.3
			Final Exam: Dec 17 Wednesday 6:30pm-9pm

 

Grades:

·      Homework: 15% (Graded for completeness only)

·      Midterm: 30% (Type C3, one aid-sheet, non-programmable calculator allowed)

·      Final Exam: 55% (Type C3, one aid-sheet, non-programmable calculator allowed.)