Course Announcements

11/2/25: 1. Sets 11 and 15 of the class notes have been modified to reflect the comments in class on a limit point and accumulation points of the sequence x_k when the norm of gradient is converging to 0 and reposted.
10/27/25: 1. Sets 17 and 18 of the class notes have been posted.
10/21/25: 1. A note on coverage of lectures relevant to your reading order of notes:

Wednesday we will finish Set 10.

We will then discuss the algorithms covered in Sets 12, 13 and 14.

The convergence results in Sets 11 and 15 will then be summarized.; 2. Set 16 on the geometry of linear programs has been posted.; 3. An extensive set of study notes on unconstrained optimization for convex quadratic cost functions has been posted. It includes Steepest Descent, General Descent and Conjugate Gradient algorithms and convergence proofs for exact line search stepsizes and related alternatives. You are not required to master this set however you should become familiar with the structure and stepsize choices for Steepest Descent and Conjugate Gradient. Note that Conjugate Gradient is derived from the point of view of a GFS on ℝⁿ and makes use of the projection theorem and related facts discussed in the first part of the course.
10/20/25: 1. Solutions for Homework 2 have been posted.
10/17/25: 1. In Problem 3 of Homework 2 the data samples are assumed to be centered, that is their mean is the 0 vector. This has been clarified in the posted file.
10/16/25: 1. The due date for Homework 2 has been extended until 11:59PM Sunday October 19. Those who already submitted solutions may update them.
10/10/25: 1. Set 8 on aliasing in the DFT/FFT reconstruction error is left as an exercise to read. Questions will be addressed in office hour meetings.; 2. Sets 9, 10, 11, 12, 13, 14 and 15 have been posted. These are very detailed notes and therefore require lectures that will assume you have prepared, by reading them, for lectures that describe the overall content and important points to which you contribute in the class discussions.; 3. Several relevant reference papers have been posted.
10/10/25: 1. Study Question Set 1 and the solutions have been posted.; 2. Graded Homework 2 has been posted.
10/9/25: 1. Comments on Program 1 and some Cholesky codes have been posted.
10/7/25: 1. Set 8 of the class notes has been posted.
10/6/25: 1. Typos on slides 18, 22, and 23 of Set 5 discussed in class today have been fixed.; 2. Sets 6 and 7 of the class notes have been posted.; 3. These sets present many details of the DFT and Cooley-Tukey FFT. The lectures and homework will concentrate on part of the topics in order to give you a sense of the origins and computations associated with the FFT. The other portions are very important for more detailed consideration of the FFT's implementation but not directly germane to this class. The distinction between topics will be made clear during the lectures.
10/1/25: 1. Set 5 of the class notes has been posted.; 2. A set of notes on the trace and determinant of a matrix has been posted.
9/29/25: 1. Your solutions to the first programming assignment are due at 11:59PM Sunday October 5, 2025
9/23/25: 1. Solutions for the written problems from Graded Homework 1 have been posted.
9/16/25: 1. Set 2 of the class notes has been modified and reposted. Material has been added covering the lecture comments about finding the nearest element of a linear variety, i.e., not the smallest normed element, in Corollary 2.6 and Theorem 2.7 (p. 29 -- 31) and the consistency of projection onto a subspace and a linear variety being consistent with the conditions for projection onto a convex set (p. 42 -- 44).
9/14/25: 1. Graded Homework 1 has been posted. It consists of a set of written problems and a programming assignment.; 2. Note that the written problems are due in one week while the due date for the programming assignment will be set when we complete the relevant lectures that begin Monday.; 3. Also, note the Canvas announcement that clarifies "grading based on effort" described in the syllabus and during the first lecture. This applies to all aspects of class, i.e., graded homework and class participation/attendance. The latter is particularly important to note since it has not been adhered to sufficiently well in the first three weeks of class lectures. Failure to adjust may adversely affect your semester grade.
8/27/25: 1. Sets 3 and 4 of the class notes have been posted.
8/26/25: 1. Set 2 of the class notes has been posted.
8/25/25: 1. The class syllabus has been posted.; 2. A description of the format for submitting program solutions has been posted.; 3. Set 1 of the class notes has been posted.

Course Announcements

11/2/25

1. Sets 11 and 15 of the class notes have been modified to reflect the comments in class on a limit point and accumulation points of the sequence xk when the norm of gradient is converging to 0 and reposted. 10/27/25

10/27/25

1. Sets 17 and 18 of the class notes have been posted. 10/21/25

10/21/25

1. A note on coverage of lectures relevant to your reading order of notes: Wednesday we will finish Set 10. We will then discuss the algorithms covered in Sets 12, 13 and 14. The convergence results in Sets 11 and 15 will then be summarized.

2. Set 16 on the geometry of linear programs has been posted.

10/20/25

1. Solutions for Homework 2 have been posted.

10/17/25

1. In Problem 3 of Homework 2 the data samples are assumed to be centered, that is their mean is the 0 vector. This has been clarified in the posted file.

10/16/25

1. The due date for Homework 2 has been extended until 11:59PM Sunday October 19. Those who already submitted solutions may update them.

10/10/25

1. Set 8 on aliasing in the DFT/FFT reconstruction error is left as an exercise to read. Questions will be addressed in office hour meetings.

2. Sets 9, 10, 11, 12, 13, 14 and 15 have been posted. These are very detailed notes and therefore require lectures that will assume you have prepared, by reading them, for lectures that describe the overall content and important points to which you contribute in the class discussions.

3. Several relevant reference papers have been posted.

10/10/25

1. Study Question Set 1 and the solutions have been posted.

2. Graded Homework 2 has been posted.

10/9/25

1. Comments on Program 1 and some Cholesky codes have been posted.

10/7/25

1. Set 8 of the class notes has been posted.

10/6/25

1. Typos on slides 18, 22, and 23 of Set 5 discussed in class today have been fixed.

2. Sets 6 and 7 of the class notes have been posted.

10/1/25

1. Set 5 of the class notes has been posted.

2. A set of notes on the trace and determinant of a matrix has been posted.

9/29/25

1. Your solutions to the first programming assignment are due at 11:59PM Sunday October 5, 2025

9/23/25

1. Solutions for the written problems from Graded Homework 1 have been posted.

9/16/25

9/14/25

8/27/25

1. Sets 3 and 4 of the class notes have been posted.

8/26/25

1. Set 2 of the class notes has been posted.

8/25/25

1. The class syllabus has been posted. 2. A description of the format for submitting program solutions has been posted. 3. Set 1 of the class notes has been posted.

1. Sets 11 and 15 of the class notes have been modified to reflect the comments in class on a limit point and accumulation points of the sequence x_k when the norm of gradient is converging to 0 and reposted.

10/27/25

1. Sets 17 and 18 of the class notes have been posted.

10/21/25

1. A note on coverage of lectures relevant to your reading order of notes:

Wednesday we will finish Set 10.

We will then discuss the algorithms covered in Sets 12, 13 and 14.

The convergence results in Sets 11 and 15 will then be summarized.

1. The class syllabus has been posted.

2. A description of the format for submitting program solutions has been posted.

3. Set 1 of the class notes has been posted.