Course Announcements

12/19/23: 1. An anonymous semester grade list has been posted. Follow the Grades tab.
12/5/23: 1. Several papers on interior point methods and convex optimization algorithms have been posted. Note that these papers also have theorems and algorithms relevant to KKT systems and quadratic programming, e.g., Forsgren, Gill and Wright SIREV Volume 44.; 2. The notation of A vs its transpose in the KKT matrix K has been made consistent, i.e., it is no longer altered to be consisten with Nocedal and Wright's textbook for the direct and iterative solution of KKT for quadratic problems with linear equality constraints. Set 21 has been reposted. reposted.
12/4/23: 1. Set 21 has had additional clarifications added and the file reposted.
11/29/23: 1. Set 21 has been altered with some clarifications and reposted.; 2. Two typos fixed on Set 19 slide 33.
11/20/23: 1. Graded Homework 4 has been posted.; 2. Two typos fixed on Set 19 slide 33.
11/15/23: 1. An alternate form of the convergence theorem for Gradient Projection has been added to Set 19 of the class notes.; 2. Sets 20 and 21 of the class notes have been posted.; 2. Set 21 is on the KKT conditions and some discussions of their use. It may be updated in the next few days.
11/14/23: 1. Set 19 of the class notes has been posted.; 2. Two early papers on Gradient Projection on a convex closed feasible set have been posted.
11/13/23: 1. Some details in Set 16 on the Simplex Method have been clarified and the set has been reposted.; 2. Some details in Set 18 on the stationarity and nondegeneracy based sufficient conditions have been clarified and the set has been reposted.
11/8/23: 1. Set 18 of the class notes has been posted.; 2. It is critical that you read ahead in these notes for next week.
11/7/23: 1. Set 17 of the class notes has been posted.; 2. Study Homework 3 and the solutions have been posted.; 3. Some additional papers relevant to least squares, LU and Cholesky updating as well as in the simplex algorithm have been added. Also the grouping of the papers on the webpage has been altered to reflect commonality and a progression from simple to more complicated and breadth of topics.
11/2/23: 1. Solutions for Study Question Homework 2 have been posted. Note that the stationary points and their classifications have been given for multiple cost functions in the solutions to the problems. These can be used as test problems for your current graded homework programming assignment.
11/1/23: 1. Study Question Homework 2 typo fixed in Problem 2.7 cost function number 7.
10/26/23: 1. Sets 15 and 16 on linear programming: geometry and the simplex algorithm, have been posted. Read ahead.; 1. Several papers related to linear programming, numerical linear algebra algorithms, and the simplex algorithm have been posted.
10/22/23: 1. Comments on the solutions of Graded Homework 2 have been posted along with related Matlab code.
10/22/23: 1. Comments on the solutions of Graded Homework 1 have been posted.; 2. Set 14 of the class notes has been posted.
10/19/23: 1. The due date of Graded Homework 2 has been updated to 11:59PM Sunday October 22. The new date has been added to the pdf on the departmental class website and on canvas.
10/17/23: 1. Subscripts have been added to the BB stepsizes on slides 36 and 37 of Set 9. This makes the relationship between the indices of the s and y vectors and the index of the BB stepsize clear. Note on slide 37 the clarification of the use of α_k-1 in the derivation of α_k for both BB1 and BB2. The modifications also makes clear the source of the comment that the BB stepsizes are used one step later than the associated steepest descent stepsize.
10/16/23: 1. Slide 30 in Set 9 has been modified to make it clear that the two initial stepsize choices on that slide are different and to correct a typo in the denominator or the second. The inner product is at x_k-1 not x_k.; 2. Slide 37 in Set 9 has been modified to make it clear that the interpretation of BB1 and BB2 from an operator point of view is related to the action on ∇f_k-1, i.e., not necessarily the action on all of ℝⁿ.
10/9/23: 1. Graded Homework 2 has been posted.
10/9/23: 1. Some typos fixed on the title slide of Set 8; 2. Sets 9, 10, 11, 12 and 13 of the class notes have been posted.; 3. Many papers on inexact Newton, quasi-Newton, parameter selection and convergence have been posted.; 4. Office hours for Wednesdays have been updated to avoid conflict with the ACM seminar.
10/5/23: 1. The discussion of the sign resulting from applying a Givens rotation to the reference vector has been updated on slide 54.
10/4/23: 1. Three problems related to Jensen's Inequality for convex/concave functions have been added to Study Homework Question Set 1 and the solutions.
10/2/23: 1. Set 8 of the class notes has been posted.; 2. Several papers on line search methods and line search termination criteria.
9/29/23: 1. The due date of Graded Homework 1, Friday October 6, 2023 has been added the assignment pdf.; 2. The Mathematics Conference room, 204A LOV, has been reserved for the makeup class on Tuesday October 3, 2023. It is reserved starting at 14:00. If there is no objection the class will start then as well.
9/28/23: 1. Solutions for Study Homework 1 have been posted.
9/25/23: 1. Sets 5, 6 and 7 have been posted. You are strongly recommended to read ahead in Sets 4, 5, 6 and 7 immediately.; 2. Three references on Kantorovich's inequality have been posted.; 3. Several papers related to CG have been posted.
9/21/23: 1. Study Questions Set 1 has been posted. Note it contains problems concerning upcoming topics as well as material already presented in lectures and notes.; 2. Graded Homework 1 has been posted. The due date will be set next week depending on when we complete the discussion of the related topics in Set 3 of the notes.
9/20/23: 1. Set 4 has been posted.
9/19/23: 1. Set 3 has been modified for typos and clarification.
9/18/23: 1. Clarifications added to Slide 47 of Set 2. They will be pointed out during the lecture.
9/13/23: 1. Additional minor clarifications concerning real vs complex Hilbert spaces have been added to Set 2. They will be pointed out during the lecture.
9/11/23: 1. Minor clarifications have been added to Set 2. They will be pointed out during the lecture.
9/7/23: 1. On September 13, 2023, office hours will not be held due to a conflict with a college committee meeting.
9/6/23: 1. Set 2 of the class notes has been modified to clarify the proofs of the two forms of the Projection Theorem.
8/28/23: 1. The class syllabus has been posted.; 2. A description of the format for submitting program solutions has been posted.; 3. Sets 1, 2, and 3 of the class notes have been posted. Two forms are posted: a lecture form that includes the incremental overlay presentation mode when displayed and a handout form that has suppresses the overlay mode of presentation.

Course Announcements

12/19/23

1. An anonymous semester grade list has been posted. Follow the Grades tab.

12/5/23

12/4/23

1. Set 21 has had additional clarifications added and the file reposted.

11/29/23

1. Set 21 has been altered with some clarifications and reposted. 2. Two typos fixed on Set 19 slide 33.

11/20/23

1. Graded Homework 4 has been posted. 2. Two typos fixed on Set 19 slide 33.

11/15/23

1. An alternate form of the convergence theorem for Gradient Projection has been added to Set 19 of the class notes. 2. Sets 20 and 21 of the class notes have been posted. 2. Set 21 is on the KKT conditions and some discussions of their use. It may be updated in the next few days.

11/14/23

1. Set 19 of the class notes has been posted. 2. Two early papers on Gradient Projection on a convex closed feasible set have been posted.

11/13/23

1. Some details in Set 16 on the Simplex Method have been clarified and the set has been reposted. 2. Some details in Set 18 on the stationarity and nondegeneracy based sufficient conditions have been clarified and the set has been reposted.

11/8/23

1. Set 18 of the class notes has been posted. 2. It is critical that you read ahead in these notes for next week.

11/7/23

11/2/23

1. Solutions for Study Question Homework 2 have been posted. Note that the stationary points and their classifications have been given for multiple cost functions in the solutions to the problems. These can be used as test problems for your current graded homework programming assignment.

11/1/23

1. Study Question Homework 2 typo fixed in Problem 2.7 cost function number 7.

10/26/23

1. Sets 15 and 16 on linear programming: geometry and the simplex algorithm, have been posted. Read ahead. 1. Several papers related to linear programming, numerical linear algebra algorithms, and the simplex algorithm have been posted.

10/22/23

1. Comments on the solutions of Graded Homework 2 have been posted along with related Matlab code.

10/22/23

1. Comments on the solutions of Graded Homework 1 have been posted. 2. Set 14 of the class notes has been posted. 10/19/23

10/19/23

1. The due date of Graded Homework 2 has been updated to 11:59PM Sunday October 22. The new date has been added to the pdf on the departmental class website and on canvas.

10/17/23

10/16/23

10/9/23

1. Graded Homework 2 has been posted.

10/9/23

10/5/23

1. The discussion of the sign resulting from applying a Givens rotation to the reference vector has been updated on slide 54.

10/4/23

1. Three problems related to Jensen's Inequality for convex/concave functions have been added to Study Homework Question Set 1 and the solutions.

10/2/23

1. Set 8 of the class notes has been posted. 2. Several papers on line search methods and line search termination criteria.

9/29/23

9/28/23

1. Solutions for Study Homework 1 have been posted.

9/25/23

1. Sets 5, 6 and 7 have been posted. You are strongly recommended to read ahead in Sets 4, 5, 6 and 7 immediately. 2. Three references on Kantorovich's inequality have been posted. 3. Several papers related to CG have been posted.

9/21/23

9/20/23

1. Set 4 has been posted.

9/19/23

1. Set 3 has been modified for typos and clarification.

9/18/23

1. Clarifications added to Slide 47 of Set 2. They will be pointed out during the lecture.

9/13/23

1. Additional minor clarifications concerning real vs complex Hilbert spaces have been added to Set 2. They will be pointed out during the lecture.

9/11/23

1. Minor clarifications have been added to Set 2. They will be pointed out during the lecture.

9/7/23

1. On September 13, 2023, office hours will not be held due to a conflict with a college committee meeting.

9/6/23

1. Set 2 of the class notes has been modified to clarify the proofs of the two forms of the Projection Theorem.

8/28/23

1. Set 21 has been altered with some clarifications and reposted.

2. Two typos fixed on Set 19 slide 33.

1. Graded Homework 4 has been posted.

2. Two typos fixed on Set 19 slide 33.

1. An alternate form of the convergence theorem for Gradient Projection has been added to Set 19 of the class notes.

2. Sets 20 and 21 of the class notes have been posted.

2. Set 21 is on the KKT conditions and some discussions of their use. It may be updated in the next few days.

1. Set 19 of the class notes has been posted.

2. Two early papers on Gradient Projection on a convex closed feasible set have been posted.

1. Some details in Set 16 on the Simplex Method have been clarified and the set has been reposted.

2. Some details in Set 18 on the stationarity and nondegeneracy based sufficient conditions have been clarified and the set has been reposted.

1. Set 18 of the class notes has been posted.

2. It is critical that you read ahead in these notes for next week.

1. Sets 15 and 16 on linear programming: geometry and the simplex algorithm, have been posted. Read ahead.

1. Several papers related to linear programming, numerical linear algebra algorithms, and the simplex algorithm have been posted.

1. Comments on the solutions of Graded Homework 1 have been posted.

2. Set 14 of the class notes has been posted.

10/19/23

1. Set 8 of the class notes has been posted.

2. Several papers on line search methods and line search termination criteria.

1. Sets 5, 6 and 7 have been posted. You are strongly recommended to read ahead in Sets 4, 5, 6 and 7 immediately.

2. Three references on Kantorovich's inequality have been posted.

3. Several papers related to CG have been posted.