Beck A. Introduction to nonlinear optimization theory, algorithms, and applications with MatLAB

N.-Y.: SIAM, 2014. - 294p.This book emerged from the idea that an optimization training should include three basic components: a strong theoretical and algorithmic foundation, familiarity with various applications, and the ability to apply the theory and algorithms on actual real-life problems. The book is intended to be the basis of such an extensive training. The mathematical development of the main concepts in nonlinear optimization is done rigorously, where a special effort was made to keep the proofs as simple as possible. The results are presented gradually and accompanied with many illustrative examples. Since the aim is not to give an encyclopedic overview, the focus is on the most useful and important concepts. The theory is complemented by numerous discussions on applications from various scientific fields such as signal processing, economics and localization. Some basic algorithms are also presented and studied to provide some flavor of this important aspect
of optimization. Many topics are demonstrated by MatLAB programs, and ideally, the interested reader will find satisfaction in the ability of actually solving problems on his or her own. The book contains several topics that, compared to other classical textbooks, are treated differently. The following are some examples of the less common issues.The Space Rn.
Inner Products and Norms.
Eigenvalues and Eigenvectors.
Basic Topological Concepts.
Exercises.
Global and Local Optima.
Classification of Matrices.
Second Order Optimality Conditions.
Global Optimality Conditions.
Quadratic Functions.
Exercises.
Solution of Overdetermined Systems.
Data Fitting.
Regularized Least Squares.
Denoising.
Circle Fitting.
Exercises.
Descent Directions Methods.
The Gradient Method.
The Condition Number.
Diagonal Scaling.
The Gauss-Newton Method.
The Fermat-Weber Problem.
Convergence Analysis of the Gradient Method.
Exercises.
Pure Newton’s Method.
Damped Newton’s Method.
The Cholesky Factorization.
Exercises.
Definition and Examples.
Algebraic Operations with Convex Sets.
The Convex Hull.
Convex Cones.
Topological Properties of Convex Sets.
Extreme Points.
Exercises.
Definition and Examples.
First Order Characterizations of Convex Functions.
Second Order Characterization of Convex Functions.
Operations Preserving Convexity.
Level Sets of Convex Functions.
Continuity and Differentiability of Convex Functions.
Extended Real-Valued Functions.
MAXIMA of Convex Functions.
Convexity and Inequalities.
Exercises.
Definition.
Examples.
The Orthogonal Projection Operator.
CVX.
Exercises.
Stationarity.
The Orthogonal Projection Revisited.
The Gradient Projection Method.
Sparsity Constrained Problems.
Exercises.
Separation and Alternative Theorems.
The KKT conditions.
Orthogonal Regression.
Exercises.
Inequality Constrained Problems.
Inequality and Equality Constrained Problems.
The Convex Case.
Constrained Least Squares.
Second Order Optimality Conditions.
Optimality Conditions for the Trust Region Subproblem.
Total Least Squares.
Exercises.
Motivation and Definition.
Strong Duality in the Convex Case.
Examples.
Exercises.
Bibliographic Notes.