Properties of the delayed weighted gradient method

Author(s)

Andreani, Roberto ^[1] ; Raydan, Marcos ^[2]

Affiliation

^[1] Univ Estadual Campinas, IMECC UNICAMP, Dept Appl Math, Rua Sergio Buarque Holanda, BR-13083859 Campinas, SP - Brazil

^[2] UNL, FCT, Ctr Matemat & Aplicacoes CMA, P-2829516 Caparica - Portugal

• Document type: Journal article
• Source: COMPUTATIONAL OPTIMIZATION AND APPLICATIONS; v. 78, n. 1, p. 167-180, JAN 2021.
• Web of Science Citations: 0

Abstract

The delayed weighted gradient method, recently introduced in Oviedo-Leon (Comput Optim Appl 74:729-746, 2019), is a low-cost gradient-type method that exhibits a surprisingly and perhaps unexpected fast convergence behavior that competes favorably with the well-known conjugate gradient method for the minimization of convex quadratic functions. In this work, we establish several orthogonality properties that add understanding to the practical behavior of the method, including its finite termination. We show that if the n x n real Hessian matrix of the quadratic function has only p < n distinct eigenvalues, then the method terminates in p iterations. We also establish an optimality condition, concerning the gradient norm, that motivates the future use of this novel scheme when low precision is required for the minimization of non-quadratic functions. (AU)

• FAPESP's process: 17/18308-2 - Second-order optimality conditions and algorithms
• Grantee: Gabriel Haeser
• Support Opportunities: Regular Research Grants
• FAPESP's process: 13/05475-7 - Computational methods in optimization
• Grantee: Sandra Augusta Santos
• Support Opportunities: Research Projects - Thematic Grants