The natural extension of the Gradient Descent Method to a certain class of (structured) non-smooth optimization problems is the so-called Forward--Backward Splitting Algorithm. The fundamental understanding of this algorithm and its accelerated variants has significantly advanced the state of the art in a large number of applications in image processing, machine learning, computer vision, and compressive sensing. Moreover, the inherent structure of Forward--Backward Splitting is the basis for several other optimization techniques. This course introduces various aspects of Forward--Backward Splitting for convex and non-convex optimization problems. More in detail, most of the following topics will be covered: Accelerations and inertial variants, relation to (proximal) quasi-Newton methods, time-continuous interpretation, several other modifications, generalizations of the convergence proof strategy, local and global convergence, Kurdyka--Lojasiewicz inequality, etc. The course will be accompanied by examples from practical application in the above mentioned fields.

The seminars will provide state of the art contributions on the application of the presented acceleration strategies to optimization methods widely used in regularization approaches to imaging problems.

• Flexible Krylov Methods for Regularization
  Lecturer: Silvia Gazzola, University of Bath

  Krylov methods are well-known and efficient iterative solvers for linear systems of equations, and they are often employed as iterative regularization methods for linear inverse problems. In this talk we will explore classical Krylov methods when applied to solve Tikhonov-like regularized problems, and introduce new flexible Krylov methods that allow to efficiently incorporate nonnegativity constraints and 1-norm penalization terms within the solution process.

• A general framework for inertial variable-metric forward-backward algorithm
  Lecturer: Valeria Ruggiero, Università degli Studi di Ferrara, Italy

  Imaging techniques based on the acquisition of data by photon counting are dominated by Poisson noise. The numerical methods for the treatment of Poisson data require to solve variational problems with special features. In this talk, I will discuss some tools to address these problems in the framework of inertial forward-backward methods. In particular, a variable metric, a variant of the inertial step and an inexact proximal gradient step are included in a generalized version of the classical method, so that the recent improved ${o}(1/k^2)$ convergence rate for the objective functions values and the convergence of the iterates are preserved. The effectiveness of the proposed approach is then validated with a numerical experience on synthetic Poisson data.

• First order algorithms for convex optimization under inexact information
  Lecturer: Silvia Villa, Politecnico di Milano, Italy

  First order methods are among state-of-the-art approaches for solving large scale optimization problems arising in a variety of applications. In many situations, the first order information is inexact. This is mainly due to two reasons: either because the objective function is not fully accessible, or because it is computationally convenient to discard some available information. The most significant example is the stochastic gradient algorithm. In this talk i will discuss more generally the design and analysis of gradient based methods to solve convex optimization problems under inexact information. In particular, I will focus on inexact computations of the proximal step, and on several variants of the stochastic gradient algorithm.

• Steplength selection strategies and variable metric approaches in gradient projection methods
  Lecturer: Luca Zanni, Università degli Studi di Modena e Reggio Emilia, Italy

  Gradient projection methods are widely used for solving optimization problems with simple constraints, due to their simplicity and low memory requirements. In the last years, very efficient gradient-based approaches have been designed, exploiting special strategies to accelerate their convergence rate. In this talk, we focus on steplength selection techniques and variable matric approaches. We provide a spectral analysis of popular steplength rules for quadratic optimization problems with box-constraints, motivated by recent studies on the connection between the steplengths and the Hessian of the objective function. This analysis suggests modified versions of the well known Barzilai-Borwein rules that improve the performance of the gradient projection methods. In the framework of the variable metric approaches, diagonal scaling techniques are discussed and the numerical behaviour of the corresponding scaled gradient projection schemes is evaluated on imaging problems.