Andreea-Alexandra Muşat

Gradient Descent Avoids Strict Saddles with a Simple Line-Search Method Too

It is known that gradient descent (GD) on a [Formula: see text] cost function generically avoids strict saddle points when using a small, constant step size. However, no such guarantee existed for GD with a line-search method. We provide one for a modified version of the standard...