menu
techminis

A naukri.com initiative

google-web-stories
Home

>

ML News

>

Convergenc...
source image

Arxiv

2d

read

268

img
dot

Image Credit: Arxiv

Convergence, Sticking and Escape: Stochastic Dynamics Near Critical Points in SGD

  • Study on convergence properties and escape dynamics of Stochastic Gradient Descent (SGD) in one-dimensional landscapes with infinite- and finite-variance noise.
  • Focus on identifying time scales for SGD to move from initial point to local minimum in the same basin.
  • SGD converges to basin's minimum unless initial point is too close to a local maximum, leading to lingering in its neighborhood.
  • Results show SGD does not remain stuck near a 'sharp' maximum and provide estimates on reaching neighboring minima, influenced by noise characteristics and function geometry.

Read Full Article

like

16 Likes

For uninterrupted reading, download the app