A naukri.com initiative
Medium
2w
120
Use cases of Hessian Matrix in Machine Learning part3
- Implementation of the standard full waveform inversion (FWI) poses difficulties as the initial model offsets from the true model. The wavefield reconstruction inversion (WRI) was proposed to mitigate these difficulties by relaxing the wave-equation constraint.
- A new approximate Hessian, called Augmented Gauss-Newton (AGN) Hessian, is developed for nonlinear term in the Hessian matrix of FWI. An updating formula resulting from the AGN Hessian on the FWI problem has an intimate connection with the WRI method.
- A fast algorithm is introduced for evaluating the Gauss-Newton Hessian (GNH) matrix for fully-connected feed-forward neural networks. The algorithm reduces the computation cost from O(Nn) to O(n+d/ε^2) work, where N is the number of parameters, n is the number of data points, d is the output dimension, and ε is the prescribed accuracy.
- The fast algorithm can be used to construct the hierarchical-matrix (H-matrix) approximation of the GNH matrix, which reduces the memory footprint and factorization work from O(N^2) and O(N^3) to O(Nro) and O(Nr^2o), respectively. The performance of the fast algorithm and H-matrix approximation is demonstrated on classification and autoencoder neural networks.
Read Full Article
7 Likes
For uninterrupted reading, download the app