Optimization techniques in deep learning are predominantly led by first-order gradient methodologies like SGD, but second-order optimization methods like Newton's GD can greatly benefit neural network training.
Matrix inversion is a major bottleneck for Newton's GD, with a time complexity of O(N^3).
The use of quantum linear solver algorithms (QLSAs) presents a promising approach to accelerate matrix inversion with exponentially reduced time complexity of O(d * κ * log(N * κ / ε)).
Q-Newton is a hybrid quantum-classical scheduler proposed to accelerate neural network training with Newton's GD by coordinating between quantum and classical linear solvers.