<ul><li>KO (Kinetics-inspired Optimizer) is a new neural optimizer inspired by kinetic theory and partial differential equation simulations.</li><li>It reimagines training dynamics as a particle system evolving based on kinetic principles, using a numerical scheme for the Boltzmann transport equation.</li><li>The approach promotes parameter diversity during optimization, preventing parameter condensation into low-dimensional subspaces.</li><li>Experiments on image and text classification tasks show that KO outperforms baseline optimizers like Adam and SGD in terms of accuracy improvement with comparable computation cost.</li></ul>

KO: Kinetics-inspired Neural Optimizer with PDE Simulation Approaches

Discover more