<ul data-eligibleForWebStory="true"><li>Learned optimizers have been a focus in research with progress towards practical optimizers.</li><li>Recent advances like VeLO are hard to access due to reliance on JAX.</li><li>The PyLO library is introduced to make learned optimizers more accessible in PyTorch.</li><li>PyLO aims to bring learned optimizers to the machine learning community through familiar workflows.</li><li>Unlike previous work, PyLO focuses on real-world large-scale pre-training tasks.</li><li>The release includes a CUDA-accelerated version of a learned optimizer architecture.</li><li>The small_fc_lopt learned optimizer architecture speeds up training ViT B/16 significantly.</li><li>PyLO allows the combination of learned optimizers with existing optimization tools.</li><li>When combined with other tools, learned optimizers show significant benefits.</li><li>The code for PyLO is available on GitHub at https://github.com/Belilovsky-Lab/pylo</li></ul>

PyLO: Towards Accessible Learned Optimizers in PyTorch

Discover more