<ul><li>Large-scale machine learning models with sparse weight matrices are widely used to decrease computation and memory costs.</li><li>Models with block-wise sparse weight matrices fit better with hardware accelerators and can further reduce costs during inference.</li><li>However, existing methods for training block-wise sparse models are inefficient and start with full and dense models.</li><li>The proposed efficient training algorithm decreases both computation and memory costs, while maintaining performance.</li></ul>

An Efficient Training Algorithm for Models with Block-wise Sparsity

Discover more