<ul><li>Researchers introduce a novel method called Hadamard-based Optimized Training (HOT) to optimize backpropagation in deep learning.</li><li>HOT focuses on matrix multiplication, the most computationally expensive part of training, and applies Hadamard-based optimizations selectively.</li><li>The method achieves up to 75% memory savings and a 2.6 times acceleration on GPUs, with minimal loss in accuracy compared to FP32 precision.</li><li>HOT includes techniques such as Hadamard quantization, Hadamard low-rank approximation, activation buffer compression, and layer-wise quantizer selection.</li></ul>

HOT: Hadamard-based Optimized Training

Discover more