<ul><li>Low-Rank Adaptation (LoRA) method is effective for parameter-efficient fine-tuning by reducing trainable parameters through low-rank matrices.</li><li>Recent research applied sinusoidal transformation to low-rank adapters to increase stable rank without adding parameters, aiming to enhance performance compared to full-rank fine-tuning.</li><li>This study explores the application of the sine-activated technique in Post-Training Quantization to preserve benefits during model compression.</li><li>The results show that using sinusoidal non-linearity even after quantization leads to highly compressed adapters with minimal performance loss in various fine-tuning tasks.</li></ul>

Compressing Sine-Activated Low-Rank Adapters through Post-Training Quantization

Discover more