<ul><li>Speculative decoding accelerates language model inference by using a lightweight draft model to propose tokens verified by a larger target model.</li><li>Applying speculative decoding to vision-language models faces challenges as small language models lack visual processing components and token predictions mismatch with larger VLMs.</li><li>MASSV introduces a method to transform small language models into effective multimodal drafters for VLMs by connecting them to a vision encoder and aligning token predictions using self-distilled visual instruction tuning.</li><li>Experiments show that MASSV improves accepted length by up to 30% and accelerates inference speed by up to 1.46x, providing a scalable method for enhancing both current and future VLMs.</li></ul>

MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models

Discover more