<ul><li>Vision-language models (VLMs) have achieved remarkable success across diverse tasks with minimal labeled data.</li><li>Knowledge distillation (KD) is a solution to deploying large models in resource-constrained environments.</li><li>A new approach called Dual-Head Optimization (DHO) simplifies and improves knowledge distillation from VLMs to compact models in semi-supervised settings.</li><li>DHO outperforms baselines in various experiments, achieving state-of-the-art performance on ImageNet with less labeled data and parameters.</li></ul>

Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ual-$\mathbf{\texttt{H}}$ead $\mathbf{\texttt{O}}$ptimization

Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ual-$\mathbf{\texttt{H}}$ead $\mathbf{\texttt{O}}$ptimization

Discover more