<ul><li>Multi-task model merging techniques aim to combine knowledge from task-specific experts into a unified model efficiently.</li><li>A new approach called Layer-wise Optimal Task Vector Merging (LOT Merging) is introduced to minimize feature drift during model merging.</li><li>LOT Merging minimizes feature differences between task-specific experts and the unified model in a layer-by-layer manner, enhancing performance.</li><li>Extensive experiments show that LOT Merging outperforms existing methods, achieving improvements of up to 4.4% on vision and vision-language benchmarks.</li></ul>

Towards Minimizing Feature Drift in Model Merging: Layer-wise Task Vector Fusion for Adaptive Knowledge Integration

Discover more