<ul><li>Mixture-of-Experts (MoE) Large Language Models (LLMs) suffer from sub-optimal expert pathways resulting in lower accuracy.</li><li>A novel class of test-time optimization methods, called C3PO, is developed to re-weight or 're-mix' the experts in different layers for each test sample.</li><li>C3PO applies optimization only to the core experts' mixing weights in critical layers, resulting in improved accuracy while saving computation.</li><li>C3PO consistently improves the accuracy of MoE LLMs by 7-15% and outperforms other test-time learning methods.</li></ul>

C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization for Test-Time Expert Re-Mixing

Discover more