<ul><li>Tencent has released Hunyuan-Large, an open-source Transformer-based MoE model with a total of 389 billion parameters and 52 billion active parameters.</li><li>Hunyuan-Large is designed to handle large contexts of up to 256K tokens and rivals other leading models in performance.</li><li>The model incorporates technical advancements such as pre-training on diverse data, mixed expert routing strategy, key-value cache compression, and expert-specific learning rate.</li><li>Hunyuan-Large outperforms existing models on NLP tasks and addresses the need for long-context understanding in AI applications.</li></ul>

Tencent Releases Hunyuan-Large (Hunyuan-MoE-A52B) Model: A New Open-Source Transformer-based MoE Model with a Total of 389 Billion Parameters and 52 Billion Active Parameters

Tencent Releases Hunyuan-Large (Hunyuan-MoE-A52B) Model: A New Open-Source Transformer-based MoE Model with a Total of 389 Billion Parameters and 52 Billion Active Parameters

Discover more