AMD Releases Instella-MoE-16B-A3B: A Fully Open Mixture-of-Experts LLM With 2.8B Active Parameters Trained On Instinct GPUs
AMD launched Instella-MoE-16B-A3B, an open-source Mixture-of-Experts LLM with 16B total parameters and 2.8B active per token. The model was trained from scratch on AMD Instinct MI300X and MI325X GPUs.
AMD has introduced Instella-MoE-16B-A3B, a fully open Mixture-of-Experts language model. The architecture features 16 billion total parameters but activates only 2.8 billion per token during inference, aiming to balance performance with computational efficiency.
The model was trained from scratch using AMD's Instinct MI300X and MI325X GPUs. Technical implementations include Gated MLA and FarSkip-Collective mechanisms to optimize the expert routing and communication overhead inherent in MoE architectures.
AMD has published the weights from every training stage, allowing researchers and developers to inspect the model's progression. This release serves as a demonstration of AMD's hardware capabilities for training large-scale AI models on its own silicon.
By providing an open-weight alternative, AMD aims to expand the ecosystem for its Instinct GPU lineup. The release highlights the growing competition in AI infrastructure, where chipmakers are increasingly validating their silicon through proprietary model training.
This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.