Model pre-training completed

Microsoft has finished pre-training its large artificial intelligence model, MAI-Base-2, using 16,000 NVIDIA GB200 GPUs over 38 days. The model has 2.8 trillion parameters and uses a mixed expert architecture (MoE), which activates around 80 billion parameters during inference.

Efficiency claims

According to internal tests, MAI-Base-2 would require 4.2 to 12.3 times less computation than the DeepSeek-V4-Pro-Base model to achieve comparable performance. However, these results have not yet been confirmed by independent evaluations.

Improved GPU usage

Microsoft claims to have improved GPU utilization for the model, increasing the rate from 24% to 35% in the relevant part of the model. This improvement could lead to more efficient training and inference processes.

Enhanced data and optimization

To enhance MAI-Base-2, Microsoft increased the volume of proprietary data used by 36% compared to its previous model, MAI-Base-1. The company prioritized internal data collection over distillation methods and used the Muon optimization algorithm and gradient adjustment techniques to improve training stability and reduce resource needs.

Multimodal capabilities

MAI-Base-2 is a multimodal model that can process text and images. It is designed to analyze documents, interpret charts, understand spatial relationships, and perform certain tasks on a computer, making it versatile for various applications.

Future development and availability

Microsoft is continuing to refine MAI-Base-2 and plans to publish more evaluation results. The company is also working on MAI-Base-3, aiming to further improve performance while optimizing infrastructure. Microsoft intends to make the model's weights available to professional customers through Microsoft Foundry, though the exact availability date has not been announced.