Today, Baidu officially opened the latest base model Wenxin 5.5 Flash Base. ERNIE 5.5 Flash Base uses the MoE architecture, with a total number of 280 billion parameters, each token activates 14 billion parameters, uses 33.5 trillion tokens to complete pre-training, the maximum length of the pre-training context is 128K, and supports up to 1 million token contexts during the fine-tuning and inference stages.

Zhitongcaijing · 2d ago
Today, Baidu officially opened the latest base model Wenxin 5.5 Flash Base. ERNIE 5.5 Flash Base uses the MoE architecture, with a total number of 280 billion parameters, each token activates 14 billion parameters, uses 33.5 trillion tokens to complete pre-training, the maximum length of the pre-training context is 128K, and supports up to 1 million token contexts during the fine-tuning and inference stages.