# Prime Intellect releases Prime Mega MoE kernel for faster training

Digest AI · Research · published 2026-10-29T18:28:57Z

Canonical: https://digestai.news/story/prime-intellect-releases-prime-mega-moe-kernel-for-faster-training

## Summary

Prime Intellect has released Prime Mega MoE, a GPU kernel extension that adds a fused, overlapping backward pass to DeepSeek’s existing Mega MoE kernel. While DeepSeek’s original kernel only supported the forward pass for inference, Prime Intellect’s addition makes the fused kernel usable for full model training. The tool is designed to reduce idle GPU time caused by communication bottlenecks in mixture-of-experts (MoE) architectures.

According to self-reported figures from Prime Intellect, the new kernel improves training efficiency significantly. On Qwen3 MoE models, full training steps run 32% to 34% faster than the prime-rl baseline. Compared to TorchTitan, the speedup reaches 48% to 55%. The most notable gain is in the forward pass, which is up to 3.11x faster on a Qwen3-30B-A3B layer, though backward pass improvements are smaller, reaching only 1.24x on larger layers.

The kernel supports BF16, MXFP8, and blockwise FP8 precision on NVIDIA Hopper and Blackwell GPUs. It is available as a dispatch backend in the prime-rl trainer, requiring a specific prebuilt DeepGEMM wheel. These performance gains translate directly into reduced GPU hours for teams training large MoE models.

## Key points

- Prime Intellect added a fused backward pass to DeepSeek’s Mega MoE kernel for training.
- Training steps run 32% to 34% faster than the prime-rl baseline on Qwen3 models.
- Forward pass speed reaches 3.11x faster, while backward pass gains are up to 1.24x.

## Why it matters

This reduces GPU hours and costs for training large mixture-of-experts models. It makes advanced MoE architectures more accessible by optimizing the communication-bound training process on standard NVIDIA hardware.

## Sources

1. [Prime Intellect Releases Prime Mega MoE: Trainable Fused MoE Kernel, 3.11x Faster Forward Pass](https://marktechpost.com/2026/10/29/prime-intellect-prime-mega-moe-fused-moe-kernel-training) (MarkTechPost, 2026-10-29)

## Cite

Digest AI, "Prime Intellect releases Prime Mega MoE kernel for faster training", 29 October 2026, https://digestai.news/story/prime-intellect-releases-prime-mega-moe-kernel-for-faster-training

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/prime-intellect-releases-prime-mega-moe-kernel-for-faster-training.json