The developers have open-sourced their Mixture-of-Kittens (MoK) MoE training megakernel for NVL72 hardware; the kernel combines expert communication and computation into a deterministic unit, and according to the announcement can be up to 2.37 times faster than the best publicly available baselines, enabling faster and more predictable modeling.
AI-generated text
Mixture-of-Kittens (MoK) megakernel released as open source for NVL72
The developers have open-sourced their Mixture-of-Kittens (MoK) MoE training megakernel for NVL72 hardware; the kernel combines expert communication and computation into a deterministic unit, and…



