Alibaba has released the model weights for Qwen3.8-Flash-Next so developers can experiment with and evaluate an early precursor to the forthcoming Qwen4 architecture. The model is a multimodal mixture-of-experts (MoE) with 176 billion total parameters, including 51 billion N-gram embedding parameters, and it activates about 6 billion parameters per token. Its native context window is 262,144 tokens and can be extended to 1 million tokens using YaRN.
Target use cases
Qwen3.8-Flash-Next is aimed at high-volume, context-intensive workloads such as agentic coding, document processing and tool-driven workflows. In very long contexts, attention computation and KV cache memory typically become bottlenecks; the model introduces architectural measures to address both constraints.
Hybrid architecture: Gated DeltaNet (GDN) and Qwen Sparse Attention (QSA)
The model uses a hybrid design that alternates Gated DeltaNet (GDN) and Qwen Sparse Attention (QSA) layers. Three out of every four layers run GDN, which continuously compresses historical context into a fixed-size recurrent state and thereby prevents KV cache growth as sequence length increases. The remaining layer uses QSA to provide precise retrieval across the full context.
QSA differs from prior sparse-attention schemes that rely on token-level indexers and grow more expensive with context length. QSA aggregates the sequence into micro-blocks, estimates importance at the block level, and selects only the most relevant regions for attention. This reduces attention, compute and indexing overhead within each layer and pairs effectively with the GDN/QSA alternation.
Benchmarks on long-context workloads
Alibaba’s published benchmarks indicate that QSA can improve efficiency for 1M-token workloads. Compared with full attention, the QSA attention kernel delivered up to 7.6× speedup during prefill and 4.9× during decoding. In a cache-heavy online serving test at 1M-token context length and a 90% prefix-cache hit rate, Qwen3.8-Flash-Next achieved 8.6× the prefill throughput of Qwen3.7-Plus.
NVIDIA support and hardware validation
NVIDIA provided best-effort Day 0 functional support via SGLang, vLLM and NVIDIA TensorRT LLM, performed validation on the NVIDIA GB300 NVL72 for inference, and supplied post-training recipes through NVIDIA NeMo AutoModel and NVIDIA NeMo RL.
The GB300 NVL72 is a rack-scale system integrating 72 NVIDIA Blackwell Ultra GPUs into a single NVLink domain. Its all-to-all communication bandwidth is 130 TB/s, which reduces bottlenecks when MoE expert traffic must traverse the interconnect. On GB300 NVL72, Qwen3.8-Flash-Next achieved peak throughput above 16K tokens per second per GPU and delivered over 200 tokens per second per user, enabling high-throughput, low-latency experimentation with agentic coding workloads.
Local deployment options
Beyond rack-scale deployment, Qwen3.8-Flash-Next can run on local NVIDIA hardware including NVIDIA DGX Station, NVIDIA DGX Spark clusters, and workstations equipped with four NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPUs. This allows developers to prototype agentic coding workflows locally and scale the same model to GB300 NVL72 for production serving.
Fine-tuning and inference ecosystem
Developers can fine-tune the model for domain-specific needs using NVIDIA NeMo AutoModel, a PyTorch-native fine-tuning library with Day-0 Hugging Face checkpoint support that enables full SFT or memory-efficient LoRA fine-tuning without model conversion. Reinforcement learning workflows are supported via NVIDIA NeMo RL recipes.
NVIDIA supports multiple inference stacks so teams can choose the runtime that fits their requirements. SGLang, vLLM and TokenSpeed provide open-source inference recipes for developers seeking fine-grained control over performance on NVIDIA-accelerated platforms.
How to get started
Qwen3.8-Flash-Next is available to try on QwenCloud. Model weights can be downloaded from Hugging Face or ModelScope so that developers can run, evaluate and fine-tune the model on their own environments.
Qwen3.8-Flash-Next combines long-context architectural innovations with rack-scale NVIDIA validation to target large-scale agentic and document-intensive AI applications.
