NaiveAI has released Naive‑N0.5‑Flash, an open‑weights model tuned for AI coding tasks and for maximizing tokens‑per‑second. Paired with the NaiveRT inference engine, the model is presented as a step toward faster engineering and research iteration and, in NaiveAI’s framing, toward a self‑improving research agent. The model is a continued development of Xiaomi’s MiMo‑V2.5 checkpoint, further trained on sparse attention layers.
Origins and training
- MiMo‑V2.5 is a 310B‑parameter Mixture‑of‑Experts model with 15B active parameters, originally trained on 48T tokens. It uses a hybrid attention design combining sliding‑window attention (SWA) and global self‑attention.
- NaiveAI started from those weights and performed multi‑stage further training to adapt the checkpoint to DeepSeek Sparse Attention layers. According to NaiveAI’s technical blog, this required roughly 3.25T additional tokens on top of the original 48T; about 3T of that extra data was trained at a fixed 1M context length with a constant learning rate.
Benchmarking setup
The ultrafast mode of Naive‑N0.5‑Flash was compared with DeepSeek V4.1 Flash 1017 and GLM‑5.3‑Flash. All three models were given the same app‑building prompts and a separate set of 30 text‑only questions each: 10 easy from MATH‑500, 10 medium from GPQA‑Diamond, and 10 hard from HLE‑Diamond. Each request used the same maximum completion token limit of 256K. Measured metrics were TPS (tokens per second), total completion tokens, and end‑to‑end response time. The experimental code is published by the benchmark author.
Intelligence index and public benchmarks
- DeepSeek V4.1 Flash 1017 Intelligence Index: 45
- GLM‑5.3‑Flash Intelligence Index: 43
- Naive‑N0.5‑Flash: Intelligence Index not yet available
Published benchmark scores reported in model cards and technical posts place Naive‑N0.5‑Flash’s coding ability between the two competitors, and ahead on NL2Repo‑Bench. Reported values: NL2Repo‑Bench — GLM 56.3, DeepSeek 68.2, Naive 71.9; DeepSWE v1.1 — GLM 63.4, DeepSeek 77.0, Naive 67.8; Agent’s Last Exam (ALE) — GLM 26.3, DeepSeek 35.4, Naive 32.4.
Speed and verbosity experiments — key findings
- Throughput: Naive‑N0.5‑Flash reports exceptionally high throughput in server‑side measurements, approaching or exceeding 1,000 tokens per second in both thinking configurations. Peak rates measured around 2,100 token/s for a single second and should be treated as upper bounds.
- Decode speeds (measured on a single eight‑GPU Blackwell node using speculative decoding in NaiveRT): roughly 984 token/s during the thinking phase and about 1,816 token/s in the answer phase. Acceptance rates for drafted tokens were 70.6% with thinking disabled and 43.4% with thinking enabled.
- Inference round latency: a full speculative round (draft, verify, sampling, commit) was reported as ~3.4 ms in NaiveRT versus ~12.3 ms in SGLang — a 72.4% reduction.
Verbosity and token use
- On hard HLE questions, Naive‑N0.5‑Flash with thinking enabled averaged ~108,500 completion tokens — roughly nine times the GLM‑5.3‑Flash (high) average and eleven times the DeepSeek V4.1 Flash 1017 (high) average.
- Distributions are skewed: the median completion for Naive with thinking on across all 30 questions was about 3,200 tokens, far below its average of over 51,000; a small number of very long reasoning traces drove the mean upward. Similar, though less extreme, patterns appeared in the other models.
End‑to‑end response times
- Median end‑to‑end times across the 30 questions: Naive (thinking) 3.1 s, GLM‑5.3‑Flash (high) 5.3 s, DeepSeek V4.1 Flash 1017 (high) 10.1 s.
- Mean times were closer: Naive mean 45.3 s, DeepSeek 45.0 s, GLM 86.9 s.
- For hard questions specifically, Naive (thinking) was slower on average than DeepSeek (high): 89.8 s vs. 79.7 s.
Caveats
- Small sample size: only 10 questions per difficulty level.
- No independent grading of answer correctness in these speed/verbosity experiments. Fast replies are useful only if they are accurate.
App experiments
Six model/setting combinations were tested on three app prompts: an interactive 3D Earth and Solar System view, a procedurally grown 3D tree, and a browser port of Ray Tracing in One Weekend. Configurations tested: Naive‑N0.5‑Flash (no thinking), Naive‑N0.5‑Flash (thinking), DeepSeek V4.1 Flash 1017 (low/high), GLM‑5.3‑Flash (low/high).
Results summary:
- Solar system: Naive (thinking) produced the most detailed scene (sunspots, Earth city lights), DeepSeek was close, GLM struggled with lighting and overly sensitive speed controls.
- Tree model: DeepSeek (high) and Naive (thinking) produced good results; Naive without thinking produced no usable app. Minor wind animation glitches appeared in both good apps; GLM deformed strongly under high wind.
- Ray tracing port: all three thinking‑on variants produced working apps. DeepSeek (high) produced the most efficient ray tracer and the best use of screen space. Naive’s implementation was correct but generated less efficient JavaScript; GLM’s interface was the simplest.
Pricing
NaiveAI’s target pricing: $0.10 per million input tokens, $0.40 per million output tokens, and $0.01 per million cache‑read tokens. This places it in the roughly $0.20–$0.60 per million output token band comparable to other Flash family models. Actual bills depend heavily on how many tokens the model emits, which in turn depends on requested thinking budget and task difficulty.
Takeaways
- Naive‑N0.5‑Flash offers very high throughput: in ultrafast mode it can peak ~2,000 token/s and sustain nearly 1,000 token/s during thinking‑heavy tasks.
- Its median response time in the tests was substantially shorter than GLM‑5.3‑Flash and DeepSeek at their high thinking settings; mean times were similar to DeepSeek.
- On complex tasks it consumed far more thinking tokens than the other two models; on easy and medium questions it consistently replied faster.
- In the app tests, Naive was more consistent than GLM (high) and competitive with DeepSeek when thinking was enabled.
For teams that prioritize rapid iteration over token economy — for example rapid prototyping, interactive development and research experiments where end‑to‑end latency matters more than per‑token cost — Naive‑N0.5‑Flash is worth trying. Further, broader evaluations that include accuracy and larger, more diverse workloads are required to judge suitability for production use.
Broader implications
NaiveAI argues that sparse attention (DSA) and speculative decoding reduce decode costs for autoregressive rollouts, a major bottleneck in large‑scale reinforcement learning. If other models were adapted similarly, rollout costs for RL could fall significantly, potentially accelerating frontier research and aligning with NaiveAI’s long‑term vision of self‑improving AI that optimizes its own inference kernels and architectures.
