Two weeks after releasing Inkling, its first open‑source language model, Thinking Machines introduced Inkling‑Small. The new model is multimodal (it accepts text, image and audio inputs and produces text) and is distributed under the permissive Apache 2.0 license.
Technical details and licensing
- Total parameters: 276 billion.
- Active parameters per token: 12 billion (compared with Inkling’s 41 billion).
- Context window: up to 1 million tokens.
- License: Apache 2.0.
Inkling‑Small uses a sparse Mixture‑of‑Experts (MoE) architecture: its 42‑layer decoder routes each token to six of 256 specialized experts plus two shared experts that are active for every token. That design explains how the model can retain a large total parameter pool but activate only a fraction during inference.
Performance: near the flagship, better on several tasks
On the Artificial Analysis Intelligence Index, Inkling‑Small scored 40 versus Inkling’s 41. This is notable given Inkling’s 975 billion total parameters and 41 billion active parameters.
Inkling‑Small outperforms the larger model on multiple evaluations:
- SWE‑bench Verified: 80.2% (Inkling: 77.6%).
- Terminal Bench 2.1: 64.7% (Inkling: 63.8%).
- It also edges ahead on SciCode, Humanity’s Last Exam, GPQA Diamond and CritPt.
However, Inkling retains advantages in factual knowledge and some agentic tasks. For example, τ³‑Banking: Inkling‑Small 15.5% vs Inkling 23.7%. Inkling‑Small’s AA Omniscience score is negative, indicating weaker factual coverage, though its reported hallucination rate is slightly lower.
For enterprises this trade‑off matters: Inkling‑Small is attractive for coding assistants, tool‑use systems, retrieval‑augmented generation, document analysis and multimodal workflows, but high‑stakes factual use cases will still require retrieval, verification and human review.
How a 276B model uses only 12B at a time
The MoE architecture is key: the model keeps a large learned capacity but activates only selected experts per token, which yields the 12 billion active parameter figure. The decoder processes images, audio and text in a shared representation rather than delegating modalities to separate external systems, making the model natively multimodal.
Thinking Machines also supports variable reasoning effort, allowing developers to trade test‑time compute for quality depending on task difficulty.
Deployment and resource requirements: “Small” is relative
Despite the name, Inkling‑Small is not a consumer desktop model. The standard BF16 checkpoint requires at least 600 GB of aggregate GPU memory; supported configurations listed include 4× NVIDIA B300 or 8× NVIDIA H200.
A quantized NVFP4 checkpoint reduces the requirement to roughly 180 GB of aggregate VRAM and can run in W4A4 mode on a single NVIDIA B300 or W4A16 on two H200 GPUs. These requirements rule out ordinary laptops, MacBooks, gaming PCs and most developer workstations. Practical deployment targets are enterprise GPU servers, cloud clusters and specialized inference providers.
Nonetheless, the reduction versus Inkling is meaningful: lower aggregate memory needs can reduce hosting costs, simplify capacity planning and broaden the set of organizations able to self‑host the model.
Availability, fine‑tuning and pricing
Thinking Machines released the full weights on Hugging Face and added fine‑tuning support through its Tinker training API. At launch the company offered a limited‑time 50% discount:
- 64K‑context Inkling‑Small API prices: $0.58 per million prefill (input) tokens, $1.44 per million sampled (output) tokens, $1.73 per million training tokens. Cached prefill requests are $0.116 per million tokens.
- A 256K‑context variant is available at higher rates.
The company supports deployments through SGLang, vLLM, TokenSpeed, Unsloth and Hugging Face tooling, giving developers multiple ways to use the model: via API, fine‑tune on Tinker, run on third‑party inference providers or host on private infrastructure.
Repeatable development pipeline
Thinking Machines has moved from a one‑off release approach to a repeatable pipeline. Researcher Horace He noted that releasing Inkling felt like it took “a village,” whereas Inkling‑Small used the same pipeline with a smaller model and felt routine. The company says improvements came from a refined pre‑training data mix, changes to the training recipe and on‑policy distillation using Inkling as a teacher, followed by two weeks of agentic coding reinforcement learning.
Implications for enterprises and builders
For organizations weighing a very large flagship against a more manageable open‑weight system, Inkling‑Small presents a compromise: nearly the same measured intelligence, stronger results on several coding and reasoning tasks, lower token pricing, a smaller hardware footprint and the commercial flexibility of an Apache 2.0 license. It is not intended for local consumer use, but for enterprises seeking control over data, model behavior and fine‑tuning it may be an appealing option.
More broadly, the release signals that Thinking Machines is iterating toward a cadence in which open‑weight multimodal systems can be produced, improved and deployed more routinely rather than as bespoke research artifacts.
