Thinking Machines releases Inkling: 1T-parameter multimodal model with 1M-token context

Thinking Machines has published Inkling, an open multimodal model on Hugging Face. Inkling is a very large model (around 1 trillion parameters) that natively accepts text, images and audio, and supports a 1,000,000‑token context window. The model is designed for cross‑modal reasoning and for domain adaptation via fine‑tuning.

Key technical facts

• Size and context: the release describes Inkling as ~1T parameters. Internally it is a decoder‑only Mixture‑of‑Experts (MoE) model with documentation citing 975 billion total parameters and roughly 41 billion active parameters at inference time (sparse MoE behavior). The maximum context window is 1,000,000 tokens.
• Training corpus: Inkling was trained on 45 trillion tokens spanning text, images, audio and video.
• Compute and quantization options: the release includes a full BF16 checkpoint and a calibrated NVFP4 quantized variant. The BF16 checkpoint requires about 2 TB of VRAM, while the NVFP4 quantized checkpoint requires roughly 600 GB of VRAM.
• Native modalities: the model accepts text, images and audio; image inputs can include an extra temporal dimension for video processing.

Architecture details

• Decoder‑only: supports causal autoregressive generation.
• Multimodal towers: image patchification uses a simple hierarchical MLP that progressively merges pixels until a single embedding per patch is produced. Audio inputs are converted into discrete mel spectrogram bins (100 ms chunks classified into mel bins whose embeddings are summed to form the final audio input).
• Relative attention: instead of RoPE, Inkling uses a relative attention mechanism where each attention layer learns a fourth projection (R) per token/per head that is adjusted with distance information and fed into the attention logits.
• Hybrid attention: decoder layers alternate between sliding‑window and global attention in a 5:1 ratio (five sliding‑window layers to one global layer); the final layer uses global attention to build richer representations.
• Short convolution (SConv): a short 1D convolution reads the current token and the previous W−1 hidden states (W is the sliding window size) to capture local context and offload locality from attention and MoE components.
• MoE specifics: the model contains 256 experts. The router performs top‑k selection over 6 experts, plus 2 shared experts that are always active.
• Multi Token Prediction (MTP): optional extra layers act as drafters for speculative decoding by predicting multiple tokens at once, enabling faster generation with minor VRAM cost; Thinking Machines ships an MTP drafter with this release.

Inference support and integrations

• Day‑0 Transformers support and compatibility with major inference engines: SGLang, vLLM and llama.cpp are supported at release.
• Transformers usage: the model IDs are "thinkingmachines/Inkling" (BF16) and "thinkingmachines/Inkling‑NVFP4" (quantized). Transformers 5.14.0 was released the same day and is recommended.
• SGLang: a custom model implementation is available; the provided launch example shards the model across 8 GPUs and serves an OpenAI‑compatible API.
• vLLM: a single vllm serve command downloads weights, shards across GPUs with tensor parallelism and starts an OpenAI‑compatible server; multi‑node deployments will typically require SLURM or similar cluster managers.
• Hugging Face Inference Providers: the model can be used via several providers; the release includes examples for using the Hugging Face router/OpenAI client. Thinking Machines covers inference costs for two hours during the release window. Note: audio support in Inference Providers was indicated as a work in progress.
• Local inference and quantized runtimes: llama.cpp and Unsloth provide quantized GGUF builds; Unsloth’s 1‑bit quantization reduces VRAM by up to ~95% and the 1‑bit GGUFs retain ~74.2% of top‑1% accuracy while being ~86% smaller.

Example usage and tooling

• The release includes code snippets for any‑to‑any pipelines, AutoModelForMultimodalLM and AutoProcessor usage in transformers for text‑only, text+image and text+audio prompts. A reasoning_effort parameter (none, minimal, low, medium, high, xhigh, max) controls internal reasoning behavior and token cost.
• Agentic and post‑training tools: Thinking Machines provides tinker, a managed tool for post‑training open‑weight models, and examples for fine‑tuning, distillation and reinforcement learning. The release references OpenEnv and an ECHO algorithm example for agentic RL where the model learns an implicit world model via next‑token prediction alongside policy learning.
• Pi agent demo: guidance is provided to add Inkling as a model in Pi’s agents.json for agentic coding demos that use tool calls to solve tasks.

Benchmarks and vibe evaluations

Thinking Machines published a set of multimodal "vibe" evaluations (expert exam questions, university entrance items) and audio tests (BigBenchAudio, GlobeAudio). Highlights from the shared material include:

• Vibe visual reasoning examples: token consumption and success rates vary with reasoning_effort; many problems were solved at medium and high efforts, with some failures at the highest or lowest settings depending on the task.
• Audio reasoning: the model typically transcribes then reasons; GlobeAudio examples reported completion token counts in the low hundreds (e.g., Russian last‑word question 130–179 tokens across efforts), and BigBenchAudio formal‑fallacy examples showed higher token use and some failure modes at the lowest effort.
• Standard benchmarks: the release includes a comparative table with many contemporary models (GLM‑5.2, Qwen3.7‑Max, Claude Opus 4.8, GPT‑5.5, Gemini 3.1 Pro, etc.). In the provided table Inkling posts strong numbers on several math and reasoning benchmarks (for example AIME 2026: 99.2), and competitive results across other reasoning and agentic tasks; the full table is provided in the release documentation.

Use cases and recommended workflows

• Fine‑tuning and post‑training: Inkling is intended for domain adaptation via fine‑tuning; the Tinker cookbook contains examples for distillation and RL‑based post‑training.
• Document and audio processing: the model is positioned for document understanding and audio reasoning tasks, and the authors recommend experimenting with reasoning_effort to trade off token usage and accuracy.
• Agentic systems: MTP drafters, tool usage examples and Pi agent demos illustrate workflows for tool‑using agents and coding assistants.

Summary

Inkling is a large open multimodal MoE model from Thinking Machines that combines near‑trillion parameter scale, a 1M token context window, native image/audio/text inputs and MoE sparsity (975B total, ~41B active). It ships with BF16 and NVFP4 checkpoints, day‑one framework support, and multiple deployment paths (cloud inference providers, vLLM, SGLang, llama.cpp with aggressive quantizations). The release includes examples, SLURM scripts, and benchmark/vibe evaluations; due to the model’s size, the team recommends using quantized checkpoints or multi‑node cluster deployments for practical inference.