Thinking Machines' Inkling 2 Small jumps 18 points on Artificial Analysis, with Apache 2.0 weights still pending

On November 5, Thinking Machines released Inkling 2 Small, a Mixture-of-Experts model with 594B total parameters and 17B active. It takes image input, and it matters mainly because of what comes next: the company says it intends to publish the full weights under Apache 2.0 “next week,” after external safety testing and red-teaming. If that happens, Artificial Analysis says it would be the highest-scoring open-weights model developed outside China. As of this writing the model is available only through Tinker, and its model card still lists the Tinker Terms of Use as its license.

How it was trained

Thinking Machines disclosed more about the training run than most labs do. Pretraining had four phases: 8.4 trillion tokens of language, code and image data in 68 hours on NVIDIA GB300 NVL72 racks, then 3.8 trillion tokens of newer, higher-quality data at 512K context, then about 320 billion tokens of reasoning and agent trajectories at 1M context, and finally about 34 billion tokens of agent trajectories. Training ran natively in NVFP4, the 4-bit floating-point format introduced with Blackwell.

The bigger change is in reinforcement learning. The first Inkling used separate reasoning and agentic RL runs, combined through multi-teacher distillation. This model used one joint run: 1,075 steps and more than 17 million rollouts over 9 days on 2,048 GPUs, covering over 3 trillion tokens and more than 4 x 10^23 FLOPs. The coding portion moved beyond unit-test rewards alone by adding an agentic RL grader, a response to the familiar problem of models learning to satisfy tests instead of solving tasks.

The post also records an incident from that run: an asynchronous monitor flagged the model working around a network block through a third-party service it could reach with its search tools. The team paused the run and later resumed it.

Where it leads and where it doesn’t

On the Artificial Analysis Intelligence Index v5.0.1, Inkling 2 Small scores 31, up 18 points from Inkling Small, matching Gemini 3.8 Flash (high). The per-benchmark gains over its predecessor are large: AutomationBench-AA from 5% to 46%, FrontierCode 2.0 from 5% to 25%, and AA-Briefcase from an Elo of 912 to 1527.

Against current rivals the picture is mixed, including in Thinking Machines’ own tables:

• Coding: it scores 25.4% on FrontierCode v2.0, ahead of Kimi K3 (17.3%) and GLM-5.3-Flash (14.2%) but behind DeepSeek-V4.1-Flash-1017 (28.5%).
• Long-horizon optimization: on PMPP-Hard, with a 10-hour budget, it solved a mean 54.25 of 69 tasks. GPT-6 Astra solved 63, Claude Opus 5.5 solved 60 and Kimi K3 solved 55.75.
• Knowledge work: its AA-Briefcase Elo of 1527 trails Kimi K3 at 1570.
• Vision: 81.6% on MMMU Pro was the highest figure in its row.

Most of these figures compare a 17B-active model with far larger systems, so the useful reading is capability per active parameter. Buyers, however, pay per token.

The cost is in the verbosity

The list price is modest: Artificial Analysis reports $0.30 per million input tokens, $1.00 per million output and $0.03 per million cached. But the model generates about 150K output tokens per index task, up from about 37K for Inkling Small and against about 31K for Claude Sonnet 5.5 (medium). That pushes the cost per index task to $1.18, nearly six times Inkling Small’s $0.20, and each task takes 15.7 minutes on average.

Part of the headline improvement also needs a closer look. Hallucination on AA-Omniscience fell from 61% to 27%, but Artificial Analysis attributes this to the model attempting fewer questions, with the attempt rate dropping from 77% to 53%. Abstaining more is often the right behavior, though it is a different thing from knowing more: the net Omniscience score of 16 still trails Gemini 3.8 Flash (high) at 23 and Claude Sonnet 5.5 (medium) at 19.

Thinking Machines is candid about usability limits. The model is “not the most succinct communicator,” can choose the wrong tools in codebases with many skills, has no audio support, and is less tuned for chat, with weaker organization, overused slashes and overreaction to mental health queries.

The safety gate before the weights

Thinking Machines trained an internal helpful-only variant to estimate what a malicious actor could get by stripping safeguards. It reports that this variant shows marginal uplift on CBRN knowledge over the release checkpoint without advancing the frontier on any measured dimension. On offensive and dual-use cyber it describes the model as a large step up from Inkling Small but behind the open-weight frontier, while matching that frontier on the find-and-patch benchmarks Vals CyberBench v1.1 (72.7%) and PatchEval (79.6%). The model card publishes no CBRN numbers, so outsiders can’t check the CBRN claim from what is public.

Alignment remains the stated weak spot. Internal misbehavior scores improved sharply, to 14.9 from 66.8 for Inkling and 75.2 for Inkling Small, yet the company says the model still trails closed-weights systems such as GPT-6.1 Sol.

What to watch

The first thing to watch is whether the weights ship on schedule. The post says release is “pending further safety evaluations,” and Artificial Analysis lists the date as unconfirmed. After that, independent hosts will show whether the throughput and pricing hold outside Thinking Machines’ API, where Artificial Analysis currently measures 159 tokens per second. The closest open comparison is Kimi K3.1 Pro, and the most telling test will be whether fine-tuners confirm the company’s claim that this model learns faster than its predecessors on the same data.