Google ships a Decisions API at $0.10 per million tokens, as Amazon shows a decider cutting a Sonnet bill by 98%

On November 11, Google launched the Decisions API, a Gemini endpoint that does not generate text. You send one shared input plus a set of closed questions, and a new model, gemini-decision-1, returns a probability and a confidence score for each. It costs $0.10 per million input tokens and nothing for output. Two days earlier, two Amazon engineers published the production numbers from swapping Claude Sonnet 4.6 for a self-hosted decision model on an internal ticket labeling system: about 98% lower cost at the same quality. Together they are the clearest evidence so far that a large share of what agents ask LLMs to do is not generation at all, and is being priced accordingly.

What Google actually shipped

The endpoint is POST /v1beta/decisions on the Gemini Developer API. A request has three parts: the model, an input (text, images, audio, video or PDFs, using the same content blocks as the Interactions API), and a questions map of 1 to 1,000 entries. According to the documentation, each question is one of three types:

• noul (yes/no): returns the probability that a condition holds, clamped to the range 0.01 to 0.99.
• choice: 2 to 255 options, each optionally described by a criterion, returning the winner and normalized probabilities for all of them.
• score: an ordered rubric of 2 to 255 levels (Google recommends 2 to 10), returning a probability-weighted average level plus the per-level distribution.

Every answer carries a confidence between 0 and 1. For choice and score it is the top outcome’s probability; for noul it is max(p, 1 - p). The whole context window, input plus all questions, is 262,144 tokens.

Google says the model scores candidate options in a single forward pass rather than decoding an answer token by token, which is where the speed comes from: 150-250 ms end to end, 50-100 ms of median model compute, and latency that stays nearly flat between 1 and 100 questions on contexts up to 16K tokens. The response’s usage block reports total_output_tokens: 0.

The billing rule matters more than the headline rate. The shared input is charged once per request no matter how many questions ride on it; each extra question adds only its own instruction and criteria tokens. Checking a 10,000-token document against 50 policy rules costs roughly one read of the document, not fifty. There are no thinking-token charges and no separate OCR or audio transcription fees.

Amazon’s case study puts numbers on the trade

The Strands Agents post, by Carr Onstott and Mehul Jaiswal, is the more useful document for anyone deciding whether to make this switch, because it is a migration report rather than a launch post. The workload: classify tickets in one of Amazon’s largest internal ticketing systems into 20 categories (customer escalations, capacity requests, automated alarms, security findings and so on). The incumbent was Sonnet 4.6 on Bedrock with a long system prompt and JSON reasoning output. The replacement was Strands Decider E4B, an open decision model built on Gemma 4, on a SageMaker ml.g5.xlarge endpoint with a single A10G GPU.

Metric	Sonnet 4.6	Strands Decider E4B
Average time per call	4,215 ms	897 ms
Input tokens per ticket	6,554	2,009
Output tokens per ticket	170	1
Cost per million tickets	about $22,200	about $500

The cost figures use published us-east-1 prices: $3 and $15 per million tokens for Sonnet, $1.408 per instance-hour for the GPU. At roughly 640 ms per request, one instance handles about 5,600 tickets an hour, which works out to $250 per million at full utilization; the team doubles that to leave burst headroom.

Three details are worth more than the headline:

1. Most of the token saving came from rewriting the prompt, not from the model. The production label descriptions alone exceeded the decider’s 4,096-token window, so each of the 20 labels was rewritten as a one or two sentence criterion, and a “none” option was added. Long tickets lose their oldest comments.
2. JSON key order moved probabilities by up to 0.19. The team now sorts keys on every request. Anyone comparing decision-model runs across clients should check this before trusting a diff.
3. The rollout was a three-way switch in AppConfig (BEDROCK, DUAL, DECIDER). In dual mode both models ran on live traffic, Sonnet’s labels were applied, and every disagreement was reviewed. The authors say this taught them more than offline evaluation, and rollback was a 45-second config change.

One caveat: the post reports no accuracy number. It says E4B “performed just as well” and beat production Sonnet on a separate ticket set after a confidence threshold was tuned. That is a claim you would need to reproduce on your own labels.

Where Google’s version fits against OpenAI’s and the open models

The category now has three hosted or self-hosted tiers, and the pricing has converged fast. OpenAI’s Decisions API, which entered public beta on October 6, also charges $0.10 per million input tokens with no output fee and uses the same three-way split of predicate, choice and score. Google has matched the price exactly and differentiated on shape: 1,000 questions per request, a 256K-class context, and native audio, video and PDF input, where the Strands model tops out at a 4,096-token text window.

Run Amazon’s own numbers through the hosted price and the gap between hosting and self-hosting nearly disappears. At 2,009 input tokens per ticket, a million tickets would cost about $200 on a $0.10 endpoint, below the $500 Amazon pays for its headroom-padded GPU fleet. Even the original 6,554-token Sonnet prompt would cost about $655 per million. That is back-of-envelope arithmetic, not a benchmark, but it suggests self-hosting a decider is now a decision about data perimeter and latency control (Amazon’s endpoint has no outbound network access) rather than cost.

Google’s own early users show the intended uses. Gemini Notebook routes queries to a model tier and thinking level with it and reports up to 40% lower latency on everyday queries. Google Search uses it as a safety layer on MCP tools. A Google Labs product picks an emoji reaction from more than 200 options. Routing, guardrails and grading: the calls that sit in an agent’s inner loop and are currently paid for at generation prices.

Limits to know before you migrate

The documentation is candid about what is missing at launch. The API is on the Gemini Developer API only (Google Cloud support is “coming soon” behind a sign-up form), with no context caching, no Batch API and no Flex or Priority inference. Multi-turn history and tool-call blocks must be serialized as text. YouTube and gs:// URIs are not accepted.

Two behaviors will catch people who treat the output as a verdict. A choice question always spreads probability across the options you supplied, so if none fits, something still wins; either add an explicit “none” option, as Amazon did, or gate on confidence. And because yes/no values are clamped at 0.99, any threshold above that never fires. Google’s guidance is to set thresholds from labeled samples of your own traffic and to weigh false positives against false negatives explicitly.

Calibration is also the claim with the least public support so far. Google says Gemini Decision 1 has the lowest calibration error across its benchmark suite and is nearly 5x cheaper than Gemini 3.5 Flash-Lite on the same tasks, but the launch post presents the results as charts without a methodology. Decision models also inherit the injection problem: when Check Point tested an earlier typed decision model, every attack configuration broke it at least once. A model that reads attacker-controlled input and emits a probability can still be steered, so a decider used as a safety gate needs the same adversarial testing as an LLM judge.

What to watch

• Independent calibration numbers. Confidence scores are only useful if 0.9 means right nine times in ten on your data. Expect third-party comparisons of Gemini Decision 1, OpenAI’s endpoint and the open deciders within weeks.
• Vertex AI availability and batch pricing, which determine whether enterprises can use this for bulk grading and moderation backlogs.
• Framework defaults. Integrations are rolling out in LangChain, Vercel’s AI SDK, LiteLLM, Pydantic AI, OpenRouter and others. If agent frameworks start sending tool selection and routing to decision endpoints by default, a meaningful slice of small-model generation revenue moves to a product that bills a fraction as much.
• Whether Amazon publishes accuracy data. The dual-run disagreement set already exists; the numbers would turn a good anecdote into a reference result.