Six AI Labs Shipped Decision Models in 17 Days

Somewhere inside almost every AI agent, a very large model is writing a very small essay that nobody will ever read.

A refund request lands in a support queue at 2:14 in the morning. “Charged twice for the same order. Please fix.” Before anything useful can happen, the agent watching the queue has to answer a question a twelve-year-old could answer in a glance: is this a billing problem or a technical one?

So the agent asks a frontier model. The model reads the ticket, plus two pages of instructions about how to read tickets. Then it starts to write. “Based on the customer’s message, this appears to be a billing issue, since the customer mentions being charged twice for…” Word by word, each word billed, for the better part of a second. Downstream, a few lines of code wait for it to finish, search the text for the word “billing”, and throw the rest away.

Now multiply that by every fork in every agent. Is this request in scope? Which tool should run next? Does this need a human? Did the last step actually work? An agent doing real work asks itself dozens of these questions per task, and the industry’s default has been to have the most expensive writer ever built compose a paragraph for each one.

On 15 September 2026 a startup called TypeSafe AI shipped a model built on a blunt observation: most of those questions do not need a paragraph. They need a number. Seventeen days later, OpenAI, Liquid AI, Amazon, Cloudflare and Perplexity had all shipped their own version of the same idea. The category now has a name, decision models, and this essay is about what they are, why they arrived all at once, and what it takes to trust one.

Seventeen days turned one startup’s bet into a category

TypeSafe AI came out of two years in stealth with a $40 million seed round led by DCVC^[4]. Its chief executive, Diogo Almeida, previously worked at Google Brain and OpenAI on the human-feedback training that made chatbots pleasant to talk to. His new company’s argument is that this same training pointed the field in a slightly wrong direction. Models were tuned to be good conversation partners. Software, it turns out, mostly needs a good switchboard operator.

The model is called Jev, after William Stanley Jevons, the Victorian economist who noticed in 1865 that more efficient steam engines made Britain burn more coal, not less^[5][13]. The name is a wager about what happens when intelligence gets cheap. Jev reads up to 32,000 tokens of input, costs $0.042 per million of them, and charges nothing for output, because there is no output in the usual sense^[4]. InfoQ reported that roughly 13% of paid teams on Vercel’s AI Gateway had used it within its first 24 hours.

Then the dominoes. At its DevDay on 29 September, OpenAI previewed a Decisions API running on a model called GPT-6 Luna^[9]. The same day, Liquid AI released d1 and claimed to be the first model to beat Jev on a public decision leaderboard^[7]. On 1 October three more arrived in a single day. Amazon distinguished engineer Marc Brooker, who had built a prototype after seeing Jev, open-sourced Strands Decider 2B along with its training data and scripts^[3]. Cloudflare released Clef and Clef-flash under Apache 2.0^[1]. Perplexity opened a Decisions API on an open 27-billion-parameter model of its own^[8].

A category does not usually form this fast. This one could, because nobody needed a new kind of neural network to build it. Cloudflare’s Clef sits on Alibaba’s Qwen models. So does Perplexity’s model, and so does Amazon’s. The hard part was never the architecture. It was the decision to stop optimising a model for chat and start optimising it for being called by other software, millions of times a day, by code that does not care how polite the answer sounds.

“Prod, not God.”TypeSafe AI’s shorthand for building models that serve as production infrastructure rather than all-knowing assistants^[5].

A decision model reads once and never speaks

To see why this is cheaper, it helps to know what a language model actually does when it answers. The work happens in two phases. First it reads the whole prompt at once, in parallel, which is called prefill. GPUs are very good at this. Then it writes, one token at a time, and every new token requires another full pass through the network. That second phase, decoding, is where most of the waiting happens. A sixty-token answer means sixty passes, one after another, none of which can start until the previous one finishes.

A decision model keeps the first phase and deletes the second. Cloudflare describes Clef as a Qwen model that reads the input once, followed by a small “joint schema head” that looks at the model’s internal representation and scores every allowed answer at the same time. A softmax turns those scores into probabilities that sum to one^[2]. Amazon’s version is even more literal. Its engineers removed the language-modelling head from a two-billion-parameter Qwen model, the part that chooses the next word, and replaced it with a “pointer head” of about a million parameters, trained with a small LoRA adapter^[3]. Put plainly, they took the mouth off a language model and gave it a finger to point with. A slower, ground-up walk through the mechanics, with worked math on softmax, calibration and thresholds, is in the plain-English guide to decision models.

Because the model can only point at options it was given, a whole class of failure disappears. It cannot invent a fourth category. It cannot answer “Billing.” with a full stop that breaks a string comparison, or wrap its JSON in a friendly sentence. The vendors have settled on three shapes of question, with slightly different names for the same things.

A request bundles the input, which the vendors call the state, with up to 64 named questions about it. A simplified Clef call looks like this^[1]:

The idea has older relatives. In 2019, researchers showed that a model trained on textual entailment could classify text into labels it had never seen, simply by checking whether “this text is about billing” followed from the input^[15]. You described the labels at request time and skipped the training. What is new in 2026 is the scale of the reading model underneath, support for images (Clef accepts up to four per request), and above all a training objective aimed squarely at the honesty of the probability.

That last part also answers an obvious objection. Every major API already offers structured outputs, where decoding is constrained so the model can only emit valid JSON. Why not ask a chat model for {"team": "billing"} and read the probability off its tokens? Format is not the problem. Confidence is. OpenAI’s own GPT-4 technical report showed that the pre-trained model’s confidence tracked its accuracy closely, and that the chat-style post-training reduced that calibration^[11]. A chat model’s token probability is a number, but not one anybody should put a threshold on without checking it first.

The cheapest token is the one nobody generates

Return to the 2:14 a.m. ticket and scale it up. Suppose a company’s agents make a million routing decisions a day, and each one reads about 1,500 tokens of ticket and instructions. Through a frontier model at GPT-6.1 Sol’s list price of $2 per million input tokens and $10 per million output^[9], with a modest 60 tokens of written answer each time, the arithmetic goes like this.

The comparison is deliberately unkind to the frontier model, and a careful team would push back on it. Prompt caching can cut the cost of a repeated instruction block by most of its list price. A small, cheap generative model would close much of the distance. Still, the shape of the bill changes. A generative model charges for reading and for writing; a decision model charges only for reading. For a question whose answer is one word, writing was always the wasteful half.

Latency tells the same story, with a twist that is worth knowing about before trusting anyone’s speed chart. On Cloudflare’s own test harness, 43 benchmark runs, Clef-flash answered in a median of 38.8 milliseconds and Jev in 524.1^[1].

Here is the twist. A separate analysis reported by InfoQ measured Jev at a median of 76 milliseconds^[4]. Same model, roughly a sevenfold difference, and the likely reason is mundane: Cloudflare was timing its own model inside its own network and timing Jev across the public internet. A developer in Italy who tested Clef from outside Cloudflare saw medians of roughly 200 milliseconds for Clef-flash^[2]. None of these numbers is wrong. They just answer different questions, and a launch-week chart rarely says which one.

Even with the caveats, the difference adds up quickly inside an agent, because agents ask their questions in sequence. Suppose a single turn passes through six checkpoints, and suppose each one costs about 0.8 seconds as a short generation.

For a chat window, 4.8 seconds is irritating. For a voice agent it is fatal. Across ten languages studied, the typical gap between one person finishing a turn and the next person starting is around 200 milliseconds^[14]. A phone agent that thinks in paragraphs leaves callers saying “hello?” into dead air. One that thinks in probabilities can afford to check itself before it speaks.

The probability is the product, so calibration is the whole game

A decision model’s entire output is a probability. That makes one property matter more than any benchmark score: whether the probability means what it says. A weather forecaster who announces a 70% chance of rain is calibrated if it rains on about 70% of those days. A decision model is calibrated if, of all the times it says 0.95, it is right about 95% of the time.

Modern neural networks are famously bad at this by default. A 2017 study found that as image classifiers grew deeper and more accurate, they also became more overconfident, saying 0.99 about answers that were right far less often^[12]. The standard way to see it is a reliability diagram: group answers by the confidence the model stated, then check how often each group was actually right.

Why this matters becomes obvious the moment someone sets a threshold. Say a support team lets the agent act on its own whenever the decision model is at least 95% sure, and sends everything else to a person. Out of 10,000 tickets a day, 6,000 clear the bar. If the model is calibrated, at most about 5% of those 6,000 are wrong: some 300 bad automatic actions. If it is overconfident in the way Figure 5 shows, the 0.95 bucket is really right 83% of the time, and the same dashboard hides 1,020 mistakes. The automation rate looks identical, and errors go up by a factor of 3.4. Nothing in the API response would give it away.

So the vendors’ central technical claim is about calibration rather than accuracy. TypeSafe calls its training method Reinforcement Learning for Calibrated Decisions, or RLCD. The underlying idea is old and sound. Score a forecaster with a proper scoring rule, one where the best possible expected score comes from reporting what you actually believe, and honesty becomes the winning strategy. The Brier score is the simplest example, and Amazon reports it for Strands Decider^[3].

Almeida’s argument is that chat-style feedback training does the opposite. Rewarding answers people prefer pushes a model to concentrate its probability on the safe, likeable response, and the honest uncertainty gets squeezed out^[5]. There is supporting evidence from before the chat era: Anthropic researchers found in 2022 that large pre-trained models were reasonably well calibrated on multiple-choice questions when asked in the right format^[10]. The reading ability was there. The question was whether training would preserve it.

The honest summary is that the theory is good and the evidence is still thin. As InfoQ noted at launch, TypeSafe had not published expected calibration error figures for Jev^[4]. And calibration is never a property of a model alone. It is a property of a model on a particular distribution of inputs. A model calibrated on banking chat intents can be overconfident on Hindi-English code-switched voice transcripts, because the inputs it is unsure about are different. The only reliability diagram that matters is the one drawn from your own traffic.

A menu is only as good as the options on it

The same design that eliminates invalid answers creates a quieter failure. A decision model always picks something from the list. If a ticket is about a data-protection request and the options are billing, technical and other, the model will spread its probability across those three and pick one, perhaps with high confidence. InfoQ put it neatly: the model cannot emit an invalid type, but it can return a completely wrong valid value^[4]. A team that forgets to include an honest “none of these” option has built a system that can never say so.

The limits in reasoning are just as clear, and the vendors say so themselves. Users report that Jev is unreliable at counting, arithmetic and comparing dates, and degrades on large, noisy inputs^[4]. On Cloudflare’s leaderboard, Clef beats Jev on the BANKING77 intent benchmark, 94.2 to 79.7, while Jev beats Clef on the graduate-level science questions of GPQA Diamond, 78.3 to 48.0^[2]. Amazon’s launch post is blunter still: the design makes Strands Decider “significantly worse at solving complex problems than reasoning models”, and unsuited to coding, chat or summarisation^[3]. On launch day, Clef’s image questions took 13 to 30 seconds to answer^[2].

Then there are the leaderboards. Every company in Figure 1 launched with a claim to be winning, and each claim was measured on a different scoreboard.

Read across the right-hand column and the pattern is plain. Each benchmark was built or chosen by the company reporting it, and nobody has yet reproduced anyone else’s numbers^[2]. Almeida himself is openly sceptical of public benchmarks, calling them “extremely gameable”^[5]. That is no reason to dismiss the category. It is a reason to treat every number in the table as a starting hypothesis.

In a working agent, decision models are the reflexes, not the brain

TypeSafe’s “System One” branding borrows from Daniel Kahneman’s split between fast, intuitive thinking and slow, deliberate reasoning. The analogy is more useful than most marketing. Nobody wants a person to deliberate over whether to catch a falling glass, and nobody wants them to sign a contract on reflex. An agent needs both kinds of thinking, and the mistake so far has been running both on the slow one.

Drawn out like this, the pattern is hard to miss. In a refund flow, most steps are questions with a short, fixed list of answers: whether the request is in scope, whether the account allows it, which tool to call, whether a person should take over^[6]. Only one step needs actual writing. The launch posts list the same jobs again and again: routing between models, choosing tools, checks before an irreversible action, guardrails, deciding what to keep in an agent’s memory, and grading other models’ output against a rubric^[3].

Voice agents show off the pattern especially well, because they live under the 200-millisecond clock. A caller says “yeah, I guess, go ahead” after hearing a cancellation fee. Did they agree? Text-matching on “yeah” says yes. A generative model will explain the nuance at length. A decision model can return a single probability that the caller consented, and the agent can follow a simple rule: act above 0.9, ask again between 0.5 and 0.9, and treat anything below as a no. The number is only useful if it is calibrated on real callers, saying real versions of “yeah”, which is the subject of the next section.

A few design habits follow from all this. Let the generative model plan and write; let decision models gate. Put a decision in front of every action that cannot be undone. Always include an escape option such as “none of these” or “unclear”. Log the probability next to every automatic action, so that when something goes wrong there is a number to audit rather than a paragraph to reinterpret. And route the low-confidence slice to people, because that slice is both where the errors live and where the next round of training data comes from.

Jevons would predict more decisions, not cheaper agents

Back to the economist on the label. Jevons’s point in The Coal Question was that making steam engines more efficient did not save coal. It made steam useful in so many new places that total consumption rose^[13]. TypeSafe named its model for that effect, and it is the right prediction here too.

When a decision costs a fiftieth of what it used to, teams will not pocket the savings and ask the same few questions. They will ask fifty times as many. Every tool call gets a pre-flight check. Every outgoing message gets scored against policy. Every memory write gets a relevance vote. The agent of 2027 may well spend more on decisions in total than the agent of 2025 did, simply because it makes thousands of them where it used to make a dozen.

That moves the bottleneck. When every decision is cheap, the expensive thing is knowing whether the decisions are right. Each new question needs a threshold, each threshold needs a reliability diagram, and each diagram needs labelled examples the model has never seen, drawn from the traffic it will actually face. The numbers are not huge, but they are not zero either. To check that the 0.95 bucket really is right 95% of the time, to within two percentage points, takes about 456 labelled examples in that bucket alone.

Multiply that by every question, every language and every channel an agent handles, and refresh it whenever the traffic changes. That labelled, held-out, human-checked data is where an agent’s reliability is actually decided, and it is the kind of evaluation data Perit builds for speech and voice-agent teams. The model vendors can make the probability cheap. Only data from the real world can make it honest.

The 2:14 a.m. ticket will still get answered. The difference is that nobody will be paying for the essay. The paragraph was never the point. The number was.

Frequently asked questions

What is a decision model in AI?

A decision model is a model that takes an input, such as a support ticket, a screenshot or an agent’s history, plus a set of typed questions, and returns a probability for each allowed answer instead of generating text. The three common question types are yes or no, choose one option from a list, and score on an ordered scale.

How is a decision model different from a classifier?

A traditional classifier is trained on a fixed set of labels and must be retrained to add one. A decision model takes the labels and their descriptions in each request, so the same model can answer a new question without training. It is closer to zero-shot classification built on top of a large language model.

Why not use an LLM with structured outputs or JSON mode?

Structured outputs guarantee the format but still pay for generation, and a chat model’s token probabilities are often poorly calibrated after its chat training. Decision models skip generation entirely and are trained so that their probabilities can be used as thresholds.

Are decision models open source?

Some are open-weight. Cloudflare’s Clef and Clef-flash and Perplexity’s pplx-decider-v1-27b are released under Apache 2.0, and AWS’s Strands Decider 2B ships with its training data and scripts. TypeSafe’s Jev, Liquid AI’s d1 and OpenAI’s Decisions API are hosted only.

When should an agent not use a decision model?

When the answer is not on a short list, or the task needs multi-step reasoning, arithmetic, counting, date logic, long noisy context or written output. Those belong to a generative or reasoning model, and a decision model can then check the result.

References and links