Decision models bring bounded answers to edge AI

Liquid AI has released d1-3B and d1-omni-600M, two open-weight decision models designed to answer named questions directly rather than generate prose. Give the model a state—text, structured data, an image or, for the experimental smaller model, a short voice clip—and define the allowed answers. One forward pass returns typed choices, scores or yes-no probabilities.[1][2][3][4]

The architecture is useful because many production calls are not writing tasks. A support system may need to select the correct queue. A vision system may need to decide whether a component should be inspected. An agent guardrail may need to score whether an intended action violates a rule. Asking a chat model to narrate its way to a JSON label adds generation time, parsing failure and language that the application may immediately throw away.

A decision model changes the interface, not the need for judgement

A generative model normally estimates one token after another. A decision model instead evaluates predefined answers against the supplied state. That can remove a large class of integration problems: there is no free-form output to coerce into a schema, no explanation to truncate and no answer sentence whose wording accidentally changes a downstream parser. It also forces the application designer to declare the decision being made.

That declaration can improve governance. ‘Which team should receive this ticket?’ is easier to test than ‘handle this request.’ But the apparent simplicity can conceal bad option design. If the correct route is absent, the model may distribute probability across the wrong choices. If two labels overlap, a high-confidence answer may reflect the ambiguity of the rubric rather than the clarity of the evidence.

Millisecond latency depends on the input

The best headline number is 8 milliseconds for one question on an RTX 4090. The same table shows why a workload estimate needs more than that figure. A 3,400-token state takes 102 milliseconds on the RTX 4090, 220 milliseconds on Jetson AGX Thor, 560 milliseconds on AGX Orin and 1,640 milliseconds on Orin Nano. On the Nano, the longer state is 32.8 times the one-question latency, calculated as 1,640 divided by 50.[2][1]

The table also shows that asking three questions about the same state can be efficient: Jetson AGX Thor moves from 16 to 20 milliseconds, and Apple M5 Pro from 30 to 41. That does not mean every extra question is nearly free. The questions, state length, image processing, model loading, preprocessing and application code all contribute to end-to-end delay.[2]

Latency numbers need a workload model

• At the reported 16-millisecond single-question latency on Jetson AGX Thor, a purely sequential arithmetic ceiling is 62.5 decisions per second.
• At 26 milliseconds on Jetson AGX Orin, the same ceiling is about 38.5 decisions per second.
• At 50 milliseconds on Jetson Orin Nano, the ceiling is 20 decisions per second.
• The separately reported AMD MI325X result is 1,106 states per second for 64 packed states; batching makes it a different measurement from the single-request ceilings.

These are not production-throughput claims. The calculated ceilings are simply 1,000 milliseconds divided by the reported single-request latency. They exclude input handling, network delay, safety logic, queues and every downstream action, while the packed accelerator result uses a different workload shape.[2]

Calibration is the product requirement hiding behind speed

A useful decision system needs probabilities that correspond reasonably to observed outcomes. If cases assigned 90% confidence are right only 60% of the time, the score cannot safely drive thresholds. Teams should evaluate calibration by decision type, subgroup and operating condition, then choose thresholds based on the cost of a false positive, a false negative and a human review.

An abstain route is especially important. A model asked to choose among Billing, Technical and Fraud should be allowed to say that the evidence is insufficient or the request does not fit. Otherwise, every uncertainty is converted into a confident-looking business action. The right metric is not merely accuracy; it is the quality of the whole policy that maps scores to actions.

The strongest use is a narrow decision with a visible escape hatch

Decision models could become a useful layer between sensors, software and slower generative agents. A small model can classify a request, rank candidates or flag an action locally; a larger model or person can handle the uncertain minority. That architecture saves time only when the escalation logic is measured and the local decision does not discard information the next stage needs.

The deeper lesson from d1 is not that generated text is obsolete. It is that generation has become the default even where the product needs a bounded answer. Separating deciding from explaining can make a system faster and easier to test. It also makes the designer responsible for defining the question, the answer space and the conditions under which the model must refuse to decide.

Report an error or suggest a correction