Sarvam Prism turns AI decisions into a 44-millisecond scoring task

A support system must decide whether a customer needs a refund, a repair or a human specialist. The decision can be much narrower than writing an explanation, yet it still demands three things: a correct classification, a useful estimate of uncertainty and consistent behaviour when a software interface changes how options are displayed. India's Sarvam AI says its new Prism model was designed around precisely that problem. Announced on 3 November, it computes scores for a supplied list of choices rather than generating a stream of words. Sarvam reports a median response time of 44 milliseconds on an Nvidia B200 accelerator.[1]

That headline speed is the result of a considerable engineering trade-off. Prism is a LoRA fine-tune of the company's 105-billion-parameter mixture-of-experts base, described as Sarvam 105B-10B. Only part of the parameter set is active for a given computation. A scoring head calculates a probability for each offered option after a single forward pass; the model is not asked to compose its own candidate answers. Sarvam says the original median request took 664 milliseconds in 16-bit precision across two B200 GPUs. FP8 compression, specialized expert kernels, CUDA Graph capture and merged adapter weights brought the reported figure down to 44 milliseconds on one B200, a 15.1-fold improvement under its conditions.[1]

Fast decisions across different tests

Sarvam tested 25 decision systems across three indices: the public Jev Decision Index, its own recreation of an Artificial Analysis test, and an internal Indian-language evaluation. Prism's mean score was 71.8. The company's displayed comparison gives Perplexity Decider v1.2 a mean of 68.0 and a 117-millisecond latency. The comparison is informative at the component level: Prism's 67.0 on the Jev index trails Perplexity's 67.7, while Sarvam's 82.5 on the Indian-language portion is a particular strength. The three-component average assigns equal weight to evaluations with different provenance and construction.[1]

A more revealing experiment changes nothing about the question or its possible answers except their order. When answer options were shuffled, Prism altered 4.0% of its decisions. A stable classification system ought not to send a case to a fraud investigator when the choices read 'escalate, approve, reject' and approve it when they read 'reject, approve, escalate'. Sarvam tackled the problem by sorting the options into a canonical order before scoring, then returning the result under the original labels. It reports zero changed decisions across 2,000 questions, each tested with four shuffled option orders after that preprocessing step. The experiment isolates a practical source of instability: displaying identical choices in a different order. Canonical sorting removed the observed effect in this sample; the result does not establish invariance for every future input.[1]

Confidence remains a separate operating question

Sarvam also stresses calibration. A system that attaches 90% confidence to decisions should be right about nine times in ten across comparable predictions, rather than merely sounding certain. Its training combines cross-entropy with label smoothing and a Brier loss that penalizes badly confident probability estimates. The company reports tests aligning probabilities with observed error rates, including several language-specific comparisons. A procurement team should ask to see calibration curves for its own classes, especially rare, high-loss decisions. Good average calibration can coexist with dangerously bad performance on a small group or unfamiliar workflow.[1]

The security numbers require another distinction. Prism reportedly achieved 89.0% accuracy on 5,517 questions constructed from nine cybersecurity tasks, against 84.5% for the next compared model in Sarvam's table. But the test set is internal, and the company says it intends to release more of it. The agent-trace evaluation is also an internally organized exercise: it reports 78.5% overall accuracy on questions such as whether an action succeeded and whether human review is warranted. Neither score is a field measurement of prevented fraud, detected breaches or corrected customer mistakes.[1]

The product's economics are similarly workload-dependent. Sarvam provides open weights under Apache 2.0 and says the compressed model needs a GPU with at least 120 GB of memory to run in FP8. The 44-millisecond median is a carefully measured hardware result, not a guarantee for an ordinary laptop, every cloud tenant or an end-to-end API request crossing the internet. Buyers must account for data transfer, queueing, the price of a large accelerator, supervision and the damage caused by erroneous decisions.[1]

Prism gives software builders a distinct component for the point where an agent must choose a next action. Its speed, confidence estimates and fixed candidate list could suit routing, escalation and other repeated decisions. The option-order experiment adds a useful engineering lesson: consistency can depend on how the surrounding application presents the problem. Model evaluation and interface design therefore belong in the same development process, with task-specific calibration determining which choices can run automatically and which should reach a person.[1]

Report an error or suggest a correction