Cloudflare traces 155 exposed AI responses to a broken memory handoff

Cloudflare has disclosed an AI infrastructure failure in which 155 responses contained data from other customers’ requests. The affected service was its GLM-5.3 model on Workers AI, between 27 August and 9 September. The company says no other hosted model was affected and that it found no evidence of deliberate exploitation. The count concerns responses, not 155 identified customers.[1]

The incident was reported by security researcher Gadi Evron of Knostic and remediated in September. Cloudflare and the SGLang maintainers published coordinated accounts on 26 October. Their explanation concerns a broken boundary between requests during model serving, rather than a chatbot retrieving information learned during training.[1][2]

The useful memory that must not cross a boundary

Generating text requires a model to attend to preceding tokens. A key-value, or KV, cache stores intermediate attention calculations so the system does not repeatedly recompute the entire history for every new token. Hugging Face’s technical documentation explains this as an inference-time optimisation. The cache is working memory for a request; it is not a new set of trained model weights.[3]

A helpful analogy is a reusable workbench. Reusing the bench saves money; letting the next job inherit the previous job’s unfinished paperwork creates a different problem. The security obligation is not simply to make the next calculation fast. It is to establish which request owns every piece of state used in that calculation.[3]

In the affected deployment, the configured context window exceeded the available prefill cache capacity. Cloudflare knew of that mismatch before launch but expected over-length requests to fail safely. Its tests checked successful responses without catching their incorrect content. A request too large to process instead progressed through the handoff with insufficient replacement state.[1]

Two stages, one mistaken assumption

SGLang’s account identifies the vulnerable path as disaggregated serving: one set of workers processes the prompt, and another generates the answer. Its single-node rejection path returned an error. In the distributed path, a rejected request was not properly terminated, allowing the receiving worker to proceed as though the required cache transfer had completed correctly.[2]

The maintainers also describe related cache-lifetime hazards, including asynchronous writes arriving after memory had been released for reuse. Their hardening work delays reuse until transfers have drained and adds checks to detect corrupted handoffs. Those are memory-ownership and concurrency fixes, not instructions telling the language model to behave more carefully.[2]

This distinction changes the engineering response. A content filter applied after generation is a poor substitute for preventing foreign request state from entering the calculation. Similarly, a successful HTTP response is not a correctness test: the service can remain available while violating the separation its customers depend on.

For operators, the relevant evidence is a combination of configuration checks, rejection-path tests and assertions about memory ownership across asynchronous work. SGLang’s disclosure supplies mechanisms and remediation guidance; it should not be flattened into a claim that every SGLang deployment suffered this exposure.[2]

The case exposes a less theatrical AI risk than an adversarial prompt, but a foundational one. A model may follow its instructions perfectly and still answer from the wrong customer’s state. In a shared inference service, trustworthy behaviour begins before the first output token—with the machinery deciding whose computation it is.

Report an error or suggest a correction