By Hamza Faisal

Cloudflare has disclosed that a cache-sizing flaw and an inference-engine bug combined to leak fragments of other customers' prompts into 155 responses from an artificial intelligence model it hosts, in an incident that ran for thirteen days before it was caught. The company published an incident report on October 26, alongside a companion post from the maintainers of the open-source SGLang inference engine.

The affected model was GLM-5.3, the agentic coding model from Chinese lab Z.ai, served on Cloudflare's Workers AI platform as @cf/zai-org/glm-5.3. It launched on the platform on August 27 with an advertised context window of 1,048,576 tokens, and every affected response went to a prompt longer than 681,280 tokens — the threshold at the heart of the incident.

A cache smaller than the context window

The first problem was a configuration gap. When a language model reads a prompt, it stores the prompt's attention state in a memory structure called a KV cache. Cloudflare split GLM-5.3 across two GPU groups: one to read the prompt (prefill) and another to write the answer (decode). The prefill KV pool was sized for 681,280 tokens, well short of the 1,048,576-token context window the model advertised.

Any prompt longer than the pool could not fit. Cloudflare's pre-launch testing checked status codes and streaming, but not whether very-long-prompt responses were actually coherent — so the failure mode went unnoticed for nearly two weeks.

How a rejected request kept running

The second problem was a bug in SGLang, the open-source engine Cloudflare uses to run the model, in its disaggregated prefill and decode mode. When a prompt exceeded the pool, SGLang marked it as rejected and swapped it for a one-token placeholder. But instead of aborting the request with an error, the system carried on and passed that placeholder's cache to the decode side, signalling that the transfer was complete.

The decode side had already reserved pages for the full prompt, so it started answering while reading pages that still held other customers' prompts and partial generations from shortly before. The customer got an HTTP 200 response with a matching token count and coherent-looking text: the failure looked exactly like success.

Security researcher Gadi Evron of Knostic reported the tenant-isolation issue to Cloudflare on September 8 through the company's HackerOne programme. Cloudflare reproduced it, took the model offline within 40 minutes of declaring an incident on September 9, fixed it, and restored the model at 19:40 UTC the same day. Further hardening rolled out to its SGLang deployments between September 11 and 19.

What Cloudflare found, and what it means for hosted AI

Cloudflare said a small number of customers received responses containing data from other customers' requests, and that it found no evidence of deliberate exploitation. Excluding its own testing and the researcher's requests, the affected traffic matched ordinary long-context use, not repeated exploitation. The company also confirmed that no other model hosted on Workers AI was affected, because every other disaggregated deployment has a KV pool larger than its context length on both sides.

For anyone running or renting hosted inference, the incident carries a blunt lesson. Disaggregated serving, pools sized to expected traffic and multi-tenant GPU sharing are the standard economics of inference in 2026 — and this incident shows the tenant-isolation boundary in that stack is enforced by configuration and engine behaviour, not by design. Cloudflare's guidance to operators is concrete: measure KV pool size against context length on every node after any configuration change, run SGLang version 0.5.22 or later, enable KV checksums and canaries where supported, and configure load balancers to avoid aborting transfers.