OpenAI says it disrupted a July 2026 campaign that attempted to extract protected model reasoning by copying encrypted reasoning into other conversations and asking the model to reproduce it. In its 30 September disclosure, the company says the operators did not break its encryption, compromise a database or directly access stored user conversations. The reported request counts measure attempted extraction, not a published total of successful recoveries.[1]
That distinction matters because “model theft” can sound like one technical event. Adversarial distillation is closer to industrial harvesting: many permitted-looking requests, distributed across accounts, are organised into a training pipeline intended to reproduce capabilities that the provider does not expose as a downloadable model.
Two disclosed campaigns show the scale and the limits of attribution
| Measure | Value | Context |
|---|---|---|
| OpenAI campaign | More than 15,000 users in related activity | OpenAI attributes a core cluster to individuals associated with Moonshot AI, but says it is unclear whether all operators belonged to one actor. It reports disrupting the observed activity by 28 July.Source 1 |
| OpenAI start point | 1 July 2026 | The disclosure concerns attempts to extract protected reasoning, not direct access to user data.Source 1 |
| Anthropic total | More than 16 million exchanges | Anthropic attributes campaigns to DeepSeek, Moonshot and MiniMax through metadata and infrastructure indicators.Source 2 |
| Anthropic accounts | Approximately 24,000 fraudulent accounts | Traffic was distributed across direct and third-party access paths.Source 2 |
| Largest Anthropic-attributed campaign | More than 13 million exchanges | Attributed to MiniMax and focused on agentic coding and tool orchestration.Source 2 |
| Proxy scale | More than 20,000 accounts in one network | Anthropic describes hydra-style replacement when accounts were removed.Source 2 |
| Success status | Attempts are not proof of successful capability transfer | OpenAI explicitly cautions that its counts describe attempted extraction.Source 1 |
The attack surface is the output contract
An API is intentionally designed to reveal useful behaviour. A customer can ask for code, analysis, tool plans and corrections. A distiller asks similar questions at a frequency, structure and breadth chosen to create a training set. No individual prompt necessarily proves abuse; coordination across accounts and concentration on high-value capabilities provide the signal.
Protected reasoning creates an additional boundary. A final answer may be licensed for use while the hidden process remains unavailable. If prompts can repeatedly reconstruct that process, the attacker receives a richer imitation target than ordinary outputs provide. Defences therefore need to examine interaction patterns, tool returns and partner-hosted routes—not only visible text.
Controls for capability-extraction campaigns
| Layer | What it can detect or prevent | Trade-off or residual risk |
|---|---|---|
| Account integrity | Fraudulent organisations, payments and identity reuse | Stronger checks can burden legitimate researchers and startupsSource 2 |
| Behavioural fingerprinting | Repetition, synchronisation and narrow capability harvesting across accounts | Attackers can vary prompts and mix traffic with ordinary usersSource 2Source 1 |
| Output boundary | Requests designed to elicit protected reasoning or internal traces | Overblocking can reduce legitimate debugging and educational valueSource 1 |
| Partner parity | Equivalent controls on cloud and reseller access | A weaker downstream route becomes the extraction pathSource 1Source 2 |
| Watermark or transformation | Reduce the usefulness of outputs for unauthorised training | May damage answer utility or fail against adaptive trainingSource 3 |
| Intelligence sharing | Join indicators seen by providers, clouds and authorities | Attribution and data-sharing rules must be reliable and lawfulSource 2 |
Legitimate distillation and adversarial distillation use the same mathematics
Providers routinely distil their own models to create smaller and cheaper systems. Researchers also use teacher outputs under licences and declared terms. The dispute is therefore not whether learning from another model is technically valid. It concerns permission, scale, circumvention, protected content and whether the resulting system preserves the safeguards attached to the source service.
This makes policy difficult. A prohibition on every form of output-based learning would collide with ordinary research and product development. A workable boundary needs concrete indicators: fraudulent access, evasion of regional or account restrictions, coordinated extraction of protected traces, and direct mapping from harvested traffic to a competing training programme.
Defence can change the product itself
OpenAI describes layered changes across model behaviour, tools, classifiers and partner deployments. Anthropic has introduced preserved-thinking controls in newer models. Research on distillation resistance explores transformations intended to remove information useful to the student while preserving answer quality, but the central trade-off remains: an answer that teaches a legitimate user may also teach another model.[1][2][3]
Adversarial distillation turns ordinary AI usage into a supply-chain question. The provider controls an interface, cloud partners extend it, resellers can obscure it, and model outputs can become training material elsewhere. Trustworthy access will depend on a record that distinguishes normal high-volume work from coordinated capability extraction without treating every ambitious user as an attacker.
Sources & further reading
- Disrupting a coordinated model distillation campaign OpenAI · 30 Sep 2026 · Accessed 11 Oct 2026
- Detecting and preventing distillation attacks Anthropic · 23 Feb 2026 · Accessed 11 Oct 2026
- Towards Distillation-Resistant Large Language Models: An Information-Theoretic Perspective Hao Fang and colleagues / arXiv · 3 Feb 2026 · Accessed 11 Oct 2026