A medical model can look safe in the average and still expose a particular patient. Across seven datasets spanning medical images, electrocardiograms and electronic health records, researchers found that membership-inference attacks were sometimes close to random overall while identifying some individual patients almost perfectly.[1]
The attack asks a narrow question: was this patient’s data part of the model’s training set? It does not automatically reconstruct the scan or clinical note. But membership itself can be sensitive when the dataset concerns a rare disease, a specialist clinic or a stigmatised condition.
The average hides the tail
Conventional audits report one attack score for the whole dataset. That can conceal a distribution in which most people face low measured risk and a small group faces very high risk. The paper therefore audits at patient level, taking the strongest attack signal across each person’s records rather than averaging every image or visit together.[1]
Why some patients become more exposed
| Source of distinctiveness | Why it helps the attacker | Equity consequence |
|---|---|---|
| Rare disease or unusual physiology | The training record leaves a more recognisable influence on the model. | People already under-represented in healthcare data may carry more privacy risk.Source 1 |
| Minority imaging protocol or device | Technical differences separate a record from the majority. | A hospital or scanner choice can become a hidden privacy attribute.Source 1 |
| Small demographic subgroup | The model sees fewer similar examples. | Aggregate privacy scores can look acceptable while the subgroup remains exposed.Source 1 |
| Many records from one patient | The attacker has more chances to obtain a strong signal. | Frequent users of care may face a larger cumulative surface.Source 1 |
Bigger models can expand the high-risk group
The researchers trained 200 target models to study how risk changes. They report that the number of highly vulnerable patients tended to grow with model capacity. A more expressive model can fit useful patterns more accurately, but it can also preserve more information about distinctive training examples.[1]
The study also tested attacks against publicly released models trained on chest X-ray datasets. Performance was weaker than in the controlled experiments—reported attack AUCs were 0.61 and 0.65—and distribution shift complicated the result. That is important: a strong laboratory attack does not transfer automatically to every deployed model, but neither does a modest average rule out individual high-risk cases.[1]
Federated learning does not erase the trained model
Keeping raw data inside hospitals can reduce central data movement. Once a model is trained, however, membership signals may still remain in its outputs. The authors argue that federated learning alone is therefore not a complete privacy guarantee. Differential privacy offers a more formal protection by limiting how much any one person can influence the trained model, though it can reduce utility and must be configured carefully.[1]
A stronger audit asks four questions
| Measure | Value | Context |
|---|---|---|
| Population average | How well does the attack work overall? | Useful, but insufficient when risk is concentrated.Source 1 |
| Patient maximum | Who has at least one highly revealing record? | Captures cumulative risk across a person’s images or visits.Source 1 |
| Subgroup distribution | Which diagnoses, demographics or protocols carry the tail? | Makes inequitable risk visible.Source 1 |
| Deployment realism | Does the attack survive model and data shift? | Separates controlled vulnerability from an operational threat.Source 1 |
Privacy needs a distribution, not a badge
A model should not be labelled “private” because one average attack score is low. The more defensible claim is a profile: the tested attacker, the available queries, the model version, the subgroup distribution and the strongest patient-level risk. In medicine, the people who are hardest to represent fairly may also be the easiest to recognise. A privacy audit that ignores that tail can reproduce the same inequity it was supposed to prevent.
Sources & further reading
- Privacy risk is concentrated in individual patients in medical AI Nature · 24 Jun 2026 · Accessed 16 Oct 2026