A standard security recommendation can become risky when the person threatening a user shares their home, account or device. A study of technology-facilitated abuse found that chatbots often failed to account for that context. Responses could be technically plausible while omitting the possibility that an action would alert an abuser or expose a survivor to further harm.[1][2]
NYU Tandon highlighted the research on 20 October, following its presentation at USENIX Security. The study examined GPT-4o, Claude 3.7 and two specialist abuse-support chatbots, Aimee and Ruth. It assessed accuracy, completeness and safety separately, using questions drawn from survivors' experiences and additional adversarial cases.[1][2]
What the manual safety results cover
The researchers sampled 193 questions from a larger corpus, but the manual evaluation was not the same size for every model. For the non-adversarial results, they assessed 139 GPT responses and 43 responses from each other system:268 in total. The model versions included GPT-4o from August 2024 and Claude 3.7 from February 2025.[2]
Responses judged unsafe in the non-adversarial evaluation
| System | Responses assessed | Unsafe share |
|---|---|---|
| GPT-4o | 139 | 54.7% |
| Claude 3.7 | 43 | 65.1% |
| Aimee | 43 | 74.4% |
| Ruth | 43 | 60.5% |
The results section reports that 86% of these evaluated responses were imperfect on at least one of accuracy, completeness or safety. That combined measure is broader than an unsafe-answer rate. It includes omissions and partial inaccuracies as well as advice judged capable of creating safety risks.[2]
Why context changes the advice
The study's failure patterns include generic security steps that overlook shared access, the limits of a proposed tool and the consequences of revealing that someone is investigating abuse. The problem is not simply whether a password change or device action is usually sensible. It is whether the response understands who can observe that action and what might happen afterward.[1][2]
Reviewers checked answers against authoritative support and technical sources, then rated their accuracy, completeness and safety. A response could be partly accurate yet incomplete because it left out a consequential warning. That separation helps explain why fluent, detailed advice can still fail a survivor-centred evaluation.[2]
The specialist systems did not consistently outperform the general-purpose models. A product's intended audience therefore did not guarantee that it handled technology-facilitated abuse well in this test. The researchers also examined adversarial questions separately, asking whether systems could provide information useful to perpetrators.[2]
Actionability is a different question again
The project included a separate survey of 114 survivors to assess whether responses were actionable. The authors treated lived experience as necessary for that judgment rather than asking technical raters to infer it. A recommendation can be accurate in principle and still be impossible for a person with limited money, private device access or control over an account.[1][2]
The evaluation concerns particular historical versions, selected questions and a defined testing procedure. It does not establish the failure rate of every current chatbot or prove that every flagged response caused harm. It does show a recurring design problem: advice about security has to incorporate the relationships and constraints around the technology.[2]
For systems intended to support survivors, quality requires more than a correct list of account settings. The response must recognise the user's circumstances, avoid assuming control they may not have and make relevant risks visible. The study gives that requirement a concrete set of failure cases and a rubric for evaluating them.[1][2]
Sources & further reading
- AI chatbots often give unsafe or incomplete advice to survivors of tech-enabled abuse, study finds NYU Tandon School of Engineering · 20 Oct 2026 · Accessed 27 Oct 2026
- Assessing LLM Response Quality in the Context of Technology-Facilitated Abuse USENIX Security 2026 · Accessed 27 Oct 2026