A new Stanford University study warns that therapy chatbots powered by large language models (LLMs) may pose serious user risks, including reinforcing harmful stigmas and offering unsafe responses. Presented at the upcoming ACM Conference on Fairness, Accountability, and Transparency, the study analysed five popular AI chatbots marketed for therapeutic support, evaluating them against core guidelines for assessing human therapists.
The research team conducted two experiments, one to detect bias and stigma, and another to assess how chatbots respond to real-world mental health issues. Findings revealed that bots were more likely to stigmatise people with conditions like schizophrenia and alcohol dependence compared to those with depression.
Shockingly, newer and larger AI models showed no improvement in reducing this bias. In more serious cases, such as suicidal ideation or delusional thinking, some bots failed to react appropriately or even encouraged unsafe behaviour.
Despite these dangers, the team doesn’t entirely dismiss the use of AI in therapy. If used thoughtfully, they suggest that LLMs could still be valuable tools for non-clinical tasks like journaling support, billing, or therapist training. As Haber put it, ‘LLMs potentially have a compelling future in therapy, but we need to think critically about precisely what this role should be.’
Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!
