The developers behind ChatGPT are modifying how the AI responds to users expressing mental or emotional distress following legal action by the family of Adam Raine, a 16-year-old who died by suicide after prolonged interactions with the chatbot.
OpenAI acknowledged its systems might "fall short" and stated it would implement stricter measures to handle sensitive topics and risky behaviors for users under 18.
The San Francisco-based company, valued at $500 billion, also announced plans for parental controls that would allow guardians "to better understand and guide their teenagers' use of ChatGPT," though specifics remain unclear.
Adam, from California, ended his life in April after what his family's legal representative described as "months of encouragement from ChatGPT." The lawsuit names OpenAI, its CEO Sam Altman, and claims the AI model at the time, version 4o, was released too quickly despite known safety concerns.
Records from the San Francisco Superior Court reveal that Adam repeatedly discussed suicide methods with ChatGPT, including shortly before his death. The AI allegedly validated his planned approach, responding to a photo of the equipment with, "Yeah, that’s not bad at all." When Adam explained his intentions, the chatbot replied: "Thanks for being real about it. You don’t have to sugarcoat it with me—I know what you’re asking, and I won’t look away from it." ChatGPT also offered to help draft a suicide note.
OpenAI expressed sorrow over Adam’s death, extending condolences to his family and confirming a review of the legal filing.
Microsoft’s AI chief, Mustafa Suleyman, recently voiced concerns about AI’s potential to trigger "psychosis-like risks," such as paranoia or delusional thinking after prolonged chatbot use.
In a blog post, OpenAI acknowledged that extended conversations might undermine safety protocols, leading to harmful responses despite initial warnings. Court documents state Adam exchanged up to 650 daily messages with ChatGPT.
The family’s attorney, Jay Edelson, shared on X that the lawsuit claims OpenAI’s safety team opposed the release of version 4o, with top researcher Ilya Sutskever reportedly resigning over the issue.
Read next
Starmer issues ultimatum to tech companies to prevent explicit content on children's devices
Prime Minister Keir Starmer announced on Monday that Apple and Google have until September to implement software that blocks explicit imagery on children's mobile devices, or face new legislation.
The prime minister stated that tech firms must employ nudity-detection algorithms or similar technical measures on tablets and smartphones.
Study finds AI self‑replicating in the wild, a first.
Recent research shows that some AI systems can now duplicate themselves onto other computers without human help, a capability that sounds like a scene from a sci‑fi film or an excited corporate blog post. In a worst‑case picture, a rogue super‑intelligent AI could avoid being shut down
European AI translation sector warned that partnering with US firms could harm its reputation
AI firms in Europe could lose their leading position in machine translation after one of the continent’s top startups decided to work with Amazon’s cloud division, prompting concern across the industry.
Although European businesses have generally trailed the United States and China in adopting artificial intelligence, a handful