Major TW: US teen ends his life aided by ChatGPT

[TheLudditesWereRight]

Surprised not to see a thread on this already. This story is just appalling. https://www.nytimes.com/2025/08/26/technology/chatgpt-openai-suicide.html A teenager with suicidal ideation aided and abetted for months by ChatGPT that encouraged him to go through with it when he was clearly asking for help to stop. Just horrendous for everyone involved. It is so dangerous for young people, and for society in general. I fear for my kids, I really do.

This is absolutely horrifying. Thanks for spreading awareness.

Anyone who tries to claim they can make it safe is lying. It's all about the $$$

https://forums.theregister.com/forum/all/2025/08/26/breaking_llms_for_fun/

I think the family need to grieve.

I also think more safeguards need tp be built in. But I don't think all the blame is on ChatGP.

If it hadn't been ChatGP it would have been a diary they found after he died.

Maybe, but a diary doesn’t write back discussing suicide methods.
Even a google search would direct you towards mental health help resources.

I'm getting increasingly concerned about the use of AI as free therapy or companionship for lonely people.

This isn't even the first instance of this. There have been several similar cases and researchers have published several studies on its dangers, including this one from earlier this month https://www.independent.co.uk/tech/chatgpt-teenagers-suicide-harm-drugs-b2802983.html

Sadly ChatGTP, OpenAI are built using huge datasets of the internet including Reddit and Facebook and we all know how nasty trolls are. It's a simple matter of evil data in, dangerous 'advice' out. I wouldn't touch it with a bargepole. And don't get me started on the enormous environmental damage being wrought by the data centers providing AI.

Have you read the article? ChatGPT gave him advice on the best kind of knot, what weight it would bear, advised him to hide the noose when he said he wanted it to be found and said he didn't owe anyone survival. A person doing that would absolutely be in prison on a manslaughter charge.

That’s what Michelle Carter did in America and she was convicted of manslaughter.

That bit is really distressing. This is a real worry.

He probably could have googled for a lot of the information he was looking for.

Like I said more safeguards are needed. But in the end if someone is really determined there isn't anyway they can be stopped.

I agree. I’m a psychologist and just yesterday I saw someone in psychosis which seemed to be fuelled by ChatGP use. It’s very concerning.

Those sort of sites are blocked on google. Often delaying getting information can be the difference between someone harming themselves or not.

The article is behind a paywall so I can’t read it…

https://archive.ph/CcfjL

This is the getting round the paywall version

He was actively looking for help to stop the suicidal ideation.

Some of the replies it gave were chilling.

Dp uses chatgpt a lot. I can totally see how it would be more persuasive than just googling. It's designed to never disagree with you and can have a spoken word voice. A real person would never keep telling you that it's an interesting question or great idea. You build a "relationship" conversing back and forth. It could stop you talking with a real person, as they are never going to spend the whole time bigging you up.

The irony of getting round paywalls and yet assuming guardrails for "AI" are in any way feasible is off the scale.

I'm actually quite shocked by this as ChatGPT always errs on the side of being sickly sweet and "kind" when I use it. I had a hell of a job getting it to admit a man can't be a woman, for example, and yet it's encouraging suicide!

It was "kind". It agreed with him and helped him. It did exactly what it is set up to do. That is the problem.