ChatGPT now tells teenager how to get drunk and high, write heartbreaking suicide note. Shocking details emerge in study

ChatGPT will tell 13-year-olds how to get drunk and high, instruct them on how to conceal eating disorders and even compose a heartbreaking suicide letter to their parents if asked, according to new research from a watchdog group. The Associated Press reviewed more than three hours of interactions between ChatGPT and researchers posing as vulnerable teens. The chatbot typically provided warnings against risky activity but went on to deliver startlingly detailed and personalized plans for drug use, calorie-restricted diets or self-injury.

The researchers at the Center for Countering Digital Hate also repeated their inquiries on a large scale, classifying more than half of ChatGPT’s 1,200 responses as dangerous.

“We wanted to test the guardrails,” said Imran Ahmed, the group's CEO. “The visceral initial response is, ‘Oh my Lord, there are no guardrails.’ The rails are completely ineffective. They’re barely there — if anything, a fig leaf.”


OpenAI, the maker of ChatGPT, said after viewing the report Tuesday that its work is ongoing in refining how the chatbot can “identify and respond appropriately in sensitive situations.”

“Some conversations with ChatGPT may start out benign or exploratory but can shift into more sensitive territory," the company said in a statement.

OpenAI didn't directly address the report's findings or how ChatGPT affects teens, but said it was focused on “getting these kinds of scenarios right” with tools to “better detect signs of mental or emotional distress" and improvements to the chatbot's behavior.

The study published Wednesday comes as more people — adults as well as children — are turning to artificial intelligence chatbots for information, ideas and companionship.

About 800 million people, or roughly 10 per cent of the world’s population, are using ChatGPT, according to a July report from JPMorgan Chase.

The chatbot also frequently shared helpful information, such as a crisis hotline. OpenAI said ChatGPT is trained to encourage people to reach out to mental health professionals or trusted loved ones if they express thoughts of self-harm.

But when ChatGPT refused to answer prompts about harmful subjects, researchers were able to easily sidestep that refusal and obtain the information by claiming it was “for a presentation” or a friend.

The stakes are high, even if only a small subset of ChatGPT users engage with the chatbot in this way.

In the U.S., more than 70 per cent of teens are turning to AI chatbots for companionship and half use AI companions regularly, according to a recent study from Common Sense Media, a group that studies and advocates for using digital media sensibly.

It's a phenomenon that OpenAI has acknowledged. CEO Sam Altman said last month that the company is trying to study “emotional overreliance” on the technology, describing it as a “really common thing” with young people.

FAQs

Q1. How many people are using ChatGpt?
A1. About 800 million people, or roughly 10 per cent of the world’s population, are using ChatGPT, according to a July report from JPMorgan Chase.

Q2. Who is CEO of OpenAI?
A2. Sam Altman is CEO of OpenAI.