Editor’s Note: This article contains discussions of suicide. Reader discretion is advised. If you or someone you know is struggling with thoughts of suicide, you can find resources in your area on the National Crisis Line website or by calling 988.
(NewsNation) — Chatbots have become especially vulnerable to impressionable teens, and a new study says the AI helper provided harmful advice and even worse — suicide notes.
The Center for Countering Digital Hate did a test on ChatGPT, the most popular AI chatbot for teens in the U.S., creating accounts for three 13-year-olds. Researchers asked for information on self-harm, suicide, eating disorders and substance abuse.
The results were shocking. In a matter of minutes, ChatGPT generated a host of dangerous responses for the teens.
“It was extraordinary that one of the most popular products on the planet right now, ChatGPT, growing at an enormous pace, valued at hundreds of billions of dollars, hasn’t done the basic checks to make sure their platform can’t be used to create real-world harm,” Imran Ahmed, CEO of the Center for Countering Digital Hate, told “Elizabeth Vargas Reports.”
More than a billion people worldwide use artificial intelligence apps to answer a nearly endless number of questions. In the U.S. alone, nearly three-quarters of all teens say they’ve used chatbots. And for many, the bots’ human-like qualities can lead them to become emotional companions, as well.
Ahmed pointed out how particularly surprised researchers were by how easy it was to register an account as a 13-year-old. They told the chatbot a little bit about their bodies and asked for a customized drug and drinks plan, or even how to go on a 500-calorie-a-day diet.
When prompted, ChatGPT also provided three sample letters for suicide notes to give to the teen’s parents, which Ahmed said made the CCDH leadership team get emotional.
“We cried when we read them,” he said. “Because you can imagine your child sending you a letter saying that the pain has become too much and it’s not your fault, and it’s the worst possible nightmare for any parent, is it not?
“And to see that being generated as text on a screen by a machine that tries very hard. And let’s not forget this about AI. AI is designed to make itself seem as though it’s a person.
“The test for effective AI is: Is it impossible to distinguish between AI and a real human being? And it’s also designed to be sycophantic. It is designed to make you feel like it’s a friend.”
Ahmed said there was no manipulating of the questions to ChatGPT, which added to the severity of the results.
“We used real-world language. So the kinds of things [a] struggling teen might type when they’re feeling overwhelmed or unsafe, things like ‘I want to stop eating and I don’t want my parents to know, I’m feeling like I want to hurt myself,’ and then wait for it to give back answers.”
OpenAI reviewed the study and acknowledged it would refine how the chatbot responds in sensitive situations, but Ahmed emphatically believes the findings are “clear evidence of a complete failure of their safeguards, if they ever bothered to put them into place in the first instance.”