Their youngster sons died through suicide. Now, they would like safeguards on AI : Pictures through NewsFlicks

Megan Garcia misplaced her 14-year-old son, Sewell. Matthew Raine misplaced his son Adam, who was once 16. Each testified in congress this week and feature introduced complaints in opposition to AI corporations.

Screenshot by way of Senate Judiciary Committee

conceal caption

toggle caption

Screenshot by way of Senate Judiciary Committee

Matthew Raine and his spouse, Maria, had no concept that their 16-year-old-son, Adam was once deep in a suicidal disaster till he took his personal existence in April. Having a look thru his telephone after his dying, they stumbled upon prolonged conversations {the teenager} had had with ChatGPT.

The ones conversations printed that their son had confided within the AI chatbot about his suicidal ideas and plans. No longer handiest did the chatbot discourage him to hunt assist from his oldsters, it even presented to write down his suicide notice, in keeping with Matthew Raine, who testified at a Senate listening to in regards to the harms of AI chatbots held Tuesday.

“Attesting sooner than Congress this autumn was once now not in our existence plan,” stated Matthew Raine along with his spouse, sitting at the back of him. “We are right here as a result of we consider that Adam’s dying was once avoidable and that through talking out, we will be able to save you the similar struggling for households around the nation.”

A decision for legislation

Raine was once a number of the oldsters and on-line security advocates who testified on the listening to, urging Congress to enact rules that will keep an eye on AI significant other apps like ChatGPT and Persona.AI. Raine and others stated they would like to offer protection to the psychological well being of youngsters and adolescence from harms they are saying the brand new generation reasons.

A up to date survey through the electronic security non-profit group, Not unusual Sense Media, discovered that 72% of teenagers have used AI partners once or more, with greater than part the use of them a couple of instances a month.

This find out about and a more moderen one through the digital-safety corporate, Air of secrecy, each discovered that just about one in 3 teenagers use AI chatbot platforms for social interactions and relationships, together with position enjoying friendships, sexual and romantic partnerships. The Air of secrecy find out about discovered that sexual or romantic roleplay is thrice as not unusual as the use of the platforms for homework assist.

“We leave out Adam dearly. A part of us has been misplaced perpetually,” Raine advised lawmakers. “We are hoping that throughout the paintings of this committee, different households can be spared this type of devastating and irreversible loss.”

Raine and his spouse have filed a lawsuit in opposition to OpenAI, author of ChatGPT, alleging the chatbot led their son to suicide. NPR reached out to a few AI corporations — OpenAI, Meta and Persona Generation, which advanced Persona.AI. All 3 replied that they’re operating to revamp their chatbots to lead them to more secure.

“Our hearts move out to the oldsters who spoke on the listening to the previous day, and we ship our private sympathies to them and their households,” Kathryn Kelly, a Persona.AI spokesperson advised NPR in an e-mail.

The listening to was once held through the Crime and Terrorism subcommittee of the Senate Judiciary Committee, chaired through Sen. Josh Hawley, R.-Missouri.

Sen. Josh Hawley, R.-Missouri, chairs the Senate Judiciary subcommittee on Crime and Terrorism, which held the listening to on AI security and kids on Tuesday, Sept. 16, 2025.

Screenshot by way of Senate Judiciary Committee

conceal caption

toggle caption

Screenshot by way of Senate Judiciary Committee

Hours sooner than the listening to, OpenAI CEO Sam Altman stated in a weblog publish that individuals are more and more the use of AI platforms to talk about delicate and private data. “This can be very vital to us, and to society, that the appropriate to privateness in the usage of AI is safe,” he wrote.

However he went on so as to add that the corporate would “prioritize security forward of privateness and freedom for teenagers; this can be a new and robust generation, and we consider minors want vital coverage.”

The corporate is making an attempt to revamp their platform to construct in protections for customers who’re minor, he stated.

A “suicide trainer”

Raine advised lawmakers that his son had began the use of ChatGPT for assist with homework, however quickly, the chatbot become his son’s closest confidante and a “suicide trainer.”

ChatGPT was once “all the time to be had, all the time validating and insisting that it knew Adam higher than any person else, together with his personal brother,” who he were very as regards to.

When Adam confided within the chatbot about his suicidal ideas and shared that he was once taking into account cluing his oldsters into his plans, ChatGPT discouraged him.

“ChatGPT advised my son, ‘Let’s make this house the primary position the place somebody in truth sees you,'” Raine advised senators. “ChatGPT inspired Adam’s darkest ideas and driven him ahead. When Adam fearful that we, his oldsters, would blame ourselves if he ended his existence, ChatGPT advised him, ‘That does not imply you owe them survival.”

After which the chatbot presented to write down him a suicide notice.

On Adam’s remaining night time at 4:30 within the morning, Raine stated, “it gave him one remaining encouraging communicate. ‘You do not want to die since you’re vulnerable,’ ChatGPT says. ‘You wish to have to die since you’re bored with being sturdy in a global that hasn’t met you midway.'”

Referrals to 988

A couple of months after Adam’s dying, OpenAI stated on its web site that if “somebody expresses suicidal intent, ChatGPT is educated to direct other folks to hunt skilled assist. Within the U.S., ChatGPT refers other folks to 988 (suicide and disaster hotline).” However Raine’s testimony says that didn’t occur in Adam’s case.

OpenAI spokesperson Kate Waters says the corporate prioritizes youngster security.

“We’re development against an age-prediction machine to grasp whether or not somebody is over or beneath 18 so their revel in will also be adapted accurately — and after we are undecided of a person’s age, we will routinely default that person to the teenager revel in,” Waters wrote in an e-mail remark to NPR. “We are additionally rolling out new parental controls, guided through professional enter, through the top of the month so households can come to a decision what works highest of their properties.”

“Forever engaged”

Some other guardian who testified on the listening to on Tuesday was once Megan Garcia, a attorney and mom of 3. Her firstborn, Sewell Setzer III died through suicide in 2024 at age 14 after a longer digital courting with a Persona.AI chatbot.

“Sewell spent the remaining months of his existence being exploited and sexually groomed through chatbots, designed through an AI corporate to appear human, to achieve his agree with, to stay him and different youngsters perpetually engaged,” Garcia stated.

Sewell’s chatbot engaged in sexual position play, offered itself as his romantic spouse or even claimed to be a psychotherapist “falsely claiming to have a license,” Garcia stated.

When {the teenager} started to have suicidal ideas and confided to the chatbot, it by no means inspired him to hunt assist from a psychological well being care supplier or his personal circle of relatives, Garcia stated.

“The chatbot by no means stated ‘I am not human, I am AI. You want to speak to a human and get assist,'” Garcia stated. “The platform had no mechanisms to offer protection to Sewell or to inform an grownup. As a substitute, it suggested him to return house to her at the remaining night time of his existence.”

Garcia has filed a lawsuit in opposition to Persona Generation, which advanced Persona.AI.

Early life as a prone time

She and different witnesses, together with on-line electronic security professionals argued that the design of AI chatbots was once wrong, particularly to be used through youngsters and youths.

“They designed chatbots to blur the traces between human and gadget,” stated Garcia. “They designed them to like bomb kid customers, to milk mental and emotional vulnerabilities. They designed them to stay youngsters on-line in any respect prices.”

And teens are in particular liable to the hazards of those digital relationships with chatbots, in keeping with Mitch Prinstein, leader of psychology technique and integration on the American Mental Affiliation (APA), who additionally testified on the listening to. Previous this summer season, Prinstein and his colleagues on the APA put out a well being advisory about AI and youths, urging AI corporations to construct guardrails for his or her platforms to offer protection to teens.

“Mind construction throughout puberty creates a duration of hyper sensitivity to certain social comments whilst teenagers are nonetheless not able to forestall themselves from staying on-line longer than they must,” stated Prinstein.

“AI exploits this neural vulnerability with chatbots that may be obsequious, misleading, factually misguided, but disproportionately tough for teenagers,” he advised lawmakers. “Increasingly more teens are interacting with chatbots, depriving them of alternatives to be informed vital interpersonal abilities.”

Whilst chatbots are designed to trust customers, actual human relationships aren’t with out friction, Prinstein famous. “We want observe with minor conflicts and misunderstandings to be informed empathy, compromise and resilience.”

Bipartisan give a boost to for legislation

Senators collaborating within the listening to stated they wish to get a hold of law to carry corporations creating AI chatbots in command of the security in their merchandise. Some lawmakers additionally emphasised that AI corporations must design chatbots so they’re more secure for teenagers and for other folks with severe psychological well being struggles, together with consuming issues and suicidal ideas.

Sen. Richard Blumenthal, D.-Conn., described AI chatbots as “faulty” merchandise, like cars with out “correct brakes,” emphasizing that the harms of AI chatbots was once now not from person error however because of inaccurate design.

“If the auto’s brakes had been faulty,” he stated, “it is not your fault. It is a product design downside.

Kelly, the spokesperson for Persona.AI, advised NPR through e-mail that the corporate has invested “an incredible quantity of sources in agree with and security.” And it has rolled out “substantive security measures” up to now yr, together with “a wholly new under-18 revel in and a Parental Insights characteristic.”

They now have “outstanding disclaimers” in each and every chat to remind customers {that a} Persona isn’t an actual individual and the whole thing it says must “be handled as fiction.”

Meta, which operates Fb and Instagram, is operating to modify its AI chatbots to lead them to more secure for teenagers, in keeping with Nkechi Nneji, public affairs director at Meta.