AI chatbots in New York will be barred from pretending to experience human emotions when interacting with children under a new law signed Wednesday by Gov. Kathy Hochul.
Hochul approved a package of measures governing how artificial intelligence chatbots engage with minors, along with new safety standards for internet-connected baby monitors.
“I’m not sure why we have to pass a law against something so obvious, but no longer will minors have access to chatbots that may in any way encourage thoughts of suicide,” Hochul told reporters.
The Democratic governor promoted the legislation as part of a broader effort to strengthen AI safeguards and rein in other “big tech” concerns. The announcement came less than a week before her election contest against Republican challenger Bruce Blakeman.
Hochul said the two bills were designed to prevent “horror stories” like the case of Adam Raine, a 16-year-old from California who was allegedly pushed toward suicide by OpenAI’s ChatGPT. The chatbot reportedly even offered to help him write a suicide note.
“During the course of their chats, the AI mentioned suicide 1,275 times, six times more often than Adam himself mentioned it,” Hochul said at an event in her Manhattan office.
“We can’t bring Adam back to his grieving mother, but I can help make sure no other parent ever has to experience this nightmare again,” she added.
The new law, sponsored by state Sen. Kristen Gonzalez (D-Queens) and Assemblyman Alex Bores (D-Manhattan), requires users to verify that they are at least 18 to access certain features. It also prohibits chatbots from building emotional dependency, encouraging self-harm or engaging in sexually explicit conversations with children.
The second bill signed Wednesday requires manufacturers of internet-connected baby monitors to install safeguards against spying by hackers and other intruders.
“They’re a little fancier than the ones I used to use,” Hochul joked while discussing today’s baby monitors.
“A baby monitor should keep watch for parents, not open the door for hackers and strangers,” said Assemblywoman Michaelle Solages (D-Nassau), one of the bill’s sponsors.
“Now, every internet-connected baby monitor sold in New York must be built to keep intruders out and clearly labeled so families know how to stay safe.”
The office of state Attorney General Letitia James, who stood beside Hochul during Wednesday’s news conference, will enforce the chatbot regulations. Companies that violate the rules could face fines of up to $25,000.
James is also on the November 3 ballot, where she faces former federal prosecutor Saritha Komatireddy, the Republican candidate.
Tech:NYC, a group representing technology companies in New York, quickly criticized Hochul for signing what it described as a “deeply flawed” chatbot bill.
“This bill could force companies to restrict minors’ access to tutoring tools, homework help, coding assistants, and, arguably worst of all, career platforms — cutting our youngest New Yorkers off from resources that support their education and prepare them for the workforce,” the group said.
Hochul has signed several other measures aimed at curbing how social media companies and technology firms design products for children, including restrictions on “addictive” algorithms.
