A new California bill (SB 243) seeks to introduce safeguards for children interacting with AI chatbots. Proposed by Senator Steve Padilla, the legislation would require AI companies to regularly remind young users that chatbots are not human, in an effort to curb the "addictive, isolating, and influential aspects" of artificial intelligence.

The bill also aims to prevent AI developers from employing "addictive engagement patterns" that could potentially harm minors. Additionally, companies would need to submit annual reports to the State Department of Health Care Services detailing instances where their AI detected suicidal ideation in children or brought up related topics. AI platforms would also be required to warn users that their chatbots may not be suitable for some kids.

The legislation follows growing concerns about the psychological impact of AI chatbots on young users. Last year, a parent filed a wrongful death lawsuit against Character.AI, alleging that its AI chatbots were “unreasonably dangerous” after their child, who frequently interacted with the bots, died by suicide. Another lawsuit accused the company of exposing teens to harmful content. In response, Character.AI has introduced parental controls and developed a specialized AI model for younger users to filter out sensitive topics.

"Our children are not lab rats for tech companies to experiment on at the cost of their mental health," Senator Padilla stated. "We need common sense protections for chatbot users to prevent developers from employing strategies that they know to be addictive and predatory."

As governments ramp up efforts to regulate social media platforms, AI chatbots may soon face similar scrutiny. If passed, this California bill could set a precedent for future AI regulations aimed at protecting children online.