Parents sue Google over AI safety risks on school Chromebooks
Emma, a seventh grader in North Carolina, had grown convinced that she had multiple mental health disorders following interactions with AI Mode The lawsuit is likely the first to claim that Google "hastily integrated" AI features into company apps used on K-12 Chromebooks, Google-powered laptops provided to millions of children by U.S. school districts, while failing to notify their parents of the potential risks. Photo: Virginia Department of Education , CC BY 2.0 , via Wikimedia Commons After chatting for months with Google's artificial intelligence on a school-issued Chromebook laptop, a 13-year-old student made plans to end her life, according to a lawsuit filed in California Superior Court on Monday, the latest in a wave of litigation over Big Tech's effects on young people. The lawsuit, which refers to the girl as Emma, said that her parents first learned she had access to Google's AI in mid-March after their daughter's school sent a message warning them that she was in "imminent danger" of self-harm. Emma's mother logged into the teen's school Google account and found her daughter composing a suicide note on Google Docs, the complaint said. Her parents, named as John and Jane Doe in the lawsuit, immediately took Emma for an emergency psychiatric evaluation, and she now goes to therapy every week. Emma, a seventh grader in Charlotte, North Carolina, had grown convinced that she had multiple mental health disorders following interactions with AI Mode, a chatbotlike feature on Google Search, the complaint said. She also searched for "what is a suicide note" and made comments like "I am planning my own funeral" to the AI, the complaint said. The lawsuit is likely the first to claim that Google "hastily integrated" AI features into company apps used on K-12 Chromebooks, Google-powered laptops provided to millions of children by U.S. school districts, while failing to notify their parents of the potential risks. The claims add to mounting public concern that Silicon Valley has hooked young people on social media apps, video game platforms and AI chatbots, putting some children and teenagers in harm's way. José Castañeda, a Google spokesperson, said that the company had worked extensively with health experts to build features to help young people find emergency resources and that it was reviewing the claims in the lawsuit. "For years, we've shown hotline information directly in search when someone may be experiencing a mental health crisis," he said in a statement, "and we've built those same protections into search AI features, connecting people with real-world support." The case underscores how public school districts have come to depend on classroom laptops, tablets and apps from companies like Google and Microsoft for teaching and learning. In recent years, some Big Tech companies have used their own infrastructure in schools to promote their AI chatbots for teachers and students. The new lawsuit argues that Google's "sprawling digital ecosystem" of laptops and apps has enabled the tech giant to spread its Gemini AI chatbot across public schools -- as well as turn on some students' access to AI features by default -- without parents' awareness or permission. While school administrators can toggle company apps like YouTube and Gemini on or off for students in different grades, school administrators have complained that Google's system has not allowed them to disable AI Mode and AI Overview, features that are embedded in Google Search. Californians are so far in favor of increasing investment in health care research but against funding restrictions on community health care clinics, according to preli…
Open article →