AI EXPERTS and child safety researchers have warned parents to be wary of AI-powered toys this Christmas, after US media including CNN Business last week reported the suspension of FoloToy’s $99 “Kumma” teddy bear after it discussed explicit sexual content and how to access knives with testers at the US PIRG Education Fund.
While conventional physical toys are required to undergo rigorous safety testing for choking hazards and toxic materials, the experts cautioned that AI toys face far less scrutiny in terms of conversational safety and age-appropriate content — something experts say is especially worrying given the level of money entering the AI sector.
Colette Mason, AI Consultant at London-based Clever Clogs AI, urged parents to be wary of the AI toys they buy this Christmas as the guardrails put in place are often flimsy: “The Kumma bear disaster exposes what AI researchers already know: current language models are trivially easy to jailbreak.
“These are by no means sophisticated hacks. They’re conversational patterns children naturally use during play. These systems simply aren’t robust or reliable enough for unsupervised child interaction.
Parents urged to be wary
“Until we have regulations requiring child-specific AI design, independent safety certification and accountability for conversational harms, parents should treat AI-enabled toys with the same caution applied to handing their child unsupervised access to the entire internet.”
She added: “Be wary of the AI toys you put under the Christmas tree.”
Mashood Ahmad, Founder at Gigabit IQ, said we need mandatory pre-market safety testing for AI toys: “AI toys need digital seat belts now. As an online safety advocate and parent, the FoloToy incident proves we’re handing children digital strangers disguised as teddy bears.
“We’d never accept toys with toxic paint, yet we tolerate ‘learning’ toys that discuss BDSM and knives with kids. Children under eight need imagination in play, not AI companions that hallucinate harmful content.
“Traditional toys let children create stories. AI toys override their narrative control with adult themes they can’t process.
“We need mandatory pre-market safety testing for AI toys matching physical toy standards, with child development expert reviews, transparent AI model disclosure and age-appropriate content filtering.
“Until AI toys undergo rigorous safety testing, parents should choose traditional Christmas toys that spark imagination without surveillance risks.”
Some presents best left on the shelf
Patricia McGirr, Founder at Burnley-based Repossession Rescue Network, said incidents such as the Kumma bear risk turning parents’ festive plans into a safeguard-free nightmare.
She continued: “When a furry teddy bear can spout X-rated content, the question isn’t why it happened, but how it ever made it to market. We rightly insist on safeguards for wiring, paint and child-safe materials, yet we have almost none for generative content that can speak directly into a child’s ear.
“AI toys need proper safety testing, firm guardrails and honest disclosure about the tech inside. Until that happens, some presents are better left on the shelf than stuffed into a stocking.”
When approached about the Kumma bear, a spokesperson from OpenAI told Newspage: “We suspended this developer for violating our policies. Our usage policies prohibit any use of our services to exploit, endanger, or sexualize anyone under 18 years old. These rules apply to every developer using our API, and we monitor and enforce them to ensure our services are not used to harm minors.”
OpenAI also said that last month it introduced an expert council focused on well-being and AI supported by experts in psychology, psychiatry and human-computer interaction.
Singapore-based FoloToy has also been approached for comment, but CEO Larry Wang told CNN that the Kumma bear has now been withdrawn from sale.
Photo by Barrett Ward on Unsplash
