{"version":1,"type":"story","url":"https://digestai.news/story/ai-teddy-makes-unprompted-i-can-t-help-with-killing-you-remark","json":"https://digestai.news/story/ai-teddy-makes-unprompted-i-can-t-help-with-killing-you-remark.json","markdown":"https://digestai.news/story/ai-teddy-makes-unprompted-i-can-t-help-with-killing-you-remark.md","slug":"ai-teddy-makes-unprompted-i-can-t-help-with-killing-you-remark","headline":"AI teddy makes unprompted 'I can't help with killing you' remark","summary":"A BBC investigation into AI toys for children revealed that a plush toy named Doubao made an unprompted and alarming statement. When asked how babies are made, the toy misinterpreted the query and replied, \"I can't help with killing you,\" before advising the user to call emergency services. The BBC was unable to identify the manufacturer of Doubao despite repeated efforts, highlighting transparency issues in the industry.\n\nThe report also tested Grem, made by US firm Curio, and My Baby Bunny. While Grem was overly chatty, My Baby Bunny provided age-appropriate answers but struggled with processing speed. A family in Newport tested the toys for a week and found the experience disappointing, noting that children wanted immediate, conversational responses rather than delays or monologues.\n\nPaul Clarke, founder of Play Tech AI, stated the industry is \"crying out\" for regulation, citing unsafe and unregulated products seen at the Shanghai Toy Fair. The UK government said it is seeking views on evolving toy safety regulations for AI-enabled products. Children's Commissioner for Wales Rocío Cifuentes urged parents to be careful, emphasizing that AI toys cannot replace human interaction and that data privacy concerns remain unresolved.","keyPoints":["Doubao AI teddy said \"I can't help with killing you\" after mishearing a question about babies.","BBC could not identify the manufacturer of Doubao, raising concerns about industry transparency.","UK government is seeking views on regulating AI-enabled toys to ensure child safety."],"whyItMatters":"The incident highlights significant safety and regulatory gaps in AI toys marketed to young children. The inability to identify manufacturers complicates accountability, while the unprompted harmful response underscores the risks of deploying LLMs in consumer devices without robust safety guardrails.","category":{"slug":"policy","name":"Policy & Regulation","url":"https://digestai.news/category/policy"},"entities":{"companies":["Curio","Play Tech AI","Mattel","Swansea University"],"models":["ChatGPT 4.1 Nano"],"people":["Paul Clarke","Rocío Cifuentes","Berry Billingsley","Kanishka Narayan","Grimes"]},"firstPublishedAt":"2026-11-06T20:08:11Z","updatedAt":"2026-11-06T20:08:11Z","sourceCount":1,"hasPrimarySource":false,"sources":[{"outlet":"bbc.com","title":"AI teddy makes unprompted 'I can't help with killing you' remark","url":"https://bbc.com/news/articles/c6x2w4ej245ko?xtor=AL-71-%5Bpartner%5D-%5Bbbc.news.twitter%5D-%5Bheadline%5D-%5Bnews%5D-%5Bbizdev%5D-%5Bisapi%5D&at_bbc_team=editorial","publishedAt":"2026-11-06T20:08:11Z","type":"press","primary":false,"lead":true}],"sourceNotes":null,"discussions":[{"site":"Reddit","url":"https://www.reddit.com/r/artificial/comments/1xpmrry/ai_teddy_makes_unprompted_i_cant_help_with/","points":null}],"thread":null,"cite":{"text":"Digest AI, \"AI teddy makes unprompted 'I can't help with killing you' remark\", 6 November 2026, https://digestai.news/story/ai-teddy-makes-unprompted-i-can-t-help-with-killing-you-remark","publisher":"Digest AI","title":"AI teddy makes unprompted 'I can't help with killing you' remark","datePublished":"2026-11-06T20:08:11Z","url":"https://digestai.news/story/ai-teddy-makes-unprompted-i-can-t-help-with-killing-you-remark"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}