Opinion: avoid using ChatGPT for legal, medical, therapy, confidential or password tasks
Sam Altman told podcaster Theo Von in July 2025 that courts could compel OpenAI to hand over ChatGPT conversations, meaning chats lack the legal privilege that lawyer‑client or doctor‑patient talks enjoy. The article notes that chats have been subpoenaed, exposed through a share button, indexed by Google and preserved by court order in the past 18 months, creating a non‑zero risk for users.
Key points
- Over 2,000 court decisions cite ChatGPT errors, with 1,175 self‑representeds and 815 lawyers, leading to fines in the thousands‑of‑dollars range.
- OpenAI estimates 0.15 % of weekly users show suicide‑planning cues, equating to more than a million people per week, prompting added crisis safeguards.
- Fast Company found nearly 4,500 shared ChatGPT conversations indexed online; OpenAI disabled the share feature after the report.
The piece lists five areas where misuse is common. Researchers led by Damien Charlotin have logged over 2,000 court decisions where ChatGPT‑generated citations or arguments were rejected, including 1,175 cases involving self‑representing litigants and 815 involving lawyers, with courts in the US, UK, South Africa, Israel, Australia and Spain imposing fines that run into the thousands of dollars. OpenAI estimated last year that 0.15 % of weekly active users show explicit suicide‑planning cues – a figure the article translates to more than a million people per week – and hundreds of thousands more display signs of psychosis or mania. Laws in Illinois, Nevada and Maine now bar AI from delivering therapy. The article also cites Madhu Gottumukkala’s accidental upload of classified DHS documents, Samsung’s 2023 ban on generative AI tools, and Fast Company’s discovery of nearly 4,500 shared ChatGPT conversations indexed online, which prompted OpenAI to remove the share feature.
Never use ChatGPT for these five tasks
engadget.com · 23 September 2026
Loading the full article…
This text was published by engadget.com and written by Luke James. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Policy & Regulation
All →- OpenAI and Anthropic investigate tens of thousands of AI agent safety incidents, Axios reports · 9 src
- Florida AG James Uthmeier seeks emergency injunction against OpenAI over ChatGPT self‑regulation · 2 src
- Trump to have private dinner with Anthropic CEO Dario Amodei at the White House on Sunday · 32 src
- OpenAI pauses training of its most powerful AI models after AI agent accessed external chatbot via DNS vulnerability · 23 src
- OpenAI agents accessed SEC and Census Bureau sites, prompting congressional scrutiny · 2 src
Comments
via GitHub Discussions