Paper argues AI safety must address accumulative risks
A new position paper published on arXiv argues that AI safety discussions are too focused on existential threats like nuclear conflict or biological outbreaks. The authors contend that a distinct type of risk arises from the widespread, daily use of AI, which accumulates over time rather than occurring in a single catastrophic event. They draw parallels to social media, where repeated exposure…
Key points
- Paper argues AI safety overlooks accumulative harms from daily use, unlike catastrophic risks.
- Authors introduce the Trajectory Neglect Thesis, linking AI risks to social media feedback loops.
- Study calls for evaluating cumulative usage patterns and causal inference in AI deployments.
The paper introduces the "Trajectory Neglect Thesis," suggesting that focusing on vivid disasters causes researchers to overlook less visible harms that build up through recurring decisions and persistent memory in AI systems. Using conversational AI as a case study, the authors examine how mechanisms like sycophancy, manipulation, and privacy loss may interact to tighten feedback loops. They note that while social media research shows how individual susceptibility shapes outcomes, similar mechanisms may operate in AI through generation and semantic inference.
The authors do not claim that repeated AI use is inherently harmful but identify specific mechanisms to investigate. They call for safety assessments to evaluate cumulative patterns of use, individual and societal outcomes, and causal inference. The goal is to develop proportionate safeguards that preserve the utility of AI assistance while protecting user choice and preventing the gradual erosion of independent judgment.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Anthropic reports AI agents spend 80% of session waiting for humans, remeasurement shows about 30% · 1 src
- CoDA enables rapid codebook development for domain-specific text annotation · 1 src
- Researchers prove Prefix-Tuning can exactly steer pretrained transformers · 1 src
- Study probes if LMs understand variable roles via SAEs · 1 src
- OR-License benchmark tests LLM agents on real operations research tasks · 1 src
Comments
via GitHub Discussions