Researchers link AI agent power to human Goal attainment in formal models
A new paper on arXiv explores how power dynamics between AI agents and humans can be quantified in socio-technical systems. The authors propose two measures: information-theoretic empowerment—how much an agent’s actions influence future states—and capability-based individual power, which gauges a human’s ability to achieve goals based on probabilistic outcomes. They derive a mathematical…
Key points
- Two formal measures of AI-agent power defined: empowerment (action influence) and capability-based individual power (goal attainment)
- Goal revelation bounded by empowerment (upper limit) and a satisfaction floor (*B(s)*) derived from Fano’s inequality
- Failure modes identified: empowerment-maximizing agents may ignore goal relevance; revelation-maximizing agents may mislabel goals
The work highlights critical failure modes: an AI maximizing empowerment might prioritize broad influence over irrelevant actions, while one maximizing revelation could mislabel goals without improving outcomes. The authors argue that only satisfaction floors—calculated from interaction logs—can reliably monitor human power retention. Simulations suggest the three metrics (empowerment, revelation, and satisfaction) correlate but are not interchangeable.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Researchers propose framework for AI-directed evolution and safety risks · 1 src
- Researchers propose TRAP policy for AI agent authorization · 1 src
- Researchers autoformalize linear regularization theory in Lean with AI agents · 2 src
- Researchers propose Two-Ledger Model for tracking AI costs in enterprise transactions · 1 src
- FlexFloor improves reinforcement learning post‑training efficiency for LLMs · 1 src
Comments
via GitHub Discussions