# Researchers find transformer prompt influence decays as power law

Digest AI · Research · published 2026-10-20T04:00:00Z

Canonical: https://digestai.news/story/researchers-find-transformer-prompt-influence-decays-as-power-law

## Summary

The paper, posted on arXiv, examines how much of a transformer model’s prompt remains after generating tokens. The authors show that the distance between hidden states of two different prompts decays approximately as a power law $t^{-alpha}$, with $alpha=1$ for a fixed prompt’s share of uniform attention. Trained models forget fast ($alphaapprox0.55$–$0.7$), untrained slowly ($alphaapprox0.2$–$0.5$). They prove an upper bound $O\big(P(\log t)^{L-1}/t\big)$ for prompt length $P$ and depth $L$, under conditions of finite‑depth pre‑LN softmax transformers with bounded attention logits and nonzero pre‑norm activations. The mechanism is geometric: prompt influence dilutes as context grows, and relay tokens can only add logarithmic factors. The authors call this the conditional no‑hair theorem, meaning that given the generated text so far, the prompt’s influence on the next token vanishes as $t$ grows. An exactly solvable linear relay model reproduces the rise of the local exponent toward 1 seen in untrained networks. Cancelling dilution with a logit bias stops the decay early in training and reverses it after, showing dilution is necessary for the forgetting measured.

## Key points

- Prompt influence decays as $t^{-alpha}$ with $alpha\approx0.55$–$0.7$ for trained models
- Upper bound $O\big(P(\log t)^{L-1}/t\big)$ holds for finite‑depth pre‑LN softmax transformers
- Conditional no‑hair theorem: prompt influence vanishes as $t$ grows, leaving only text to carry prompt properties

## Why it matters

Understanding prompt forgetting helps explain transformer behavior and informs design of more efficient models, potentially reducing reliance on long prompts and improving interpretability.

## Sources

1. [A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt](https://arxiv.org/abs/2610.24133) (arXiv cs.AI, 2026-10-20, primary source)

## Cite

Digest AI, "Researchers find transformer prompt influence decays as power law", 20 October 2026, https://digestai.news/story/researchers-find-transformer-prompt-influence-decays-as-power-law

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/researchers-find-transformer-prompt-influence-decays-as-power-law.json