Study finds causal control in subliminal prompting varies by model depth
Researchers from arXiv cs.CL present a study on subliminal learning, where language models transmit hidden traits through seemingly unrelated outputs. The paper challenges the common explanation of "token entanglement" by distinguishing between four different measurements: output co-variation, fixed vector alignment, state readability, and causal control. Using a fixed animal-number prompting…
Key points
- Fixed output-vector similarity predicts behavior less well in larger Llama-3.1 models, with a correlation change of -0.080.
- Donor-control AUC rises from 0.254 to 0.540, showing causal control increases with depth in Llama-3.1.
- Per-token averaging in Qwen models creates a length confound that masks the true one-token association.
The results show that fixed output-vector similarity predicts behavior less effectively as model size increases, with a paired mean correlation change of -0.080. However, causal control measurements reveal a different pattern. When copying temporary answer-position states between prompts, the donor-control AUC rises from 0.254 to 0.540, a significant increase of +0.286. This causal control remains strong even with eight transformer blocks remaining, suggesting that the mechanism is not simply a matter of static geometry.
The study also examines Qwen models and finds that scoring every digit in sequence does not recover the positive one-token association. Instead, per-token averaging creates a false positive association that disappears when controlling for number width, indicating a length confound. The authors conclude that fixed geometry, observational readability, causal timing, and multi-token measurement are distinct properties. While these findings constrain token-level explanations, they do not yet identify the exact mechanism of training-time trait transfer.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us.
More in Research
All →- New framework optimizes LLM inference costs via adaptive model activation · 4 src
- Qwen3.5-4B outperforms larger LLMs on new user-side conflict benchmark · 1 src
- Neo-Classic benchmark evaluates linguistic-aesthetic reasoning in Classical Chinese poetry · 1 src
- Study finds trust and friction issues in major generative AI app reviews · 1 src
- Study finds PCA can detect stylistic axes in LLM activations without training · 1 src
Comments
via GitHub Discussions