LLMs struggle with planning problems as symbolic heuristics outperform learned models
A new arXiv study shows that large language models (LLMs) have difficulty solving classical planning tasks, and that traditional symbolic heuristics can predict problem difficulty better than learned approaches. The zero‑training delete‑relaxation heuristic \(h{add}\) reaches a mean Spearman correlation of 0.931 on held‑out STRIPS domains, beating ridge regression (0.707), simple object counting…
Key points
- Zero‑training delete‑relaxation heuristic h_add achieves 0.931 mean Spearman correlation, outpacing ridge regression (0.707), object count (0.657) and ARC (0.400).
- Renaming all predicates drops LLM planning success from 67.5% to 40.0% while refusal rate remains at 0%.
- Hybrid LLM‑symbolic planning benefits more from calibrating to LLM outcomes than from choosing a structural difficulty predictor.
To isolate the effect of semantic understanding, the researchers progressively renamed predicates while keeping states, transitions and optimal plans unchanged. LLM planning success falls from 67.5% to 40.0% as renaming moves from 0% to 100%, with refusal rates staying at 0%. Supplying an LLM with ARC’s conditioning information improves difficulty prediction (mean (|\rho|=0.826)) compared with a trained ARC model. The paper concludes that improving hybrid LLM‑symbolic planners requires better modeling of LLM behavior sensitive to predicate semantics rather than creating new structural difficulty predictors.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Researchers propose construction method using ChatGPT 5 and reinforcement learning · 1 src
- Navalia improves cowork agent performance over Qwen3.6-35B-A3B on finance benchmarks · 1 src
- OpenAI and Ironclad report GPT-6 Astra scores 55% on contract task benchmark · 2 src
- OpenAI brings back Aleksander Madry to focus on AI safety · 1 src
- Claude designs protein minibinders against scorpion neurotoxins · 1 src
Comments
via GitHub Discussions