Llama.cpp achieves up to 42x faster prompt lookup drafting
Benchmarks were run on an Apple M4 Pro (14 cores, 48 GB RAM) using the WikiText‑103 corpus (541 MB) and smaller subsets (25 MB‑200 MB). Median latency per drafted token and memory figures were measured over three runs.
Key points
- Up to 42× faster prompt‑lookup drafting reported on Apple M4 Pro hardware
- Switching outer map to ankerl::unordereddense segmentedmap drives most speed gains
The work builds on earlier contributions by Daniel Lemire, Martin Ankerl, and a PR from Johannes Gaessler that added a static cache to llama.cpp. All code and results are publicly available in the repository, and the author emphasizes that acceptance rates remain unchanged compared with the original implementation.
42x Faster Prompt Lookup Drafting in llama.cpp
jadidbourbaki.github.io · 27 September 2026
Loading the full article…
This text was published by jadidbourbaki.github.io and written by Hayder Tirmazi. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Reddit discussionreddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- OpenAI details self-replicating prompt injections spreading across AI agents · 1 src
- OpenAI Astra and Anthropic Claude Opus 5 decode unsolved Enigma messages · 3 src
- OpenAI discloses AI agents accessed US government websites in unexpected ways · 8 src
- OpenAI data shows AI automates coding, monitoring over decision-making · 2 src
- Google DeepMind’s Hassabis and Dean step down amid AI industry shift · 1 src
Comments
via GitHub Discussions