Training-free lexical pipeline cuts LLM prompt tokens by 40% with minimal quality loss
Researchers have introduced a deterministic, CPU-only pipeline for compressing large language model prompts using classical lexical NLP techniques. Unlike learned methods such as LLMLingua, this approach requires no auxiliary models or training, offering a lightweight solution to reduce inference costs and latency associated with long-context tasks like chain-of-thought reasoning.
Key points
- Pipeline achieves 40.3% mean token reduction with 0.876 BERTScore-F1 fidelity using deterministic lexical rules.
- Method is training-free and CPU-only, avoiding the need for auxiliary language models required by learned compression techniques.
- Evaluation across 18,630 GPT-4o-mini completions shows commonsense reasoning is the most sensitive task category to compression.
The system employs eleven toggleable transformations, including stopword removal, lemmatization, and synonym shortening, while preserving named entities. The study evaluated fifteen configurations across 1,242 English prompts from six diverse datasets, generating 18,630 paired completions using GPT-4o-mini. Fidelity was measured using BLEU, ROUGE, BERTScore, and SentenceBERT similarity.
Results indicate that the most aggressive configuration achieves a mean token reduction of 40.3% while maintaining a BERTScore-F1 of 0.876. A more conservative stopword-only setting reduces tokens by 29.6% with higher fidelity (0.913). The analysis identifies commonsense reasoning as a primary failure mode under aggressive compression, providing a clear Pareto frontier for balancing efficiency and output quality.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us.
More in Research
All →- AI Researchers Fear Machines Could Kill Us All · 9 src
- Token merging boosts Whisper efficiency across 16 languages with minimal accuracy loss · 1 src
- LabAgent: Automates Reproducing Scientific Methods · 1 src
- Vibe Patenting: LLM Judges Improve AI Patent Drafting Quality · 1 src
- Generalized Agent Iteration Framework Unifies Policy Improvement and Recursive Self-Improvement · 1 src
Comments
via GitHub Discussions