AQL-Lite improves Arabic LLMs with curated data and partial-parameter training
Researchers introduced AQL-Lite, a framework designed to efficiently adapt multilingual large language models (LLMs) to Modern Standard Arabic (MSA). The method addresses challenges like noisy web data, dialectal variation, and high continual pretraining (CPT) costs by combining a quality-first data curation pipeline with partial-parameter training. This includes structural cleaning,…
Key points
- AQL-Lite uses curated Arabic data and partial-parameter training to cut costs and improve MSA LLM performance
- At 1B tokens, AQL-curated data boosts benchmark scores by 3.51 points over raw web text
- Scaled to 40B MSA tokens, the 12B model scores 75.3 on the Open Arabic LLM Leaderboard v2
At a 1B-token budget, CPT on AQL-curated data improved performance by 3.51 points across four Arabic benchmarks compared to raw web text, which showed no gain over the base model. In a 5B-token comparison, AQL-Lite achieved 94% of the gain from full-parameter CPT in 52% of the wall-clock time, outperforming adapter-only methods on all benchmarks.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- MIT historian says early AI engineers added randomness to mimic human creativity · 1 src
- Researchers introduce MemPlan for text-based planning with PDDL · 1 src
- Lighthouse chatbot to our knowledge launched for Civil Rights history research · 1 src
- Researchers introduce MemPABench to evaluate personal agent memory · 1 src
- BELT reduces LLM reasoning tokens by 41% on Qwen3-1.7B · 1 src
Comments
via GitHub Discussions