Researchers release Pearl-Chat, a Luganda language model with 8.5M parameters
A team of researchers introduced Pearl-Chat, an 8,473,122‑parameter causal decoder‑only transformer built for Luganda, a Bantu language spoken by over 10 million people. The model was engineered from first principles in JAX and Flax NNX, using explicit functional state decomposition and deterministic resumable data streaming via Google Grain.
Key points
- Pearl-Chat has 8,473,122 parameters and a 8,192‑token BPE tokenizer with 1.60 tokens per word fertility.
- Training used 59,501 pre‑training sequences and 510 fine‑tuning dialogues across 4,000 steps.
- Benchmark scores: 0.9792 ROUGE‑L and 0.9405 BLEU on a three‑tier diagnostic set.
Training proceeded in two stages: pre‑training on 59,501 sequences from diverse corpora, followed by instruction fine‑tuning on 510 dialogues. Over 4,000 optimization steps the validation cross‑entropy loss fell from 9.4695 (perplexity: 12,958.98) to 0.1818 (perplexity: 1.20). On a held‑out benchmark Pearl-Chat achieved 0.9792 ROUGE‑L and 0.9405 BLEU. The authors released a Python Server‑Sent Events streaming runtime and open‑sourced all weights, tokenizers, and code.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Generative AI & Models
All →- OpenAI lists three aurora-alpha models in its API · 2 src
- Researchers swap Gemma 3 270M embedding table for 16 MB frozen vocabulary · 1 src
- Anthropic’s Claude Opus 5.5 tops Arena AI model rankings · 3 src
- DeepSeek releases V4.1-Flash-1017 update with lower cost and higher DeepSWE score · 20 src
- KillSwitch language challenges LLMs, Claude Opus scores highest · 1 src
Comments
via GitHub Discussions