Laya, a 322M-parameter decision engine, gains 26,639 GitHub stars in nine days

The project ships three checkpoints on Hugging Face, including a 421M-parameter English model and a 322M-parameter multilingual model supporting over 100 languages. The repository claims a latency of 33ms per question on a T4 GPU, dropping to 7.2ms when batched. A hands-on test on a CPU-only VM confirmed that the system runs locally without API keys or GPU requirements, though CPU inference is slower at roughly 0.7 seconds per question. The tool includes a built-in abstention mechanism that flags low-confidence answers, allowing developers to route uncertain cases to human review or larger models.

While the speed and cost efficiency are significant, the documentation warns that the shipped checkpoints are over-confident and require users to fit temperature parameters on their own data before relying on the probability scores. Laya is positioned for high-volume, bounded decisions like ticket triage and moderation, where deterministic, fast responses are preferred over the nuanced reasoning of frontier models.