Researchers release COILD corpus for Indian language machine translation
A team of researchers has introduced COILD, a new parallel corpus and benchmark for machine translation between Indian languages. The dataset includes over 1.16 million human-translated and verified sentence pairs across 20 language pairs, covering families like Indo-Aryan, Dravidian, Tibeto-Burman, and Austro-Asiatic. The data comes from licensed repositories and spans eight real-world domains,…
Key points
- COILD corpus contains 1.16 million human-translated sentence pairs across 20 Indian language pairs
- Data sourced from licensed repositories covering eight real-world domains like legal and medical texts
- Fine-tuning IndicTrans2-Distilled and NLLB-200 models improved translation accuracy per researchers
The researchers also created a domain-centric benchmark of 2,000 expert-verified sentences for consistent evaluation. Testing fine-tuned models—IndicTrans2-Distilled and NLLB-200—showed improved translation quality across metrics and human reviews. The work aims to support better multilingual AI for Indian languages, where existing datasets often lack depth or cultural relevance.
The story so far
2 episodes →- Researchers release COILD corpus for Indian language machine translationthis story
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- arXiv study finds 30% of AI research agents hack rewards without instructions · 2 src
- Researchers release META, an episodic-memory trading agent for financial decisions · 1 src
- Epydemix Agent Framework automates epidemic modeling with AI agents · 1 src
- Researchers introduce tracer, a user simulator for AI behavior alignment · 1 src
- Researchers introduce Adversarial Closed-Loop training for Role-Playing AI agents · 1 src
Comments
via GitHub Discussions