{"version":1,"type":"story","url":"https://digestai.news/story/researchers-release-intlawner-dataset-for-international-law-ner","json":"https://digestai.news/story/researchers-release-intlawner-dataset-for-international-law-ner.json","markdown":"https://digestai.news/story/researchers-release-intlawner-dataset-for-international-law-ner.md","slug":"researchers-release-intlawner-dataset-for-international-law-ner","headline":"Researchers release IntLawNER dataset for international law NER","summary":"The paper introduces IntLawNER, a new named‑entity‑recognition dataset covering 2,987 gold‑annotated sentences and 8,094 entity spans drawn from International Court of Justice decisions, UN Security Council resolutions, and European Court of Human Rights judgments. The data use seven institution‑specific entity types and were built with a hybrid algorithmic‑agentic pipeline that trimmed 468 k source sentences through candidate retrieval, LLM‑based vetting, and human review, leaving 89.6% of gold spans unchanged from the silver layer.\n\nBenchmark results show that aggregate agreement metrics can be misleading: Cohen’s kappa reaches 0.964 on boundary‑matched spans, yet macro‑F1 drops to 0.753 when missing entities, boundary errors, and label corrections are considered. Zero‑shot GLiNER collapses on function‑based entity types with only 0.243 micro‑F1, while fine‑tuned transformers struggle with rare labels. Providing carefully selected few‑shot examples improves all LLMs, and Claude Opus 4.6 attains the highest score of 0.873 micro‑F1. The authors release IntLawNER as a reusable benchmark for extracting references in international legal texts.","keyPoints":["IntLawNER contains 2,987 gold‑annotated sentences and 8,094 entity spans from ICJ, UN Security Council, and ECtHR texts.","The hybrid pipeline reduced 468 k source sentences to the final set, with 89.6% of gold spans unchanged from the silver layer.","Claude Opus 4.6 achieved the highest benchmark score, 0.873 micro‑F1, while zero‑shot GLiNER scored only 0.243 micro‑F1."],"whyItMatters":"A dedicated NER benchmark for international law enables more accurate extraction of legal entities, supporting research, compliance, and automated analysis of treaties, court decisions, and UN resolutions.","category":{"slug":"research","name":"Research","url":"https://digestai.news/category/research"},"entities":{"companies":[],"models":["Claude Opus 4.6","GLiNER"],"people":[]},"firstPublishedAt":"2026-09-22T04:00:00Z","updatedAt":"2026-09-22T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"IntLawNER: A Named Entity Recognition Dataset and Benchmark in International Law","url":"https://arxiv.org/abs/2609.22529","publishedAt":"2026-09-22T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Researchers release IntLawNER dataset for international law NER\", 22 September 2026, https://digestai.news/story/researchers-release-intlawner-dataset-for-international-law-ner","publisher":"Digest AI","title":"Researchers release IntLawNER dataset for international law NER","datePublished":"2026-09-22T04:00:00Z","url":"https://digestai.news/story/researchers-release-intlawner-dataset-for-international-law-ner"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}