CoDA enables rapid codebook development for domain-specific text annotation
CoDA is an open‑source assistant that lets domain experts build annotation codebooks together with large language models. The system iteratively reviews model prediction errors, incorporates expert feedback, and updates the codebook, aiming to replace the weeks‑long manual process traditionally required.
Key points
- CoDA cuts codebook creation time from several weeks to two hours while needing five times fewer annotated examples
- In health and wildlife‑trafficking case studies, CoDA matched expert codebook performance on downstream classification
- Using CoDA, researchers flagged 2,410 health‑related case notes and 10,196 negative‑sentiment wildlife‑trafficking posts for review
The authors tested CoDA on two real‑world tasks: extracting health indicators from homelessness case‑management notes and classifying sentiment in wildlife‑trafficking social‑media posts. In both cases, the automatically generated codebooks achieved classification performance comparable to expert‑crafted ones while shrinking development time from several weeks to two hours and using five times fewer manually labeled examples. Scaling the approach, CoDA identified 2,410 case notes containing health discussions and 10,196 negative‑sentiment posts that may warrant further investigation.
By streamlining codebook creation, CoDA aims to help practitioners deploy language‑model‑based analysis in niche domains faster and at lower cost, expanding the practical reach of AI‑driven text analytics.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- Researchers introduce parade to improve demographic Survey simulations with LLMs · 1 src
- Open-source LLMS over-cooperate in prisoner's dilemma, study shows · 1 src
- Researchers introduce SURE-Act to improve Tool-Using agents · 1 src
- Google and Meta release TorchTPU for native PyTorch on TPU · 1 src
- Researchers find transformer prompt influence decays as power law · 1 src
Comments
via GitHub Discussions