Researchers test how well ontologies check AI agents’ answers on knowledge graphs
A new study on arXiv examines whether ontologies—formal schemas that define relationships in data—can reliably verify answers from AI agents using knowledge graphs. The paper compares three types of ontologies: curated by humans, generated by large language models, and derived directly from the data itself. Across tests on public and private graphs (including one with nearly a billion triples),…
Key points
- Curated ontologies flagged more correct answers as errors than LLM-generated or data-derived ontologies on UniProt
- Structural checks missed some errors, and combining ontology sources showed mixed results for agent accuracy
- Researchers tested ontologies on public and private graphs, including one with ~$10^9$ triples
The study also explores whether combining ontology sources or giving agents specific feedback improves accuracy. Key questions remain about whether the cost of curated ontologies justifies their use as a reliable checker, given their trade-offs between false positives and missed errors.
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Research
All →- AI startups aim to automate formal software verification with $365M funding · 2 src
- Researchers find instruction exposure unlocks rhetorical patterns in LLMs · 1 src
- New guide introduces LLMs for scientists and professionals · 1 src
- Researchers introduce Seq@K to test language models with sequential feedback · 6 src
- Researchers propose rank-aware method for private listwise LLM alignment · 1 src
Comments
via GitHub Discussions