# DeepInstructor agentic framework improves idea evaluation using experience graph from 58,607 peer reviews

Digest AI · Research · published 2026-09-22T04:00:00Z

Canonical: https://digestai.news/story/deepinstructor-agentic-framework-improves-idea-evaluation-using-experi

## Summary

Researchers introduced DeepInstructor, an agentic AI framework designed to evaluate scientific ideas by reasoning over structured scholarly experience. The system builds an Experience Graph from 58,607 peer reviews and uses a ReAct-based agent to retrieve evidence for traceable evaluation across dimensions like novelty and feasibility. It was tested on a new dataset called DeepInstruct, which contains controlled pairwise comparisons. According to the paper, DeepInstructor outperforms existing baselines, improving Hit@1 alignment with human judgments by 24.4% and Hit@2 by 29.7%. The authors state that this approach grounds idea evaluation in explicit reasoning over experience rather than relying solely on parametric knowledge or unstructured retrieval.

## Key points

- DeepInstructor constructs an Experience Graph from 58,607 peer reviews
- It uses a ReAct-based agent to retrieve dimension-specific evidence for traceable evaluation
- Experiments show 24.4% improvement in Hit@1 and 29.7% in Hit@2 alignment with human judgments

## Why it matters

This work shifts AI-assisted scientific discovery from idea generation to better-grounded evaluation, potentially improving the quality and reliability of AI-generated research hypotheses by anchoring them in real academic review experience.

## Sources

1. [DeepInstructor: An Agentic AI Instructor for Experience-Driven Idea Evaluation](https://arxiv.org/abs/2609.22104) (arXiv cs.CL, 2026-09-22, primary source)

## Cite

Digest AI, "DeepInstructor agentic framework improves idea evaluation using experience graph from 58,607 peer reviews", 22 September 2026, https://digestai.news/story/deepinstructor-agentic-framework-improves-idea-evaluation-using-experi

---

Written by Digest AI's editorial model from the linked sources; the sources are the record. Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse
JSON: https://digestai.news/story/deepinstructor-agentic-framework-improves-idea-evaluation-using-experi.json