DeepInstructor agentic framework improves idea evaluation using experience graph from 58,607 peer reviews
Researchers introduced DeepInstructor, an agentic AI framework designed to evaluate scientific ideas by reasoning over structured scholarly experience. The system builds an Experience Graph from 58,607 peer reviews and uses a ReAct-based agent to retrieve evidence for traceable evaluation across dimensions like novelty and feasibility. It was tested on a new dataset called DeepInstruct, which…
1 source primary source
Key points
- DeepInstructor constructs an Experience Graph from 58,607 peer reviews
- It uses a ReAct-based agent to retrieve dimension-specific evidence for traceable evaluation
- Experiments show 24.4% improvement in Hit@1 and 29.7% in Hit@2 alignment with human judgments
Read the original at arXiv cs.CL · by Rongcan Pei, Fang Guo, Qinglin Qi, Qi Zhu, Yun Luo, Jianhao Yan, Minjun Zhu, Qiujie Xie, Dehong Zheng, Yue Zhang primary sourceOpen source ↗
Topics · follow one to build your own front page
DeepInstructor
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us.
More in Research
All →- Researchers propose zero-inference prospective term that boosts personal memory recall · 1 src
- PsyAgentBench tests if LLMs truly mimic human psychological biases · 1 src
- Indy Autonomous Challenge features first road-course overtakes, Unimore wins · 1 src
- Researchers unveil TangleDiff to design entangled protein hydrogels · 1 src
- Anthropic reports Claude leads 26% of its AI research and development tasks · 3 src
Comments
via GitHub Discussions