{"version":1,"type":"story","url":"https://digestai.news/story/contracteval-improves-procedural-instruction-conformance","json":"https://digestai.news/story/contracteval-improves-procedural-instruction-conformance.json","markdown":"https://digestai.news/story/contracteval-improves-procedural-instruction-conformance.md","slug":"contracteval-improves-procedural-instruction-conformance","headline":"ContractEval: Improves Procedural Instruction Conformance","summary":"As LLM agents move from answering questions to executing procedures, failures can be subtle. The article introduces **ContractEval**, a diagnostic framework that makes the active obligations in procedural instructions explicit. It matches query-active obligations against response or trace evidence to identify and localize structural failures such as omissions, wrong branches, ordering errors, extra actions, invariant breaches, and output-contract violations. On a suite of audited procedural contracts, **ContractEval** outperforms both output-only and trace-aware LLM judges by detecting all the injected structural failures. The framework is not a guarantee but makes conformance auditing more transparent.","keyPoints":["Introduces ContractEval for procedural instruction conformance","Matches query-active obligations against response or trace evidence","Detects and localizes structural failures in procedures"],"whyItMatters":"Improves the reliability of LLM agents executing complex procedures by making their actions more transparent.","category":{"slug":"models","name":"Generative AI & Models","url":"https://digestai.news/category/models"},"entities":{"companies":[],"models":[],"people":[]},"firstPublishedAt":"2026-09-11T04:00:00Z","updatedAt":"2026-09-11T04:00:00Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"arXiv cs.AI","title":"ContractEval: Query-Conditioned Execution Matching for Procedural Instruction Conformance","url":"https://arxiv.org/abs/2609.09458","publishedAt":"2026-09-11T04:00:00Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"ContractEval: Improves Procedural Instruction Conformance\", 11 September 2026, https://digestai.news/story/contracteval-improves-procedural-instruction-conformance","publisher":"Digest AI","title":"ContractEval: Improves Procedural Instruction Conformance","datePublished":"2026-09-11T04:00:00Z","url":"https://digestai.news/story/contracteval-improves-procedural-instruction-conformance"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}