{"version":1,"type":"story","url":"https://digestai.news/story/amazon-adds-skill-selection-and-instruction-following-evaluators-to-be","json":"https://digestai.news/story/amazon-adds-skill-selection-and-instruction-following-evaluators-to-be.json","markdown":"https://digestai.news/story/amazon-adds-skill-selection-and-instruction-following-evaluators-to-be.md","slug":"amazon-adds-skill-selection-and-instruction-following-evaluators-to-be","headline":"Amazon adds skill selection and instruction-following evaluators to Bedrock AgentCore","summary":"Amazon Web Services announced that the Strands Evals SDK and Amazon Bedrock AgentCore now include skill‑focused evaluators. The new metrics—Skill Selection Accuracy and Skill Instruction Following—measure whether an agent picks the right reusable skill and how completely it follows the skill’s prescribed steps. A deterministic check called SkillInvoked confirms that a named skill was loaded successfully.\n\nThe evaluators work on recorded agent trajectories in Strands Evals or on OpenTelemetry traces captured by AgentCore. Skill Instruction Following rates execution on a five‑level scale (Fully Followed 1.0, Mostly Followed 0.75, Partially Followed 0.5, Minimally Followed 0.25, Not Followed 0.0). Developers can run these checks in unit‑test pipelines, batch over stored sessions, or continuously on live traffic using the AgentCore CLI. Prerequisites include Python 3.10+, an AWS account with Bedrock access, and enabled CloudWatch observability.\n\nThe blog post provides step‑by‑step examples for an HR‑assistant agent, guidance on interpreting per‑skill results, and best practices for combining deterministic and judge‑based metrics. It emphasizes keeping routing and execution failures separate to diagnose issues quickly and to block releases when skill regressions are detected.","keyPoints":["Strands Evals SDK and Bedrock AgentCore now offer Skill Selection Accuracy and Skill Instruction Following evaluators.","Skill Instruction Following rates execution on a five‑level scale from Fully Followed (1.0) to Not Followed (0.0).","Evaluations can run on recorded trajectories, OpenTelemetry traces, or continuously on live traffic via the AgentCore CLI."],"whyItMatters":"Separating skill routing errors from incomplete execution lets teams catch hidden failures, improving reliability and safety of AI agents deployed in production.","category":{"slug":"agents","name":"Agents & Tools","url":"https://digestai.news/category/agents"},"entities":{"companies":["Amazon","Strands"],"models":[],"people":["Ritvika Pillai","Vincent Chen","Qiaoxuan Xue","Shoaib Javed"]},"firstPublishedAt":"2026-09-22T17:18:13Z","updatedAt":"2026-09-22T17:18:13Z","sourceCount":1,"hasPrimarySource":true,"sources":[{"outlet":"AWS Machine Learning Blog","title":"Evaluate skill-equipped agents with Strands Evals and Amazon Bedrock AgentCore","url":"https://aws.amazon.com/blogs/machine-learning/evaluate-skill-equipped-agents-with-strands-evals-and-amazon-bedrock-agentcore","publishedAt":"2026-09-22T17:18:13Z","type":"primary","primary":true,"lead":true}],"sourceNotes":null,"discussions":[],"thread":null,"cite":{"text":"Digest AI, \"Amazon adds skill selection and instruction-following evaluators to Bedrock AgentCore\", 22 September 2026, https://digestai.news/story/amazon-adds-skill-selection-and-instruction-following-evaluators-to-be","publisher":"Digest AI","title":"Amazon adds skill selection and instruction-following evaluators to Bedrock AgentCore","datePublished":"2026-09-22T17:18:13Z","url":"https://digestai.news/story/amazon-adds-skill-selection-and-instruction-following-evaluators-to-be"},"generatedBy":"Written by Digest AI's editorial model from the linked sources; the sources are the record.","license":"Headlines, digests and key points are written by Digest AI and may be quoted with a link to the story page. Linked articles belong to their publishers. Terms: https://digestai.news/terms#reuse"}