Embers framework improves robustness testing of deep research systems

Using these insights, EMBERS generates stage‑targeted test inputs through defect‑guided mutation, web‑search‑assisted fidelity checks, and diversity‑aware seed maintenance. Across 16 framework–model–dataset combinations, the framework discovers failures 3.21 times faster than the strongest baseline while keeping the same testing budget. Human reviewers and LLM‑assisted checks confirm that most generated inputs preserve the original information need and contextual constraints.

The study also shows that fine‑tuning DR models with EMBERS‑generated data consistently improves answer stability, suggesting the framework can both expose hidden failure modes and help strengthen models. Researchers and developers of autonomous research agents can use EMBERS to uncover hidden failure modes and accelerate robustness improvements.