Surge AI releases dayjob: Tax benchmark testing AI agents on 50 real tax tasks
Surge AI introduced DAYJOB: Tax, a new benchmark evaluating AI agents on 50 professional tax assignments. The test covers eight tax practice areas, requiring agents to handle incomplete client requests, conflicting evidence, and consequential deadlines—skills beyond simple calculations or legal questions.
Key points
- DAYJOB: Tax tests AI agents on 50 real tax tasks across eight practice areas, including individual filings and corporate transactions
- Claude Sonnet 5.5 leads with a 10.8% pass rate, but models often miss consequential deadlines or conflicting evidence
- Benchmark includes 1,789 files (PDFs, Excel, emails) and was reviewed by tax experts from PwC, Deloitte, and Exxon
Models like Claude Sonnet 5.5 (10.8% pass rate) and Opus 5.5 (9.2%) struggled with real-world complexities. For example, an agent missed a critical deadline for a tax exclusion in Anna’s condo sale scenario, while another failed to verify a couple’s marital status despite conflicting records. The benchmark includes 1,789 workspace files (PDFs, Excel, emails) and was designed by tax experts from firms like PwC and Deloitte. DAYJOB contrasts with narrower benchmarks like TaxBench or TaxCalcBench, which focus only on defined problems or calculations, not end-to-end professional judgment.
Model pages: Claude Sonnet 5.5 → · Claude Fable 5.1 → · Claude Opus 5.5 →
Dayjob: Tax – evaluating AI agents on 50 professional tax assignments
surgehq.ai · 28 October 2026
Loading the full article…
This text was published by surgehq.ai and written by Surge AI. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1source- Hacker News discussion · 60 pointsnews.ycombinator.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Agents & Tools
All →- Anthropic study finds AI agents sway experts’ decisions 86% of the time · 4 src
- Google expands Gemini and Search integrations to third-party apps · 1 src
- Google opens Android Studio to 40+ third-party AI agents via BYOA feature · 1 src
- Microsoft opens Copilot harness to developers and enterprises via Foundry · 3 src
- Plato raises $16M Series A to build AI agent training simulations · 1 src
Comments
via GitHub Discussions