Dayjob: Tax – evaluating AI agents on 50 professional tax assignments

The benchmark spans 1,789 files across eight areas of tax practice, from individual and family returns and small business to partnerships, estates and trusts, international tax, corporate transactions and tax disputes. Agents investigate records, reconcile conflicting information, identify consequential issues, and produce returns, workpapers, technical reviews, and client advice. Claude Sonnet 5.5 passes 10.8% of attempts at a 95% rubric threshold. Fable 5.1 scored 9.6% and Claude Opus 5.5 scored 9.2%. Several open models scored at or near zero. Surge AI attributes most failures to judgment errors rather than bad math. One example is an issue the client never raised, which agents overlook and then repeat. This is the third DAYJOB release, following healthcare and finance benchmarks that comprised 130 expert assignments. The full benchmark is mostly held out, and access must be requested from Surge AI by email. On Hacker News, commenters said top tier tax professionals can do pretty well on this benchmark, better than agents. For founders building agentic products, the result points toward agents that gather documents and draft work while a qualified person signs off. Evaluation should target end-to-end deliverables, since agents can perform the individual steps yet still fall short on professional judgment and reliable workflows.