The best AI model scores 12% on a new test of long agent tasks. Set pilot expectations for estimating and back office accordingly
SemiAnalysis's new GDPx benchmark gives the top model 12% on multi-hour professional tasks, and zero on finance and legal under pass/fail grading. For a contractor planning an agent pilot, that is a useful calibration.
The best AI model currently available completes about 12% of a new set of long, realistic professional tasks. That is the headline of GDPx, a benchmark SemiAnalysis released on November 2, and it is a useful reality check for estimators and PMs planning to hand an AI agent a whole workflow.
What is GDPx and what did it find?
According to Unite.AI's summary, version 1.0 has 206 tasks across 11 domains, built with 14 specialized data partners. Tasks run in a realistic environment, with a default limit of eight hours, and scores average three attempts.
| Model | Overall score |
|---|---|
| Claude Opus 5.5 | 12.0% |
| Gemini 4 Carbon | 10.9% |
| GPT-6 Astra | 9.6% |
Per OfficeChai, Opus 5.5 costs roughly $51 per task, while Gemini 4 Carbon costs about a third of that for a score a point lower. SemiAnalysis also notes that safety classifiers pull down some models' scores in cyber, biology and alignment.
Why do finance and legal results matter to a contractor?
Construction is not one of the 11 domains, so GDPx says nothing direct about takeoff or scheduling. But two domains resemble back-office work. Unite.AI reports every model scored zero under pass/fail grading in both finance and legal, so SemiAnalysis published partial scores instead. The finance tasks mimic a private-equity deal spread across roughly 4,000 anonymized files.
A pay app, a change order log or a subcontract review is smaller than that. But it has the same shape: many documents, hidden dependencies, and an answer that is either right or wrong. If a deal-sized pile of files produces zeros, a bid package with 40 addenda and conflicting spec sections deserves caution.
What should an estimator or PM do with this?
- Pilot steps, not workflows. Give the agent one checkable task, such as pulling alternates out of an addendum, and compare against your own work.
- Keep a person on the output. Treat agent results as a draft. An error that survives into a bid is yours.
- Price the attempts. At about $51 per long task for the leader, cost per completed task matters, and a failed run still costs money.
- Track pass/fail, not impressions. Count how often the agent's output survives review without edits.
What are the limits?
This is one benchmark, designed by its authors to be hard, and its scores come from SemiAnalysis's own methodology. A 12% score on eight-hour frontier tasks does not mean a model fails 88% of short, well-defined ones. It also does not test construction documents. The authors plan new versions every three to six months, so these numbers will move.
Related: OpenAI's GPT-6.1 Terra and the pay app test shows how to run a narrow, measurable pilot.
Takeaway: Before buying an agent for estimating or back office, pick one repeatable step from your last bid, run it five times, and count how many outputs you would send unedited. That number, not a benchmark, decides the pilot.
Construction AI Brief covers AI for contractors three times a week. Subscribe at constructionaibrief.com.
- What is the GDPx benchmark?
- GDPx is an agentic benchmark released by SemiAnalysis on November 2, 2026. Version 1.0 has 206 tasks across 11 professional domains, built with 14 data partners, with a default time limit of eight hours per task.
- How well do today's best AI models score on GDPx?
- Claude Opus 5.5 leads at 12.0%, followed by Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%. SemiAnalysis says safety classifiers lower some scores in cyber, biology and alignment.
- Does GDPx test construction work?
- No. Its domains include finance, legal, chip design and software engineering, but not construction. It is a calibration for long, messy professional tasks, not a measure of estimating or project management.
- What does a low benchmark score mean for a contractor piloting AI agents?
- It suggests agents should be piloted on narrow, checkable steps with a person reviewing the output, rather than handed whole multi-day workflows like a full bid package.