A preconstruction estimator opens a 30-by-42-inch architectural drawing. Their job looks simple on paper. Calculate the net boundary area of nine carpeted office suites, deducting columns and wall partitions. Trace a continuous domestic water loop through an exterior wall chase. Measure a duct run through four transitions without skipping vertical drops.
General AI benchmarks test whether models can read schedules or count symbols. On those tests, models regularly score between 70% and 100%. That leads teams to believe automated takeoff is solved. It isn’t. When Google Research published Physical-Engineering’s Last Exam (arXiv:2610.30537), frontier models scored up to 51% on engineering formulas, spreadsheets, and design reviews. But that benchmark explicitly skipped drawing takeoff.
To find out how frontier AI actually performs on construction plans, we ran 325 real drawing tasks across 1,950 blind attempts at strict 2% linear and volumetric tolerances. We evaluated Anthropic’s Claude Sonnet 5.5 and Google’s Gemini 2.5 Pro head to head.
The result is a wake-up call for preconstruction. On single attempts (Pass@1), Claude Sonnet 5.5 achieved 40%. Gemini 2.5 Pro scored 5%. Across three independent attempts (Pass@3), Claude reached 52% and Gemini reached 11%. Exactly 197 out of 325 tasks proved hard on both models.

Why drawing takeoff defeats general vision
Construction estimating isn’t an image recognition task. It’s a continuous topological navigation problem.
When an estimator calculates linear footage, they don’t draw diagonal lines between room corners. They follow the pipe centerline through fittings. They track elbows, tees, reducers, and elevation changes. They verify wall thickness assemblies across callout details.
Frontier models fail here because they treat plans as unstructured pixels. They spot equipment tags, but they lose the connected line. When faced with complex corridor crossings, models take visual shortcuts. They draw direct lines between endpoints, under-measuring actual linear footage by 30% to 50%.
Boon is built specifically for preconstruction. We don’t build generic document wrappers, and we aren’t an AI takeoff tool. We build spatial reasoning engines that parse construction drawing networks. To evaluate the state of the art, we built a reproducible benchmark suite across four core trades.
The evaluation protocol and tolerances
Every task in this benchmark comes from real municipal, commercial, and healthcare plan sets. We evaluated four primary trades: Flooring (151 tasks), Wall partitions (61 tasks), Mechanical Piping (65 tasks), and HVAC Ductwork (48 tasks).
Each model ran under blind conditions:
- Inputs: The model received a 1568-pixel overview image plus 1500-by-1500-pixel quadrant tiles cropped at 150 DPI.
- Attempts: Each model ran three independent attempts per task (975 attempts per model family, 1,950 attempts total).
- Tooling: Zero access to external code interpreters, bash scripts, or ground truth files.
- Grading Tolerances: Linear run lengths evaluated within 2% of truth. Net boundary areas evaluated within 2% of truth. Topological traces required exact network matches.
A task is classified as hard when a model passes at most one of its three attempts.
Head to head across trades
The benchmark revealed stark performance differences across trades and model architectures:
| Trade System | Tasks | Evaluated Checks | Claude Sonnet 5.5 Pass@1 | Claude Sonnet 5.5 Pass@3 | Gemini 2.5 Pro Pass@1 | Gemini 2.5 Pro Pass@3 | Hard on Both Models |
|---|---|---|---|---|---|---|---|
| Flooring Area | 151 | 453 | 40% | 55% | 2% | 5% | 93 |
| Wall Partitions | 61 | 183 | 20% | 26% | 8% | 18% | 49 |
| Mechanical Piping | 65 | 195 | 52% | 66% | 12% | 26% | 31 |
| HVAC Ductwork | 48 | 144 | 48% | 58% | 2% | 2% | 24 |
| Combined Total | 325 | 975 | 40% | 52% | 5% | 11% | 197 |
Claude Sonnet 5.5 significantly outperformed Gemini 2.5 Pro across every discipline. Claude’s highest pass rates came on mechanical piping (52% Pass@1, 66% Pass@3). But on wall partitions, Claude collapsed to 20% Pass@1, failing 49 of 61 tasks.
Gemini 2.5 Pro struggled across all drawing categories. Despite generating up to 20,000 hidden reasoning tokens per task, it achieved just 2% on flooring and 2% on HVAC ductwork, finishing with an overall Pass@1 rate of 5%.

What 150 DPI resolution proved
In early benchmark sweeps, critics argued that models failed simply because drawing rasters were blurry. To test that hypothesis, we re-rendered every plan sheet at 150 DPI with dedicated zoomed quadrant tiles.
The resolution upgrade made a dramatic difference, but only on specific trades:
- The Resolution Flips: Exactly 91 tasks flipped from hard to passing under 150 DPI tiles. HVAC ductwork saw the biggest jump, with 21 previously failed tasks becoming solvable once linework callouts were sharp.
- The True Spatial Wall: Wall partition takeoffs barely moved. Out of 61 wall tasks, 49 remained hard even with crystal-clear 150 DPI crops.
- The Primitive Floor: When we broke down pass rates by geometric primitive, the baseline collapsed. While discrete branch length checks passed at 65%, net boundary area fell to 39%, continuous runs dropped to 22%, and hidden linear run lengths scored just 17%.
High resolution fixes optical character recognition. It doesn’t fix spatial reasoning. Once lines are legible, models still don’t know how to track partitions through door openings or trace pipe elevation drops.
Where models break down
By analyzing failure logs across the 197 mutually hard tasks, we identified three consistent error patterns:
1. The diagonal hypotenuse shortcut
When tracing mechanical pipe between two terminal units, models regularly connect equipment ports with straight diagonal lines. In real installations, pipes follow orthogonal building grids through 90-degree elbows. The model’s shortcut cuts linear takeoff by a third.
2. Demolition and screened layer blindness
On renovation sheets, existing lines appear as dashed or screened layers. In our mechanical demolition tests, models routinely dropped entire systems. Claude Opus 5.5 identified heating hot water piping but completely missed a 137-foot chilled water loop running parallel in the same chase.
3. Boundary leakage around openings
On architectural wall takeoffs, models struggle to establish closed polygons. When an interior drywall partition meets a door frame or storefront glass, the model leaks into the adjoining corridor. That turns an 800-square-foot suite takeoff into an incorrect 2,400-square-foot estimate.
What estimators should demand
If a software vendor claims their AI automates construction takeoffs, don’t ask for a demo on a fixture schedule or a door count. Simple text extraction is solved.
Ask them to run a continuous linear takeoff through fittings at 2% tolerance. Ask them to calculate net interior wall partitions without leaking into hallways. Ask them to trace a hydronic loop across three sheets without dropping half the return run.
General vision models are impressive, but they aren’t preconstruction estimators. Building reliable takeoff intelligence requires dedicated spatial graph pipelines trained on engineering linework. Until AI architectures treat drawings as topological coordinate spaces, human estimators remain essential.
Related: Boon Agent · Preconstruction series hub · Book a demo