← Field Notes

Trends

What frontier AI models can and can't do on a jobsite in 2026

Josue Hernandez · September 2026 · 11 min read

Every month there's a new model and a new demo. Most of it doesn't matter to a framer. Some of it matters a lot. This is our attempt to translate the current state of the art into what it means for a crew, a GC, or a developer, sorted into three buckets: real now, real but not ready for the field, and wait.

01Voice: real now

Speech recognition is the quiet success story. Current models transcribe a tech describing a water heater install, in a mechanical room, with a compressor running, well enough to be useful most of the time. Not perfect. Useful. Two years ago that was not true.

What that unlocks is documentation that happens in the moment instead of at a desk three hours later. Talk through the job as you do it; get a structured record at the end. That is the shift that makes field AI worth building now, not in 2023.

Where it still breaks: trade vocabulary the model hasn't seen (it will hear a part number wrong), heavy accents under noise, and crosstalk between two people. The fix is a review step, not a better microphone. Design for "mostly right, quick to correct," not "perfect."

What to do: if your techs are still writing notes on paper or not at all, voice capture is the first thing to try. It's cheap and the downside is small.

02Phone and text agents: real now, with rules

A model can hold a phone conversation, book a slot on a calendar, and text a confirmation. This works today for the bounded version of the job: answer, collect details, book, hand off. It does not work for the unbounded version: negotiate, diagnose, promise.

The difference between a pilot that sticks and one that dies is almost entirely whether the person deploying it understood that line. We wrote a whole note on it: why most contractor AI pilots die in month three.

What to do: one agent, one workflow, narrow handoff rules. The 30-day plan is the sequence.

03Photo and video understanding: real, not field-ready for judgment calls

Models can look at a photo of a panel, a rough-in, or a foundation pour and describe what's there with surprising accuracy. They can flag "this looks like the wrong wire gauge" or "there's standing water at the footing." That's genuinely new and it's useful for documentation: tag the photo, describe the condition, file it against the job.

What they cannot do reliably is decide. Is that crack structural? Does this pass inspection? Is the slope on that drain enough? The models will answer confidently. They are not reliably right, and the cost of being wrong on a jobsite is not a typo. For anything with a safety or code consequence, vision AI is a second set of eyes, not the eyes.

What to do: use it to make photo documentation automatic and searchable. Do not use it to skip an inspection. If a vendor says it can, ask them to put that in the contract.

04On-device models: real and underrated

Smaller models now run directly on a phone with no signal. That matters more in construction than in almost any other industry, because jobsites are where cell coverage goes to die: basements, mechanical rooms, rural lots, steel-framed buildings.

On-device also means the audio and photos from a job don't have to leave the phone to get processed, which is the right answer for customer privacy and for a lot of commercial clients' data rules. The strongest field tools run a first pass on-device for exactly this reason and send only what's needed to the cloud for the heavier structuring.

What to do: when evaluating any field tool, ask "what happens with no signal?" If the answer is "it doesn't work," it's an office tool, not a field tool.

05Estimating and document work: real for drafts, not for the number

Models are good at reading a set of plans or a scope document and producing a first-pass takeoff, a draft proposal, or a list of what's missing from an RFI response. That's real time saved on the office side, especially for GCs drowning in paperwork.

They are not good at the number. Local material pricing, crew productivity, what the inspector in that jurisdiction actually enforces: that's tacit knowledge the model doesn't have and will confidently make up. Let it draft; keep a human on the price.

06Multi-agent "AI teams": real but not ready for most shops

The frontier right now is agents that coordinate: one handles intake, one handles scheduling, one handles billing, and they pass work between each other. It works in controlled settings. In a five-person shop it's premature; you'll spend more time managing the agents than they save.

Where it starts to make sense is a multi-crew operation with real volume, where the office is the bottleneck and the workflows are already written down. That's the Ops Autopilot tier on our rate sheet, and we'd tell most shops not to buy it yet.

07Robotics: wait, mostly

Layout robots that mark floors from a plan are real and in use on commercial sites. Drywall finishing, overhead drilling, rebar tying, and site-scanning robots exist and work in demos and on a handful of large projects. For residential and small commercial, the economics mostly don't close yet: the setup time, the site prep, and the cost are built for a 200,000-square-foot floor plate, not a kitchen remodel.

The interesting near-term move for smaller contractors is not owning a robot. It's being the crew that knows how to work alongside one when a GC brings it to the site, and having your documentation and layout data clean enough that a robot can use it. That's a two-year head start that costs nothing today.

What to do: if you're a GC or developer with repetitive, dangerous, or labor-starved tasks across multiple active sites, it may be worth a real assessment. If you're a trade shop, watch, and get your data in order.

08The one-line version

Voice and bounded phone agents: do it now. Vision for documentation: yes; for judgment: no. On-device: insist on it. Estimating: drafts only. Agent teams: not yet unless you're big. Robots: watch, unless you're a large GC. And every single one of these depends on your data being somewhere a machine can read it, which is the least exciting and most valuable project on this whole list.

This is our read of the state of the art as of September 2026, informed by our work with current models and by what we see working and failing in the field. The models change monthly; the buckets change slower. We'll revise this note when something moves between them.