How the estimation engine actually works
Sheet 1 says what was built. This sheet says how: the untangling that turned a 23-minute sequential analysis into a parallel one that had to prove it produced the same numbers, the discipline that keeps money deterministic, and the faults the validation gates caught before users could. The client is under NDA, so the platform is not named; every figure on this page is our own measurement of our own build.
← Back to sheet 1, the case studyThe constraint that shaped everything
The estimate this pipeline produces feeds quotes, bid packages and a generated contract. So when the client asked for the analysis to be faster, there was one non-negotiable: a faster pipeline that produces different numbers is not an optimisation, it is a defect. Speed had to come from restructuring the work, never from cutting it.
That constraint set the acceptance gate before any code changed: every iteration of the parallel pipeline re-ran the same real blueprint and was compared with the sequential baseline, stage by stage and trade by trade. More than one iteration failed that gate and never shipped. Section 6.0 covers what the failures taught us.
- Outputs must match the sequential baseline on a real blueprint, not a toy
- Stages that share no data run in parallel; stages that do run in ordered waves
- Every stage persists its own result, so a missing piece is detectable
- Cost totals are summed in code from persisted outputs, never re-asked of the model
Untangling a stateful pipeline
The pipeline began life as one long stateful conversation: each stage read everything before it, which made the output coherent and the whole thing impossible to parallelise naively. The untangling started by asking, per stage, what it actually needs. The head stages genuinely build on each other, so they stay sequential. The sixteen trade analyses each need the project context and the building's measurements, but not each other, so the measurements were promoted into an explicit quantity basis stage: computed once, handed to every trade as authoritative context, and launched alongside the head so its cost disappears into wall-clock time the pipeline was already spending. Because every trade prices from the same basis, the trades cannot quietly disagree about the building they are pricing.
The trades then run in parallel under a bounded concurrency cap, and the tail, the stages that read the assembled report, runs in six dependency-ordered waves. Two anchors are enforced in code rather than asked of the model: the cost rollup is summed from the persisted trade outputs, and the executive summary is locked to that rollup, so no stage can invent a second total. Each stage persists its result as it completes, which is what makes a partial failure something the system can detect instead of something a reader has to notice.
Keeping money deterministic
Language models intermittently drop forced markers and section labels, and under parallel load they drop them more. Every parser that keyed on a literal label eventually broke on some trades, so the rule is now uniform: find a table by its headers, find a section by its shape, and treat labels as decoration.
Once an estimate exists, its figures live in a canonical store where recalculation is a fixed formula, not a model call. Editing a line item is maths. AI is reserved for the edges, suggesting a replacement product or regenerating a narrative report, and never recomputes a figure a user has touched.
The same total appears across documents as a raw number, a grouped figure and a truncated abbreviation. When an edit changes it, substitution handles each shape, applies the longest match first so a fragment cannot corrupt a fuller figure, and reproduces the document's own style.
Substitution into prose and contract text is fenced: a percentage is only rewritten when the sentence names the rate it belongs to, abbreviations that could collide with a product name are skipped, and an ambiguous match is suppressed entirely. A safe miss is acceptable; a corruption is not.
Measured, with the method stated
The pipeline numbers come from persisted evidence, not stopwatches. Every stage writes its result to the database as it completes, so a run's duration is the span from its first persisted stage to its last, and the same query measures any run, old or new. Speed was never accepted on its own: the validated parallel run carried all 32 stages and all 16 trades, and its material total landed within 0.1% of the sequential baseline.
The platform figures use the same discipline. Backend query counts come from replaying an identical session against the logs before and after the change; the API-call counts come from a scripted browser session on the same job, with the pre-fix code restored from source control so the comparison is honest. One lesson is worth stating: on a small, far-away development database, milliseconds are mostly network noise, so we count queries rather than seconds. A count carries over to production; a stopwatch reading does not.
Tested against real artefacts
The dangerous parsers never meet production first. Before a parser changes, the new logic is replayed in simulation over stored production reports alongside the old logic, and the comparison is row counts and totals, not opinion. One such replay showed the old path recovering a single trade section from a report where the new path recovered seventeen; only with that evidence did the change ship.
The substitution logic that edits contract prose has a permanent test suite: fourteen checks covering the cases that make find-and-replace dangerous, such as two items sharing a rating where only one was edited, or a manufacturer name that reads like an abbreviated number. The suite links the production code directly rather than a copy, so the tests cannot silently drift from what ships.
Faults the process caught
The first parallel runs produced a material total of $128,466 against a $467,415 baseline. Not a costing error: under parallel load the model had stopped emitting the section labels the cost rollup keyed on, so 11 of 16 trades summed to zero. The A/B gate caught it before anything shipped. The fix parses tables by structure instead of labels, and the rerun landed within 0.1% of the baseline.
Under the parallel burst, shared document references began failing, and per-trade error isolation did its job too well: a run completed with 7 of 16 trades absent. Caught because every run is verified against the full expected stage list in the database, never judged by whether it finished. The fix falls back to inlining documents on the first failure and latches for the rest of the run, so degradation is graceful instead of silent.
Two overlapping first-loads of the same job each decided it was unseeded, and the takeoff total doubled. The application-level guard could not have fired: it read state once, before the race began. The durable fix is a unique database index that makes the duplicate impossible, backed by recovery that serves the winning rows and request de-duplication in the client. The guard that matters is the one no race can slip past.
Operations for long-running AI
An analysis is a long-running background operation, so nothing about it depends on a browser staying open. Runs execute as queued background jobs on Azure, progress is broadcast over SignalR and written to the database at every sequential seam, and a page refresh resumes from the persisted state rather than guessing. Concurrency is capped and configurable, sized to the AI provider's quotas rather than to optimism.
The prompts that drive the pipeline are versioned in blob storage per environment, not baked into the build, so prompt changes deploy and roll back independently of code. And because every stage's output is persisted individually, a support question weeks later starts from evidence: what ran, when, and what it produced.
- Analysis as queued background jobs; no browser dependency
- Progress broadcast over SignalR and persisted, so refreshes resume honestly
- Concurrency capped and configurable, sized to provider quotas
- Prompts versioned in blob storage per environment
- Per-stage persistence: every run is auditable after the fact