Benchmark
Solve the escalations that bounce between teams for days
Watch BuildWright trace a real customer-reported bug to the exact file and method the shipped fix changed, using nothing but the customer’s description.
The problem · ERPNext
“The invoice totals never reconcile…”
Off by a few cents, only sometimes, and nothing in the ticket says where to look. This kind of ticket bounces between teams for days.
“On some multi-item invoices, the item-wise tax breakup doesn’t add up to the total tax. It’s off by a few cents, only on certain rate and quantity combinations. The header total looks right; the per-item breakdown doesn’t reconcile.”
The solution
BuildWright investigated
BuildWright traced the missing cents to lines 540 to 556 of the set_item_wise_tax() method in taxes_and_totals.py.
Tax was rounded on each line separately, so small differences drifted from the header total. ERPNext pull request #53925 changed the lines BuildWright named.
- File
- erpnext/
controllers/ taxes_and_totals.py - Method
- set_item_wise_tax()
- Lines
- 540–556
- Cause
- Tax rounded on each line separately. Small differences add up and drift from the header total.
Matched the shipped fix. ERPNext PR #53925 changed the same lines.
Technical detail
It works the way your team would
A read-only index of the code, related knowledge, and a grade on the evidence. The only input was the customer’s description.
With its knowledge layer switched off, BuildWright’s production agent matches the reference agent on τ²-bench, Sierra Research’s public benchmark for AI support agents.
| τ²-bench domain | BuildWright | Reference agent |
|---|---|---|
| Retail | 0.82 | 0.81–0.83 |
| Airline | 0.79 | 0.80 |
Resolution rate across 50 tasks per domain, graded automatically.
Results
BuildWright doesn’t mark its own homework
Not enough evidence to name a root cause
no indexed code repository
Connect the repository for this service, or add the error log from the time of the report, and I will investigate again.
A tool that says “I don't know”
In one run it reported “no indexed code repository” instead of inventing a file path.
BuildWright used a past resolution
Same rounding issue, different customer
Learned resolutions surface the facts it needs roughly 3x more reliably the next time.
Faster with every fix
Once it learns a resolution, the facts it needs surface roughly 3x more reliably the next time.
- ShadowDrafts only. Nothing posts.
- CopilotA person approves every post.
- AutopilotOnly for proven case types.
Read-only by design
It never changes code, data or production. Your team sets the trust level.
Design partners
Test BuildWright against your own ticket
The proof that matters is your tickets. We build a demo around your real escalations and score BuildWright's diagnoses against how your team resolved them.
Not ready for a call? Send us a note
BuildWright could not support a diagnosis