DeployGuard incident CRUD
First external-project pilot of FORGE on a real application task, run on 10 September 2026.
Question
Does a FORGE-assisted AI coding workflow show less repository-discovery noise and less implementation rework while preserving successful build, type-check and automated-test outcomes?
Task
Add PostgreSQL-backed incident management to DeployGuard: create, list, get, update and delete endpoints; title and description; severity levels; incident status; validation; tests; and preservation of the existing health endpoint.
Observed results
| Metric | Plain AI control | FORGE-assisted | Observation |
|---|---|---|---|
| Unique files inspected | 7 | 5 | 28.6% fewer |
| Clearly unnecessary files inspected | 3 | 0 established | Lower discovery noise |
| Failed commands | Several | 0 during implementation | Cleaner assisted run |
| Retries / rework | Several | 0 | Cleaner assisted run |
| Automated tests | 3 / 3 passed | 3 / 3 passed | Same tested outcome |
| Build | Passed | Passed | Same |
| Type-check | Passed | Passed | Same |
| Feature completion | Partial* | Yes at implementation level* | Verification scopes differed |
| Provider tokens / cost | Not available | Not available | No cost claim possible |
*The control attempted live Docker verification and encountered unresolved connection resets/timeouts. Docker runtime verification was not rerun on the FORGE branch, so completion labels are not directly comparable.
FORGE telemetry
The assisted run reported 3 preparation events, 0 cache hits, 263 ms average preparation duration, 695 of 2,748 context characters selected, 514 estimated context tokens avoided, 74.7% estimated context reduction, and 15 deterministic steps completed.
Why this is labelled a pilot
- The control started from commit
ab59830, while the FORGE-assisted run started from638be04. - The plain run could still inspect FORGE-generated files such as
AGENTS.md,AI-EFFICIENCY.mdandGIT.md, so it was not a completely FORGE-free control. - Docker/runtime verification was attempted on the control and not repeated on the assisted branch.
- Exact provider tokens, model identity, cost/credits and total tool-call counts were unavailable.
DG-BM-001 is therefore evidence that the measurement workflow works and a promising signal — not proof of a universal performance or cost reduction.
What we learned about FORGE itself
The pilot also found a classifier weakness: a new incident CRUD capability was classified as a bug rather than a feature. External testing is being used to turn these observations into product improvements rather than hiding them.
Next benchmark
Future runs will use identical application starting states and equal acceptance criteria across both branches. Planned categories include authentication/security, database changes, CI/CD, and Kubernetes/infrastructure.