spec tracking

Did we ship the spec? Spec tracking for agencies and delivery teams

The most expensive surprise in client work is the missing feature found at the demo. Here's how tracking every pull request against the agreed spec turns “are we done?” into a scorecard.

Drafted with AI assistance and reviewed by the CodeReviewer team. How we use AI

If you run delivery for clients, you know this meeting. The demo is going well. The client scrolls, nods and then asks: “Where’s the bulk export for the accountants?”

It was in the statement of work. Nobody built it. Nobody meant to skip it. It sat on page four of a document that everyone read at kickoff and nobody reopened. Now it’s a schedule slip, an awkward email and a small dent in the relationship.

This isn’t a talent problem. It’s a visibility problem, and it gets worse the more projects you run at once.

Where the gap comes from

Scope and work live in different places:

  • The scope is a document: a spec, a statement of work, a requirements list. It’s written once, agreed and mostly left alone.
  • The work is in pull requests, spread across repositories and developers, each describing a slice of a feature in the author’s own words.

Nothing connects the two automatically. Project managers bridge the gap by hand, with spreadsheets, stand-up questions and “is this done?” messages. That works on one small project. It fails across five clients, three squads and a mix of permanent staff and contractors.

Requirements slip through in predictable ways:

  • Half-done. The feature shipped without one of its parts, such as refunds on the statement or tax lines for one country.
  • Forgotten. A requirement nobody picked up, usually a small or dull one.
  • Drifted. The team built something close to the requirement, but not quite what was agreed.

Put the spec where the work happens

CodeReviewer’s spec tracking connects the document to the pull requests.

1. Upload the spec. Add the agreed specification to the project as markdown. CodeReviewer splits it into individual requirements and classifies each as a must, should or may, so priorities are explicit.

2. Every pull request is checked against it. As part of review, CodeReviewer works out which requirements a pull request advances, and whether it contradicts any of them. Scope issues show up as findings like any other: “the spec says exports include refunds; this one doesn’t.”

3. The project keeps score. The scope page shows progress against the spec: which requirements have shipped, which are partly done and which nobody has touched. Each requirement links to the pull requests that delivered it, so you have the evidence.

The change we see in delivery teams is small but useful. Before a milestone review, the delivery lead doesn’t call a meeting to ask what’s finished. They open the scorecard.

The open requirements are right there. The half-done ones say what’s missing. The forgotten ones are visible before the client sees them, while there’s still time to schedule the work or agree a change of scope.

After delivery, the same scorecard becomes an asset. When a client asks “did we get everything we paid for?”, you can show them each requirement next to the pull request that delivered it.

Access that fits agency teams

Agency teams are rarely just employees. They include contractors, client-side developers and specialists who move between accounts. CodeReviewer handles that with per-project membership: people join only the projects they work on. Admins control what members can do, such as merging pull requests, applying AI fixes, triggering AI runs or managing specs and test suites. A client’s developer can follow the findings on their own project without seeing your other clients or merging anything.

Consistent quality across squads

Spec tracking tells you what shipped. Review determines how well. In a multi-squad agency, review quality usually depends on which senior engineer is free that day. Putting every pull request through the same whole-codebase review first gives each client the same baseline, and it frees your senior engineers for design questions and client conversations.

Start with the next engagement

You don’t need to change your process to try this. Take the spec for your next milestone, upload it and let the scorecard fill in as pull requests land. The first time it flags a forgotten requirement two weeks before the demo, it will have paid for itself.

Give every PR the review it deserves.

Start your 15-day free trial. We onboard a few teams every week.