We Drive a Ferrari and Carry It Every 300 Metres

Context: an enterprise AI-agent platform at a large German telecom IT provider, plus our own product, 37lab Core.

Tool: semantic-diff — open source, MIT, pip install semantic-diff

The problem nobody puts on the slide

Recent changes in AI-augmented software development give me a strong vibe of 1960s science fiction: we can do fantastic things, but sometimes with the most ridiculous retro artifacts. For us, that artifact was the plain merge request review — the basic seat belt you still need while trying to reach escape velocity.

Our loop used to look like this: open a merge request, ask a reviewer, sit back and watch the grass grow. The reviewer runs their own agent, the agent writes comments, and we read them — or ask our agent to read them. Two AI agents talking to each other through human middleware (not flattering, I know).

The car is fast. But every 300 metres you have to get out and carry it.

What we built

semantic-diff started small: show the intent behind a commit, not just the lines, and flag what could be dangerous. Early on it caught the boring, expensive things — plain IDORs among them.

Then the work changed. Merge requests on the platform grew larger, spanned many commits and had to clear compliance and security requirements, not only "does it work". The tool grew with them. Today the name undersells it — it started as a semantic diff and outgrew it:

• Merge requests as a whole, merge-base to head — the same diff the reviewer sees, with the MR title and description as stated intent.
• Agentic review: an agent reads around the change — whole functions, callers, the file before the change, history — with read-only tools over git objects, never the working tree.
• Every finding checked in code: does the cited line exist, and did the model actually read that file? Findings that fail are flagged in the report.
• Hard limits: turns, input tokens, diff budget. When a limit hits, the model gets one last turn to submit, and the report says what cut it short.
• Honest coverage: large diffs are packed riskiest-first; the report names every file the model did not see.
• Model-agnostic: Anthropic, or any OpenAI-compatible gateway — including the corporate LLM hubs regulated companies actually have to use.
• Lives in CI: GitLab MR pipelines, a GitHub Action, or a pre-push hook.

Why we check every citation

Because the tool is model-agnostic, and many models confabulate. And because even the best ones take liberties: a generous reading of a line, a conclusion drawn one file too early. A review finding that points at code which is not there is worse than no finding — it spends the one thing review is short of, human attention. So the model's claims are verified by code.

What changed

We moved the baseline review out of people's hands and into the CI layer. Every merge request gets it automatically, as an advisory job: it reports, it does not block.

Four-eyes review did not go away — where it is required, it stays. But the second reviewer now gets a cleaner batch: their own AI is not busy with the slips semantic-diff already caught, and can spend its attention on what needs judgement.

Dogfooding? Of course. We run it on every merge request of our own product, 37lab Core: 208 commits in eight weeks.

Why open source

Because we sell privacy architecture and compliance, and "trust us" is not an argument. The code that reviews our code is public. Read it, run it on your repository, see what it finds.