Per-engineer PR throughput has gone up a lot in the last year. Reviewer capacity hasn’t. My team has been running an experiment for the last few weeks to find out what to do about that, and we have some early results worth sharing.
A short history
For most of the first two decades of commercial software development, the primary owner of code quality was the person who wrote the code. You shipped it, you owned it. Review, when it happened at all, was informal: a colleague glancing over your shoulder, a senior engineer eyeballing a diff before it hit production. There was no ceremony because there was no need for one. Teams were small. Systems were legible. One person could hold most of it in their head.
Formal code review as a standard practice emerged from two distinct traditions. The first was Fagan inspection. In 1976, Michael Fagan formalized "code inspection" at IBM in a paper that’s still cited today. This was a meeting-driven defect hunt with assigned roles (moderator, author, reader, recorder) that turned up impressive bug counts and earned a reputation as one of the more cost-effective interventions in software engineering. It was also slow, ceremonial, and largely incompatible with how most teams ship now.
Open source quietly invented a second, lighter form. Asynchronous patches on mailing lists, then patches in version control, then pull requests on GitHub. The defect hunt narrowed and the social function broadened. By the mid-2010s, "two-approval PR review" was the unspoken standard at most professional engineering organizations. That standard worked because the cost of producing a change and the cost of reviewing one were roughly balanced. Both were bound by the same thing: an engineer’s attention.
What made code review stick wasn’t just defect detection. It was also ownership diffusion. A review signature says: I read this, I understood it, and I accept some share of responsibility for what it does. That’s the social contract that made team codebases legible over time and gave engineers a stake in each other’s work. Review wasn’t only about finding bugs. It was about building shared mental models of a system and distributing accountability across the people who maintain it.
Then the cost of writing code dropped to near zero. Claude Code, Copilot, Cursor, and their successors didn’t just speed up coding; they changed its economic structure. The constraint shifted. Writing is no longer the bottleneck. Review is. And a process that was already struggling to keep up with human-authored output is now being asked to handle a volume of AI-generated code that no review culture was designed for.
Why the review process hasn’t caught up
While AI assistants have made writing code dramatically cheaper, they’ve also taken on a real share of the review burden; most teams now have at least one AI reviewer commenting on every PR, and the comments are often substantive.
What hasn’t kept up is the policy layer around review. Most organizations still require comprehensive human review on every change, regardless of how many AI reviewers have already weighed in. So authoring throughput keeps climbing, AI review absorbs some of the load on the way in, and then the queue piles up at the one part of the pipeline that hasn’t adapted: the human approval requirement.
The visible symptoms are familiar: older PRs sitting in the queue for days, authors pinging reviewers to ask for a look, reviewers context-switching constantly. The more subtle failure mode is rubber-stamping. As the queue gets longer, the temptation to skim grows, and a culture of two-approval rigor can get hollowed out without anyone noticing.
Writing code is cheap. AI review is cheap. Getting a human to sign off with real attention is the new constraint, and our SDLC policies haven’t adapted to that. We ran an experiment to see whether AI-assisted risk analysis, plus a review process change, could alleviate some of this pressure and widen the review bottleneck.
What we tried: A two-part approach
The first piece of the experiment is an internal bot we call Moxly. Built on Claude Code, Moxly inspects every PR in our repo and applies one of four risk labels: risk: low, risk: medium, risk: high, or risk: critical. The label is its estimate of the blast radius (what the change touches, how broadly, and how dangerous a regression would be). It also leaves a short review comment explaining how it got there. It’s explicitly instructed to focus on production incident prevention; it is not intended to enforce style or other cosmetics. It is solely focused on changes that could cause incidents, data corruption, or service degradation. Broadly (and non-exhaustively), it’s focused on risk factors like:
-
Database query changes and migration safety
-
Impacts to shared infrastructure
-
Potential side-effects from refactoring
-
Test coverage
-
API contract changes
The second piece is a process change. On PRs labeled risk: low, a human reviewer is explicitly allowed to skip the line-by-line read and review the change from a higher conceptual/system level instead. Does the problem framing make sense? Is the approach reasonable? Does the shape of the solution fit? On anything medium or above, our existing line-by-line norm stands.
The bot on its own would not have changed anything. We already had AI review running before Moxly; what AI review couldn’t do was change human reviewer behavior. What actually changed behavior was the explicit policy update giving reviewers permission to do less when the bot said it was safe to do less.
The pilot ran across a small team over a few weeks. By the end the bot had labeled north of 97% of our PRs. Nobody manually overrode a label during the pilot. When a reviewer disagreed with the bot, they handled it inside their review, not by relabeling.
Curious about Moxly? Tune in to the latest Mews R&D Check-In on YouTube or Spotify. Prefer reading over listening? We’ve got you covered with a written version here.
What we saw
We had two questions:
-
Did the queue actually move?
-
Were reviewers genuinely thinking less, or just looking less attentive?
The queue moved. P75 time-to-first-approval dropped from about 70 hours to 44. The share of PRs approved in under 24 hours went from 54% to 63%, and the share merged in under 24 hours from 51% to 70%. The long tail (PRs older than 72 hours with no approval) was cut roughly in half, from 24% of the queue to 11%.
Those numbers are also what you’d see if everyone had just started rubber-stamping, so we ran a counter-test. We looked at the rate of "fast" approvals — PRs that received an approval within an hour of being opened, which is the timing signature of a skim-and-stamp. If reviewers were getting lazier, that rate should have gone up. It went down: from 22% to 10%. Unique human reviewers per PR was unchanged. "Changes requested" review counts were unchanged. Cycle time on medium-sized PRs (the bucket where the bot is doing the most work) was about twice as fast as before.
So the queue compressed without the worst failure mode showing up. We still couldn’t tell from git data alone whether reviewers were doing less line-by-line work or just doing the same work faster with more confidence. So we asked them, and what we found, we weren’t expecting.
The part we didn’t expect
When we polled the team, each person had quietly built their own personal validation layer on top of the bot’s label. None of them coordinated. None of them were the same.
One person skips the line-by-line on labeled PRs, as expected, but always runs a quick mental check first: can I convince myself this PR really is low risk? If yes, approve. If no, read the diff. They reported they haven’t disagreed with the bot yet, but the check itself keeps them honest.
Another doesn’t do the line-by-line either, but runs a separate local AI tool on the branch and pastes anything substantive into the PR. Usually that surfaces missing test coverage. Rarely, this additional review surfaces a real bug.
A third reads the PR description, the unresolved comments, and the bot’s review comment, and approves on those three signals. They still read carefully when a change touches something they consider integration-shaped, regardless of what the label says.
None of this was designed. Each engineer evolved their own version within the first couple of weeks. The pilot’s mechanism (a label plus a permission) did exactly what it was supposed to do. The emergent guardrail is the more interesting outcome: a team-wide culture of "low risk plus my own check" rather than "low risk equals skip."
What the bot is bad at
A few patterns showed up where the bot’s risk call was weaker than it should have been:
-
CI/CD and infrastructure. AI assessment of release pipelines, container builds, and infrastructure-as-code is meaningfully worse than its assessment of application code. The bot tends to underweight the blast radius of a release-system change because the diff itself looks small. While the risk here is not external (a failure here is a failure in our CI/CD pipelines, not production), getting this wrong still means reversions, investigations, fixes; all of the makings of an incident, just without the customer impact.
-
UI changes. The bot doesn’t see the UI. It reads the diff. UI regressions that are obvious to a human looking at a screenshot are invisible at the diff level.
-
Cross-repo standards. When the bot labels a PR low in a repo with strong but undocumented local conventions, the social cost of skipping line-by-line falls on the author, not the bot.
The common thread is integration points: places where a change has to cooperate with something outside the diff. The bot is good at the average case and weak at the edges.
Where we go from here
The pilot worked. The queue compressed. Reviewer attention got reallocated to the changes that needed it. The rubber-stamp rate went down instead of up. Each engineer built their own verification gate on top of the permission.
So, where do we go from here? We see three distinct points of progression.
-
Fix the blind spots. AI risk assessment is generally strong and trustworthy but can start to fail when needing to reason across systems or at integration points. Providing the agent with as much context as we can about the big picture, per-repo guidelines and rules, the change itself, and why it was made allows it to better reason about the change’s impact and risk and provide more accurate analysis and scores. Having all of this front-and-center helps reviewers trust the output.
-
Expand to the rest of the team. Given our positive results, expanding to the rest of our team makes sense. Expanding helps us uncover and correct more blind spots, make more process improvements and, most importantly, re-confirm that our findings hold on a larger scale.
-
Allow low-risk PRs to automatically merge. The natural progression of this process and line of thinking is to consider, in areas where our AI counterparts are generally very successful, allowing changes in those areas to merge automatically.
The broader lesson, at least so far: AI has already taken on a lot of review work, but most SDLC policies still treat human review as the only review that counts. That policy lag is what creates the queue. The cheapest defense seems to be moving the trust signal closer to the reviewer, letting AI assessment shape the kind of human review that’s required, instead of bolting human review on top of AI review as if neither knew the other existed. Telling a person "this is safe to skim" turned out to be more valuable than we expected, not because the bot was always right, but because it let each engineer build their own version of trust on top of it.
We don’t have the full answer, nor is what we have perfect. But, what we have is working, and it has bought back a meaningful chunk of attention. We’re going to keep testing, keep experimenting, and keep learning as we figure out how to navigate this new world.
