AI Can Write the Code. Who Verifies It?

Automated Software Verification in the Age of AI Coding Agents

We are living in peak AI, with more and more developers shipping code without actually writing (or sometimes reviewing) it themselves. According to a 2026 study by Faros, which analyzed two years of telemetry from 22,000 developers across 4,000 teams, engineering throughput is increasing, while bugs, rework, review load, and production incidents are rising even faster in organizations with high AI adoption.

The numbers are striking: Faros reported 54% more bugs per developer, a roughly fivefold increase in median PR review time, and more than three times as many incidents per PR when comparing periods of lower and higher AI adoption. These are correlations, not proof that AI caused every one of those outcomes. A subsequent Q3 report also found that some of the earlier effects were easing, while verification, remediation, and operational costs continued to pile up downstream.

We need to slow down enough to understand, and improve, what we are doing here. One cannot expect a small team to meaningfully review enormous AI-generated diffs. At the speed at which AI can now push out code, even ordinary review practices are struggling to keep up.

So how do we preserve the speed of AI-assisted development without sacrificing software quality, reliability, and maintainability?

Simple: automated software verification. With the same clever tools, skills, and harnesses we use as AI engineers to push code at breakneck speed, we can build elaborate testing and verification automations.

Automated software verification that builds justified trust and confidence is the way forward. Not because any system can guarantee bug-free software, but because we need more ways to catch problems before they reach users.

What’s available right now?

There are a few concepts circling around at the moment.

Superpowers

One concept that has caught on is Superpowers, an agentic development methodology and collection of skills that works with coding agents and supported harnesses. It encourages your agent to behave more like a disciplined software engineer instead of immediately jumping into code.

For example, it can help refine what is actually being built instead of just assuming things. It emphasizes planning, test-driven development, debugging, and reviews, including having separate agents review the work.

This is a great start. However, depending on the task, following the whole methodology can slow down development, and it still doesn’t replace the broader checks needed to gain confidence in our code’s correctness and security. No plugin alone makes software bug-free or vulnerability-free.

RoboRev

Another interesting framework, RoboRev, comes from Wes McKinney, the developer behind the widely used pandas library. RoboRev brings continuous AI code review into the local development loop.

The idea is compelling: just like in-person peer reviews, we can have another LLM review code that our first LLM wrote. Having a separate reviewer, potentially using a different model, may catch mistakes or architectural problems that the authoring agent missed. It is another perspective, though not a guarantee of an unbiased or correct review.

In a typical RoboRev setup, you commit your code and a local daemon automatically reviews the commit using a configured coding agent. It surfaces findings that you can feed back into your current LLM and coding harness, fixing issues while the context is still fresh.

RoboRev also supports follow-up review workflows and targeted review tasks. The appeal is that verification becomes part of the coding loop, not something you remember to do at the very end. Its documentation covers the available commands and integrations.

But the concept of code review is not new. In 2026, providers such as Claude, Codex, GitHub Copilot, Cursor, Greptile, CodeRabbit, and Qodo offer AI-assisted code reviews.

Explaining all of them would put even the most terminally online engineer to sleep, so instead let us focus on the main ones.

Claude Code Review

Anthropic’s hosted Code Review is a heavy-duty PR reviewer. When triggered on a GitHub pull request, it sends multiple specialized agents to investigate changes in parallel and look for logic errors, security vulnerabilities, regressions, and missed edge cases, among other things.

The goal is a deeper review that may take more time and cost more than a quick local pass. As of October 2026, Anthropic lists this hosted service as a research preview for Team and Enterprise plans, with separate usage-based billing.

There is also an Anthropic-maintained /code-review plugin for Claude Code. It runs a review from your coding workflow and can post findings to a PR. It is a different workflow from the hosted review service, so I wouldn’t treat them as identical in depth, setup, or pricing. Check your plan and usage terms before assuming the review is included.

OpenAI Codex Review

Codex code review takes a similar approach to reviewing pull requests against the repository’s context and instructions. You can request a review with @codex review on a connected GitHub PR or configure automatic reviews. Codex also supports reviewing local changes before a PR exists.

You can be more descriptive about what you want checked: a diff, a PR, security concerns, or compliance with the specification. If the review finds a problem, you can ask Codex to prepare a fix, inspect the resulting change, and repeat the review. That gives you a useful review-and-repair loop, although review and implementation still need independent verification.

GitHub Copilot Code Review

GitHub has one of the simplest, most hands-free approaches. You can configure Copilot Code Review to automatically review PRs, follow repository instructions, and suggest fixes that can be applied or passed to the Copilot coding agent.

As seamless as it is, it introduces problems if you start blindly accepting its feedback. Copilot normally leaves comments rather than an approving review, though GitHub supports configuring approvals in some setups. Either way, automated review is not the same as a safe merge decision.

A tradeoff is that the review experience is more managed than a DIY agent workflow: you can customize its instructions, but you don’t have the same direct, per-run model configuration you get when invoking your own coding agent.

Cursor Bugbot

Cursor Bugbot automatically reviews GitHub PRs, looking for logic bugs and other problems, and lets you act on the findings in Cursor. Its Autofix feature can also propose fixes through cloud agents.

Its biggest advantage is convenience for teams already using Cursor. The tradeoff is that much of the model selection and review orchestration is handled by Cursor rather than exposed as a fully customizable workflow. Cursor says it uses a combination of frontier and in-house models.

One reviewer isn’t enough

I like having a strong coding model write the code and a separate strong model review it. Different models can bring different perspectives, but model diversity isn’t magic: two agents can agree on the same wrong answer, overlook the same requirement, or confidently flag a bug that doesn’t exist.

The practical takeaway is to combine AI review with checks that don’t rely on an LLM’s judgment. And ideally, keep a human accountable for the final decision, especially on high-risk changes.

Okay, so we can review code. But there is much more we can do to increase our confidence that the code meets spec, doesn’t introduce bugs or vulnerabilities, and doesn’t accidentally expose secrets.

That’s what we talk about next.

The CI/CD pipeline in the age of AI and automated software development

Here is a comprehensive verification pipeline, built as a superset of what is possible. Each step can add confidence that the code works as intended, though not all teams will need every check, and no set of checks can prove software is perfect. The broader philosophy is well established in software engineering and aligns with frameworks such as NIST’s Secure Software Development Framework.

Not a fan of walls of text? There’s a diagram of the whole pipeline at the end of this section.

Code quality & build checks

• Linter checks: Check code for style violations and common mistakes.
• Formatter checks: Make sure formatting, such as indentation or spacing, follows our rules.
• Type checking: If using a typed language such as TypeScript, verify that types are used correctly.
• Compilation/build checks: Make sure the application compiles or builds successfully.

Automated tests

• Unit tests: Test whether individual units of functionality work as intended.
• Integration tests: Test whether multiple components work together, such as a payment flow that touches an API, database, and payment provider.
• End-to-end (E2E) tests: Verify important user workflows across the application from start to finish.

Security & dependency checks

• Static application security testing (SAST): Scan source code for patterns associated with potential security vulnerabilities.
• Secret scanning: Look for passwords, keys, or sensitive credentials accidentally included in the repository.
• Dependency vulnerability scanning: Find known vulnerabilities in package dependencies.
• Dependency freshness and support status: Identify outdated or unsupported packages that may need attention.

Agentic code review

• General bug scanning: Use AI agents to find and flag likely bugs.
• Security-focused review: Use AI agents to investigate potential security risks beyond what a rules-based scanner might catch.
• Review against spec: Check whether the latest PR or feature branch implements the requested behavior, including important edge cases.

Infrastructure & database checks

• Container and infrastructure configuration scanning: Look for insecure settings or misconfigurations in containers and cloud infrastructure.
• Database migration validation: Assess whether schema changes are safe to deploy, might cause data loss or downtime, or make rollback difficult.

Architecture & maintainability review

• Bloat, complexity, and architecture simplification checks: Use AI agents to identify unnecessary code, overengineering, excessive complexity, and opportunities to simplify the implementation.

Other checks worth remembering

Depending on the application, add performance and load testing, accessibility checks, privacy and compliance checks, and software license / supply-chain checks. A secure deployment can still be slow, inaccessible, or mishandle data. These checks don’t have to run on every commit, but they should have a place in the overall process.

Putting it all together, how can we configure these checks?

Most of the verification patterns we’ve discussed have been established practices for decades, and major platforms already provide tools to automate them. GitHub Actions, GitLab CI/CD, and Jenkins are common ways to orchestrate them.

Take GitHub Actions, for example. Developers define automated workflows in YAML files stored within their repositories. These workflows can be triggered by pushing code, opening a pull request, or merging changes. GitHub then starts an execution environment (a runner) and executes the workflow steps.

These steps often involve installing dependencies and running deterministic checks such as linters, tests, and security scanners. Increasingly, they can also invoke AI agents for more sophisticated, nondeterministic reviews. See GitHub’s Actions documentation and the official Codex GitHub Action.

A simplified workflow might look like this, in .github/workflows/verification.yml:

name: Code Verification

on:
  pull_request:

permissions:
  contents: read

jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - run: ./ci/quality.sh

  tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - run: ./ci/tests.sh

  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - run: ./ci/security.sh

  infrastructure:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - run: ./ci/infrastructure.sh

  agent-review:
    # This example skips external forks and Dependabot-triggered runs;
    # bot reviews need separate setup.
    if: github.event.pull_request.head.repo.full_name == github.repository && github.actor != 'dependabot[bot]'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
        with:
          fetch-depth: 0
          persist-credentials: false

      - uses: openai/codex-action@v1
        with:
          openai-api-key: ${{ secrets.OPENAI_API_KEY }}
          permission-profile: ':read-only'
          prompt: |
            Review the changes in this PR relative to
            base commit ${{ github.event.pull_request.base.sha }}.
            Use git diff to inspect the changes.

            Check for:
            1. Logic bugs and missed edge cases
            2. Security vulnerabilities
            3. Compliance with the project specification
            4. Unnecessary complexity or overengineering

            Report actionable findings with file paths
            and line numbers. Do not modify files.

This is an illustrative workflow, not a drop-in CI configuration. The ./ci/*.sh scripts are placeholders for scripts you would write in your repository. The agent job runs a review but does not automatically post the findings back to the PR; that requires an additional reporting step. In a real project, also pin third-party actions to trusted commit SHAs, restrict permissions, and carefully separate code from untrusted contributors and workflows that can access API keys. GitHub and OpenAI both publish guidance on secure PR workflows and Codex Action security.

Note that with an agentic review, you can configure your own LLM provider and credentials in GitHub Actions. The official Codex Action uses an OpenAI API key; using another provider would require that provider’s corresponding integration.

However, running this whole pipeline for every PR can get expensive, especially for teams practicing continuous integration with many PRs throughout the day. Some checks and agentic reviews take minutes rather than seconds, and the cost of the AI calls can add up.

It is much faster and more convenient to run cheap checks early and save the heavier ones for later stages or higher-risk changes.

This is where the pre-commit, post-commit/push, pull request, pre-production, and production stages come in.

Ideally, in a mature testing workflow, we categorize the checks by stage and by the risk of the change.

Still text-avoidant? Another diagram is waiting for you at the end of this section.

Pre-commit

This is the stage before code is committed locally.

What should happen here: Formatting, linting, type checking, focused unit tests, build checks when fast enough, and secret scanning. Some of these can run through Git hooks; others can run directly in your editor or coding-agent harness.

Post-commit / push

This is the stage after code is committed and pushed to a remote repository.

What should happen here: Broader unit and integration tests, SAST, and dependency vulnerability scanning. Local post-commit review tools can also give fast feedback after each commit.

Pull request (PR)

This is where proposed changes are reviewed and verified before merging into the main branch.

What should happen here: Relevant automated tests, agentic bug scanning, agentic security reviews, specification compliance reviews, and architecture/complexity assessments. More comprehensive checks should be reserved for higher-risk changes. Require key checks to pass before merge using repository rulesets.

Pre-production / deployment

This is where code is verified before being released into the production environment.

What should happen here: Full integration and E2E tests where appropriate, infrastructure configuration scanning, database migration validation, deployment smoke tests, and final security checks. For risky releases, consider staging or canary deployments and a tested rollback plan.

Production / post-deployment

This is the stage after code has been deployed and is running in production.

What should happen here: Application health checks, error monitoring, performance monitoring, security monitoring, smoke tests, anomaly detection, and automated rollback when it is safe to do so.

How big tech runs AI software factories

Some large enterprises have been tinkering with this topic for years now, as they have the talent to pull it off and the deep pockets to tokenmax. As Adam Faik lays out in How to build an AI-native software factory, Uber, Shopify, DoorDash, Spotify, and Stripe all offer a glimpse into where the state of the art lives right now in this constantly evolving software factory lifecycle.

In short, teams generally get a gateway or front door that routes their requests, those requests are qualified through proper permissions and governance, then pushed through a largely automated pipeline. This pipeline has access to the code, company context, and internal tools it needs to research the request, land on a spec, propose a plan, build the thing, test the thing, and have proof to show for it. Diffs, screenshots, tests, automated reviews, and a whole slew of analytics, which are then reviewed by developers before code is actually merged or pushed to production.

In other words, developers increasingly focus on coming up with the problem to be solved, providing some input on how that might happen, and letting the agentic workflow handle much of the implementation and verification. Once done, developers serve as glorified reviewers and final gatekeepers before the PR merge or production push. Of course, the degree of autonomy varies between companies, and humans are still involved in important decisions along the way.

There is a lot more to this process, particularly around security, isolated environments, verification, and costs, and many details I’ve left out, but I think Adam’s article is a fascinating read for anyone curious about the different ways these AI software factories are configured and run at scale.

The practical agentic software development workflow I use

So, for those of you who have gotten this far, allow me to walk through one PR to bring the case home.

A diagram summarizing my full pipeline follows at the end of this section.

Say I have a new request to create a profile page for my app so users can see and update their profiles.

Using a strong coding model (often Claude in my setup) I explain in as much detail as I can muster what the profile page should do and look like.

My agentic workflow returns a spec. We go back and forth a couple of times, I review it carefully, and we update it until I’m satisfied. I then ask the agent to create an implementation plan broken into strategic, practical, sequential steps that can logically be divided into commits.

I also ask it to run the plan by a second top-tier model, such as Codex, to see if anything was missed or could be improved. The final plan includes the unit, integration, and end-to-end tests we intend to build.

I review the plan, iterate on it if needed, and approve it.

At that point, my main coding agent starts working on the plan, step by step or commit by commit. After each step, it asks another strong model to review the changes.

If the change is high-stakes, for example if it touches payments or sensitive production flows, I also review each commit in the meantime. Otherwise, I may wait until the whole feature is finished before doing my deep human review.

Once the coding agent gets suggestions for improvement, either from the second LLM or from me, it incorporates those changes and moves on to the next step.

This continues commit by commit, with changes to the frontend, backend, or any other code necessary. Along the way, the agent runs unit tests, integration tests, and E2E tests as appropriate. For the software I build and maintain, many of these tests are relatively cheap and fast to run. If anything fails, it gets flagged, and we correct it.

Finally, when we get to the last commit, I ask another model to review the entire change—all commits and all diffs—against the original spec and flag anything that was missed. If something is flagged, we investigate and fix it. Otherwise, we move on.

Now we are at the PR stage. We’ve implemented, reviewed, and re-reviewed the code against the spec and run tests along the way to ensure everything still works.

I then review the full diff myself, giving it attention commensurate with the gravity of the change. The more important the change, the more careful the review. I also run the other automated checks: SAST, dependency scanning, agentic bug review, and agentic security review. Some of these use AI agents; others use deterministic tools.

Finally, I ask the agent that built the feature to come up with short and simple manual tests I can do to confirm that the change works.

I launch the code locally or in a test environment, run those checks, and deploy to production if everything passes.

If the change is very complex or touches sensitive data, I may also run a limited, safe set of smoke tests in production, without exposing real user data or making destructive changes.

This flow allows me and my team to ship code quickly while building confidence that what we ship meets spec and has gone through meaningful testing and security checks. Not perfect confidence (software is never guaranteed bug-free or impossible to hack), but far more confidence than blindly merging whatever an agent generates.

And this is really the point: if AI makes writing code cheap, verification has to become a first-class part of how we build software.

If you’re curious about how I run this lightweight workflow harness, let me know and I’d be happy to share it with you.

Until next time, may your tests pass and your context windows never run out.

~Mihai

---

Sources and references

The sources below support the research findings, product capabilities, and CI/CD/security guidance mentioned above. Product features, plan availability, and pricing can change.

1. Faros — The Acceleration Whiplash (Q2 2026): 2026 AI engineering report. Study scope, throughput, review load, bugs, and incidents.
2. Faros — The Speed Trap (Q3 2026): Latest report takeaways. Nuance on later AI-adoption patterns and downstream costs.
3. Superpowers (Jesse Vincent / obra): GitHub repository. Skills-based development workflow, planning, TDD, debugging, reviews.
4. RoboRev: Official website and documentation. Local continuous code review, Git hooks, review lifecycle.
5. Wes McKinney: GitHub profile. pandas creator and Kenn Software founder.
6. Anthropic: Set up Code Review for Claude Code. Hosted review architecture, eligibility, pricing, and triggers.
7. Anthropic: Claude Code /code-review plugin. Plugin features and workflow.
8. OpenAI: Review GitHub pull requests with Codex and Codex PR review help. PR reviews, prompts, and follow-up actions.
9. GitHub: Using GitHub Copilot code review. Automatic reviews, suggested changes, review status, and customization.
10. Cursor: Bugbot and Bugbot Autofix. PR review and assisted fixes.
11. NIST: Secure Software Development Framework (SSDF), SP 800-218. Security practices across the software-development lifecycle.
12. GitHub: GitHub Actions documentation, repository rulesets, and secure pull_request_target usage. Workflows, status checks, and trust boundaries.
13. OpenAI: Codex GitHub Action and security guidance. Action configuration, read-only permissions, and secret-handling cautions.
14. Adam Faik, The AI Thinker: How to build an AI-native software factory (October 2026). How Uber, Shopify, DoorDash, Spotify, Stripe and others run coding agents at scale, from intake to merge.