Building Change Confidence in an AI-Native SDLC

An engineer can give Claude, Cursor, Devin, Codex, or another coding agent a task and increasingly get back not just code, but tests, documentation, and a pull request.

That changes an assumption our engineering processes have been built around for decades:

Generating the change is no longer necessarily the expensive part. Establishing confidence in the change is.

I recently came across framing of engineers as "Indexers" and "Instrumenters."

An Indexer relies heavily on engineers maintaining a deep mental model of the codebase. An Instrumenter externalizes more of that knowledge into systems that provide evidence about whether software behaves correctly.

I find myself leaning toward the Instrumenter side.

But I think there's a harder question underneath it:

What exactly should we instrument for?

Passing Tests Doesn't Mean Safe to Ship

Tests are an obvious answer.

Did the unit tests pass?
Did integration tests pass?
Did the build succeed?
Did static analysis find anything?

All useful signals.

But consider two changes.

PR A modifies an internal Tier-4 service with few dependencies and limited customer impact.

PR B modifies a Tier-1 service sitting on a critical customer path, has twelve downstream consumers, and touches an area associated with two previous P1 incidents.

Both PRs could have:

all tests passing,
high test coverage,
successful builds,
clean AI reviews.

Yet they clearly don't carry the same risk.

The difference isn't necessarily visible in the code.

It's in the context surrounding the change.

And that's where I think code review needs to evolve.

The Unit of Review Should Be the Change

Most code-review systems are fundamentally diff-oriented.

Pull Request
   |
   +-- Files changed
   +-- Lines changed
   +-- Tests
   +-- Reviewer comments

This model made sense when humans wrote most of the code and reviewers could reasonably build context by reading it.

That becomes harder when agents can generate substantial changes in minutes.

Instead of asking only:

What changed in the code?

we should also be able to ask:

What does this change mean to the system?

That requires enriching the PR with context outside the repository.

Developer / AI Agent
        |
        v
      Intent
     (Jira)
        |
        v
   Pull Request
        |
        v
 Services Changed
        |
        +-- Service Tier
        +-- Ownership
        +-- Dependency Graph
        +-- Blast Radius
        +-- Incident History
        +-- Test Evidence
        +-- Deployment Strategy
        +-- Runtime Health

Now the review isn't just about a diff.

It's about a change entering a running system.

What Would That Look Like?

Imagine opening a PR and seeing:

Change Context

Services:
  checkout-api       Tier 1
  pricing-service    Tier 2

Dependency impact:
  3 upstream services
  11 downstream services

Operational history:
  2 P1 incidents in last 12 months

Change characteristics:
  API contract modification
  Database schema change

Required evidence:
  Contract tests
  Integration tests
  Canary deployment
  SLO validation during rollout

Notice what this isn't.

It isn't an AI saying:

"Risk score: 87. Do not merge."

I'm skeptical of reducing complex engineering risk to an opaque number.

Instead, the system should assemble evidence and use deterministic policies where appropriate.

That gives both humans and AI reviewers substantially better context for making decisions.

Risk Should Influence the Path

Once we understand the context of a change, the SDLC itself can become adaptive.

Today, many organizations effectively have:

Change
  ↓
Review
  ↓
Test
  ↓
Deploy

with roughly the same controls applied to every PR.

But why should every change take the same path?

A low-risk change might require:

Review → Tests → Deploy

A higher-risk change might require:

Deep Review
    ↓
Integration / Contract Tests
    ↓
Ephemeral Environment
    ↓
Canary
    ↓
SLO Validation
    ↓
Progressive Rollout

The important part is that this decision shouldn't be driven primarily by something simplistic like lines changed.

A two-line configuration change affecting traffic routing for a Tier-1 service can be far more dangerous than a 500-line isolated refactor.

The better signals are things like:

service criticality + dependency impact + type of change + operational history + available evidence

So Where Does Code Review Fit?

Code review remains extremely important.

But I don't think its job should be to provide all of our production confidence.

Review is good at answering:

Does the implementation match the intent?
Does this violate an architectural boundary?
Is an API contract changing?
Are there concurrency or failure-mode concerns?
Are security or data assumptions changing?
Is the implementation unnecessarily complex?

Some of those questions are excellent candidates for AI assistance.

But even the best reviewer can't completely determine how a distributed system will behave in production by inspecting a diff.

So confidence should accumulate across the lifecycle.

Code Review

Validate intent, implementation, architecture and obvious risk.

Pre-production

Validate actual behavior through integration tests, contract tests, ephemeral environments, performance testing, or whatever evidence the change warrants.

Deployment

Control exposure through feature flags, canaries, progressive delivery, health gates and automated rollback.

Production

Validate reality through SLOs, errors, latency, telemetry and customer-impact signals.

This gives us something closer to:

Intent
   ↓
Change
   ↓
Review Evidence
   ↓
Pre-production Evidence
   ↓
Deployment Evidence
   ↓
Production Evidence
   ↓
Learning

No individual stage needs to prove that a change is safe.

Confidence accumulates.

Incident History Is an Underused Engineering Signal

This is the part I find particularly interesting.

Most incident data lives far away from the pull request.

PagerDuty knows which services caused incidents.

Datadog knows what happened at runtime.

Postmortems describe failure modes.

The service catalog knows criticality and ownership.

The dependency graph knows blast radius.

GitHub knows what is changing.

Jira knows why.

But the engineer reviewing the next change often sees only a fraction of this information.

Imagine instead that a reviewer sees:

This PR modifies the retry mechanism in checkout-api. The same component was involved in two previous P1 incidents caused by retry amplification.

That immediately changes the review conversation.

More importantly, it might automatically change the required evidence:

Previous incident pattern detected
          ↓
Failure-mode integration test
          ↓
Canary required
          ↓
Stricter SLO gate

Now incident management is no longer purely retrospective.

Production history becomes an input into future engineering decisions.

Close the Loop

This leads to what I think is the more interesting architecture.

AI Agent / Developer
        |
        v
      Intent
        |
        v
    Pull Request
        |
        v
   Change Context
        |
        v
Required Evidence
        |
        v
      Deploy
        |
        v
Production Signals
        |
        v
Incident / Success
        |
        +--------------------+
                             |
                             v
                       Future Changes

Suppose a particular type of database migration repeatedly causes incidents.

Future changes matching that pattern should automatically require stronger evidence.

Conversely, suppose thousands of changes of a particular type deploy successfully with the same controls.

Perhaps some unnecessary friction can eventually be removed.

The system starts learning not from whether code was generated by AI, but from what actually happened when changes reached production.

That is a much more useful feedback loop.

AI Makes This More Necessary — and More Possible

None of the individual pieces here are new.

We already have:

GitHub for changes,
Jira for intent,
service catalogs for ownership,
dependency graphs for topology,
CI systems for test evidence,
PagerDuty for incidents,
observability platforms for runtime health,
deployment platforms for progressive delivery.

The problem is that they mostly operate as separate systems.

AI makes this interesting for two reasons.

First, AI increases the throughput of changes, making human attention increasingly scarce.

Second, AI is very good at synthesizing context across systems.

The opportunity isn't another dashboard where engineers need to manually correlate ten tools.

It's bringing the relevant evidence to the point where the decision is being made:

the change itself.

Human Review Becomes Scarcer — and More Valuable

This doesn't mean removing engineers from code review.

Quite the opposite.

If AI-generated changes increase significantly, human attention becomes one of the most expensive resources in the SDLC.

We should spend it deliberately.

An engineer probably shouldn't spend twenty minutes verifying boilerplate that deterministic systems can validate.

But that same engineer absolutely should spend twenty minutes questioning an architectural change affecting a critical customer path with a history of incidents.

The goal isn't:

AI replaces human review.

It's:

Use instrumentation to determine where human judgment matters most.

From Code Confidence to Change Confidence

For years, engineering systems have optimized heavily around validating code.

But what enters production isn't just code.

It's a change to a running system with dependencies, customers, historical failure modes and operational consequences.

As AI makes generating that change increasingly inexpensive, I think the scarce resource shifts.

It becomes evidence that the change can be trusted.

So perhaps the question we ask before merging shouldn't only be:

Has someone reviewed this code?

It should increasingly become:

What evidence do we have that this change is safe to ship?

That shift—from code review to change confidence—may become one of the more important changes in how we design developer platforms for an AI-native SDLC.

Beyond Code Review