What is shift-left testing, and what cannot move?

Shift-left testing means finding defects earlier in the process — while writing the code rather than in a testing phase afterwards. The reasoning is that a defect found immediately is cheaper to fix than one found in a release candidate, which is true and not the whole story.

The part that gets lost is that some kinds of problem genuinely cannot be found early, and a team that has internalised "test earlier" as a principle tends to either stop looking for those or pretend an early test covers them.

What we'll cover

• What the idea actually claims
• What genuinely shifts left
• What cannot move
• The pyramid, honestly
• Contract testing, the underused one
• Shifting right as well
• When it becomes developers doing everything
• Test data, which is where the effort actually goes
• Automating the feedback loop with CodeWords
• What to get right first
• Frequently asked questions

What the idea actually claims

A defect costs more the later it is found, because more has been built on top of it and the person who wrote it has moved on.

So move the finding earlier, from a dedicated phase after development into the development itself.

That is a claim about feedback latency, which is the useful framing — the value is in the gap between making a mistake and learning about it, not in any particular test type.

It also implies quality is not a phase, which is the cultural half of the idea and usually the more consequential one.

What it does not claim is that every kind of defect can be found early, though it is frequently heard that way.

What genuinely shifts left

Type errors and obvious mistakes, through types and linting, found as you type.

Unit-level logic, tested in seconds.

Contract mismatches between services, covered below and genuinely shiftable.

Security basics — dependency vulnerabilities, secrets in code, common patterns — all of which are cheap checks at commit time.

Accessibility basics, which automated checks catch a meaningful share of.

Requirements problems, which shift furthest left of all. A conversation about an edge case before implementation is the cheapest defect prevention available and is not usually thought of as testing.

What cannot move

The honest list, and each of these is a real category.

Performance under real load, which needs the real system, real data volumes, and real concurrency.

Integration with systems you do not control, where a sandbox behaves differently from production in ways that only show up in production.

Usability. A person using the thing tells you something no automated check can.

Emergent behaviour. How several services interact under failure is not visible from any one of them.

Data-dependent behaviour, where the production data has shapes that no fixture contains.

Anything involving real users — whether they understand it, whether they use it as expected, whether it solves the problem.

Pretending these shift left is how they stop being tested at all, which is the failure mode worth guarding against.

The pyramid, honestly

Many fast unit tests, fewer integration tests, few end-to-end tests is the conventional shape and it is broadly right.

The reasoning is feedback speed and stability, not a belief that unit tests are better.

End-to-end tests are slow and flaky in proportion to how much they touch, which is why few is correct rather than zero.

Few is not none. A handful covering the critical paths catches the integration failures that unit tests cannot, and teams that removed them entirely tend to reintroduce them after an incident.

Integration tests are undervalued. The layer between unit and end-to-end — a service against a real database, two components together — catches a great deal at moderate cost.

The shape should follow your architecture. A system whose complexity is in the interactions needs proportionally more integration testing than the pyramid's conventional ratios suggest.

Contract testing, the underused one

The technique that shifts the most valuable thing left, and the one fewest teams use.

Each side tests against an agreed contract: the provider asserts what it produces, the consumer asserts what it requires.

A breaking change fails the provider's own test suite, before deployment, rather than in an environment afterwards.

It removes the need for a full integration environment to catch interface mismatches, which is where a lot of end-to-end testing effort goes.

It works across teams and across deployments, which is exactly where integration problems live.

Mocks drift and contracts do not. A hand-written mock reflects what the other service did when somebody wrote it; a contract is verified against the real provider.

It requires agreement between teams, which is the reason it is underused — the technical part is straightforward and the organisational part is not.

Shifting right as well

The complement, which is not a contradiction.

Some things can only be learned in production, so learn them there deliberately rather than by accident.

Feature flags let a change reach a subset of users, with the rest unaffected.

Canary releases expose a small share of traffic and watch what happens.

Synthetic monitoring runs the critical paths against production continuously, which catches what nobody's test suite covers.

Good observability is testing in production — it is how you find out that something is wrong with real traffic and real data.

The two directions serve different purposes. Shift left for what is knowable early; shift right for what is only knowable with real users, real data, and real load.

When it becomes developers doing everything

The organisational failure, and it is common.

"Shift left" becomes "no testing specialists", and the exploratory testing nobody had time for stops happening.

Developers test what they thought of, which by definition excludes what they did not — and that is precisely what a second person finds.

Specialist skills go missing. Performance, security, and accessibility testing are disciplines, and expecting every developer to cover them produces shallow coverage of all three.

Exploratory testing has no automated equivalent. A person using the system with intent finds things no assertion was written for.

The workable version is specialists moving earlier, into design and into building the tools, rather than being removed. That is what shifting left was supposed to mean.

Test data, which is where the effort actually goes

The unglamorous obstacle, and the one that determines whether early testing is realistic.

Fixtures do not resemble production. Real data has null fields, unexpected encodings, records from a schema version three years old, and shapes nobody would invent.

Copying production is the obvious answer and carries obligations, since a test environment holding real customer records is real customer data with weaker access controls.

Anonymise properly if you do. Replace rather than mask, keep the shape so the data stays useful, map consistently so joins still work, and handle free-text fields where personal information hides.

Generate data that matches the real distribution, not the convenient one — including the awkward records rather than a tidy sample.

Keep the failures you find. Every production bug should become a fixture, which is the cheapest way a suite gets better over time.

Seed deterministically. Tests that depend on randomly generated data fail unpredictably, which is flakiness with a respectable-sounding cause.

Automating the feedback loop with CodeWords

Much of what makes early feedback useful is connective — routing failures to the right person, reporting flakiness, telling somebody when a check that normally passes stops. CodeWords connects to more than 3,000 integrations.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: which signal, and who should hear about it.
3. Describe the exceptions: a known flaky failure, a check that has stopped running.
4. Connect the systems involved once and reuse the connections.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

What to get right first

Reduce the time between writing a defect and hearing about it, which is the actual claim underneath the phrase. That usually means making the existing suite faster and less flaky rather than adding tests, because a suite people wait for provides feedback and one they work around does not.

Two habits make the difference. Name explicitly what you are not testing early — load, integration with third parties, usability — and decide where it will be covered instead, or those categories quietly stop being tested at all. And keep a small number of end-to-end tests on critical paths, since the teams who removed them entirely tend to reintroduce them shortly after an incident.

Frequently asked questions

What does shift-left actually mean?

Reducing the gap between making a mistake and finding out, by moving checks into development rather than a separate phase afterwards. It is a claim about feedback latency, not about any particular test type being superior.

What cannot be tested early?

Performance under real load, integration with systems you do not control, usability, emergent behaviour across services, and anything involving real users. Pretending these shift left is how they stop being tested at all.

Should we have end-to-end tests?

Few, on critical paths — not none. They are slow and flaky in proportion to how much they touch, which is why few is right. Teams that removed them entirely tend to reintroduce them after an incident.

What is contract testing and why does it matter?

Provider and consumer each test against an agreed contract, so a breaking change fails the provider's own suite before deployment. It catches interface mismatches without a full integration environment, and mocks drift while contracts do not.

Does shift-left mean no testers?

No, and that reading is the common organisational failure. Developers test what they thought of; a specialist finds what they did not. The workable version is specialists moving earlier, into design and tooling, rather than being removed.

Is testing in production a contradiction?

No, a complement. Some things are only knowable with real users, real data, and real load, so learn them deliberately — with flags, canaries, synthetic checks, and observability — rather than by accident.

Where should I start?

By making the existing suite faster and less flaky rather than by adding tests. A suite people wait for provides feedback; one they work around provides none, however comprehensive it is.

Why do tests pass and production break?

Test data. Fixtures are tidy and real data is not — null fields, odd encodings, records written by a schema version from years ago. Generating data that matches the real distribution, including the awkward records, closes most of that gap.

Can we copy production data into test environments?

With obligations attached, since it remains real customer data under weaker access controls. Anonymise by replacing rather than masking, keep the shape, map consistently so joins work, and handle free-text fields where personal information hides.

What is the cheapest way to improve a test suite over time?

Turn every production bug into a fixture. It costs minutes at the moment you have the full context, and it guarantees the suite gets better at exactly the things that have actually gone wrong.

Related reading

• CI/CD pipelines explained
• Canary deployment
• Automating regression test triggering