Automating vulnerability scanning: the triage problem

Turning on a scanner takes an afternoon and produces four hundred findings. Six weeks later the number is six hundred, nobody has fixed anything, and the security dashboard has become a thing people close rather than a thing people use.

The scanning was never the hard part. The hard part is triage: deciding which of those findings can actually hurt you, in what order, and how to stop the queue growing faster than it drains. Automating that decision — rather than automating the scan — is what makes the difference between a security programme and a dashboard.

What we'll cover

• Why scanners produce unusable output
• Reachability changes everything
• Severity in your context, not the advisory's
• Building it with CodeWords
• Stopping the backlog growing
• Secrets, which are a different problem
• Routing a finding to someone who can fix it
• Scanning the things people forget to scan
• Making it survive
• Limits worth knowing about
• What to fix first
• Frequently asked questions

Why scanners produce unusable output

Transitive dependencies dominate. Most findings are in packages you never chose, several levels down, and often in code paths you never execute.

Severity is context-free. A rating is assigned assuming the worst deployment, which is rarely yours.

Development dependencies are counted alongside production ones, and a vulnerability in a build-time tool is a genuinely different risk.

The same issue appears repeatedly across services, branches and images, arriving as separate findings.

Nothing expires. A finding raised eight months ago sits alongside one raised yesterday with equal weight.

So the number grows monotonically, and a number that only grows is one people stop looking at — which is the actual security failure.

Reachability changes everything

The single most useful filter, and the one most scanners do not apply by default.

Is the vulnerable function called from your code, at all, through any path? Frequently it is not.

Is the vulnerable code path in production, as opposed to a test helper or a CLI you do not ship?

Is the input attacker-controlled? A parsing vulnerability reached only with your own configuration file is a different proposition.

Reachability analysis is imperfect — dynamic dispatch, reflection and configuration-driven calls defeat it — so treat it as strong ranking evidence rather than as proof of safety.

Unreachable does not mean ignore. It means a lower tier with a slower clock, since a refactor can make it reachable without anybody noticing.

It routinely removes most of the queue, which is what makes the remainder actionable.

Severity in your context, not the advisory's

Is the service internet-facing? The same library in an internal batch job is a different risk.

What data does it touch? A service handling payment or personal data raises the stakes of everything in it.

Is there a mitigating control — a gateway rule, a network boundary, an authentication requirement — that changes the practical exploitability?

Is it being exploited in the wild? Known active exploitation should outrank a higher nominal score with no exploitation.

Is a fix available? A finding with no patch needs a mitigation decision rather than a place in a fix queue.

Combine these into your own priority, computed automatically, and stop sorting by the advisory's number — which is the single change that makes the backlog tractable.

Building it with CodeWords

CodeWords connects to more than 3,000 integrations, so the scanners, the service catalogue, the tracker and your channels can meet in one pipeline.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: which scanners, how findings should be scored for your context, and where each tier goes.
3. Authorize the connections — the scanners or their output, the service catalogue or ownership data, and the tracker.
4. Describe the exceptions: development-only dependencies, accepted risks with an expiry, services being decommissioned.
5. Run it against the current backlog and see how many findings survive the filters.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

Stopping the backlog growing

Block on new introductions, not on the existing backlog. Preventing new high-severity reachable findings is achievable; fixing four hundred is not.

Automate the trivial upgrades — patch-level bumps with passing tests — since a meaningful share of findings resolve with no human judgement at all.

Batch the routine ones into a weekly dependency update rather than a pull request per finding.

Give accepted risks an expiry, always, so an acceptance is revisited rather than permanent.

Set a time budget per team rather than a target number, because a queue that can only be cleared by stopping feature work will not be cleared.

Report the trend, not the total. Whether the backlog is growing or shrinking is the number that tells you if the process works.

Secrets, which are a different problem

Treat a found secret as already compromised, regardless of where it was found, because the repository history and every clone contain it.

Rotation is the fix, not removal from the code. Removing it from the latest commit changes nothing.

Scan history as well as the current tree, since the interesting ones are usually older.

Block at commit time where you can, which is far cheaper than finding it afterwards.

Alert immediately and directly, since this is the one category where the clock genuinely matters.

Expect false positives on test fixtures and example values, and tune for them rather than letting them train people to dismiss the alerts.

Routing a finding to someone who can fix it

Ownership, not a security queue. A finding in a service goes to that service's team, since they are the only people who can act on it.

Say what to do, specifically — the version to upgrade to, or the configuration change — rather than restating the advisory.

Include why this one matters in your context, which is what distinguishes it from the four hundred that were filtered out.

Group by fix, not by finding. One dependency upgrade may resolve a dozen findings, and presenting it as a dozen tickets wastes everybody's time.

Give a deadline proportionate to the tier, and escalate on that rather than on the raw severity.

Keep the security team in the loop rather than in the path, because a queue that must pass through one team is a queue.

Scanning the things people forget to scan

Dependency and container scanning get set up first and cover a fraction of the surface. The rest is usually unscanned because nobody owns it.

Infrastructure configuration. Open storage buckets, permissive security groups, and overly broad roles produce more real incidents than library vulnerabilities do.

Container base images, on a schedule rather than at build time, since an image built three months ago has accumulated findings while sitting still.

Your own code, through static analysis, which finds a genuinely different class of problem from anything a dependency scanner sees.

Third-party scripts on your web pages, which execute in your customers' browsers with your origin's privileges and change without notice.

Abandoned services. Anything still deployed, still reachable, and no longer maintained is the most likely route in, and it is invisible to a scanner pointed at active repositories.

Start with an inventory. The most common finding of this exercise is not a vulnerability but a service nobody remembered was running.

Making it survive

Deduplicate across scanners, which will overlap substantially and disagree on severity.

Handle scanner outages by reporting them, since a scanner that silently stopped looks exactly like a clean result.

Keep the finding's identity stable across runs so a re-raised finding is not treated as new.

Store the triage decision with its reason and expiry, since the same finding will be triaged repeatedly otherwise.

Report the outcome: findings ingested, filtered, routed, fixed, and accepted — with the trend on each.

Limits worth knowing about

Reachability analysis varies by language and is considerably weaker in dynamic ones.

Version detection is imperfect with vendored, patched or repackaged dependencies.

Container scanning reports the base image, so findings frequently belong to a base image update rather than to the application team.

Advisory databases disagree on severity and on affected ranges, and the differences are not small.

A scanner finds known vulnerabilities in known components, which is a genuinely useful and genuinely narrow thing — it says nothing about the security of the code you wrote.

What to fix first

The filter, not a finding. Take the current backlog, apply reachability, production-versus-development, internet exposure and fix availability, and see what remains. The filtered list is usually small enough to act on, and having it is what converts a dashboard nobody opens into a queue somebody works.

Two habits make the difference. Block new introductions while leaving the existing backlog on a slower clock, since that stops the number growing and is achievable this quarter. And put an expiry on every accepted risk, because an acceptance without one is a decision that quietly becomes permanent and is never revisited.

Frequently asked questions

Why do vulnerability scanners produce so many findings?

Transitive dependencies dominate, severity is rated context-free, development and production dependencies are counted together, the same issue repeats across services, and nothing ever expires. The number only grows, so people stop looking.

What is reachability and why does it matter?

Whether the vulnerable function is actually called from your code through any path. It typically removes most of the queue, which is what makes the remainder actionable.

Does unreachable mean safe to ignore?

No — it means a lower tier with a slower clock. Reachability analysis is defeated by dynamic dispatch and reflection, and a refactor can make something reachable without anybody noticing.

How should findings be prioritised?

By your context: internet exposure, the data the service touches, mitigating controls, known active exploitation, and whether a fix exists. Sorting by the advisory's score is what makes the backlog intractable.

How do I stop the backlog growing?

Block new introductions rather than the existing backlog, auto-upgrade trivial patch bumps with passing tests, batch routine updates weekly, and report the trend rather than the total.

What should happen when a secret is found?

Rotate it. Treat it as compromised regardless of where it was found, since the history and every clone contain it. Removing it from the latest commit changes nothing.

Who should a finding be routed to?

The team that owns the service, with the specific fix, the reason it matters in your context, and findings grouped by fix rather than individually. Security should be in the loop, not in the path.

Should accepted risks expire?

Always. An acceptance with no expiry is a decision that becomes permanent without anybody choosing that, and it is where the genuinely dangerous items end up.

What gets forgotten in vulnerability scanning?

Infrastructure configuration, base images on a schedule rather than at build time, static analysis of your own code, third-party scripts running in customers' browsers, and abandoned services that are still deployed and reachable.

What is usually the biggest finding?

Not a vulnerability — an inventory gap. A service nobody remembered was running, still deployed and no longer maintained, is the most likely route in and is invisible to a scanner pointed at active repositories.

Related reading

• Automating regression test triggering
• Automated code review workflows
• GitHub and CodeWords