AI automation for security teams: what not to auto-remediate

Security teams have more alerts than they can investigate, which makes automation obviously necessary and makes automated response obviously tempting. The second does not follow from the first, and conflating them is how a team ends up with an automation that disabled a production account at two in the morning because of a false positive.

Triage and enrichment are where automation returns the most for the least risk. Response is where it needs the most care, and the distinction between containing something and remediating it is the one worth holding onto.

What we'll cover

• Alert volume is the starting condition
• Enrichment before decisions
• Triage that a person can check
• Building it with CodeWords
• Containment and remediation are different
• What must keep a human in the path
• The work around an incident
• Automating the security team's own obligations
• The alerts worth deleting before automating anything
• Making it survive
• Limits worth knowing about
• What to build first
• Frequently asked questions

Alert volume is the starting condition

Most alerts are not incidents, and in most environments the great majority are not.

The cost is attention, not compute, and attention is the constrained resource.

Tuning is the first intervention, before automation — an alert nobody has ever acted on should be removed rather than processed faster.

Count actions per alert type, over a quarter, which is the number that tells you what to delete.

Volume hides the real one. The purpose of reducing volume is not efficiency, it is that the important alert is currently invisible.

Automating triage on an untuned stream scales the noise rather than removing it.

Enrichment before decisions

The highest-value automation in this function, and the lowest-risk.

Attach the asset context: what this machine is, who uses it, what it runs, how critical it is.

Attach the identity context: whose account, what role, what they normally do, whether they are on leave.

Attach the history: has this fired before, what was it last time, what was the resolution.

Attach the external context: reputation data, known indicators, recent advisories.

Present it as a brief, not as a dump, since an analyst reading eight bullet points decides faster than one reading a JSON blob.

This changes nothing and saves most of the time, which is why it is the right place to start.

Triage that a person can check

Classify and rank, do not close, at least until the classification has been measured.

Show the confidence and the reasoning, so an analyst can disagree with a specific step rather than the conclusion.

Route by type to whoever knows that area, rather than to a shared queue.

Measure the false negative rate deliberately by sampling what was deprioritised, since that is the number nobody collects and the one that matters.

Let analysts correct it, and treat those corrections as the primary feedback.

Auto-closing requires evidence, meaning a measured error rate on a sample, not a general impression that it is working.

Building it with CodeWords

CodeWords connects to more than 3,000 integrations, and the connection is made once and reused.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: which alerts, what context to gather, how it is ranked, and what requires approval.
3. Authorize the connections — the detection tools, the asset and identity sources, the ticketing system and your chat.
4. Describe the exceptions: an executive account, a production system, a detection with a known false positive pattern, an alert during a change window.
5. Run it in enrichment-only mode for a fortnight and measure how much analyst time it returns.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

Containment and remediation are different

Containment limits damage and is usually reversible: isolating a host, revoking a session, blocking an address temporarily.

Remediation changes state and frequently is not: deleting files, rebuilding a machine, disabling an account permanently.

Automate containment selectively, with a bounded blast radius and an expiry.

Do not automate remediation, which destroys evidence and is the action you cannot take back.

Time-box every automated containment, so a false positive resolves itself rather than requiring somebody at three in the morning.

Preserve evidence first. An automated action that wipes the artefact you needed for the investigation is a net loss even when the threat was real.

What must keep a human in the path

Anything affecting production availability, since an automated containment that takes a service down is an outage you caused.

Executive and privileged accounts, which are both high-value targets and high-cost false positives.

Anything involving a person — an insider concern, an HR matter — which has process requirements beyond the technical.

Actions with legal or contractual implications, including notifying third parties.

Anything irreversible, as a general rule.

Escalations to law enforcement or regulators, which follow a defined process and a defined authority.

The work around an incident

Opening it properly: the channel, the record, the right people, the timeline started.

Assembling the evidence that responders always need, before they ask for it.

Recording the timeline from what actually happened, since reconstructing it afterwards is unreliable.

Tracking the actions to completion, which is where post-incident work quietly fails.

Notification obligations have clocks attached in several regimes, and a reminder is worth more than a policy document.

The report, assembled from the record rather than written from memory a fortnight later.

Automating the security team's own obligations

Access reviews, which are periodic, tedious and frequently done badly under time pressure.

Certificate and secret expiry, which fails eventually if it depends on memory.

Vulnerability triage feeding, so the prioritised list arrives rather than being assembled.

Control evidence collection for audits, gathered continuously rather than in the fortnight before.

Policy exception tracking, with expiry, since an exception without one becomes permanent.

Onboarding and offboarding verification, checking that access actually changed rather than that a ticket was closed.

The alerts worth deleting before automating anything

Tuning precedes automation and is skipped because deleting a detection feels like reducing coverage. It usually is not.

Count actions per detection over a quarter. A detection that has never produced an action is producing nothing but load.

Find the detections everybody suppresses individually, which is deletion happening informally and inconsistently.

Look at what fires during deployments, which is frequently a detection reacting to normal change.

Check the ones that fire constantly on one asset, since a single noisy source often accounts for a large share of total volume.

Delete rather than filter where the decision is that nobody should see it, because a filter is the same outcome hidden in a config file.

Record what was removed and why, so a future incident can be checked against it rather than producing an argument about whether coverage was reduced.

Making it survive

Fail closed on decisions, open on enrichment. A missing context lookup should not block an alert reaching a person.

Make every action idempotent and logged with its trigger and its authorisation.

Do not run response automation on the infrastructure it may need to contain, which is the obvious trap.

Alert on the automation failing, through a separate path.

Report the outcome: alerts processed, enriched, auto-closed, escalated, and actions taken with their approver.

Limits worth knowing about

Detection quality bounds everything. Automation cannot compensate for a poorly tuned detection stack.

Classification is uncertain, and a system that hides its uncertainty will be trusted until somebody checks.

An automated responder is itself a target, since something holding broad privileges to act across an estate is worth compromising.

Evidence handling has requirements if anything might become a legal matter, and an automated action can compromise it.

Automation does not add expertise, and a team without the skills to investigate will investigate faster and no better.

What to build first

Enrichment. Every alert arriving with the asset, the identity, the history and the external context attached, presented as a short brief. It changes no decision, it carries no risk, and it removes most of the time analysts currently spend gathering context before they can even begin to judge.

Two habits make the difference. Separate containment from remediation and never automate the second, since remediation destroys the evidence you need and is the action you cannot take back. And measure the false negative rate by sampling what was deprioritised, because everybody measures the alerts they investigated and nobody measures the ones they dismissed.

Frequently asked questions

Should security alert triage be automated?

Enrichment and ranking, yes. Auto-closing requires a measured error rate on a sample rather than an impression that it is working, and the stream should be tuned first — processing noise faster does not remove it.

What is the highest-value security automation?

Enrichment: every alert arriving with asset, identity, history and external context attached as a short brief. It changes no decision and returns most of the analyst time currently spent gathering context.

What is the difference between containment and remediation?

Containment limits damage and is usually reversible — isolating a host, revoking a session. Remediation changes state and frequently is not — deleting files, rebuilding a machine. Automate the first selectively; do not automate the second.

Why time-box automated containment?

So a false positive resolves itself rather than requiring somebody at three in the morning. An expiry on every automated containment bounds the cost of being wrong.

What must keep a human in the path?

Anything affecting production availability, executive and privileged accounts, anything involving a person, actions with legal or contractual implications, anything irreversible, and escalation to regulators or law enforcement.

Which number does nobody collect?

The false negative rate. Everybody measures the alerts they investigated; almost nobody samples what was deprioritised to find out what was missed.

Is the response automation itself a risk?

Yes. Something holding broad privileges to act across the estate is worth compromising, and it should not run on infrastructure it may need to contain.

Can automation compensate for poor detection?

No. Detection quality bounds everything downstream, and automating triage on an untuned stream scales the noise rather than reducing it.

Should I tune detections before automating triage?

Yes. A detection that has produced no action in a quarter is pure load, and automating triage over an untuned stream scales the noise. Count actions per detection — the list of candidates for deletion is usually short and obvious.

Is deleting a detection reducing coverage?

Not if nobody was acting on it. Record what was removed and why, so a future incident can be checked against the list rather than producing an argument.

Does a noisy detection stack justify hiring instead?

It justifies tuning first. Adding analysts to an untuned stream buys more capacity to process noise, and the important alert stays invisible either way.

Should automated actions be logged differently from analyst actions?

They should be logged the same way and additionally record what authorised them. During an investigation, "what changed and who changed it" needs one answer covering both.

Related reading

• Automating security vulnerability scanning
• Automating Slack alerts from monitoring tools
• Workflow automation for DevOps