Register your interest: Tag @Cody, get an agent
BlogResources

Workflow automation for DevOps: the automation nobody owns

Automation is the job, which is why DevOps teams accumulate scripts that became infrastructure. What to automate, what that encodes, and how to stop it rotting.

Rithul PalazhiRithul Palazhi11 min read

Summarize with AI

Workflow automation for DevOps: the automation nobody owns
On this page

Every other function automates things that were previously manual. DevOps automates for a living, which produces a different problem: a layer of scripts, pipelines and jobs that were written for a reason nobody present remembers, which now constitute infrastructure and which have no owner.

The question for this team is not what to automate. It is what an automation encodes, who maintains it, and how you tell the difference between removing toil and cementing a bad process into something nobody can safely change.

What we'll cover

Automating a bad process makes it permanent

Manual steps are visible friction, and friction is what eventually prompts somebody to fix the underlying thing.

Automation removes the prompt. A painful approval chain that now runs itself is a painful approval chain forever.

The script becomes the documentation, and reading it is harder than reading the process it replaced.

Changing it gets risky once several things depend on it, so the encoded process outlives the reason for it.

Ask what the process is for before automating it, which occasionally ends with deleting the process instead.

Automate the thing that should exist, not the thing that does, where those differ and the difference is small.

Which toil is worth removing

Frequency times duration is the obvious measure and it is not enough on its own.

Interruption cost matters more. A five-minute task that arrives unpredictably costs far more than its five minutes.

Error rate counts. A manual step people get wrong occasionally is worth automating even when it is quick.

Out-of-hours work should be automated ahead of anything else of similar size, because its real cost is not in the minutes.

Work that blocks somebody else multiplies its own cost by however many people are waiting.

Leave the rare and complex alone. A quarterly task that requires judgement is a bad automation candidate and a good documentation one.

Ownership, which is the actual failure

Every automation needs a named owner, an individual, recorded where somebody can find it.

Unowned automation is the default end state, reached by the original author moving team.

Automation is production. If it runs against production systems, it deserves the same review, testing and alerting as anything else you would call a service.

Put it in version control, always, including the things that feel like scripts rather than software.

Give it an expiry or a review date, since the alternative is discovering in three years that a job has been running for two with no effect.

Track the inventory. Most teams cannot list what they have running, which is the honest starting point for this whole subject.

Building it with CodeWords

CodeWords connects to more than 3,000 integrations, and the connection is made once and reused.

  1. Open CodeWords and start a new automation.
  2. Describe what should happen in plain language to Cody, the automation builder: the trigger, the steps, and what should stop it.
  3. Authorize the connections — the repository, the cloud provider, the monitoring tools and your chat.
  4. Describe the exceptions: a failure mid-run, a change freeze, a missing approval, an environment that is not responding.
  5. Run it against a non-production environment and check the audit trail before pointing it anywhere that matters.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

The work around incidents

Not the response itself, which is judgement, but everything surrounding it.

Opening the incident: the channel, the document, the timeline, the right people paged, without somebody doing it under pressure.

Gathering the context that responders always need — recent deploys, current alerts, related incidents, the relevant dashboards — assembled before they ask.

Recording the timeline automatically from what actually happened, since nobody reconstructs it accurately afterwards.

Status updates on a cadence, to the right audiences, which is the thing most often dropped during a real incident.

Closing the loop: the review scheduled, the actions tracked, and somebody checking they were done.

Not the diagnosis or the fix, which are exactly the parts that need people.

Requests from other teams

Access requests are the highest-volume category in most organisations and the most automatable.

Environment provisioning, which is a request queue that should be self-service.

Data restores and exports, where the request is routine and the execution is risky enough to want a controlled path.

Certificate and secret rotation, which fails eventually if it depends on somebody remembering.

Route the exceptions to a person rather than building a decision tree that handles every case badly.

Measure the queue, since the volume by category tells you which self-service tool would remove the most work.

What should stay manual

Anything destructive without a review step — deletions, scale-downs, data removal.

Production deployment approval, where your risk profile requires one, since automating the approval removes the control rather than the work.

Anything where the rollback is worse than the problem.

Changes during an incident, where the ordinary automation should be suspended rather than competing with the responders.

Decisions that need context the system does not have, which is most decisions worth making.

Anything nobody understands. An automation whose behaviour no current team member can explain should be read before it is relied on, not after.

Deleting automations, which is the missing practice

Teams add automations continuously and remove them almost never, so the layer only grows. Making removal ordinary is what keeps it maintainable.

Log every run, including the no-op ones, which is what makes the next four points possible.

Find the jobs that do nothing. A scheduled task that has processed zero records for six months is either broken or finished, and both warrant action.

Find the jobs that always fail and have been failing long enough that somebody muted the alert.

Check what triggered each one last. An automation whose trigger no longer fires is already dead and is still holding credentials.

Retire with a grace period: disable, wait, then delete, so the thing you were wrong about surfaces before the code is gone.

Report removals as a number, alongside additions, since a team that only ever adds is accumulating a problem rather than reducing one.

Making it survive

Make everything idempotent, since re-running is the normal recovery action.

Fail loudly and safely. A partial run that leaves systems in a mixed state is worse than a clean failure.

Alert on the automation itself, through a different path, since a broken job is silent by nature.

Keep an audit trail of what ran, when, triggered by what, and with what result.

Report the inventory: what exists, who owns it, when it last ran, and what has not run in a year.

Limits worth knowing about

Automation is a dependency and adds a failure mode of its own, which should be counted against the toil it removes.

Credentials need managing, and an automation with broad standing access is a larger risk than the manual process it replaced.

Rate limits and quotas on cloud and vendor APIs constrain what can run in parallel.

Test coverage for operational code is usually poor, which is why these things break in ways the team finds surprising.

Cultural fixes are not technical. If the process is bad because of how decisions are made, automating it changes nothing about that.

What to build first

The inventory. Every scheduled job, pipeline and script currently running against production, with an owner name against each and the date it last ran. It changes nothing and most teams cannot produce it, which is itself the finding — the things nobody can name are the ones that will fail confusingly.

Two habits make the difference. Ask what a process is for before automating it, since the occasional answer is that it should not exist and automating it makes that permanent. And treat anything running against production as production code, in version control with an owner and alerting, because the alternative is a layer of infrastructure that nobody wrote and nobody can safely change.

Frequently asked questions

Why is automating a bad process a problem?

Because manual friction is what eventually prompts somebody to fix the underlying thing. Automating it removes the prompt, and the encoded process outlives the reason for it — permanently, and increasingly riskily as other things come to depend on it.

How do I choose what toil to remove first?

Not frequency alone. Weight interruptions, error-prone steps, out-of-hours work, and anything that blocks other people — all of which cost more than their duration suggests.

What is the most common failure in DevOps automation?

Ownership. The original author moves team and the automation becomes unowned infrastructure that nobody can safely change or confidently delete.

Should operational scripts be treated as production code?

If they run against production, yes — version control, review, testing and alerting. The scripts that feel too small for that are exactly the ones that break confusingly.

What parts of incident response can be automated?

Everything around the response: opening the incident, assembling context, recording the timeline, sending status updates, and scheduling the review. Not the diagnosis or the fix.

What should never be automated?

Anything destructive without a review step, anything where the rollback is worse than the problem, changes during an incident, and anything the current team cannot explain.

Does automation always reduce risk?

No. It is a dependency with its own failure modes, and an automation holding broad standing credentials can be a larger risk than the manual process it replaced. Count that against the toil it removes.

Why can most teams not list their automations?

Because they accumulate one script at a time, each too small to register as infrastructure. Producing the inventory is usually the most informative hour this subject offers.

How do I know which automations to delete?

Log every run including no-ops. Jobs processing zero records for months, jobs failing since somebody muted the alert, and jobs whose trigger no longer fires are all dead — and still holding credentials.

Should deletion be immediate?

Disable, wait, then delete. The grace period is what surfaces the case where you were wrong, while the code still exists to restore.

How should access requests be handled?

As the highest-volume category, which in most organisations they are. Self-service with a controlled path beats a queue, and the volume by category tells you which self-service tool would remove the most work.

What happens to ordinary automation during an incident?

It should be suspended rather than left competing with the responders. Scheduled jobs firing during an incident add noise and occasionally change the thing being investigated.

Get started today

Your first workflow is free to build.

Describe what you need. Cody handles the build, the connections, and the deployment.