AI automation examples: including the ones that went wrong
Worked examples are usually written as successes. These are described with what they cost, what broke, and what the second version had to change.
Isha Maggu11 min read
On this page
- What we'll cover
- Why examples are usually misleading
- Inbound request routing
- Document intake
- Building it with CodeWords
- Reporting distribution
- Lifecycle messaging
- Reconciliation between systems
- What the failures have in common
- What each of these actually cost to run
- Making it survive
- Limits worth knowing about
- What to build first
- Frequently asked questions
- Related reading
Published automation examples describe what was built and how long it took. They rarely describe what broke afterwards, which is the part that determines whether the thing survives its first quarter.
These are described differently: what the automation does, what its second version had to change, and what it costs to keep running. The failures are more instructive than the designs, and they recur.
What we'll cover
- Why examples are usually misleading
- Inbound request routing
- Document intake
- Building it with CodeWords
- Reporting distribution
- Lifecycle messaging
- Reconciliation between systems
- What the failures have in common
- What each of these actually cost to run
- Making it survive
- Limits worth knowing about
- What to build first
- Frequently asked questions
Why examples are usually misleading
They describe the first version, which is the easy one.
The maintenance is invisible and is the majority of the lifetime cost.
The context is missing. The same automation is worthwhile at one volume and not at another.
The failures are omitted, since the people writing them are usually selling something.
The time saved is self-reported and rarely measured against a baseline.
Read for the shape and the failure mode, not for the promised outcome.
Inbound request routing
What it does: classifies incoming requests and sends them to the right team with context attached.
What the first version got wrong: it forced every request into a category, so anything unusual went to whichever team matched least badly and sat there.
The second version added an unrouteable queue that a person watches, which turned out to hold the most interesting requests.
Ongoing cost: reviewing the reroutes monthly, which is the feedback that keeps classification current as the mix changes.
Where it genuinely helps: response time, because a request reaching the right person in minutes rather than a day is most of the outcome.
When it is not worth it: below a volume where a person can read everything, which is a higher threshold than people assume.
Document intake
What it does: extracts fields from documents arriving in varied formats and creates records from them.
What the first version got wrong: it treated extraction confidence as a formality and wrote everything, so a proportion of records were quietly wrong.
The second version set a threshold from measured accuracy and routed anything below it to review, which reduced coverage and made the output trustworthy.
Ongoing cost: the review queue, which never reaches zero and should not be expected to.
Where it genuinely helps: removing transcription, which is both slow and error-prone when done by people under time pressure.
When it is not worth it: when documents vary so much that most items need review anyway.
Building it with CodeWords
CodeWords connects to more than 3,000 integrations, and the connection is made once and reused.
- Open CodeWords and start a new automation.
- Describe what should happen in plain language to Cody, the automation builder: the process, what needs judgement, and where a person stays in the path.
- Authorize the connections the workflow needs.
- Describe the exceptions: an ambiguous case, a low-confidence result, an unavailable system, an unusual volume.
- Run it against historical inputs where you know the right answer before it touches anything live.
You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.
Reporting distribution
What it does: assembles a recurring report from several sources and sends it to the people who need it.
What the first version got wrong: a source failed and the report sent anyway, missing a section, and nobody noticed for three weeks.
The second version failed loudly on a missing source and said so in the output rather than omitting silently.
Ongoing cost: almost none, which makes this the highest-return category once the failure handling is right.
Where it genuinely helps: consistency and timing, since a report that arrives reliably gets built into how people work.
When it is not worth it: when nobody acts on it, which a fortnight of not sending it will establish definitively.
Lifecycle messaging
What it does: contacts people at defined points based on what they have done.
What the first version got wrong: it had no global frequency cap, so people in several sequences received everything at once.
The second version added a cap across every automated message, with a priority order deciding what wins.
Ongoing cost: reviewing the suppression and the priority order as campaigns are added, which nobody owns by default.
Where it genuinely helps: reaching people at the moment something is relevant rather than on a schedule.
When it is not worth it: when the underlying product or service is the problem, since better messaging about something that does not work is not an improvement.
Reconciliation between systems
What it does: compares two systems and reports what disagrees.
What the first version got wrong: it tried to fix the differences automatically, which propagated errors in both directions.
The second version reported rather than resolved, which made it useful and considerably safer.
Ongoing cost: somebody working the difference list, which is real work and is the point.
Where it genuinely helps: finding the disagreements before they become a dispute, an audit finding, or a customer complaint.
When it is not worth it: when nobody will act on the differences, in which case you have built a report of problems you are choosing to ignore.
What the failures have in common
Forcing an answer where "unknown" was the correct output.
Silent degradation, where something failed and the output still looked complete.
No global view, where each automation was sensible and the total was not.
Acting rather than reporting, in cases where reporting was sufficient and much safer.
No absence alerting, so the thing that stopped working was discovered by accident.
Every one of these is a design decision taken by default rather than deliberately, which is why they recur.
What each of these actually cost to run
The build is quoted and the running is not, which is why automation portfolios grow past what a team can maintain without anybody deciding to let that happen.
Somebody has to own it, named, and that person's attention is the largest recurring cost.
Review queues do not empty. Any automation with a human fallback creates a standing workload proportional to its volume.
Upstream changes break things. A system it reads from changes, and the fix is unplanned work arriving at an inconvenient moment.
Every automation is a dependency in whatever process it serves, which means its failure is now that process's failure.
Monitoring is not free, and an automation with no alerting has traded a visible cost for an invisible risk.
Budget the maintenance at roughly the build cost per year, which is a rough figure and is much closer than zero, which is what most plans assume.
Making it survive
Report, do not resolve, until you have measured that resolving is safe.
Fail loudly rather than producing partial output that looks whole.
Alert on absence for every automation without exception.
Keep an escape hatch — an unknown category, an unrouteable queue, a review path — since forcing a decision is the most common failure here.
Report the outcome: what ran, what it did, what it could not decide, and what needed a person.
Limits worth knowing about
Volume determines viability, and an automation worthwhile at scale is a distraction below it.
Maintenance is the real cost, and it is absent from every example including the ones above.
Measured baselines are rare, so most claims about time saved are estimates.
Your data is worse than the example's, in essentially every case.
Nothing here fixes an unclear process, which is the precondition rather than an outcome.
What to build first
Whichever of these is closest to a problem you already have, built to report rather than to act. The reporting version is faster to build, safe to run, and answers whether the acting version is worth building — which is a question most implementations skip and then discover the hard way.
Two habits make the difference. Give every automation a way to say "I do not know", since forcing an answer is the single most common cause of the failures above. And alert on absence from the beginning, because every one of these dies quietly rather than loudly and the discovery is otherwise accidental and late.
Frequently asked questions
Why are published automation examples misleading?
They describe the first version, omit the maintenance that dominates lifetime cost, leave out the context that determines whether it was worthwhile, and are usually written by somebody selling something.
What do the common failures have in common?
Forcing an answer where "unknown" was correct, degrading silently while still producing complete-looking output, no global view across automations, acting where reporting would have sufficed, and no alerting on absence.
Why start with a reporting version?
It is faster to build, safe to run, and it answers whether the acting version is worth building at all — a question most implementations skip and then learn the hard way.
What went wrong with automated reconciliation?
The first version fixed differences automatically and propagated errors in both directions. Reporting rather than resolving made it both useful and considerably safer.
What is the most common document intake mistake?
Treating extraction confidence as a formality and writing everything, which makes a proportion of records quietly wrong. A threshold set from measured accuracy reduces coverage and produces output you can trust.
When is an automation not worth building?
Below the volume where a person can read everything, which is a higher threshold than most people assume — and whenever nobody will act on what it produces.
What is the hidden cost in every example?
Maintenance, which is the majority of lifetime cost and appears in none of the published examples, including the ones here.
What single habit prevents most of these failures?
Giving the automation a way to say it does not know, and routing that somewhere a person looks. Forcing a decision is the root of nearly all of them.
What does an automation cost to keep running?
A named owner's attention, a review queue that never empties, unplanned work when upstream systems change, and monitoring. Budgeting roughly the build cost per year is rough and much closer than the zero most plans assume.
Why do automation portfolios outgrow their teams?
Because the build is quoted and the running is not. Each addition looks small, nobody totals the maintenance, and the limit is discovered when something breaks and there is no capacity to fix it.