What is intelligent automation, and when should a model decide?

Intelligent automation means automation that makes judgements rather than only following rules — classifying a document, deciding where a request should go, extracting a value from something unstructured, interpreting a message.

That changes the design problem in one specific way. Rule-based automation is either correct or it has a bug you can fix. Judgement-based automation is right most of the time, and the design question becomes what happens the rest of the time — which is a question about process rather than about the model.

What we'll cover

• Rules and judgement
• Where judgement is actually needed
• Confidence, and what to do with it
• Designing the escalation path
• The cost of being wrong
• Evaluating before deploying
• Watching it after deployment
• Explaining a decision afterwards
• Building it with CodeWords
• What to get right first
• Frequently asked questions

Rules and judgement

Rules are explicit and auditable. If the amount exceeds a threshold, route to a manager. You can read the rule, test it, and explain a decision afterwards.

Judgement handles what rules cannot express. Whether a complaint is about billing or delivery, what a supplier's invoice actually says, whether a message is urgent.

Rules should be preferred wherever they suffice. They are cheaper, faster, deterministic, and explainable, and a great deal of work that gets pointed at a model was expressible as rules by somebody who knew the domain.

The mistake in both directions is common: forcing judgement into a hundred-branch rule tree nobody can maintain, or putting a model in front of a decision that was three clear conditions.

Most real processes need both, with rules handling the bulk and judgement handling the part that varies.

Where judgement is actually needed

Unstructured input. Documents, emails, messages, images — anything where the information is not already in fields.

Classification with fuzzy boundaries. Routing by topic, sentiment, or intent, where the categories are clear and the edges are not.

Extraction from varied formats. Fifty suppliers' invoices carry the same facts in fifty layouts.

Summarising for a person, which is genuinely useful and low risk because a person reads the result.

Matching and deduplication, where "the same company" is a judgement rather than a string comparison.

Not calculation. Not anything with a defined correct answer. Not anything where a rule is available and legible.

Confidence, and what to do with it

The mechanism that makes judgement-based automation operable rather than a gamble.

Get a confidence signal where the model provides one, or construct a proxy — agreement between two approaches, the presence of expected fields, a plausibility check.

Set a threshold above which the automation acts, and below which it escalates.

Tune it against the cost of each error. A wrong routing decision costs a few minutes; a wrong payment costs considerably more, and the threshold should differ accordingly.

Expect confidence to be imperfectly calibrated. Models are frequently confident when wrong, so the threshold is a risk control rather than a guarantee.

Measure the escalation rate. Too high and the automation is not saving anything; too low and you are probably acting on decisions you should not be.

Report both the confident errors and the escalations, since the first number is the one that tells you whether the threshold is set correctly.

Designing the escalation path

The half that determines whether this works in practice, and the half that gets built last.

Somebody must own the queue. An escalation queue with no owner is where automated processes fail silently while every dashboard shows success.

Give the person what they need to decide — the input, the model's suggestion, and the reason it was uncertain — rather than making them start from nothing.

Make the decision cheap. If reviewing an escalation takes as long as doing the task manually, the automation has saved nothing on that share of cases.

Feed decisions back. What people decide on escalations is the most useful evaluation data available, and it is free.

Age the queue. Escalations that sit are work that is not happening, and nothing else surfaces that.

Staff it with the people who did the work before, since they know which cases are genuinely difficult and which merely look unusual.

The cost of being wrong

The analysis that should precede any decision to automate a judgement.

Reversible and cheap: route a ticket to the wrong team. Somebody forwards it. Automate at a modest threshold.

Reversible and expensive: misclassify a customer's plan, sending the wrong communication. Recoverable, embarrassing, and worth a high threshold.

Irreversible: pay the wrong amount, delete a record, send something to a customer. A person confirms, regardless of confidence.

Asymmetric errors are common. Wrongly approving and wrongly rejecting rarely cost the same, so the threshold should not be symmetric either.

Consider the aggregate. A one per cent error rate on forty cases a month is manageable; on forty thousand it is four hundred wrong outcomes, which is a different conversation.

Evaluating before deploying

Build a set of real examples with the answers you consider correct. Fifty well-chosen cases, including the awkward ones, is worth more than any benchmark.

Include the cases that are genuinely ambiguous, and record what a person decided and why, since those are where the threshold matters.

Measure the errors separately by type, since a false positive and a false negative usually have different costs.

Compare against the current process, not against perfection. If people currently get it right most of the time and the automation matches that at a fraction of the effort, that is the relevant comparison.

Re-run on every change to the prompt, the model, or the surrounding logic, because all three move quality and none announces it.

Keep the set current. Real inputs change, and an evaluation set from a year ago measures a distribution you no longer have.

Watching it after deployment

Track the escalation rate over time. A rising rate means the inputs have changed, which is worth knowing before the errors follow.

Sample and review confident decisions, periodically, since the ones that did not escalate are exactly the ones nobody is looking at.

Watch for drift. Suppliers change invoice formats, customers change how they phrase things, and a model that was accurate becomes less so with nothing reporting it.

Log inputs, outputs, and confidence for anything consequential, because when a decision is questioned, reconstructing it afterwards is otherwise impossible.

Give people a way to report a wrong outcome, and read those reports. They are the cheapest error detection available.

Explaining a decision afterwards

A requirement that arrives later than it should, usually from somebody outside the project.

Somebody will ask why. A customer, a regulator, an auditor, or a colleague — and "the model decided" is not an answer anybody accepts.

Log the input, the output, the confidence, and the version of whatever made the decision. Without those four, reconstruction is guesswork.

A rule is self-explaining and a judgement is not, which is one more reason to prefer rules wherever they suffice.

Record the reason where the approach supports it — the extracted evidence, the matched passage, the fields that drove the classification.

Know your obligations. In some jurisdictions and some domains, automated decisions affecting people carry specific transparency and review requirements, which is a question for whoever advises you rather than one to assume.

Keep the logs as long as the decision matters, which is frequently longer than an ordinary application log retention policy assumes.

Building it with CodeWords

CodeWords connects to more than 3,000 integrations, and where a decision needs judgement, the escalation and the exception handling are part of describing the automation rather than separate systems.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder, including which decision needs judgement.
3. Describe what should happen when the automation is unsure: who decides, what they should see, and by when.
4. Connect the systems involved once and reuse the connections.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

What to get right first

The escalation path, before the model. A judgement-based automation is a system that will be wrong some of the time by design, so the route to a person is not a fallback — it is half the design, and building it last is why these projects stall at the pilot stage.

Two habits make the difference. Decide the cost of each type of error before choosing a threshold, since a threshold set without that is a number somebody picked. And feed escalation decisions back as evaluation data, because it is the most relevant data you will ever have and it arrives free.

Frequently asked questions

When should a model make the decision rather than a rule?

When the input is unstructured or the boundaries are genuinely fuzzy. Rules are cheaper, faster, deterministic, and explainable, so prefer them wherever they suffice — and a lot of work pointed at models was expressible as rules by somebody who knew the domain.

How do I set a confidence threshold?

From the cost of each type of error, which is usually asymmetric. A wrong routing decision and a wrong payment justify very different thresholds, and a number chosen without that analysis is arbitrary.

Is confidence reliable?

Imperfectly. Models are frequently confident when wrong, so treat a threshold as a risk control rather than a guarantee, and sample confident decisions periodically — they are the ones nobody is reviewing.

What makes these projects stall?

The escalation path being built last. A judgement-based automation is wrong some of the time by design, so the route to a person is half the design rather than a fallback, and without an owner the queue silently absorbs the failures.

How much evaluation data do I need?

Fifty real examples, chosen to include the awkward and ambiguous ones, with the answers you consider correct. That beats any benchmark, because the benchmark measures a distribution that is not yours.

How do I know it is still working?

Escalation rate over time, periodic review of confident decisions, and a route for people to report wrong outcomes. Drift is silent — inputs change, accuracy degrades, and nothing errors.

What should never be automated with judgement?

Anything irreversible without a person confirming — payments, deletions, customer communications. Confidence is not certainty, and the asymmetry between a saved minute and an unrecoverable action is not close.

How do I explain a decision after the fact?

Log the input, the output, the confidence, and the version of whatever decided, plus the supporting evidence where the approach provides it. Without those, reconstruction is guesswork and "the model decided" satisfies nobody.

Are there rules about automated decisions affecting people?

In some jurisdictions and domains, yes — including transparency and review requirements. It is worth raising with whoever advises you before deployment rather than after somebody asks.

Related reading

• Hyperautomation
• Retrieval-augmented generation
• Robotic process automation