Workflow automation platform: evaluating one for an organization
The buyer's questions rather than the builder's: governance, access control, reliability, support, vendor risk, and the total cost of running automation across a company.
Rebecca Pearson11 min read
On this page
Choosing a platform for one team is mostly about whether it can build what you need. Choosing one for an organization is a different exercise, and the questions that decide it rarely appear in feature comparisons: who is allowed to build, what happens when the person who built something leaves, whether an auditor can be answered, and what it costs when three hundred people are using it rather than three.
This page covers that evaluation. For how the categories of tool differ, see workflow automation tools; for cost modelling specifically, see the cheapest workflow automation tools.
What we'll cover
Platform or tool
The words are used interchangeably in marketing and the distinction is real. A tool builds automations. A platform also answers the questions that arise once many people are building many automations.
Five things separate them, and none is about building.
Access control beyond one shared account. Who may create, who may edit what others created, who may connect a new system, who may see the data passing through.
Environments. Somewhere to build and test that is not production. Its absence is tolerable for one team and untenable across a company.
Visibility. A view of everything running, who owns it, when it last ran, and what failed, without asking each person what they built.
Credential management. Shared connections held centrally rather than each builder entering their own password, which is the arrangement that produces an unpleasant surprise when somebody leaves.
A support relationship. Someone to contact whose answer is a commitment rather than a forum post.
If your automation is three workflows in one team, you want a tool and the platform questions are overhead. If it is heading toward becoming how the company runs, these are the questions that will matter in eighteen months.
Who is allowed to build
This is the decision with the largest consequences and the one most often left to default.
Nobody but IT is the conservative answer. It produces consistent, well-built, well-understood automations, and a queue. The business processes that would benefit most are owned by people who are not in that queue's priority order, so the work does not get done, and shadow automation appears in spreadsheets and personal tools instead.
Anyone at all produces a great deal of automation quickly, and a mess within a year: duplicated workflows, nobody sure which is authoritative, credentials scattered, and processes depending on something built by a summer intern.
The arrangement that works is usually a boundary rather than a rule. Anyone may build within their own team's systems and data. Anything touching customer data, finance, or another team's systems goes through review. A small central group owns the shared connections and keeps the register of what exists.
Whichever you choose, the platform has to be able to express it. A tool where every user is an administrator cannot implement a boundary, whatever your policy says.
What happens when people leave
Automation built by individuals becomes organizational dependency, and departure is when that becomes visible.
Ownership should be a field, not a memory. Every automation needs a named owner and a named team, recorded where anyone can see it. The question "who owns this" should never require asking around.
Connections should belong to the organization. A workflow running on a departing colleague's personal account stops when that account is disabled, which tends to be discovered at the worst moment. Shared service connections, managed centrally, avoid a whole category of incident.
What it does should be legible. Another person should be able to open an automation and understand its purpose without archaeology. This favours platforms where the logic is readable over those where it is buried in configuration, and it is a real argument for conversational building: a process described in plain language stays readable to whoever inherits it.
Offboarding must include automation. Alongside accounts and devices, a departing person's automations need reassigning. Most organizations discover this the first time something important stops.
Reliability you can describe to someone else
Once a business process depends on automation, "it usually works" stops being an acceptable answer.
Ask what happens to a failed run. Is it retried, how many times, with what spacing? Is the failure visible without someone going to look? Can it be replayed after the cause is fixed, or is the work lost?
Ask about the queue. When a downstream system is unavailable for two hours, do the triggers queue and drain, or are those events simply gone? This is the difference between a delay and a data loss, and it is rarely on the feature page.
Ask about ordering and duplication. Whether events are processed in order, and whether a retry can produce the action twice. Both matter enormously for anything touching money.
Ask for the actual uptime record, not the target in the contract. Vendors with status pages showing real history are telling you something about their confidence.
Ask what you can see. Execution history, retention of logs, and the ability to search them. Debugging a workflow that failed last Tuesday requires that last Tuesday still exists somewhere.
The questions an auditor will ask
Worth knowing in advance, because the answers are difficult to retrofit.
Who can access this data, and how is that enforced? Role-based access, not an honour system.
What changed, when, and who changed it? A change history on automations. Platforms without one make this question unanswerable.
Where is data processed and stored? Region matters for several regulations, and some platforms let you choose while others do not.
What third parties are involved? Every connector is a data flow to someone, and the list is usually longer than anyone expects.
How long is data retained? Execution logs frequently contain the data that passed through, which means your retention policy applies to them.
Can you demonstrate the control worked? Not that it exists, but that it was operating throughout the period. This needs logging that survives long enough to evidence.
If your organization faces any formal compliance regime, put these to a prospective vendor before the technical evaluation. They eliminate candidates faster than feature comparisons do.
Vendor risk
What happens if they are acquired, or close? Both happen. The practical mitigation is knowing how you would export your automations and how long rebuilding would take.
What does the pricing look like at three times your volume? Get it in writing. Growth is when leverage moves to the vendor, and the conversation is easier before you depend on them.
How portable is what you build? Logic expressed in a vendor's configuration format transfers poorly. Logic expressed as a description of the process transfers well, because it can be rebuilt from the description. This is worth weighing explicitly.
Is there a community and an ecosystem? Documentation, forums, and people you could hire who already know it. A platform only your one enthusiast understands is a risk regardless of its quality.
Running the evaluation
Bring three real processes, not a checklist. One simple, one with branching and an exception, one touching a system you expect to be difficult. Build all three on each candidate. This surfaces more in two days than a month of feature comparison.
Make a non-technical colleague build one. If the platform is meant to let operations teams build, test that rather than assuming it. Watching someone try is unambiguous.
Break things deliberately. Revoke a credential, send malformed data, take a system offline. How the platform reports failure is the thing you will live with.
Model the cost properly. Your real step counts, your real volumes, at the scale you expect in two years. Include the administrative time.
Involve security early. They will have questions, and getting them at the end of an evaluation is how a preferred candidate gets rejected after everyone has agreed.
For organizations where the constraint is that the people who understand the processes cannot build them, the evaluation should weight that heavily. On CodeWords you describe a process in plain language and Cody, the automation builder, builds it, connects it to your systems, and deploys it, which puts building in reach of the person who owns the process. Automations connect to more than 3,000 integrations. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.
Moving from a tool to a platform
Organizations rarely start with a platform. They arrive at one having outgrown something, and the migration is where the cost sits.
Inventory before you move anything. What exists, what it does, who owns it, when it last ran, and whether anyone would notice if it stopped. This is the step people skip and the one that determines whether the migration is worth doing, because a meaningful proportion of what you find will turn out to be unused.
Sort into keep, rebuild, and retire. Some automations are still valuable and should move as they are. Some solve a problem worth solving in a way that has aged badly and should be rebuilt rather than ported. Some are no longer needed and moving them is pure cost. Expect the third category to be larger than anyone predicts.
Migrate the important ones last. Counterintuitive and correct: move something low-consequence first so the team learns the platform on work that can afford a mistake. Moving payroll first is how a migration acquires a reputation.
Run both in parallel where you can. For anything critical, have the old and new versions running together with outputs compared for a cycle. This is the cheapest insurance available.
Set a date to turn the old one off. Migrations that run indefinitely leave you paying for and maintaining two systems, and the second one quietly becomes permanent. A date, with the inventory as the checklist, is what finishes it.
Frequently asked questions
How do I know when we have outgrown a simple tool?
The usual signals are nobody being able to say what automations exist, a workflow breaking because someone left, and a request from security or audit that you cannot answer. Any of those means the governance questions have arrived.
Should we standardize on one platform?
Broadly yes, for cross-system automation, because several tools in the same layer make failures hard to trace and spread credentials around. Built-in features inside individual products are a different layer and should continue to be used.
How much should we budget?
The subscription is rarely the largest component at organizational scale. Budget for administration, review of what people build, and the time to migrate what already exists. A platform costing twice as much that removes a maintenance burden is frequently cheaper overall.
Who should own the platform internally?
Someone accountable for it working, with enough proximity to the business to know which processes matter. Pure IT ownership tends toward a queue; pure business ownership tends toward a mess. A small group spanning both, owning standards and shared connections, is the arrangement that usually holds.
What about automations people have already built elsewhere?
Inventory them before migrating. Some are important and undocumented, some duplicate each other, and a meaningful proportion are no longer used. The inventory alone typically justifies the exercise, and migrating everything without it moves the mess rather than resolving it.
How long should an evaluation take?
Two to four weeks for most organizations, with real processes built on a shortlist of two or three. Longer than that and the requirements drift; shorter and you are choosing on demonstrations rather than on experience.
Should the platform decision wait until we have a strategy?
No, and waiting is how shadow automation accumulates. Start with a platform that can express a boundary on who builds what, and let the strategy develop from what people actually build. A policy written before anyone has automated anything tends to describe a problem you do not have.