Redis and CodeWords: state for automations that need it
Caching expensive lookups, deduplicating retried webhooks, rate limiting against someone else API, and holding the small amount of state an automation needs between runs.
Aymeric Zhuo11 min read
On this page
Most automations are stateless by design, and that is usually right: a run starts, does its work, and ends. The trouble arrives at the edges. Did I already process this webhook? What did that expensive lookup return last time? How many requests have I made to this API in the past minute?
Those questions need somewhere to keep a small amount of information between runs, and Redis is a reasonable answer because it is fast, it expires things on its own, and the operations you need are atomic.
What we'll cover
- The four problems that need state
- What Redis gives you that a database does not
- Connecting it to CodeWords
- Six patterns worth building
- Choosing keys and expiry
- Building it so it survives
- Limits worth knowing about
- Frequently asked questions
The four problems that need state
Duplicate processing. Webhooks retry, people click twice, and a queue redelivers. An automation that charges a card, sends an email, or creates a record needs to know whether it already did that for this event.
Expensive repeated lookups. Enriching a company from an API that charges per call, geocoding an address, or asking a model to classify text that has not changed. Doing it once and remembering the answer is both faster and cheaper.
Someone else's rate limit. When an automation calls an API with a per-minute ceiling, something has to count. Without a shared counter, parallel runs each believe they are the only one.
Anything that spans runs. A cursor recording how far a sync got, a running total, a short-lived lock preventing two runs doing the same job. None of it belongs in the automation itself, because the automation ends.
What Redis gives you that a database does not
Expiry as a first-class feature. Set a value with a time to live and it removes itself. Most automation state should expire, and a store that forgets on its own saves you writing the cleanup you would otherwise forget.
Atomic operations. Increment, decrement, set-if-not-exists. These matter because the alternative — read, decide, write — has a gap in the middle where another run can do the same thing. Nearly every bug in automation state lives in that gap.
Speed that makes the check free. A lookup fast enough that you stop weighing whether it is worth checking is a lookup you will actually use.
Data structures that fit the job. Counters, sets for membership, sorted sets for windows, lists for simple queues. Expressing the problem directly beats emulating it in table rows.
What it does not give you is durability guarantees as strong as a database, or anything resembling a query language. Redis is for small, hot, expendable state. Anything you would be upset to lose belongs elsewhere.
Connecting it to CodeWords
CodeWords connects to more than 3,000 integrations, and the connection is made once and reused.
- Open CodeWords and start a new automation.
- Describe what you want in plain language to Cody, the automation builder, including what should be remembered and for how long.
- Authorize the connection with your Redis endpoint and credentials, using a TLS connection and a user scoped to the keys this automation needs.
- Describe the failure behaviour explicitly: if Redis is unreachable, should the automation proceed, stop, or retry?
- Test the duplicate case deliberately by running the same event twice.
You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.
Six patterns worth building
Idempotency keys. Before acting, attempt to set a key derived from the event identifier, only if it does not already exist. If the set succeeds, this is the first time and you proceed. If it fails, you have seen this event and you stop. Give the key a lifetime comfortably longer than any retry window. This one pattern prevents the majority of duplicate-action incidents.
Cache with expiry. Check the cache, return the value if present, otherwise do the expensive thing and store the result with a time to live matched to how fast the underlying answer changes. Company data can live for a week; a stock level cannot live for a minute.
Rate limiting with a counter. Increment a counter keyed to the current minute, set it to expire after a minute, and pause when it exceeds your ceiling. Because the increment is atomic, parallel runs share one honest count instead of each keeping its own optimistic one.
Distributed locks. Before a job that must not run twice concurrently, acquire a lock by setting a key with an expiry. Release it when done. The expiry matters more than the release, because a run that dies without releasing would otherwise hold the lock forever.
Sync cursors. Store the timestamp or identifier where the last run finished, so the next run picks up from there rather than reprocessing everything. Write the cursor only after the work succeeds, or a failure will skip records.
Deduplication windows. Keep a set of recently seen identifiers with an expiry, to suppress repeated alerts about the same underlying problem. This is what stops a flapping condition generating two hundred notifications.
Choosing keys and expiry
Two decisions that determine whether this stays comprehensible.
Namespace every key. Something like automation-name:purpose:identifier. When you are looking at a Redis instance wondering what is in it, the key names are all the documentation that exists.
Derive idempotency keys from something stable. The event's own identifier is ideal. A hash of the payload works where no identifier exists, and be careful that the payload does not contain a timestamp that changes between retries, which would make every retry look new.
Match expiry to meaning. An idempotency key should outlive the retry window and not much more. A cache entry should expire when the underlying value plausibly changes. A lock should expire slightly after the longest the job could legitimately take.
Never set state without an expiry unless you have a specific reason. Keys without expiry accumulate forever, and the instance you never think about becomes the instance that fills up.
Assume any key may be gone. Redis can evict under memory pressure and instances get restarted. Code that treats a missing cache entry as a cache miss is fine; code that treats a missing cursor as "start from the beginning" may reprocess a year of data.
Building it so it survives
Decide what happens when Redis is down. For a cache, proceed without it and accept the slower path. For an idempotency check, stopping is usually safer than proceeding, because the alternative is duplicate action. These are opposite answers and both are right in their place.
Do not put anything irreplaceable in it. If losing a value would mean losing data, it belongs in a database.
Keep values small. Redis holds everything in memory. Caching large documents works and gets expensive quickly.
Use TLS and scoped credentials. An automation's Redis connection frequently sits alongside credentials for everything else it touches.
Log the cache hit rate. A cache nobody measures is a cache that might be doing nothing. A rate near zero usually means the key includes something that varies, such as a timestamp.
Limits worth knowing about
Memory is the binding constraint, and the eviction policy decides what happens when it is reached. Check that the policy matches your use: evicting the least recently used keys is right for a cache and wrong for locks.
Persistence is configurable and not free. A restart without persistence loses everything, which is acceptable for a cache and unacceptable for a cursor. Know which mode you are on.
Latency depends on placement. An automation calling a Redis instance in another region pays that round trip on every operation, which can outweigh the saving from caching at all.
It is not a queue with delivery guarantees, despite lists making a passable simple queue. If you need acknowledgement and redelivery semantics, use something built for it.
Managed providers impose connection limits, which matters when many automation runs connect concurrently.
Frequently asked questions
Do I need Redis for most automations?
No, and most do not. Reach for it when you have one of the four problems above: duplicate processing, expensive repeated lookups, rate limiting, or state spanning runs. Adding it without one of those is infrastructure you now maintain for no benefit.
Could I use a database table instead?
Yes, and for low volume it is perfectly reasonable and one fewer system. What you give up is expiry, which you then implement as a cleanup job, and atomic operations, which you then have to get right with transactions. At low volume that is a fair trade; at high volume Redis is much less trouble.
What is the right expiry for an idempotency key?
Longer than the longest retry window of whatever sends you events, with a margin. A day is a common default and generous. Too short risks a late retry being treated as new; too long only costs memory.
How do I stop two automation runs doing the same job?
A lock: set a key only if it does not exist, with an expiry longer than the job could take. Acquiring it means proceed, failing to acquire means another run has it. The expiry is what prevents a crashed run from holding the lock indefinitely.
Is Redis safe for storing anything sensitive?
Treat it as a cache rather than a vault. Use TLS, scope credentials, and avoid putting personal data in it where you can, since its retention and access controls are coarser than a database's. Where you must, keep the expiry short and deliberate.
What happens if my cache returns stale data?
Whatever your expiry allows, which is why matching the expiry to the rate of change matters. For anything where staleness would be harmful — prices, stock, permissions — either do not cache it or cache it for seconds rather than hours.
Can CodeWords handle this without me managing Redis?
For most automations the platform's own execution handles retries and state without you standing anything up. Bring your own Redis when you need control over it, when other systems share the same state, or when the volume justifies it.
How do I know whether the cache is actually working?
Log hits and misses and look at the ratio. A hit rate near zero almost always means the key includes something that varies between runs, most often a timestamp. A hit rate near one hundred percent on data that should change means your expiry is too long and you are serving stale answers confidently.
What is the most common mistake?
Setting keys without an expiry. It works perfectly, costs nothing to begin with, and quietly fills the instance over months until something evicts the keys you actually needed. Set a time to live on everything unless you have a specific reason not to, and know your eviction policy.
Can several automations share one instance?
Yes, and namespace the keys by automation so it stays comprehensible. The thing to watch is that one automation's bulk write can evict another's working set under memory pressure, which produces a failure in a place that changed nothing.
What to build first
Start with an idempotency key on whichever automation you would least like to run twice. It is a handful of lines, it prevents the most embarrassing category of automation failure, and it costs nothing when it is not needed.
Two things make the difference between a first automation that sticks and one that gets abandoned. Run it alongside whatever happens now for a cycle or two and compare the outputs, since the differences are nearly always genuine discoveries rather than bugs. And have it report that it ran even when it found nothing, because an automation whose only output is silence is one nobody can distinguish from a broken one.