Replicate and CodeWords: running models inside automations

Replicate exists for the models that are not language models. Image generation, background removal, upscaling, transcription, audio separation, video work. Running these yourself means GPUs, and the reason Replicate is useful in an automation is that it turns a GPU workload into an API call you pay for by the second.

That last detail shapes everything about how you should build. A prediction that takes ninety seconds is not a step you wait on the way you wait on a database query, and a per-second bill behaves differently from a per-request one.

What we'll cover

• What Replicate is actually for
• What the Replicate API reaches
• Connecting it to CodeWords
• Seven automations worth building
• Handling predictions that take minutes
• Keeping the cost sensible
• Building it so it survives
• Frequently asked questions

What Replicate is actually for

Worth being clear, because it is frequently reached for when something simpler would do.

Use it when you need a model that is not text: generating or editing images, transcribing or separating audio, processing video, running a computer vision model, or using an open-source model that no major provider hosts.

Do not use it for ordinary text generation, classification, or extraction. Those are better served by a language model provider directly, and on CodeWords model access is already available without managing a separate account.

Be careful with it for anything latency-sensitive. Models that are not kept warm have a cold start, and a prediction that usually takes eight seconds can take sixty on the first call after a quiet period.

The sweet spot is media work inside a background automation, where a minute of latency costs nothing because no one is waiting.

What the Replicate API reaches

Predictions are the core object. You create one against a model version with inputs, and it moves through states from starting to processing to succeeded or failed. Outputs are returned as URLs for media.

Models and versions can be listed and read. Pinning a version matters, and the section on durability covers why.

Webhooks can be attached to a prediction so Replicate tells you when it finishes, rather than your automation asking repeatedly.

Files can be supplied as URLs or uploaded, and outputs are served from URLs with a limited lifetime, which is the detail that catches people.

Deployments let you run a model with configured scaling, including keeping instances warm, which is how you avoid cold starts when latency matters enough to pay for it.

Connecting it to CodeWords

CodeWords connects to more than 3,000 integrations, and the connection is made once and reused.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: what media comes in, which model processes it, and where the result should end up.
3. Authorize the connection with your Replicate API token.
4. Describe the exceptions: a prediction that fails, an input that is the wrong format, an output that needs storing before its URL expires.
5. Run it on a handful of real inputs and look at the outputs before scheduling anything.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

Seven automations worth building

Product image processing. Background removal, consistent cropping, and resizing applied to every new product image as it arrives, with results written back to the store or the asset library. This is the most common genuinely useful case, because it is repetitive, visual, and nobody enjoys it.

Transcription pipelines. Audio or video transcribed, then the transcript summarised and filed. The transcription is the part Replicate does; the summarising belongs to a language model afterwards.

Asset variants. One source image turned into the sizes and aspect ratios each channel needs, named consistently, delivered to the right folders.

Upscaling archives. Old or low-resolution images improved in bulk, as a scheduled background job rather than an interactive one.

Generated imagery with review. Draft images produced from a brief, delivered somewhere a person picks. Generation is the assembly; choosing is the judgement, and that split matters more here than almost anywhere.

Content moderation. Images checked against a classifier before publication, with anything uncertain routed to a person rather than blocked outright.

Document image handling. Deskewing, cleaning, and preparing scanned documents before they reach an extraction step, which materially improves what the extraction gets right.

Handling predictions that take minutes

This is the part that distinguishes building against Replicate from building against an ordinary API.

Use webhooks rather than polling. Create the prediction with a webhook URL, and let Replicate tell you when it is done. An automation that polls every two seconds for ninety seconds is spending its own runtime doing nothing.

Split the automation in two. One part submits the prediction and ends. Another receives the webhook and handles the result. This is more robust than one long-running automation, and it costs nothing in complexity once you are describing the behaviour rather than wiring it.

Store the output promptly. Output URLs have a limited lifetime. An automation that returns a link and leaves it is handing somebody a link that will stop working. Download and store it somewhere permanent as part of the same step.

Expect failures as a normal state, not an exception. Predictions fail for malformed inputs, content filters, and capacity. Handle the failed state explicitly rather than assuming success and discovering otherwise.

Budget for cold starts. If a model has not run recently, the first call is slower, sometimes dramatically. For background work this is irrelevant. For anything a person is waiting on, either keep it warm with a deployment or choose a different architecture.

Keeping the cost sensible

Billing is per second of compute, which behaves differently from per-request pricing and rewards different optimizations.

Match the model to the job. A large, slow model producing a marginally better result at five times the runtime is a poor trade for most bulk processing. Test the smaller option on real inputs before assuming you need the larger one.

Resize before you send. Processing a 6000-pixel image when the output will be displayed at 800 wastes most of what you paid for.

Cache by input. The same image processed twice costs twice. A hash of the input as a cache key removes a surprising amount of repeat work, particularly in pipelines that re-run.

Batch where the model supports it, since per-call overhead is repeated otherwise.

Set a ceiling and alert on it. A loop that misbehaves against a per-second model is the kind of mistake that is visible on an invoice rather than in a log. A spend alert well below your tolerance turns that into a Tuesday afternoon problem.

Watch retries. An automation retrying a failing prediction three times pays for three predictions. Retry on transient failures, not on a malformed input that will fail identically every time.

Building it so it survives

Pin the model version. Models are updated, and outputs change when they are. An automation that names a version produces consistent results; one that names only a model can behave differently tomorrow for no reason you changed.

Validate the output. Check the file exists, is the expected type, and is a plausible size. A prediction can succeed and return something unusable, and a zero-byte image will propagate cheerfully through the rest of your pipeline.

Keep a person on anything published. Generated or heavily processed media going in front of customers warrants a look, particularly where a model can produce something subtly wrong in a way no automated check will catch.

Log the inputs, the version, and the cost. When an output is questioned weeks later, reproducing it requires all three.

Handle the content filter case explicitly. Some models refuse some inputs. That is a distinct outcome from a technical failure and deserves different handling, usually routing to a person rather than retrying.

Frequently asked questions

Should I use Replicate for text generation?

Generally no. Language models are better served directly by a provider, and on CodeWords model access is already available without a separate account. Replicate earns its place for image, audio, and video work, and for open-source models nobody else hosts.

How long do predictions take?

From seconds to several minutes depending on the model and the input, plus a cold start if the model has not run recently. Design for the long case rather than the typical one: use webhooks, split submission from handling, and do not build anything that assumes a fast return.

What happens to the output files?

They are served from URLs with a limited lifetime. Download and store them somewhere permanent as part of the same automation, rather than passing the link onward. This is the most common mistake in Replicate automations and it surfaces days later when the link has gone.

How do I stop a runaway bill?

Set a spend alert well below your tolerance, cache by input hash, resize inputs before sending, choose the smallest model that does the job, and be careful with retries. Per-second billing punishes loops in a way per-request billing does not.

Why did my outputs change without me changing anything?

Almost certainly an unpinned model version. Pin the version explicitly and update it deliberately, with a comparison against your previous outputs, rather than inheriting whatever is current.

Can I keep a model warm?

Yes, through deployments with configured scaling, and you pay for the idle capacity. That is worth it when a person is waiting and not worth it for background processing, which is most automation.

What should I do when a prediction fails?

Distinguish the causes. Transient capacity failures are worth retrying with backoff. A malformed input will fail identically however many times you try, so it should be flagged rather than retried. A content filter refusal is a third thing and usually belongs with a person.

How do I choose between models that do the same thing?

Run twenty real inputs through each and look at the outputs yourself. Benchmarks measure something, and rarely the thing you care about. Include your awkward cases — the badly lit photo, the noisy recording — because that is where models actually differ and where the demo inputs never go.

Should generated media go straight to customers?

No, not without a look. A model can produce something technically valid and subtly wrong in a way no automated check catches, and published media is the category where that is most visible and least recoverable. Generate, review, then publish.

What is the most common mistake?

Not storing the output before its URL expires. It works perfectly in testing, because you look immediately, and fails days later when somebody follows a link that has gone. Download and store as part of the same step, every time.

What to build first

Start with whichever image task somebody currently does by hand in bulk, usually background removal or resizing. The inputs are plentiful, the output is visually obvious, and nothing is published without a look.

Two things make the difference between a first automation that sticks and one that gets abandoned. Run it alongside whatever happens now for a cycle or two and compare the outputs, since the differences are nearly always genuine discoveries rather than bugs. And have it report that it ran even when it found nothing, because an automation whose only output is silence is one nobody can distinguish from a broken one.

How do I keep quality consistent over time?

Pin the model version, keep a small set of reference inputs, and re-run them whenever you change anything. Outputs drift when versions change, and without reference inputs you will notice only when somebody complains about a result that looks subtly worse than it used to.

Related reading

• AI automation tools: what the AI layer adds
• CodeWords integrations and templates