How to automate synthetic monitoring that stays worth maintaining

Synthetic monitoring runs a scripted journey against production on a schedule (sign in, search, add to basket, check out) and alerts you when the journey stops working. It catches failures that no server-side metric sees: a button that does nothing, a form that silently rejects input, or a third-party script that breaks the page.

Synthetic monitoring also has the highest maintenance cost of any monitoring you can run, because a script that drives an interface depends on an interface that changes. The discipline is to script only the few journeys you can afford to maintain.

What we'll cover

• What it catches that nothing else does
• Why these break so often
• Choosing what to script
• Building it with CodeWords
• Writing a check that survives a redesign
• Test data in production
• Alerting without false alarms
• Measuring the journey, not only whether it worked
• Keeping checks working
• Limits worth knowing about
• What to build first
• Frequently asked questions

Key takeaways

• Synthetic monitoring catches client-side, third-party and journey failures that server metrics cannot see.
• Script three to five journeys for most products, because a large suite goes unmaintained and then reports nothing.
• Dedicated test attributes on interface elements remove most of the maintenance that breaks scripts.
• Alert only after consecutive failures from more than one location, and track the false alarm rate for every check.
• Synthetic checks do not replace real-user monitoring, because they exercise one path with one account.

What it catches that nothing else does

Synthetic monitoring catches six kinds of failure that server-side metrics miss.

Client-side failures. A script error that breaks a button leaves the server healthy, and only a browser-driving check notices.

Third-party breakage. A payment widget, chat tool or analytics script that fails to load can block a page, and none of it appears in your logs.

Journey failures instead of endpoint failures. Each step can work alone while the sequence fails, and the sequence is what a customer does.

Rendering and interaction failures. Server responses cannot describe them. Content that arrives and never displays looks like success everywhere else.

Real network conditions, including DNS, certificates and CDN (content delivery network) behaviour, seen from outside your infrastructure.

Absence of traffic. A journey nobody has attempted since a release is a journey nobody has tested, and synthetic checks exercise it regardless of demand.

Why these break so often

Synthetic checks break because they depend on an interface that changes. Stating this up front prevents an abandoned suite.

Selectors change. A class name, a label or a button position changes in any redesign and breaks scripts that depend on it.

Timing. The check acted before the page finished loading. This is the most common cause of a false failure, and it is intermittent and environment-dependent.

Unexpected interruptions. A cookie banner, a survey prompt or a maintenance notice stops any script that was not told about it.

Content changes. An assertion on text fails when marketing rewords the heading.

Test accounts degrade. A password expires, a trial ends, or an account accumulates state that changes what it sees.

None of these is a fault in the product. A noisy suite trains people to assume a failure is the monitor and not the site, and then one day the site is the cause.

Choosing what to script

Script very few journeys: three to five for most products. A suite of forty is a suite nobody maintains, and an unmaintained check is worse than no check because it eventually reports nothing.

Pick by revenue and by blast radius. Choose sign-in, the primary transaction, and whatever the business would notice within an hour.

Skip the long tail. Edge cases belong in a test suite that runs before deployment, not against production every five minutes.

Write one journey per check, so a failure names what broke and nobody has to read a log to find the failing step.

Keep each check short. A twelve-step script has twelve chances to break and tells you less than three three-step scripts.

Review the list quarterly and remove what no longer matters. Journeys outlive their importance, and the maintenance continues regardless.

Building it with CodeWords

CodeWords connects to more than 3,000 integrations (CodeWords pricing page, 2026), so a check can assert against your own systems and route failures to the people who can act. To build a check:

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: which journey, what a successful outcome looks like, and how often to run it.
3. Authorize the connections it needs, including wherever the alert should go.
4. Describe the exceptions: a slow step, an unexpected dialog, a failure during a known deployment.
5. Break something deliberately in a staging environment and confirm the check notices.

You describe the outcome, and Cody builds it, connects it and deploys it. The Free plan costs $0, Pro costs $39 per month and Business costs $100 per month (CodeWords pricing page, 2026). Details are on the pricing page.

Writing a check that survives a redesign

A check survives a redesign when it targets stable elements and waits for conditions. These practices separate a suite people maintain from one they turn off.

Use dedicated test attributes. An element marked explicitly for automation survives every visual change, while a class name does not. This habit has the highest return, and it needs cooperation from whoever writes the interface.

Fall back to accessible roles and labels where attributes are not available. They change less than styling and are meant to be stable.

Never match on text a marketer might edit, and never match on position.

Wait for conditions, not durations. Waiting until an element is present is stable. Waiting two seconds is a timing bug that fails on a slower day.

Dismiss known interruptions explicitly and fail on unknown ones, so the check reports a new banner instead of silently working around it.

Capture a screenshot on failure. The failure is visual, so a screenshot is the most useful diagnostic.

Test data in production

Test data in production needs six decisions, because these checks run against the real system.

Use a dedicated account, clearly marked, and never a real customer's.

Exclude the account from analytics and reporting. Otherwise your conversion numbers include a robot completing checkout every five minutes.

Decide about writes. A check that places an order creates real data that someone must clean up, and a check that never writes cannot verify the transaction that matters.

Clean up deliberately, and monitor the cleanup. Debris accumulates and eventually confuses anyone reading real data.

Suppress side effects. A checkout check can send an email, charge a card or notify a fulfilment team. Suppress each effect for the test account.

Rotate the account's credentials like any other, and alert distinctly when the check fails on authentication. Otherwise an authentication failure looks like a broken site.

Alerting without false alarms

Alert only on failures that persist. These rules cut false alarms.

Require consecutive failures from more than one location before alerting. A single failure is normal.

Retry once immediately before counting a failure, because most false failures are transient.

Alert differently for slow and for broken. The responses differ.

Suppress alerts during deployments with a time window, so every release does not produce alerts.

Track the false alarm rate per check. A check that alarms falsely more than occasionally costs more attention than it protects. Fix it or delete it.

Report recoveries, so nobody assumes a problem is still ongoing.

Measuring the journey, not only whether it worked

A check that drives the whole journey also collects timing data, and most teams discard that data. Use it in five ways.

Record the duration of each step, not only the total, so a slow journey points at the slow part.

Track durations over time. A step that has doubled since last month is a regression nobody reported, because it still works.

Compare against the same hour on the same weekday. Load patterns make raw comparisons misleading.

Watch the trend before the threshold. A step that creeps upward for three weeks is a better signal than the day it finally breaches the limit.

Correlate with releases. A step that slows sharply the day after a deployment tells you where to look.

Report the results monthly to whoever owns the journey, since performance degradation is gradual and nobody notices it from inside.

Keeping checks working

Checks keep working when you treat them as code and run them from outside your infrastructure.

Run the check from outside your infrastructure, so an outage does not take the monitoring down with it.

Treat scripts as code. Keep them in version control and review them with the same care as tests.

Update scripts with the interface. A redesign that breaks three checks should fix them in the same change, which requires whoever ships the redesign to know the checks exist.

Delete instead of muting. A muted check still runs, still costs money and still looks like coverage while catching nothing.

Report the outcome: checks run, passed, failed, false alarms and duration trends.

Limits worth knowing about

Synthetic monitoring has five limits.

Frequency costs money, per check per run. A five-minute interval across many journeys adds up faster than people expect.

Synthetic traffic is not real traffic. It exercises one path with one account, so it will not find a bug that affects one customer's data.

Browser checks are slower and more fragile than API checks. Use browser checks only where the browser is the point.

Geographic coverage is limited to where your provider runs, which can differ from where your customers are.

Synthetic checks do not replace real-user monitoring. Real-user monitoring sees what people experience, including the segments a script never represents.

What to build first

Build one check for your single most important journey. Run it from outside your network, agree test attributes with whoever owns the interface, and alert after two consecutive failures. One reliable check beats a suite of twelve flaky ones, and the attribute agreement determines whether the check still works in six months.

Two habits make the difference. First, ask for dedicated test attributes instead of matching on class names or text, since that one change removes most of the maintenance. Second, track the false alarm rate per check: a check that alarms falsely costs more attention than it protects, so fix it or delete it.

Frequently asked questions

How many synthetic checks should I have?

Three to five journeys for most products. A large suite goes unmaintained, and an unmaintained check eventually reports nothing while still looking like coverage.

Why do synthetic checks break so often?

Checks break because they depend on an interface that changes. Selectors, timing, unexpected banners and reworded content all break a script without any fault in the product. Test attributes reduce this problem the most.

What is the single best practice?

Add dedicated test attributes to the elements your checks interact with. They survive redesigns that break class names and text matching, and agreeing them with whoever builds the interface removes most of the ongoing maintenance.

Should checks write data in production?

It depends on what you need to verify. A read-only check cannot confirm the transaction works. A writing check creates data and side effects that you must suppress and clean up. Decide deliberately instead of by default.

How do I stop test traffic polluting analytics?

Exclude the test account explicitly everywhere: analytics, conversion reporting and every operational dashboard. Otherwise a robot completing checkout every five minutes appears in numbers that people quote.

When should I delete a check?

Delete a check when it alarms falsely more than occasionally and you will not fix it, or when the journey no longer matters. Muting leaves the check running and costing money while it catches nothing.

Do synthetic checks replace real-user monitoring?

No. Synthetic checks exercise one path with one account on a schedule. Real-user monitoring sees what actual people experience, including the segments and devices a script never represents. The two answer different questions.

What should I do with the timing data?

Record timing per step instead of per journey, track the trend, and correlate it with releases. A step that has slowly doubled is a regression nobody reported, because it never failed. It only got slower.

How do I spot gradual degradation?

Compare each step against the same hour on the same weekday and watch the trend instead of the threshold. Performance decays slowly enough that nobody inside notices, so send a monthly report to the journey's owner.

Related reading

• Automating API health checks
• Automating website uptime monitoring
• Shift-left testing