How to automate API monitoring: watching an API you publish

Most API monitoring watches whether the service is responding. That is necessary and it is not what breaks an API relationship. What breaks it is a single consumer whose integration stopped working three weeks ago, against an API whose aggregate metrics never moved.

Monitoring an API you publish means monitoring it per consumer, and watching the things that only matter when somebody else depends on you: the contract, the versions in use, the deprecations nobody has acted on, and the quota somebody is about to hit.

What we'll cover

• Aggregate health hides consumer failure
• Monitoring per consumer
• Contract drift, which nothing catches
• Building it with CodeWords
• Version usage and deprecation
• Quota, before it is exceeded
• Telling the consumer
• What the consumer should be able to see themselves
• Making it survive
• Limits worth knowing about
• What to build first
• Frequently asked questions

Aggregate health hides consumer failure

One consumer is a small fraction of traffic. Their integration failing entirely may not move the overall error rate at all.

Error budgets are averages. A 99.9% success rate is compatible with one customer receiving nothing.

The failure is often theirs, not yours — a malformed request, an expired credential, a field they stopped sending — and it produces a 4xx that your monitoring correctly ignores.

They may not tell you. A consumer whose integration breaks frequently silently stops using the API rather than filing a ticket.

The first signal is usually churn, which is the most expensive possible way to find out.

So the unit of monitoring is the consumer, not the endpoint, and that single change is what makes API monitoring useful to the business rather than to the on-call rota.

Monitoring per consumer

Error rate by API key or client id, compared against that consumer's own baseline rather than a global threshold.

Traffic disappearance. A consumer whose calls stop is either finished with you or broken, and both are worth knowing about within a day.

4xx patterns per consumer, since a spike in 401s means an expired credential and a spike in 422s means they changed something.

Latency as they experience it, which differs by geography and by the endpoints they actually call.

Endpoint mix. A consumer suddenly hammering one endpoint is either a new feature or a retry loop, and the second is worth an early conversation.

New consumers in their first week, which is when integration problems concentrate and when a small amount of attention buys the most.

Contract drift, which nothing catches

The changes that break consumers are rarely the ones anybody labelled breaking.

A field that becomes optional breaks a consumer who was relying on it being present.

An enum gains a value and a consumer's switch statement falls through.

A default changes, which is invisible in the schema and visible in their output.

Ordering stops being stable, which nobody documented as a guarantee and several consumers depended on.

Nullability changes on a field somebody was parsing strictly.

Catch it with recorded real traffic: replay a sample of actual consumer requests against the new version and diff the responses. It is the only approach that finds the behaviours you never wrote down.

Building it with CodeWords

CodeWords connects to more than 3,000 integrations, so the gateway logs, the customer records, the issue tracker and your communication channels can meet in one automation.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: which consumers, which signals, and who gets told when one is in trouble.
3. Authorize the connections — your API gateway or log source, the CRM, and Slack or email.
4. Describe the exceptions: a consumer in a trial, an internal client, a known migration in progress.
5. Run it over last month's traffic first and see how many consumer-level problems it would have found.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

Version usage and deprecation

Know who is on what, per consumer, continuously. Most teams find this out by asking, at the point where it is already urgent.

Deprecation needs a real date and a count of who is still affected, otherwise it is a suggestion.

Track movement, not just usage. A version whose consumer count is falling is on track; one that is flat six weeks before the cutoff is not.

Alert on new adoption of a deprecated version, which happens more than anyone expects and is entirely preventable.

Report the long tail by name, since the last few consumers on an old version are usually a short list that somebody can simply contact.

Sunset headers help and are not enough on their own, because nobody is reading response headers until after something has broken.

Quota, before it is exceeded

Warn at a proportion of the limit, per consumer, with enough time for them to do something.

Watch the trend, not the instantaneous figure, since growing usage is the case where the warning is genuinely useful.

Distinguish a burst from growth. A retry storm and a successful product launch look identical in a rate counter and need opposite responses.

Tell them what to do — a higher tier, a batching endpoint, a caching suggestion — rather than only that they are near a limit.

Surface their own numbers to them where you can, since a consumer who can see their usage raises fewer tickets and is less surprised.

Rate limiting should be visible in the response, with the limit, the remaining count and the reset, so a competent consumer can self-manage.

Telling the consumer

Reach a person, not a support queue. The technical contact for an integration is a specific individual and knowing who they are is worth keeping current.

Lead with what they will observe, since they may not have noticed yet.

Include the request detail — an example failing request id, the error, the timestamp — because without it your message generates a round trip.

Say whether it is yours or theirs, honestly and early, rather than hedging.

Do not automate the message for a significant incident. A templated notification about a serious problem reads badly and generates more work than it saves.

Do automate the small ones. An expiring credential, a deprecated version, an approaching quota — these are exactly the messages that should send themselves.

What the consumer should be able to see themselves

Every alert above is something you noticed on a consumer's behalf. The higher-leverage version is giving them the same visibility, because a consumer who can answer their own question does not need you to notice.

Their error rate and recent failures, with request ids, so their engineer can debug without opening a ticket.

Their usage against the limit, current and trending, which removes the most common surprise.

Which version they are on and when it stops being supported, stated as a date rather than a policy.

A machine-readable status for their own monitoring, since sophisticated consumers want to alert on your API inside their own systems.

Webhooks for the events that matter to them — credential expiring, version sunsetting, quota threshold — so they can automate their side.

This is a product decision, not a monitoring one, and it is usually the cheapest reduction in support load available to an API team.

Making it survive

Do not run this inside the API it is monitoring, which is the obvious trap.

Sample rather than process everything where traffic is high, and be explicit about the sampling rate in what you report.

Handle consumer identity changes. Keys are rotated and clients are renamed, and a consumer who looks new is often the same customer.

Respect data protection in the logs you retain, since request bodies contain more than you want to keep.

Report the outcome: consumers checked, anomalies raised, notifications sent, and which ones were acted on.

Limits worth knowing about

Gateway logs vary in what they retain and for how long, and what you can monitor is bounded by that.

Attribution is imperfect when consumers share keys or proxy through one another.

4xx is ambiguous. Some are the consumer's fault, some are your documentation's fault, and telling them apart needs the request detail.

Low-volume consumers are statistically awkward — their baselines are noisy and threshold alerting on them produces mostly false positives.

Replay testing needs care with side effects, so it belongs against a staging environment with recorded traffic rather than against production.

What to build first

The consumer-disappearance alert: any consumer whose traffic drops to zero or near it, compared against their own recent baseline, reported by name with their contact. It is read-only, it is cheap, and it catches the failure that costs most — an integration that broke silently and a customer who was quietly deciding to stop using you.

Two habits make the difference. Compare each consumer against their own baseline rather than a global threshold, since a global one either ignores your small consumers or drowns you in their noise. And keep the technical contact for each integration current, because every alert this system produces is useless if it ends in a support queue nobody routes.

Frequently asked questions

Why is aggregate API monitoring not enough?

Because one consumer's integration can fail completely without moving the overall error rate. A 99.9% success rate is entirely compatible with one customer receiving nothing at all.

What should be monitored per consumer?

Error rate against their own baseline, traffic disappearance, 4xx patterns, latency as they experience it, endpoint mix, and their first week of integration.

What kinds of change break consumers unexpectedly?

A field becoming optional, an enum gaining a value, a default changing, ordering ceasing to be stable, and nullability changes. None of these is labelled breaking and all of them break somebody.

How do I catch contract drift?

Replay a sample of real recorded consumer requests against the new version and diff the responses. It is the only method that finds the undocumented behaviours consumers actually rely on.

How should deprecation be tracked?

By consumer, continuously, with a real cutoff date and a named list of who is still affected. Watch whether the count is falling, and alert on any new adoption of a deprecated version.

When should quota warnings fire?

At a proportion of the limit, based on the trend rather than the instantaneous figure, with enough time to act and a specific suggestion for what to do about it.

Should consumer notifications be automated?

The small ones, yes — expiring credentials, deprecated versions, approaching quota. Not significant incidents, where a templated message reads badly and generates more work than it saves.

Why do 4xx errors matter if they are the consumer's fault?

Because some of them are your documentation's fault, and because a consumer whose integration returns errors for three weeks frequently churns rather than filing a ticket.

Should consumers see their own API metrics?

Yes, and it is usually the cheapest support-load reduction available to an API team. Their error rate with request ids, usage against the limit, their current version and its sunset date, and webhooks for the events that affect them.

Why give consumers a machine-readable status?

Because sophisticated consumers want to alert on your API inside their own monitoring. If you do not provide it they will build something fragile by scraping, and it will page them incorrectly.

Related reading

• Automating API health checks
• Automating Slack alerts from monitoring tools
• CodeWords integrations and templates