How to connect Elasticsearch to Slack with queries as alert definitions

To connect Elasticsearch to Slack, define each alert as a query, a threshold and a schedule, and have CodeWords run the query on that schedule and post the result to a Slack channel. The query selects the documents that matter, the threshold decides when the count is abnormal, and the schedule decides how often to ask.

Most of the difficulty is in the query, because a query is a statement about the shape of your data, and that shape changes. Queries that survive are narrower and more explicit than the first ones people write. Alerts that get read answer a specific question and do not try to catch anything unusual.

What we'll cover

• An alert is a query, a threshold, and a schedule
• Queries that stay true
• Building it with CodeWords
• Aggregations, which do the work
• Alerting on absence
• Keeping the query cheap
• What the message should contain
• Reviewing the alert suite
• Making it survive
• Limits worth knowing about
• What to set up first
• Frequently asked questions

Key takeaways

• An Elasticsearch alert has three parts: a query, a threshold and a schedule, and each should be written down.
• Match on structured fields, not message text, because reworded logs make a query stop matching with no error.
• Use aggregations with size zero so the cluster does the counting and the alert shows which services or customers are involved.
• Alert on absence as well as on spikes, because a stopped log stream looks identical to a healthy system.
• Review the alert suite quarterly, and delete alerts that never fire or never lead to action.

An alert is a query, a threshold, and a schedule

An alert is a query, a threshold and a schedule, and naming the three parts makes each decision clearer.

The query selects. Examples are errors from one service, failed logins, requests above a latency, or orders that failed to process. The narrower the query, the better.

The threshold decides. It can be a count, a rate, or a comparison against the same window last week. Absolute counts are the easiest to set and the most brittle.

The schedule asks. Running every minute costs more and is rarely needed. Most useful alerts run every five to fifteen minutes.

The window matters too. A count over five minutes and a count over an hour behave very differently. Match the window to how quickly the thing you are watching develops.

Write each part down. If an alert's definition lives only in a configuration file nobody reads, nobody can evaluate it when it fires at three in the morning.

Queries that stay true

Queries stay true when they match on stable fields, and stale queries are the reason log alerts decay.

Match on structured fields, not on message text. A query that looks for a phrase in a log line breaks the day someone rewords the message. Nothing tells you; the alert stops firing.

Require the fields you filter on to exist. A filter on a field that stopped being emitted matches nothing, which looks the same as a healthy system.

Be explicit about the index pattern. A pattern that stops matching after an index naming change produces zero results and no error.

Avoid wildcards at the start of a term. They are expensive and are often not what you meant.

Test the query against a period you know had the condition. An alert that has never matched anything is usually broken, not lucky, and this check finds it.

Review quarterly. Logging changes and services get renamed, and an alert suite nobody audits turns into decoration.

Building it with CodeWords

CodeWords connects to 3,000+ integrations (CodeWords integrations directory, 2026), and you make each connection once and reuse it.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: which query, what threshold, how often, and who should hear about it.
3. Authorize the connection with credentials for a role that can read only the relevant indices, and authorize your Slack workspace.
4. Describe the exceptions: a query returning nothing when it should return something, a cluster that is slow to respond, a threshold crossed during a known deployment.
5. Test against a historical window where you know the condition occurred.

You describe the outcome; Cody builds it, connects it, and deploys it. The Free plan covers light use. Pro costs $39 per month and Business costs $100 per month as usage grows (CodeWords pricing page, 2026); details are on the pricing page.

Aggregations, which do the work

Use aggregations, not document searches, to count in Elasticsearch alerts. Returning documents so your automation can count them is the common approach and usually the wrong one.

Aggregate in the cluster. A count, a terms aggregation or a date histogram returns a small result instead of thousands of documents.

Set the document size to zero when you want only the aggregation. This avoids transferring hits you will discard.

A terms aggregation shows the shape, not only the count: which endpoints, customers or hosts produce the errors. That is the difference between an alert someone can act on and one that says a number went up.

A date histogram shows whether it is a spike or a level. The response differs for each.

Watch cardinality. A terms aggregation over a high-cardinality field is expensive. Ask for the top ten, not everything.

Alerting on absence

To alert on absence, set a threshold on a count below a minimum, not only above a maximum. A search cannot express absence directly, and absence is the condition that catches real outages.

A query for documents that are not there returns zero, which looks identical to a healthy system with no errors.

Alert on a count below a threshold. A service that normally logs continuously and has stopped is either idle or dead.

Include a minimum expected volume per index or per service, derived from its own history.

Account for the schedule. A batch job that logs once an hour should not alert during the other fifty-nine minutes of silence.

This alert catches an ingestion pipeline failure. When logs stop arriving entirely, every other alert goes green.

Keeping the query cheap

Keep alert queries cheap, because alerts run constantly and an expensive one competes with everything else that uses the cluster.

Filter by time first and narrowly. Searching the last five minutes is far cheaper than searching the last day.

Use filter context instead of query context where you do not need scoring. Filters are cacheable, and scoring is wasted work for an alert.

Restrict the query to the indices that matter, not every index a pattern matches.

Do not run every alert every minute. Stagger schedules so twenty alerts do not run their queries at the same moment.

Watch the cost of the alerting itself. A suite of alerts that queries a busy cluster every minute is real load, and it grows as people add alerts.

What the message should contain

The Slack message should contain six things.

The count and the comparison. State the threshold and the current value, so the reader can judge severity at once.

The shape from the aggregation. List the top few contributors. This turns the alert into a starting point.

A representative example. Include one trimmed document, because an error message is worth more than a count.

A link to the query in your search interface, pre-filtered to the same window, so investigation takes one click.

Whether the count is rising or steady. An alert that has fired for an hour and one that just started need different responses.

Not the raw documents. A message with fifty log lines is unreadable, and the link covers it.

Reviewing the alert suite

Review the alert suite every quarter, because alert suites decay in one direction and only a deliberate review reverses it.

Count what fired and what was acted on. An alert that fires weekly and has never changed anyone's behaviour costs attention for nothing.

Find the alerts that have never fired. Some wait for a rare condition, but most are broken queries nobody noticed. Test each against a historical window that should have matched.

Find the alerts that fire constantly. The cause is a threshold problem or a real problem nobody is fixing. Decide which, instead of muting.

Check ownership. Nobody feels responsible for an alert whose owner has left, so people ignore it.

Delete instead of muting. A muted alert still runs, still costs cluster time, and still looks like coverage on a list.

Keep the review short. It takes about an hour, and it is the only thing that stops an alerting suite turning into decoration.

Making it survive

Deduplicate. An alert that fires every five minutes for the same ongoing condition should produce one message with updates, not twelve messages.

Resolve explicitly. Say when the condition clears, or people assume it is still broken.

Handle an unavailable cluster as its own alert, sent to a different place. A monitoring query that cannot run is a failure that looks like silence.

Use a read-only role scoped to the relevant indices.

Report the outcome. Alerts evaluated, fired, and suppressed as duplicates.

Limits worth knowing about

Deep pagination is capped by default, which matters when an alert tries to return everything it matched. Check the limits in the Elasticsearch documentation for your version.

Aggregations have memory limits. A high-cardinality terms aggregation can be refused instead of running slowly.

Index patterns resolve at query time, so a naming change silently changes what is searched.

Field mappings matter. A field mapped as text behaves differently in aggregations from one mapped as keyword, and a wrong mapping produces confusing results, not errors.

Watcher and alerting features depend on your licence in some deployments. Check your licence before deciding whether to build outside Elasticsearch.

What to set up first

Set up an absence alert on your most important log stream first: a count below the minimum you expect for that service at that time of day. This alert catches an ingestion failure, and an ingestion failure turns every other alert you have into false reassurance.

Two habits make the difference. Derive the minimum from the service's own history for each hour, because a flat number fires every night. And route the alert to a different channel from your other alerts, because when logging breaks, the alerts that depend on logging are the ones you cannot trust.

Frequently asked questions

Why did an alert stop firing?

The query usually stopped matching. Three causes are common: a reworded log message, a renamed field, or an index pattern that no longer resolves. All three produce zero results and no error, which looks the same as a healthy system. Test the query against a historical window that should have matched, and fix the field or pattern.

How do I alert on something not happening?

Alert on a count below a minimum, derived from the service's own history and aware of the time of day. A search for absent documents returns zero, which looks exactly like everything being fine. Set the minimum per hour, so a quiet night does not fire the alert, and route it to a separate channel.

Should the alert return documents or an aggregation?

Return an aggregation with size zero. An aggregation transfers a fraction of the data, and it shows the shape: which endpoints or customers are involved. That detail makes the alert actionable instead of just a number going up. Use a terms aggregation for the contributors and a date histogram for spike or level.

How often should alerts run?

Run most alerts every five to fifteen minutes. Running every minute is rarely necessary, and it multiplies the load on the cluster. Once you have more than a handful of alerts, stagger the schedules so they do not all run their queries at the same moment.

Why is our alerting slowing the cluster?

The alerts are themselves load on the cluster, and that load grows as people add alerts. Filter narrowly by time, use filter context instead of scoring, restrict each query to the relevant indices, and stagger the schedules so alerts do not all run at the same moment.

What should the Slack message contain?

Include the count against the threshold, the top few contributors from the aggregation, one trimmed example document, a link to the same query in your search interface, and whether the count is rising or steady. Leave out the raw documents, because fifty log lines in a message are unreadable and the link covers them.

Should I use built-in alerting instead?

Yes, use built-in alerting where your deployment includes it and it covers the case. Build outside Elasticsearch when you need to join with data from other systems, route by business logic, or send to a destination the built-in connectors do not reach. Check your licence first, because some alerting features depend on it.

How do I tell which alerts are worth keeping?

Count how often each alert fired and how often anyone acted on it. Alerts that never fire are usually broken queries, so test each against a window that should have matched. Alerts that fire constantly need a threshold change or a fix to the underlying problem. Keep the alerts that lead to action.

Is muting an alert enough?

No, muting is not enough. A muted alert still runs, still costs cluster time, and still appears as coverage on a list while catching nothing. If the alert is not worth acting on, delete it. If it is worth keeping, fix the threshold or query so people respond to it.

Related reading

• Elasticsearch and CodeWords
• Automating log analysis
• CodeWords integrations and templates