Skip to content
lube

Alerting

Tag policiesSlack / Teams / ChatWebhookSSL expiry

Page the right people, not everyone.

Route monitor and heartbeat failures — and SSL certificate expiry — to the right chat channel or webhook with tag-based policies. The right people are paged, and no one drowns in all-or-nothing noise.

Start free

alert dispatch

firing
checkout-api is DOWNpolicy: payments-oncall · tag_matchp1
  • #payments-incidentsslack
  • Ops · Reliabilityteams
  • SRE roomchat
  • pager.internalwebhook

outbound only · one-way delivery

An alert leaving for its channels

Its place in the loop

Alerting is how reliability reaches a person.

One connected loop, held on the stage this capability serves. The other stages stay as context so you can see what feeds in and what comes next.

BD
Discovery
Feedback
Product
Engineering
Release
Reliability
Evidence

Stage inventory

Reliability

Know whether it held.

Web performance

Review Lighthouse report evidence against configured targets.

Available now

Test intelligence

Investigate test outcomes and flakiness evidence across runs.

Available now

Coming later reflects product vision, not a delivery commitment.

How routing works

A failure never blasts everyone. Each policy decides what it listens for — all targets, or just the ones carrying your tags (AND or OR) — and which monitors, heartbeats, or both it scopes to. Every matching failure fans out to exactly the channels that policy binds, and nothing else.

The policies list

Every route, on one screen.

The alerts roster shows each policy with its enabled state, its match summary, the targets it scopes to, and the channels it binds — so you can read who gets paged for what at a glance, and mute a policy without deleting it.

alert policies3/4 enabled
  • payments-oncallenabled
    tag_match · ANDmonitors + heartbeats
    #payments-incidentsOps · Reliability
  • edge-degradationenabled
    tag_match · ORmonitors
    SRE roompager.internal
  • nightly-jobsenabled
    tag_match · ANDheartbeats
    #data-alerts
  • catch-alldisabled
    all targetsmonitors + heartbeats
    #ops-firehose
Alert policies for the workspace

Tag-based policies, not a single firehose

Match on tags, fan out to channels.

A policy matches all targets or a tag set (AND or OR across tags), scopes to monitors, heartbeats, or both, and every matching failure fans out to the channels that policy binds.

payments-oncall

tag_match · ANDmonitors + heartbeats
#payments#tier-1
  • #payments-incidentsslack
  • Ops · Reliabilityteams
  • SRE roomchat
  • pager.internalwebhook

matched targets → every bound channel

Tag policy routing to channels

Precision is the noise control

Fewer pages, by design.

Tag-matching is how the noise stays down. The same failing workspace routed through a match-all firehose pages everyone; scoped to a tag set, the same event pages just the room that owns it. No all-or-nothing switch.

match: all

every target fires, everyone is paged

24/24 targets → #ops-firehose

match: tag_match tier-1

only tagged targets fire

6/24 targets → #payments-incidents

Routing reach, two ways

What the routing engine gives you

Four outbound channels, every failure condition worth a page, per-monitor overrides for the one noisy service, and a one-command connect flow. All of it one-way — lube sends, it never listens.

Inside the engine

Channels, triggers, overrides, and connect.

Channels

Four ways out, all outbound.

Connect Slack, Microsoft Teams, or Google Chat over OAuth and pick a channel, or point a generic webhook at your own endpoint. Delivery is one-way — lube sends, it does not listen. Discord routing is coming soon.

slackmicrosoft teamsgoogle chatwebhook
channeldelivery
  • slack#payments-incidentsworkspace channelout
  • teamsOps · Reliabilityteam channelout
  • chatSRE roomspaceout
  • webhookpager.internalhttps endpointout
Bound alert channels

Triggers

Every condition that warrants a page.

Monitors alert when they go down or degraded, heartbeats alert when a ping is missed or explicitly failed, and certificates alert before they expire.

monitor down / degradedheartbeat missed / failedssl expiry
  • Monitor downhttp / tcp
  • Monitor degradedhttp / tcp
  • Heartbeat missedheartbeat
  • Heartbeat failedheartbeat
  • SSL certificate expiryhttp
Alert triggers

Per-monitor assignment

Pin channels to a single monitor.

Beyond policies, any monitor can carry its own channel assignment, so one noisy service can page a dedicated room without rewriting a policy.

monitor → channelspolicy or directno rewrite
monitorslackteamschatwebhook
  • api.lube.work●––●
  • checkout-api●●●●
  • nightly-sync–––●
Monitor → channel assignment

Connect

Connect once, then route.

Authorize a chat platform over OAuth, search its channels to bind the right one, or register a webhook by name, URL, and secret. From there every matching failure dispatches automatically.

oauth connectchannel searchwebhook config
  1. 01

    Connect

    OAuth: Slack / Teams / Chat

  2. 02

    Find

    search integration channels

  3. 03

    Bind

    to a policy or a monitor

  4. 04

    Route

    matched failures dispatch

Connect a channel

The config surface

Every knob a policy exposes.

Policies and channels are configured, not hardcoded. This is the full shipped surface behind the routing engine.

Policy
Name it, enable or disable it, and bind one or more channels. Matching and scope decide what it listens for.
nameisEnabledmatchTagIdschannels[]
Match type
Route on all targets, or only those carrying the policy's tags.
alltag_match
Match logic
When tag-matching, require every tag or any tag to match.
andor
Target types
Scope the policy to monitors, heartbeats, or both.
monitorsheartbeatsboth
Chat platforms
Connected chat channels, single account each, over OAuth with channel search. Outbound alerts only, never two-way apps. Discord is coming soon.
slackmicrosoft_teamsgoogle_chat
Webhook config
A generic outbound endpoint you own — name it, point it at a URL, and sign requests with a secret.
nameurlsecret
Per-monitor channels
Assign channels directly to a single monitor, independent of any policy, for a dedicated room.
PUT /monitors/:id/channels
Delivery
Every channel is send-only. lube dispatches the alert out; it does not read replies or act on inbound messages.
outbound onlyone-way

Outbound, by design

Slack, Microsoft Teams, and Google Chat are alert-delivery integrations, not two-way apps. lube sends notifications out to a channel or webhook; it does not run bots, read replies, or act on messages coming back. There is no PagerDuty, Opsgenie, or Datadog integration and no acknowledgement loop — routing ends when the message is delivered.

Connected context

What the alert is about.

Uptime

The monitors

The checks that fire.

Incident management

The response

Coordinate once it fires.

Status pages

The public view

Tell everyone else.

Send the alert where it belongs.

Start free

Connect a channel and tag a policy to start routing.