Alerts

Alerts go to your team when something happens to the validators your organization monitors. This page lists every alert type with its trigger and severity, explains how offline keys become one outage with one recovery, and documents rules, destinations and the webhook format.

How alerts work

Four pieces, all under My Alerts and My Validators once you are signed in:

Monitor: What to watch: a Lido operator, or a withdrawal address (every validator withdrawing to it). See Create a monitor.
Event: Something Roundbeat saw on chain for a monitored validator or operator: a missed proposal, a key offline, an exit request, a bond penalty.
Rule: Which events, from which monitors, at what minimum severity, go to which destinations, and how often the same alert may repeat.
Destination: Where alerts go: a Telegram chat, a Slack channel, a PagerDuty service or a webhook.

An event reaches a destination only when a rule routes it there. A monitor with no rule collects events in your feeds but sends nothing. Severities are fixed per type: Critical means act now, Warning needs attention soon, Info is routine.

Alert types

These are the types the rule editor offers, in its groups and order. The name is what the rule editor and your alert history show; the code below it is the event_type a webhook receives. Some types have a different name in the public event feed, noted under the name.

Validator

Beacon chain events for any validator a monitor covers, in every module.

Validator alert types
Alert	Severity	Fires when	How often
Block missed`block_missed`Shown in feeds as Missed proposal	Critical	A validator had a proposal duty and no block was produced for its slot.	Once per missed slot. A second miss inside the cooldown still alerts: each slot is its own alert.
Attestation missed`attestation_missed`Shown in feeds as Likely offline	Critical	A key missed its attestation in 3 consecutive epochs (about 19 minutes). Critical at any stake. See offline outages.	Once per key per offline spell, grouped into one outage per operator. Not repeated while the key stays offline.
Slashing`slashing`Shown in feeds as Slashed	Critical	A validator’s beacon status became slashed.	Once per validator.
Proposal hit`proposal_hit`Shown in feeds as Upcoming proposal	Info	A validator is scheduled to propose. The beacon chain publishes proposers 1 epoch ahead, so this arrives at most about 13 minutes before the slot.	Once per scheduled slot.
Validator exited`validator_exited`	Info	A validator reached an exited status (voluntary, requested by Lido, or triggered through EIP-7002). Curated v1 validators consolidating into Curated v2 are not exits and do not send this.	Once per validator.
Validator status change`validator_status_change`	Info	A validator’s beacon status changed, for example from `pending_queued` to `active_ongoing`.	Once per transition per validator.
Sync committee`sync_committee`	Info	A validator is a member of the current or the next sync committee.	Once per validator per sync committee period.
Validator consolidation`validator_consolidation`Shown in feeds as Consolidation queued	Info	Curated v1 validators entered the beacon chain’s consolidation queue on their way into a Curated v2 validator.	One message per operator, epoch and target operator, covering all of that target operator’s validators. The later steps (source exited, completed) are shown on the pages but never alert.
Exit request`eth.exit_request`	Info	Lido’s Validator Exit Bus asked one of the operator’s validators to exit. The alert names the exit deadline, the same deadline as the reminders. Routine on its own: the deadline reminders below carry the urgency.	Once per request.
Exit deadline approaching`exit_deadline_approaching`	Warning	A validator with an exit request has not started exiting. Reminders 48 h, 12 h and 2 h before its exit deadline; they stop as soon as the exit starts.	Up to three reminders per request.
Exit deadline passed`exit_deadline_passed`	Critical	A requested validator still has not started exiting when its exit deadline passes.	Once per request, sent within 24 h of the deadline.

Curated v2 migration alerts

Alerts derived from the Curated v1 to v2 migration. Each one is also a toggle on the operator’s monitor (all on except Consolidation stake received); a rule still has to route it.

Curated v2 migration alerts alert types
Alert	Severity	Fires when	How often
Overfill risk`migration_overfill_risk`	Warning	A Curated v2 validator would pass 2,048 ETH of effective balance once its pending consolidations land. The excess would be withdrawn instead of compounding.	When a target validator newly becomes at risk.
Stuck consolidation request`consolidation_stuck`	Warning	A Curated v1 operator has a consolidation request still unprocessed after 32 epochs (about 3.4 h) while its source validator is active.	When the operator goes from no stuck request to at least one.
Rejected consolidation request`consolidation_rejected`	Warning	A consolidation request was still unprocessed after 225 epochs (about a day) with the source still active: the beacon chain dropped it, and it needs a new request.	Once per rejected request.
Migration pair change`migration_pair_changed`	Info	Lido’s ConsolidationMigrator allowed, disallowed or re-allowed a Curated v1 to Curated v2 operator pair.	Once per change, to both operators.
Migration milestone`migration_milestone`	Info	A Curated v1 operator reached 25%, 50% or 100% of its validators consolidated into Curated v2.	Once per milestone.
Consolidation stake received`consolidation_received`	Info	A Curated v2 validator received the balance of a consolidated Curated v1 validator.	One message per operator per epoch. Off by default on each monitor, because it fires every epoch while a target fills up.

Curated v2 Penalty Framework

Curated v2 only.

Curated v2 Penalty Framework alert types
Alert	Severity	Fires when	How often
Out-of-order exit`out_of_order_exit`	Critical	A Curated v2 validator started exiting without an exit request from Lido, and it is neither consolidating nor ejected for performance strikes. The Curated v2 Penalty Framework treats this as an incident.	Once per validator. An out-of-order exit that Roundbeat’s exit watch first sees more than a day after the indexer recorded it (because the watch was down) is kept in history but never alerted.

CSM & Curated v2 on-chain

On-chain events of the CSM and Curated v2 contracts, read from the chain logs. In the rule editor this group starts folded.

CSM & Curated v2 on-chain alert types
Alert	Severity	Fires when	How often
Penalty reported`lido.sm.penalty_reported`	Critical	A delayed penalty was reported against the operator: the penalty and fine are locked from bond.	Once per on-chain event.
Penalty settled`lido.sm.penalty_settled`	Critical	A delayed penalty was settled: the locked bond was burned.	Once per on-chain event.
Penalty cancelled`lido.sm.penalty_cancelled`	Info	A delayed penalty was cancelled and its bond lock released.	Once per on-chain event.
Penalty compensated`lido.sm.penalty_compensated`	Info	A delayed penalty was compensated and its bond lock released.	Once per on-chain event.
Slashing reported`lido.sm.slashing_reported`	Critical	A slashing of one of the operator’s validators was reported on chain.	Once per on-chain event.
Slashing penalty applied`lido.sm.slashing_penalty_applied`	Critical	A slashed validator’s withdrawal was reported and the slashing penalty charged from bond.	Once per on-chain event.
Validator withdrawn`lido.sm.validator_withdrawn`	Info	A validator’s withdrawal was reported on chain; its bond is released and can be claimed.	Once per on-chain event.
Bond locked`lido.sm.bond_lock_changed`	Critical	Bond was locked, or a lock increased, for a reported penalty.	Once per on-chain event.
Bond lock reduced`lido.sm.bond_lock_reduced`	Info	Part of the locked bond was released.	Once per on-chain event.
Bond lock removed`lido.sm.bond_lock_removed`	Info	No bond is locked any more: the lock was settled, compensated, cancelled or expired.	Once per on-chain event.
Locked bond compensated`lido.sm.bond_lock_compensated`	Info	The locked penalty was covered from bond and released from the lock.	Once per on-chain event.
Bond burned`lido.sm.bond_burned`	Critical	Part of the operator’s bond was burned as a penalty.	Once per on-chain event.
Bond debt increased`lido.sm.bond_debt_increased`	Critical	The bond no longer covers the operator’s obligations. Top up the bond.	Once per on-chain event.
Bond debt covered`lido.sm.bond_debt_covered`	Info	New bond or rewards paid off part or all of the bond debt.	Once per on-chain event.
Bond charged`lido.sm.bond_charged`	Warning	A fee, such as an exit-delay or EL request fee, was charged from bond.	Once per on-chain event.
Exit delay fee`lido.sm.exit_delay_fee`	Critical	A requested exit missed its deadline and an exit-delay fee was recorded against bond.	Once per on-chain event.
Triggered exit`lido.sm.triggered_exit`	Critical	A validator was force-exited through EIP-7002 and the EL request fee recorded against bond.	Once per on-chain event.
Forced exit target`lido.sm.forced_exit_target`	Critical	The operator’s target validator limit was set to forced mode: Lido will request exits above the limit.	Once per on-chain event.
Ejection requested`lido.sm.ejection_requested`	Info	A voluntary ejection was requested through the Ejector.	Once per on-chain event.
Bad performer ejection`lido.sm.bad_performer_ejection`	Critical	A validator collected too many performance strikes and will be ejected.	Once per on-chain event.
Strikes penalty`lido.sm.strikes_penalty`	Warning	A performance-strike penalty was applied to a validator.	Once per on-chain event.
Vetted keys decreased`lido.sm.vetted_keys_decreased`	Warning	Some uploaded keys were invalidated (duplicate, wrong withdrawal credentials and similar). Upload replacements to keep keys depositable.	Once per on-chain event.
Keys deposited`lido.sm.keys_deposited`	Info	Lido deposited to new keys of the operator. Make sure the keystores are loaded.	Once per on-chain event.
Rewards distributed`lido.sm.rewards_distributed`	Info	New reward distribution data was published; the operator’s share can be claimed.	Once per on-chain event.

Curated v1 → v2 migration contract events

Raw events of the ConsolidationMigrator contract.

Curated v1 → v2 migration contract events alert types
Alert	Severity	Fires when	How often
Pair allowed (contract event)`lido.consolidation.pair_allowed`	Info	The ConsolidationMigrator contract allowed a Curated v1 to Curated v2 pair. This is the raw contract event; Migration pair change is the derived alert.	Once per on-chain event.
Consolidation submitted (contract event)`lido.consolidation.submitted`	Info	Curated v1 validators were submitted to the ConsolidationMigrator for consolidation into a Curated v2 operator.	Once per on-chain event.

Events that never alert

Some events show on the pages and in the feeds but are never sent to a rule:

Low depositable keys (warning): a Curated v1 operator below 100 depositable keys, or a SimpleDVT cluster with more than 50 keys and none left. Feed only.
Consolidation exited and Consolidation completed (info): the later steps of a consolidation. Completions arrive a few per epoch for weeks, so they are tracked on the pages instead.
Back online (info): a key that was offline attested again. It closes the key’s part of an outage; the outage’s single recovery is what gets sent.

Offline keys and outages

Offline keys rarely come alone: an upgrade, a client bug or a lost machine takes many keys of one operator down at once, often in waves. Roundbeat groups them into one outage per operator and ends it with one quiet recovery.

When a key counts as offline

A key that misses its attestation in 3 consecutive epochs raises Likely offline in the feeds, sent to rules as Attestation missed (attestation_missed). It is Critical whatever the key’s balance: a 32 ETH key counts the same as a 2,048 ETH one, and the alert shows the key’s effective balance.

A key alerts once per offline spell. While it stays offline it is not alerted again, not even after a Roundbeat restart. It comes back once it has attested in 2 consecutive epochs; if it then goes offline again, it alerts again. A key that exits while offline is let go: it will never attest again.

One outage per operator

Every offline key of an operator joins your organization’s open outage for that operator. Validators of a withdrawal-address monitor form one outage per organization.
Keys come back one by one, silently. Nothing is sent as each key recovers.
A quiet period of 30 minutes. When the last key is back, the quiet period starts. Another key of the operator going offline inside it joins the same outage: no recovery, no new PagerDuty incident.
A new wave more than 2 h after the outage’s last alert starts a new outage, with a new PagerDuty incident that pages, even while a straggler keeps the old outage open. A wave within 2 h joins the open outage. So a key stuck offline for days, for example in the exit queue, cannot hide your next real outage. The old outage ends on its own once its keys are back; its recovery says that keys which went offline since are a separate outage.

The first alert of a new outage is never held back by a rule’s cooldown. Later alerts of the same outage follow the cooldown like any alert. Nothing re-pages on a timer, and a straggler alone never pages again.

The recovery

After the quiet period the outage is over. Each destination that received one of its alerts gets exactly one recovery, even if your organization has deleted the rule since. A destination that never received one (its rule filtered the alert out, or every attempt failed) gets none, and so does a destination that was deleted. An outage nobody was alerted about ends silently.

The recovery is always the outage’s last message to a destination. An alert of the outage that Roundbeat still holds (grouping, or being sent) goes out first. An alert that failed and is still waiting for a retry when the recovery goes out is dropped and marked Not sent (“the outage was over and its recovery went to this destination”): sent after the recovery, it would reopen the PagerDuty incident with nothing to close it.

The recovery names the operator, the number of keys, their total effective balance (when every key’s is known) and how long the outage lasted, from the first alert to the last key back. On Telegram, for a made-up operator, Acme Staking:

Telegram recovery

✅ Acme Staking: all 12 keys back online after 41 min

👤 Acme Staking  ·  Curated v1
🔑 12 keys back online: each attested in 2 consecutive epochs
💰 384 ETH effective balance
⏱ Offline 41 min: first alert 2026-10-03 08:23 UTC, last key back 2026-10-03 09:04 UTC
🔕 No key of the operator went offline for 30 min since: this outage is over.

ℹ️ INFO

Other titles read ✅ Acme Staking: 1 key back online after 25 min or ✅ Acme Staking: 9 keys back online, 1 no longer active, after 2 h 10 min. Recoveries appear in your alert history as Offline outage over.

What each channel receives

Outage alerts and recoveries per channel
Channel	Each wave’s alert	The recovery
PagerDuty	A `trigger` with `dedup_key` `roundbeat-outage-<outage id>`. Every wave uses the same key, so a later wave updates the open incident instead of paging again.	A `resolve` with the same `dedup_key`: it closes the incident and pages nobody.
Telegram	A message.	One message, sent without a notification sound.
Slack	A message.	One message.
Webhook	The alert event.	One event with `event_type` `attestation_recovered` (see Recovery event).

Back-online events in your feeds

Each key’s return shows only in your own feeds: My Validators’ events and the withdrawal-address monitor pages. It reads Back online, or No longer active for a key that exited. It is info, so turn on info events to see it. The public feed, the operator pages and the overview counts leave these events out.

Rules

Create rules under My Alerts → Rules with Add rule. Owners and admins can edit rules; viewers can read them. A rule has:

Rule name: Any name, for example “Ops on-call: critical”.
Scope: All monitors (every monitor in your organization, including ones added later) or one monitor. To send each operator’s alerts to a different place, give each monitor its own rule.
Events: All events, or Specific types picked from the groups above. The two cannot be mixed.
Min severity: Drops events below it: Info: receive everything, Warning: drop info-level events or Critical: only critical events. The default is Warning for an all-events rule and Info when you pick types. The editor refuses a minimum that would drop every type you picked.
Destinations: One or more. A rule needs at least one.
Cooldown (minutes): How long the same alert waits before it can be sent again. Default 5 minutes, from 0 to 1,440 (24 hours). 0 means no cooldown: every repeat is sent, and the rules list shows the cooldown as None.

Cooldowns

A cooldown counts per rule and per alert: the same type for the same validator (for proposals, the same slot; for sync committees, the same period; for status changes, the same transition). Another validator, slot or transition is a different alert and is not held back. A held-back alert is not sent and leaves no row in History. The first alert of a new offline outage ignores the cooldown.

Migration alerts on a monitor

On a Curated v1 or Curated v2 operator, the Monitor button on the operator page shows Curated v1 → v2 migration alerts toggles, one per migration type. They decide which migration alerts the monitor produces; a rule must still route them, and the popover says so when no rule does. Consolidation stake received is off by default.

Rule status

The rules list says whether each rule can actually send:

Active: Enabled, its minimum severity lets its events through, and it has a destination that delivers.
Disabled: Turned off, or turned off because its monitor was deleted.
Never fires: The minimum severity drops every event type it lists.
Not delivered: All its destinations are email, which is not sent yet.
No destination: Its destinations were deleted.

Destinations

Add them under My Alerts → Destinations with Add destination. Each has a name and one target.

Add @RoundBeatBot to your group, or start a direct chat with it.
Send any message in the chat so the bot can see it.
Get the chat ID, for example by forwarding a message to @getmyid_bot. A positive ID is a direct chat, a plain negative one a group, and -100… a supergroup or channel. A group upgraded to a supergroup gets a new -100… ID.
Paste the Chat ID and add the destination. The Bot token field is optional: leave it blank to send from @RoundBeatBot.

Slack

Create an incoming webhook in Slack and paste its Webhook URL. It must be an https://hooks.slack.com/… URL. Messages use Slack blocks with the severity, type and chain.

PagerDuty

Paste an Integration key of a PagerDuty service using the Events API v2 integration. Each alert is a trigger with the alert’s severity, source roundbeat, its labels as custom details and links to the validators on beaconcha.in. Offline alerts share one incident per outage; the recovery resolves it.

Webhook

Paste a URL starting with https://. Deliveries go only to HTTPS addresses on the public internet: the form accepts a URL that points at a private network, but its test and every delivery then fail. Roundbeat sends a POST with the alert as JSON (see Webhook payload) and counts any 2xx answer as sent.

Email

Email is not delivered yet. The form shows it, but an email destination is marked Not delivered and rules that only use email never send.

Test sends

Press Test on a destination’s row to send 🧪 Roundbeat Test Alert. A test the destination accepts marks it Verified; a failed one shows the error and marks it Unverified. Tests are limited to one per target every 60 seconds and 10 per organization every 10 minutes; past that, the app tells you how long to wait. Owners and admins can send tests.

Grouping, late events and retries

Grouping

Alerts of the same type for the same operator are held briefly and sent as one message: 1 minute for warning and critical alerts, 7 minutes for info alerts (longer than one epoch, so a whole epoch’s events land together). A grouped message is titled like 🔗 12 × attestation missed — Ethereum · Curated v1 · Acme Staking and lists the validators.

Late events

If Roundbeat falls behind and catches up, it does not replay old news as live. A beacon-chain event whose epoch started more than 2 h ago is recorded but not alerted, with three exceptions for critical events:

a slashing, an exit deadline passed or an out-of-order exit is always sent;
a missed proposal is sent up to 24 h late;
an offline key is never delivered late: its streak may be long over.

On-chain contract events follow their own cut-off: they alert up to 96 h after their block, and the penalty, slashing and bond events (penalties reported, settled, cancelled or compensated; slashing reported and slashing penalty applied; bond locks, burns, debt and charges; exit-delay fees, triggered exits and strikes) up to 60 days, since a bond lock lasts that long. A late alert’s title starts with ⏱ Delayed: and its body says when the event happened.

Retries and history

A Failed delivery is retried while it is less than an hour old. A Retrying one whose attempt has not finished within 10 minutes is retried whatever its age. Either way a delivery gets at most 5 attempts; after the last failed one it is marked Not sent. An offline alert is also marked Not sent once its outage’s recovery has gone to the destination, even before its last attempt.

My Alerts → History lists every alert Roundbeat tried to send, one row per destination, with one of these statuses:

Sent: Handed to the destination without an error.
Failed: The destination refused it or could not be reached. It may be retried.
Retrying: The first attempt failed; it is being retried.
Not sent: Given up after failing, or dropped after its outage’s recovery; it will not be retried.

The filter chips above the list group them: Sent shows Sent rows and Failed shows Failed and Not sent rows. Retrying rows show only with no chip selected.

History opens with the latest 50 rows. Load older adds 50 more at a time, up to 200. If loading fails, the rows already shown stay, and the footer says Couldn’t load older alerts with a Retry button.

Webhook payload

A webhook destination receives an HTTP POST with Content-Type: application/json and User-Agent: Roundbeat-Alerter/1.0. The body is one alert. The examples below use the made-up operator Acme Staking.

Alert event

POST body: an offline key

{
  "org_id": "5b0e3c1a-2f4d-4c7e-9a61-0d8f3b2c7e14",
  "monitor_id": "c2a9e7d4-81f3-4b6a-a0e5-7f19d4c3b862",
  "validator_index": 1205584,
  "event_type": "attestation_missed",
  "severity": "critical",
  "title": "⚠️ Offline (3 missed epochs) — Acme Staking",
  "body": "…Telegram HTML…",
  "slack_body": "…Slack mrkdwn…",
  "labels": {
    "chain": "ethereum",
    "module_id": 1,
    "module_name": "Curated v1",
    "operator_id": 41,
    "operator_name": "Acme Staking",
    "validator_index": 1205584,
    "epoch": 480797,
    "effective_balance_gwei": 32000000000,
    "outage_id": "6a1e0c9e-8a7c-4b8e-9d1f-0f3b2d1c4e5a"
  },
  "fingerprint": "eth:1:41:eth.likely_offline:1205584"
}

org_id: Your organization.
event_type: The rule type, as in the alert types tables.
severity: critical, warning or info.
validator_index: The validator, or 0 for an alert about several validators (a grouped alert) or about an operator.
title: Plain text with emoji, the same title the other channels show.
body: The message as Telegram HTML.
slack_body: The message as Slack mrkdwn.
labels: Context for filtering and routing. Every alert has chain and module_id; one about an operator also operator_id and operator_name.
Beacon-chain events (offline keys, proposals, slashings, exits, status changes, sync committees, exit deadlines): always validator_index, epoch and module_name. Offline, validator exited, exit-deadline and out-of-order exit alerts add effective_balance_gwei; offline alerts also outage_id. Sync committee alerts add sync_period, period_start_epoch, period_end_epoch, period_start_time, period_end_time and is_current; a grouped one that spans several periods has sync_periods instead.
On-chain contract events (exit requests, CSM and Curated v2 events): block_number, transaction_hash, log_index (practically always), contract, event, and validator_index when the event names one. An exit request adds requested_at, deadline, allowed_exit_delay_s (its exit window in seconds), exit_delay_fee_estimate_eth (when Roundbeat can estimate it) and pubkey.
Curated v2 migration and consolidation alerts (one per operator): epoch, event and module_name; consolidation alerts add validator_count and the target.
A late alert has "delayed": true. A grouped alert carries the labels of its first event, except a sync committee group, whose period labels cover the whole group.
fingerprint: Identifies the alert for cooldowns and de-duplication. A grouped alert has a hash of its members’ fingerprints.
monitor_id: The monitor the event reached your organization through. A retried delivery carries the zero UUID.

Validators of a withdrawal-address monitor have module_id 0, operator_id 0 and operator_name "Custom validators".

Recovery event

The end of an offline outage, sent once to each webhook that received one of its alerts. Match it to its alerts by labels.outage_id.

POST body: an outage recovery

{
  "org_id": "5b0e3c1a-2f4d-4c7e-9a61-0d8f3b2c7e14",
  "monitor_id": "c2a9e7d4-81f3-4b6a-a0e5-7f19d4c3b862",
  "validator_index": 0,
  "event_type": "attestation_recovered",
  "severity": "info",
  "title": "✅ Acme Staking: all 12 keys back online after 41 min",
  "body": "…Telegram HTML…",
  "slack_body": "…Slack mrkdwn…",
  "labels": {
    "chain": "ethereum",
    "module_id": 1,
    "module_name": "Curated v1",
    "operator_id": 41,
    "operator_name": "Acme Staking",
    "outage_id": "6a1e0c9e-8a7c-4b8e-9d1f-0f3b2d1c4e5a",
    "outage_status": "resolved",
    "keys": 12,
    "keys_back_online": 12,
    "keys_no_longer_active": 0,
    "effective_balance_gwei": 384000000000,
    "first_alert_at": "2026-10-03T08:23:00Z",
    "all_back_at": "2026-10-03T09:04:00Z",
    "duration_seconds": 2460
  },
  "fingerprint": "outage:6a1e0c9e-8a7c-4b8e-9d1f-0f3b2d1c4e5a:recovered"
}

effective_balance_gwei is present only when every key’s effective balance is known.
keys_no_longer_active counts keys that exited while offline.
monitor_id is the zero UUID when the outage’s keys reached your organization through more than one monitor.

Protect your endpoint

Requests from a webhook destination created in the app are not signed and carry no custom headers. Treat the URL as a secret: make it long and hard to guess. Every request, the test included (event_type test), carries your organization’s org_id, so your endpoint can also turn away bodies with any other.

Written for Roundbeat 1.28. Facts checked against the app source on 13 Oct 2026.