Alerts and notification rules

What raises an alert

Lifecycle

open → acked → resolved. Re-occurrence bumps a count on the same alert; only an escalation (warning to critical) re-announces. The first notification waits ALERT_NOTIFY_DELAY_SEC (default 60) so BE AI's assessment from the automatic investigation can ride along.

Notification rules

Per company, under Companies → manage → alerts. A rule is: which machines (by tag; empty means all), which kinds (empty means all), a minimum severity, and contacts. Several rules can match; each contact is told once. A test button sends a sample through the rule.

Channels

Unconfigured channels are skipped and say so. Every delivery, sent, failed or skipped, is a row under the company's notifications.

Acknowledging and discussing

Ack an alert to stop re-announcements while you work on it. Discuss opens the machine's conversation with BE AI with the alert as context.

Accepting a condition

Some alerts describe a state that is known and acceptable: a laptop whose 61 GB drive is always 92% full, a check that fails by design until a migration is done. Acknowledging only quiets the messages; BE AI would still investigate it every morning and propose the same 30 MB clean-up. Accept is the decision that nothing is wanted.

Press accept on a disk, check or network alert (downtime and reboots are events, not conditions). Write why it is acceptable in your own words: BE AI reads that sentence. For a disk, set where it should wake you again: a used percentage, a free-space floor in GB, or both, and optionally an end date. The hub refuses a disk acceptance with no way back at all, and a re-arm point below where the disk already is.

While accepted: no investigation starts for that condition on that machine, nobody is notified when it fires again (the alert quietly counts the re-occurrences), the alert shows accepted with the note and the re-arm point, it leaves the attention counts, and every BE AI run on the machine is told the condition is known and that no fixes or clean-ups are wanted for it.

It re-arms itself: when a heartbeat shows the disk past the percentage or under the free-space floor, or when the end date passes, the acceptance is released, the alert goes back to open, and one message goes out marked WORSE, whatever the severity, so the silence you asked for is over the moment it stops being true. An acceptance also goes away on its own when the condition clears. Press release on the alert to end it by hand. Every acceptance and release is in the audit trail with who, why and the re-arm point.

Webhooks and the API

Email, SMS and phone calls reach people. A webhook contact reaches systems: a ticketing tool, a chat channel, an on-call service, or your own code. Add one under a rule's contacts with the channel webhook / API, pick the format the receiver wants, and use test to prove it before an alert does.

Headers. Up to ten extra headers travel with every call, which is how the receiving system's own authentication works: an Authorization: Bearer … or an X-Api-Key. They are stored with the rule and visible to the company's admins.

Signing. With a secret set, every call carries X-BE-Timestamp (Unix seconds) and X-BE-Signature: sha256=…, the hex HMAC-SHA256 of timestamp + "." + body under the secret. Verify it before trusting a call, and reject timestamps older than a few minutes:

import hmac, hashlib
def verify(secret, headers, body):            # body = the raw request bytes
    ts, sig = headers["X-BE-Timestamp"], headers["X-BE-Signature"]
    mac = hmac.new(secret.encode(), (ts + ".").encode() + body, hashlib.sha256).hexdigest()
    return hmac.compare_digest("sha256=" + mac, sig)

Delivery. Every call also carries X-BE-Event (opened, escalated, acked, resolved or test) and X-BE-Alert (the alert id). A network error or a 5xx answer is retried after 2 and 10 seconds; a 4xx is final, since the receiver has rejected the call. Each attempt's outcome is a row under the company's recent notifications, with the receiver's status code and the first line of its answer when it failed.

The generic payload, version 1.

{
  "version": 1,
  "event": "opened",                       // opened | escalated | acked | resolved | test
  "sent_at": "2026-09-08T02:10:44.120Z",
  "alert": { "id": "al_…", "key": "offline", "kind": "offline", "severity": "crit",
             "title": "srv-books is offline", "summary": "…", "status": "open",
             "opened_at": "…", "resolved_at": null, "acked_by": null, "count": 1,
             "assessment": "BE AI's assessment, when one was attached" },
  "machine": { "id": "…", "name": "srv-books", "hostname": "SRV-BOOKS", "display_name": null,
               "os": "windows/Microsoft Windows Server 2022", "version": "0.9.14",
               "tags": ["site/Melbourne", "role/DC"], "online": false },
  "company": { "id": "t_…", "name": "Hillcrest Dental" },
  "links": { "console": "https://hub.[yourdomain]/ui#fleet/…", "alerts_api": "https://hub.[yourdomain]/alerts/al_…" },
  "subject": "[BE Hub] CRIT · srv-books is offline · Hillcrest Dental",
  "text": "the same plain-text summary the email carries"
}

The other direction. Anything the console shows is also on the HTTP API with an API key: GET /alerts?status=active lists open alerts for the companies the key can see, POST /alerts/:id/ack and POST /alerts/:id/resolve close the loop from a ticketing system, and GET /notifications lists deliveries. See the API reference.