Stop polling. Subscribe to events.
Register a URL with one POST. We push a signed JSON envelope to it the moment an event fires anywhere in your org — typically under 500 ms from API action to your receiver. Retries are automatic, signing is HMAC-SHA256, broken receivers auto-disable.
TL;DR
Register a URL with one POST. We push a signed JSON envelope to it the moment an event fires anywhere in your org — typically under 500 ms from API action to your receiver. Retries are automatic, signing is HMAC-SHA...
Stop polling. Subscribe to events.
Register a URL with one POST. We push a signed JSON envelope to it the moment an event fires anywhere in your org — typically under 500 ms from API action to your receiver. Retries are automatic, signing is HMAC-SHA256, broken receivers auto-disable.
Distinct from the per-monitor alert webhook (one URL on a monitor row, fires when that monitor goes down). Subscriptions are org-scoped, fire on event types, and carry a structured envelope. Same signing scheme — the same receiver code verifies both.
Event types
incident.opened— monitor crossed its alertThreshold and an incident was auto-created (or a log-alert rule fired).incident.acknowledged— a team member acked an incident without resolving.incident.resolved— incident closed (manually OR auto on monitor recovery). Payload'sauto_resolvedflag tells you which.monitor.status_changed— a monitor'slastStatustransitioned (up → down → degraded → up). Fires on every real transition, not just incident-opening ones.log_alert.fired— a log-alert rule matched its threshold and opened an incident.detection.fired— a Sigma detection rule (community ruleset, evaluated by the detection engine) matched an event and opened an incident; payload carriesrule_id,rule_title,severity,mitre_technique, and the openedincident_id.correlation_rule.fired— a multi-event correlation rule (sequence or cardinality) matched across the log stream and opened an incident.analyst.verdict— the AI analyst finished triaging an incident; payload carries the disposition, confidence, summary, and recommended actions.analyst.escalation— the AI analyst confirmed a true-positive threat above the escalation threshold; payload adds the cited evidence.analyst.enforcement_proposed— the analyst proposed one or more enforcement actions (e.g. block IP) for human approval; no action is taken until approved.sensor.offline— a Linux Sensor host stopped shipping telemetry past the staleness window; payload carrieshostname,sensor_host_id, and the openedincident_id.sensor.online— a previously-offline Sensor host resumed shipping; the offline incident is auto-resolved and this fires with the same host identifiers.storm.detected— many related incidents sharing one root cause were auto-collapsed into a single case; payload carriescase_id,shared_entity, and the groupedincident_ids.metric_alert.fired— a metric-threshold rule breached (CPU / latency / interface util / SNMP); payload carries themetric_name, observedvalue,comparator/threshold, and the openedincident_id.
An empty eventTypes: [] on the subscription means "send me
every event type the org produces." Useful for the first-day
configuration; agents tend to narrow it later.
Create a subscription
curl -X POST https://api.24observe.com/api/v1/webhook-subscriptions \
-H 'Authorization: Bearer obs_<pat-with-webhooks:write>' \
-H 'Content-Type: application/json' \
-d '{
"url": "https://agent.example.com/24observe-events",
"eventTypes": ["incident.opened", "incident.resolved"],
"description": "incident-bot in prod"
}'
# Response (201)
{
"id": 42,
"url": "https://agent.example.com/24observe-events",
"eventTypes": ["incident.opened", "incident.resolved"],
"description": "incident-bot in prod",
"enabled": true,
"disabledAt": null,
"consecutiveFailures": 0,
"lastDeliveryAt": null,
"lastDeliveryStatus": null,
"lastDeliveryStatusCode": null,
"createdAt": "2026-05-23T18:00:00.000Z"
}
Required scope: webhooks:write. Required role:
owner or admin. SSRF preflight runs at write
time — URLs resolving to private/loopback/link-local addresses are
rejected with 400 WEBHOOK_URL_UNSAFE; DNS failures get
400 WEBHOOK_URL_UNRESOLVABLE.
Delivery envelope
POST https://agent.example.com/24observe-events
Content-Type: application/json
User-Agent: 24observe-webhooks/1.0
X-24Observe-Event: incident.opened
X-24Observe-Delivery-Id: sub-42-incident.157.opened.1
X-24Observe-Timestamp: 1716480000
X-24Observe-Signature: t=1716480000,v1=<64-char-hex>
{
"id": "incident.157.opened",
"type": "incident.opened",
"created": 1716480000,
"data": {
"incident_id": 157,
"monitor_id": 7,
"monitor_name": "checkout-api",
"monitor_url": "https://api.example.com/healthz",
"status": "investigating",
"error_message": "HTTP 503",
"opened_at": "2026-05-23T18:00:00.000Z"
}
} Verify the signature exactly as for per-monitor alert webhooks — same HMAC-SHA256 scheme, same org-level secret. Working Node / Python / Go examples on /docs/webhooks/.
Retry + auto-disable
- 5 attempts per event with exponential backoff and 30% jitter (≈ 30 s, 5 min, 30 min, 2 h, then dead).
- Any non-2xx from your receiver counts as a failure. We retry.
- Timeout: 10 s per attempt. Long-running receivers — return 202 + queue locally + process async.
- 10 consecutive failed deliveries across events = subscription
auto-disabled. We flip
enabledto false and setdisabledReasonso your dashboard shows why. A POST that succeeds resets the consecutive counter to zero.
Re-enable via PATCH after fixing the receiver. The
consecutiveFailures counter is cleared and any new event
fan-out lands again.
List subscriptions + delivery history
# List all subscriptions for your org (bare array — no envelope)
GET /api/v1/webhook-subscriptions
# Disable / re-enable / change eventTypes / change URL
PATCH /api/v1/webhook-subscriptions/:id
{ "enabled": false }
{ "enabled": true } # also clears disabledAt + disabledReason + consecutiveFailures
{ "eventTypes": ["incident.opened"] }
{ "url": "https://new-receiver.example.com/x" }
# Delete (cascades the delivery rows)
DELETE /api/v1/webhook-subscriptions/:id
# Inspect the last 50 delivery attempts for one subscription
GET /api/v1/webhook-subscriptions/:id/deliveries
# Returns: id, eventType, eventId, attempt, status, statusCode, errorMessage,
# payloadJson (exact bytes we sent), responseBodyExcerpt
# (first 1 KB of receiver's response), deliveredAt, createdAt
The payloadJson field on each delivery row is the exact
bytes we POSTed to your receiver — including the timestamp the signature
was computed against. If your receiver's signature check is failing, this
is the byte sequence to verify against.
Required scopes
webhooks:read— list subscriptions, read delivery history.webhooks:write— create / update / delete subscriptions.
Plus role: owner or admin on writes. Members and
viewers can list but not modify.
Reference receiver (Node)
Minimal Express receiver that verifies signatures and logs deliveries.
Drop in, set OBSERVE24_WEBHOOK_SECRET from GET /api/v1/me/webhook-secret,
and you're done.
import express from 'express';
import { createHmac, timingSafeEqual } from 'node:crypto';
const SECRET = process.env.OBSERVE24_WEBHOOK_SECRET;
const app = express();
app.post('/24observe-events', express.raw({ type: 'application/json' }), (req, res) => {
const sig = req.header('x-24observe-signature') || '';
const parts = Object.fromEntries(sig.split(',').map(p => p.split('=')));
const t = Number(parts.t);
if (Math.abs(Date.now() / 1000 - t) > 300) return res.sendStatus(400); // replay defense
const expected = createHmac('sha256', SECRET).update(`${t}.${req.body.toString('utf8')}`).digest();
const actual = Buffer.from(parts.v1, 'hex');
if (actual.length !== expected.length || !timingSafeEqual(expected, actual)) {
return res.sendStatus(401);
}
const envelope = JSON.parse(req.body.toString('utf8'));
const eventType = req.header('x-24observe-event');
console.log(`[${eventType}] ${envelope.id}`, envelope.data);
// Return 200 within 10s. Heavy work belongs in a job queue.
res.sendStatus(204);
});
app.listen(3000); Common mistakes
- Parsing the body before verifying.
JSON.parsemay reorder keys; the HMAC was computed against the raw bytes we sent. Use the raw body for the verification step. - Returning slow. The 10 s timeout is per-attempt. If your handler does heavy work synchronously you'll get retried mid-flight.
- Forgetting
X-24Observe-Event. The event type is in the header AND the envelope'stypefield. If you only branch on the JSON body, missing or malformed bodies look like an unknown event type. - Re-enabling without fixing. Auto-disable triggers when your receiver fails 10 in a row. Re-enabling resets the counter; the next 10 failures will trip it again. Fix the receiver first.
When NOT to use this
- You want per-monitor alerting. Use the per-monitor
alertWebhookUrlfield instead — different shape, same signing. - You want a push notification to a human (Slack, Discord, Telegram). Use the per-monitor alert channels; they include rich message formatting our subscriptions don't.
- You need delivery guarantees stronger than at-least-once. Subscriptions are at-least-once (BullMQ retries). For exactly-once semantics, dedupe on
envelope.idin your receiver — every event has a stable id, retries land with the same id.
Related
- /docs/webhooks/ — signature verification recipe (Node / Python / Go).
- /docs/api-for-agents/ — the broader agent API story.
- /docs/api/ — full reference, PAT scopes, error codes.
- Launch post — why we built this surface for agents first.