Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
cded88b
test: a slack destination posts {"text": message} with only the Conte…
V3RON Oct 20, 2026
1b255ea
feat(alerts): a slack destination posts a message for each alert even…
V3RON Oct 20, 2026
abfd9fd
test: each of the eleven events posts its message word for word (#720)
V3RON Oct 20, 2026
f6d8b43
feat(alerts): slack messages show ages and byte sizes in readable uni…
V3RON Oct 20, 2026
73bd767
test: names the worker by the label workerLabel answers, and by the i…
V3RON Oct 20, 2026
91fb1b3
feat(alerts): slack messages name the worker by its label (#720)
V3RON Oct 20, 2026
b80bffe
test: builds a message when it leaves the queue and resends the same …
V3RON Oct 20, 2026
d34bf77
test: worker.rejected names the claimed label, else the claimed id, e…
V3RON Oct 20, 2026
e3dd681
feat(alerts): a slack refusal message adds how many came before (#720)
V3RON Oct 20, 2026
2863e54
test: formats an age in seconds, minutes, and hours with minutes (#720)
V3RON Oct 20, 2026
779c423
test: formats bytes in GiB and MiB with one decimal rounded half up, …
V3RON Oct 20, 2026
1d13da8
test: cuts every variable part to 300 code points, then escapes &, < …
V3RON Oct 20, 2026
45bc197
test: a relayed payload with missing fields posts the text the Known …
V3RON Oct 20, 2026
ed08f1a
test: redactConfig masks a slack destination's url and adds no secret…
V3RON Oct 20, 2026
b7dcae4
test: config.get's schema keeps a slack destination without a secret …
V3RON Oct 20, 2026
963169a
test: a slack destination is tried again on 5xx like a generic one (#…
V3RON Oct 20, 2026
c4229aa
test: loadConfig accepts a slack destination, refuses a secret on it,…
V3RON Oct 20, 2026
f65ca49
feat(config): a slack destination takes no secret (#720)
V3RON Oct 20, 2026
69b26f0
chore(contract): protocol 50 for the slack alert format (#720)
V3RON Oct 20, 2026
a922bbc
test: a worker joining a gateway with a slack destination makes a loo…
V3RON Oct 20, 2026
24bc451
feat(daemon): the gateway names a worker in Slack messages by its lab…
V3RON Oct 20, 2026
8728bcd
docs(alerts): the slack format, its messages and the known gap (#720)
V3RON Oct 20, 2026
ed8910a
refactor(config): check a destination's secret in its own function (#…
V3RON Oct 20, 2026
05d0169
test: slack units and cut sit on their boundaries (#720)
V3RON Oct 20, 2026
c8521fb
docs(alerts): name the slack files by path from the module (#720)
V3RON Oct 20, 2026
7ce2f50
test: escaping reaches reason, state and attempts and holds at the cu…
V3RON Oct 20, 2026
61cba54
test: escaping is proved at every variable part of every slack messag…
V3RON Oct 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 33 additions & 3 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -342,16 +342,16 @@ receiver hears about them without anyone watching the console. Set
| Field | Rule |
|---|---|
| `name` | Lowercase letters, digits and dashes, starting with a letter, at most 32 characters; no two destinations share one. |
| `format` | `generic`, the only format so far. |
| `format` | `generic` (the signed JSON envelope) or `slack` (a written message for Slack). |
| `url` | `https://`, or `http://` to `localhost`, `127.0.0.1` or `[::1]`; no user name or password in it. |
| `events` | A non-empty list, each event once, from: `worker.connected`, `worker.disconnected`, `worker.rejected`, `worker.drain-started`, `worker.drain-ended`, `device.quarantine-abandoned`, `device.quarantine-stranded`, `device.recovery-failed`, `device.stalled-transition-detected`, `disk.pressure-detected`, `component.install-failed`. |
| `secret` | A non-empty string the gateway signs each message with. |
| `secret` | A `generic` destination needs a non-empty string the gateway signs each message with. A `slack` destination takes none; a `secret` key on it is refused. |

An event a worker reports reaches the destination too, with `workerId` in its
payload. A key not listed above is warned about and ignored, and
`simlock config set` refuses it.

**What the receiver gets.** One `POST` per event, with the event as the body: the
**What a `generic` receiver gets.** One `POST` per event, with the event as the body: the
same JSON `simlock events --json` prints (`id`, `seq`, `timestamp`, `event`,
`payload`, `module`). Headers:

Expand All @@ -367,6 +367,36 @@ To check a message, compute the HMAC-SHA256 of the bytes you received with the
secret and compare it to the header value in constant time. Use the raw body,
not a re-serialised copy.

**Slack.** A `slack` destination gets `{"text": "<message>"}` for each event it
lists, written for a person, through the same queue, tries and limits as
`generic`, with only the `Content-Type` and `User-Agent` headers (no `X-Simlock-Event`, `X-Simlock-Delivery` or `X-Simlock-Signature`, and no body but the `text`). Slack is the
only service promised; other services that take Slack's incoming-webhook body
may work. `<worker>` is the worker's label as the gateway knows it when the
message is sent, or its id when it has no label. Every variable part is cut to
300 characters (`…` added) and has `&`, `<` and `>` written as `&amp;`, `&lt;`
and `&gt;`, so a label like `<!channel>` never pings a channel. A field a worker
left out prints as `undefined`, and a size or age that is not a number prints as
`?`.

| Event | Message |
|---|---|
| `worker.connected` | `🟢 Worker <worker> connected (simlock <version>).`, or `🟢 Worker <worker> connected.` without a version |
| `worker.disconnected` | `🔴 Worker <worker> disconnected.`, or `… disconnected while holding 1 lease.`, or `… while holding <n> leases.` |
| `worker.rejected` | `⛔ A worker was refused (<reason>): <claimed>.` and, when `count` is above 1, ` <count> more refusals like it came before this one.` `<claimed>` is the label the connection claimed, else its worker id, else `an unknown worker`. |
| `worker.drain-started` | `⏸ Worker <worker> is drained: it takes no new leases.` |
| `worker.drain-ended` | `▶️ Worker <worker> takes new leases again.` |
| `device.quarantine-abandoned` | `⚠️ Device <deviceId> on <worker> was retired after <attempts> failed clean-ups.` |
| `device.quarantine-stranded` | `🛑 Device <deviceId> on <worker> could not be cleaned up or deleted after <attempts> tries: <error>. It needs a person.` |
| `device.recovery-failed` | `⚠️ Device <deviceId> on <worker> crashed during lease <leaseId> and could not be brought back: <reason>.` |
| `device.stalled-transition-detected` | `⏳ Device <deviceId> on <worker> has been <state> for <age>.` |
| `disk.pressure-detected` | `💾 Worker <worker> is low on disk: <freeBytes> free (threshold <threshold>).` |
| `component.install-failed` | `❌ Installing <platform> <componentId> on <worker> failed: <error>.` |

`<age>` reads `45 s`, `14 min` or `1 h 5 min` (`2 h` when no minutes are left),
rounded down. `<freeBytes>` and `<threshold>` are in GiB (1024³ bytes), or MiB
under 1 GiB, with one decimal rounded half up and `.0` dropped: `8.2 GiB`,
`10 GiB`, `512 MiB`.

**Delivery.** Each destination sends one message at a time, in the order the
gateway saw the events, and a slow destination never delays another. A try
ends after 10 seconds. A network error, a timeout, `429` or a `5xx` is tried
Expand Down
30 changes: 23 additions & 7 deletions src/alerts/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,14 @@ Imported by: `../daemon/main.ts` only.

## Files

| File | Role |
| ------------------ | ------------------------------------------------------------------------ |
| `index.ts` | `createAlerts`: one subscription per listed event, `stop()` |
| `queue.ts` | `DestinationQueue`: one message at a time, tries, timeout, bounded wait |
| `formats/types.ts` | `AlertFormatDefinition`: `build` turns an envelope into headers and body |
| `formats/index.ts` | `ALERT_FORMAT_REGISTRY`, typed over core's `ALERT_FORMATS` |
| `formats/generic/` | The envelope as JSON, signed with HMAC-SHA256 |
| File | Role |
| ------------------ | ---------------------------------------------------------------------------------------------------------- |
| `index.ts` | `createAlerts`: one subscription per listed event, `stop()` |
| `queue.ts` | `DestinationQueue`: one message at a time, tries, timeout, bounded wait |
| `formats/types.ts` | `AlertFormatDefinition`: `build` turns an envelope into headers and body |
| `formats/index.ts` | `ALERT_FORMAT_REGISTRY`, typed over core's `ALERT_FORMATS` |
| `formats/generic/` | The envelope as JSON, signed with HMAC-SHA256 |
| `formats/slack/` | `{text}` from a message table (`formats/slack/messages.ts`), `formats/slack/units.ts` cuts, escapes, sizes |

## How it works

Expand All @@ -28,3 +29,18 @@ Imported by: `../daemon/main.ts` only.
Logs name the destination, never its URL or secret; a failed try is logged by code, never by an
error's message. A format is added by a directory under `formats/`, an entry in the registry and in
core's `ALERT_FORMATS`; the queue never names a format.

## Slack messages

`formats/slack/messages.ts` is a `Record<AlertEventName, …>`: an event added to `ALERT_EVENTS` does not
compile until it has a message. Every variable part goes through `part()` (cut to 300 code points, then
`&`, `<`, `>` escaped), and `worker.rejected` names what the connection claimed, never the registry.
`createAlerts` takes `workerLabel(workerId)` (the gateway's view of the worker, from `../daemon/main.ts`)
and falls back to the id; the message is built when it leaves the queue, so it names the worker as known
then.

## Known gaps

- A worker's relayed payload is not checked (the gateway replaces a non-object with `{workerId}`): a Slack
message built from a missing field prints `undefined`, and an age or size that is not a finite number
prints `?`.
Loading
Loading