Skip to content

Event Streams

Every long-running operation in Backend.AI GO reports progress as an event: squad executions, agent chat turns, discussion rooms, data ingest and folder imports, embedding runs, memory consolidation. The Management API publishes those events on one bus and offers two transports over it.

Transport Route Scope Resume and lag reporting Use it when
Server-Sent Events GET /api/v1/events admin yes A read-only tail is enough and the client is an EventSource.
WebSocket GET /api/v1/events/ws admin yes The filter has to change while connected, or the client needs to resume after a drop.
WebSocket GET /api/v1/squads/{id}/ws agent_read yes Following one squad from a script that must survive a disconnect.
Server-Sent Events GET /api/v1/squads/{id}/events agent_read no Following one squad from a browser, when losing events on a drop is acceptable.
Server-Sent Events GET /api/v1/squads/events agent_read yes Watching every squad at once, or watching before the ids are known.
WebSocket GET /api/v1/squads/ws agent_read yes Watching every squad with a filter that can change while connected.
Server-Sent Events GET /api/v1/schedules/events agent_read yes Following automation runs and configuration changes.
WebSocket GET /api/v1/schedules/ws agent_read yes Following automations with a filter that can change while connected.
Server-Sent Events GET /api/v1/memory/events memory_read yes Refreshing a memory view without polling.
WebSocket GET /api/v1/memory/ws memory_read yes Following memory changes with a filter that can change while connected.
Server-Sent Events GET /api/v1/data/events data_read yes Following ingest, folder import, and embedding progress.
WebSocket GET /api/v1/data/ws data_read yes Following Data Hub work with a filter that can change while connected.

The two global routes carry every event on the bus, which is why they require admin. Every other route is pre-filtered to one domain and carries that domain's own read scope. The filter is what makes the narrower scope sound: a domain stream never delivers a name from outside its family, replayed or live, and ?types= narrows within the family and is rejected with 400 when it names anything outside it.

GET /api/v1/squads/{id}/events predates this feature and still ignores Last-Event-ID and ?since=, and still drops events silently when a consumer falls behind. Use GET /api/v1/squads/{id}/ws or either cross-squad route when that matters. Everything below about resuming, gaps, and lag applies to the routes marked yes.

Event shape

Both transports carry the same event. Over WebSocket it arrives as one JSON text frame:

{
  "id": 4213,
  "type": "squad:task-completed",
  "payload": { "squadId": "8f2c...", "taskId": "t-19" },
  "timestamp": "2026-08-22T10:00:00Z"
}

Over SSE the same three facts arrive as the standard fields, so an EventSource client reads them without parsing an envelope:

event: squad:task-completed
data: {"squadId":"8f2c...","taskId":"t-19"}
id: 4213

id increases by one per emitted event and is the resume cursor. type is the event name; the per-domain pages list the names each domain emits.

Resuming after a disconnect

The server keeps the last 2048 events. A client that reconnects and supplies the last id it saw receives everything after that id, then continues live with no duplicate and no hole.

Supply the resume point in one of two ways:

  • Last-Event-ID: 4213 as a request header. A browser EventSource resends this by itself.
  • ?since=4213 as a query parameter, for clients that cannot set headers.

The header wins when both are present.

curl -N -H "Authorization: Bearer $KEY" \
     -H "Last-Event-ID: 4213" \
     http://127.0.0.1:8001/api/v1/events

Gaps and lag are reported, never silent

Three reserved event names carry stream conditions rather than product events. A subscription filter never removes them: a client that asked for one event type still has to hear that it lost events. Do not name them in ?types= or in a subscription frame on a domain-scoped route: they are not part of any domain, so naming one there is refused like any other out-of-domain name, and there is nothing to gain by asking for what arrives anyway.

Name Meaning Payload Top-level id
stream:ready Sent once after a WebSocket upgrade, before any replayed or live event. {"subscribed": [...] or null, "lastId": 4213} the requested cursor when resuming, or the current bus anchor on a fresh connection
stream:gap The resume point cannot be honored exactly. {"droppedBefore": 2170} null
stream:lagged This connection fell behind and events were dropped for it. {"dropped": 37} null

No synthetic frame advances the resume cursor past an event the client still needs. On a resumed connection, stream:ready carries the resume point you asked for rather than the newest id the server holds because the backlog is written after it. On a fresh connection there is no backlog, so the frame carries the current bus id as a safe anchor. Recording that anchor closes the hole where the socket drops before its first product event. The newest id is always reported as lastId in the payload.

Two things produce stream:gap. The usual one is a resume point older than the oldest buffered event, so events between the two were evicted. The other is a resume point newer than anything the server has emitted, which is what a cursor saved before a server restart looks like: event ids start again from zero with the process. In that case the server replays everything still buffered in the new process, and the first real event establishes the client's new cursor epoch. A stream:ready whose lastId is below the cursor you asked for is the same signal, seen one frame earlier.

The replay buffer holds more events than the broadcast channel a connection reads from (2048 against 1024), on purpose. A client that receives stream:lagged can reconnect with the last id it did see and recover the dropped events from the buffer.

Changing the subscription while connected

WebSocket clients send control frames as JSON text. SSE clients cannot: an EventSource filter is fixed at connect time by ?types=.

Frame Effect
{"op":"subscribe","types":["squad:task-completed"]} Adds names to the filter. An empty list means every event the socket is scoped to.
{"op":"unsubscribe","types":["squad:task-completed"]} Removes names from the filter.
{"op":"set","types":["squad:task-completed"]} Replaces the filter. An empty list means every event the socket is scoped to; this is how a client narrows an unfiltered stream.
{"op":"ping"} Answered with {"op":"pong"}.

Every accepted subscription change is acknowledged with {"op":"subscribed","types":[...] or null} carrying the resulting filter, so a client knows when the change took effect without guessing. An unknown op or a malformed frame is answered with {"op":"error","message":"..."} and the connection stays open.

subscribe adds; it cannot narrow. From an unfiltered stream (no ?types= and no prior set), subscribe has nothing to add, and unsubscribe is refused with an error rather than silently narrowing. Use set to choose an explicit list.

A frame naming a type outside the socket's domain is refused the same way, with {"op":"error"} naming the rejected values, and the filter is left as it was. The two squad sockets accept squad:*, the schedule socket accepts schedule:*, the memory socket accepts memory:*, and the Data Hub socket accepts data:*. An out-of-domain name in ?types= is refused earlier still, with 400 on the upgrade, so no socket is created. GET /api/v1/events/ws carries the whole bus, so no name is out of domain there.

The socket is receive-plus-subscription. There is no control frame that starts, stops, or changes anything in the product; every such operation stays an authorized REST call.

Limits

Limit Value
Client control frame 16 KiB; larger closes the connection with code 1009
Event types in one subscription 256
Length of one event-type name 128 bytes
Replay buffer 2048 events
Server ping interval 15 s; two unanswered pings close the connection with code 1001

The server also closes with 1001 when it stops, so a client can distinguish a planned stop from a network failure. That includes a settings change that restarts the embedded Management API: reconnect on 1001 rather than treating it as an error.

On every domain socket the ids and the lastId are the server's global event counter, not a per-domain sequence, so consecutive events in one domain normally have non-consecutive ids. Resume works on the global cursor; do not read the numbers as a count of anything.

Authentication

The upgrade request is an ordinary protected request and goes through the same authentication and scope check as every other route. Authorization happens on the upgrade, before the socket exists; a caller without the route's scope receives 401 or 403 and no socket is created.

Scripts and CLIs send the key as a header:

websocat -H "Authorization: Bearer $KEY" \
  "ws://127.0.0.1:8001/api/v1/events/ws?types=squad:task-completed"

A browser cannot set headers on a WebSocket upgrade, so the key rides the subprotocol list instead. Offer two protocols: the key-bearing token and the bare marker the server echoes back.

const socket = new WebSocket(
  "ws://127.0.0.1:8001/api/v1/events/ws",
  [`aigo-key.${key}`, "aigo-key"],
);

The server selects aigo-key, which is one of the two the client offered, as RFC 6455 requires. The key-bearing token is read only on a request that actually carries Upgrade: websocket, so it is not a second way to authenticate ordinary REST calls.

A key in the query string is never accepted. Query strings are recorded by access logs, reverse proxies, and browser history, so ?key=, ?token=, and every similar spelling are ignored and the request is rejected with 401. Use a header or the subprotocol.

A socket is authorized once, at upgrade. Nothing a client sends afterwards widens what it receives: the subscription frames adjust a type filter only, and the squad predicate on /squads/{id}/ws comes from the path and is not reachable from any frame, so since replay is filtered the same way the live stream is. The converse also holds: a key revoked mid-connection keeps its stream until the socket closes, exactly as the SSE streams behave. Close the socket to end access immediately.

Unix socket transport

The Management API serves the same router over its Unix domain socket, so the upgrade works there unchanged. This is the usual transport for a local script, because it needs no port and inherits the socket's filesystem permissions.

websocat --unix-socket "$SOCKET" \
  -H "Authorization: Bearer $KEY" \
  "ws://localhost/api/v1/events/ws"

From the aigo CLI

The CLI consumes these routes directly, so a script does not have to speak either transport itself. aigo events follows the global bus; aigo squad events, aigo schedule events, aigo memory events and aigo data events follow one domain each; and the follow modes (aigo squad execute --follow, aigo squad execution --follow, aigo squad message --wait, aigo squad discussion watch, aigo data ... --follow) ride the same streams while reporting one run.

Every one of them takes --types, --since, --raw, and --transport; SSE is the default. Reconnection, the resume cursor, and the stream:gap / stream:lagged reporting are handled by the client. See the CLI reference.

Per-domain event names

Each domain page lists the events it emits and what their payloads carry: