Network Node Model Control¶

Backend.AI GO allows the head node to run and stop models on connected network nodes without requiring direct access to those machines. This completes the distributed inference workflow: you can connect nodes, route inference requests across them, and now manage what models are running on each node — all from a single interface.

Overview¶

When a network node is registered, the head node stores its base URL and API key. Remote model control uses this information to send model management commands to the network node's Management API on your behalf.

The following operations are supported:

• Run a model: instruct a network node to run a model
• Stop a model: stop a model that is running on a network node
• List running models: query which models are currently active on a network node

API Reference¶

All three operations are available as both Tauri IPC commands (desktop app) and REST API endpoints (headless mode).

Run a Model on a Network Node¶

Tauri IPC: load_model_on_remote_node

REST: POST /api/v1/nodes/{fingerprint}/models/load

Request body:

{
  "modelId": "llama-3-8b-q4",
  "modelPath": "/path/to/model.gguf",
  "contextLength": 4096,
  "gpuLayers": -1
}

Response:

{
  "success": true,
  "modelId": "llama-3-8b-q4",
  "nodeFingerprint": "fp_abc123"
}

On failure, success is false and an error field describes the problem.

Stop a Model on a Network Node¶

Tauri IPC: unload_model_on_remote_node

REST: POST /api/v1/nodes/{fingerprint}/models/unload

Request body:

{
  "modelId": "llama-3-8b-q4"
}

Response:

{
  "success": true,
  "modelId": "llama-3-8b-q4",
  "nodeFingerprint": "fp_abc123"
}

List Models on a Network Node¶

Tauri IPC: get_remote_node_models

REST: GET /api/v1/nodes/{fingerprint}/models

Response:

{
  "data": [
    {
      "id": "llama-3-8b-q4",
      "path": "/models/llama-3-8b-q4.gguf",
      "endpoint": "http://127.0.0.1:8081",
      "healthy": true
    }
  ]
}

List the Models a Network Node Can Run¶

Tauri IPC: get_remote_node_loadable_models

REST: GET /api/v1/nodes/{fingerprint}/models/loadable

The Run on node dialog in the Distributed pool tab reads this list after you choose a node and offers it as a dropdown, leaving out the models already running there. The head node asks the network node for GET /api/v1/loadable, which the node's key reaches with the same permission that lists its running models. The node answers with every chat and embedding model in its library, sorted by name; speculative-decoding drafters and image generators are left out.

Response:

{
  "data": [
    {
      "id": "llama-3.2-3b-instruct-q4_k_m",
      "name": "Llama 3.2 3B Instruct",
      "format": "gguf",
      "sizeBytes": 2019377152,
      "quantization": "Q4_K_M",
      "parameters": "3B"
    }
  ]
}

A network node on a version without GET /api/v1/loadable answers 404, and the head reports an error containing REMOTE_LOADABLE_UNSUPPORTED. The dialog then asks for the model ID as text, as it does when the list cannot be read at all. The head keeps at most 500 entries from one node and strips control characters from the names.

Real-Time Updates¶

When a model is successfully run or stopped on a network node, the head node emits a distributed-pool-changed event. The frontend nodeStore subscribes to this event and automatically:

1. Refreshes the model list for the affected network node (fetchRemoteModels).
2. Refreshes the overall distributed pool status (getDistributedPoolStatus).

This keeps the UI in sync without requiring manual page refreshes.

In headless mode the same event is broadcast over the Server-Sent Events (SSE) stream at GET /api/v1/events.

Timeouts¶

Operations use conservative timeouts to account for large models:

If a timeout occurs, the response contains "success": false with an error message indicating the timeout duration.

Error Handling¶

Model IDs are validated before any network request is made. Empty model IDs and IDs containing .. sequences are rejected to prevent URL path manipulation.

Frontend Store¶

The useNodeStore Zustand store exposes the following selectors and actions for network node model control:

Stopping a model performs an optimistic update: the model is removed from the local cache immediately, and the distributed-pool-changed event triggers a full refresh from the backend.

Security¶

All requests to network nodes use Authorization: Bearer <api_key> headers. The API key is stored in the local node registry and never exposed in logs or the UI. Model IDs are percent-encoded in URLs to prevent path injection.

Pipeline-Stage Worker Control¶

For running a model across a cluster with pipeline parallelism (epic #3413), the head node can launch, stop, and poll a local mlxcel-server stage process on a registered sub-node over the same authenticated peer channel. The sub-node writes the cluster TOML locally and spawns the stage; the head polls the transport-port readiness before the coordinator connects.

Launch a Stage on a Sub-Node¶

Tauri IPC: spawn_stage_on_remote_node

REST: POST /api/v1/nodes/{fingerprint}/distributed/stages

The body is a SpawnStageRequest containing the GO-generated clusterId, the full clusterToml, the sub-node's nodeId, the launch parameters from the #3414 builder, and an optional layerRange. The response is a StageStatus (see below). The API key in the launch is a file path only; no secret is transmitted.

Stop a Stage on a Sub-Node¶

Tauri IPC: stop_stage_on_remote_node

REST: DELETE /api/v1/nodes/{fingerprint}/distributed/stages/{cluster_id}

Idempotent: stopping an already-gone stage returns stopped: false with no error.

Poll Stage Status on a Sub-Node¶

Tauri IPC: get_stage_status_on_remote_node

REST: GET /api/v1/nodes/{fingerprint}/distributed/stages/{cluster_id}

Returns 404 when no stage for that cluster is tracked on the sub-node. The StageStatus payload includes:

The sub-node handles these three routes at POST/DELETE/GET /api/v1/distributed/stages[/{cluster_id}], authenticated with the same ak_ node key as the model-control routes.

Timeouts¶

Related Pages¶

• Distributed Routing — Automatic request routing across nodes
• Manual Registration — Connect to network nodes
• Auto-Discovery — Discover nodes on the local network