Skip to content

Auto-Discovery

Under Development

Peer-to-peer auto-discovery is currently under development. While local model discovery is functional, automatic discovery of other GO instances on the local network will be available in a future update.

In a distributed environment, manually configuring every single model or service would be tedious. Backend.AI GO utilizes an Auto-Discovery mechanism to simplify resource management.

Service Discovery within the Mesh

Once you establish a connection to a node (be it a Cluster or another GO instance), you don't need to manually register every model available on that node.

  1. Handshake: When a connection is added, Backend.AI GO performs an initial handshake.
  2. Catalog Exchange: The remote node sends a list of its currently active "backends" (loaded models, services).
  3. Dynamic Updates:
    • If a new model is loaded on the remote cluster, your local client is notified (or discovers it on the next refresh).
    • These remote models appear in your chat interface's model selector, grouped by the provider/node name.

Local Model Discovery

On your local machine, "Auto-Discovery" works by monitoring the file system and active processes:

  • Folder Monitoring: When you download a model to the models/ directory, Backend.AI GO automatically detects the file type (GGUF, safetensors) and determines the appropriate runner (llama.cpp, vLLM, etc.).
  • Process Binding: When a model loads, it registers itself with the local Continuum Router. The Router then announces this capability to the frontend UI.

Troubleshooting Discovery

If remote models are not appearing:

  • Check Network: Ensure the connection status in Settings > Mesh is green.
  • Permissions: On a Cluster, ensure your Access Key has permission to view the specific Session or vFolder where the model is hosted.
  • Refresh: Sometimes a manual refresh (or restarting the client) is needed to trigger a full catalog sync.