Cluster Integration¶
Backend.AI GO is fully integrated with the Backend.AI Enterprise ecosystem. This allows individual researchers or developers to seamlessly transition from local prototyping to large-scale cluster inference without changing their workflow.
Architecture¶
When you connect to a Backend.AI Cluster, your local GO instance acts as a smart client.
- Authentication: Unlike simple API tokens (like OpenAI), Cluster connections use HMAC-SHA256 signatures. This ensures that every request is cryptographically signed with your Secret Key, preventing replay attacks and ensuring authenticity without sending the secret over the wire.
- Session Management: The cluster dynamically provisions "Compute Sessions" or "vFolders" based on your request.
- Routing: The Continuum Router inside GO handles the decision of whether to run a prompt locally or forward it to the cluster.
Key Differences from Cloud Providers¶
While you can add "OpenAI" or "Anthropic" as providers, "Backend.AI Cluster" is treated as a first-class mesh citizen:
| Feature | Cloud Provider (e.g., OpenAI) | Backend.AI Cluster |
|---|---|---|
| Protocol | Standard HTTP/REST | Backend.AI RPC / Continuum Protocol |
| Auth | Bearer Token (API Key) | HMAC-SHA256 (Access/Secret Keys) |
| Discovery | Static Model List | Dynamic Model Discovery |
| Privacy | Data sent to public cloud | Data stays within enterprise private cloud |
Security¶
- Zero-Trust: No persistent connections are assumed secure. Every request is signed.
- Key Storage: Your Secret Key is stored in your operating system's secure keychain (macOS Keychain, Windows Credential Locker, Linux Secret Service), not in plain text files.
Usage Scenarios¶
- Fine-tuning: Use local resources for small-scale testing, then switch to the cluster to fine-tune a 70B parameter model.
- Batch Processing: Offload heavy batch inference jobs to the cluster while keeping your local machine responsive.
- Sensitive Data: Process strictly confidential data on an on-premise Backend.AI Cluster instead of sending it to public APIs.