Skip to content

OpenAI-Compatible Services

Backend.AI GO's architecture is built around the OpenAI API standard. This means you can connect not just to OpenAI itself, but to any service or local server that speaks the "OpenAI dialect."

What works with this?

You can connect Backend.AI GO to a vast ecosystem of tools and providers, including:

  • Local Inference Servers: Ollama, LocalAI, LM Studio, Text Generation WebUI (Oobabooga).

  • Third-Party APIs: Groq, DeepSeek, Together AI, Perplexity, Anyscale.

  • Enterprise Gateways: MLflow, Azure OpenAI (via proxy), and internal corporate gateways.

How to Connect

  1. Go to the Providers tab.

  2. Click Add Provider.

  3. Select OpenAI Compatible (or "Custom OpenAI").

  4. Enter the details:

    • Base URL: The full URL to the API endpoint (usually ending in /v1).

      • Example (Ollama): http://localhost:11434/v1
      • Example (DeepSeek): https://api.deepseek.com/v1
      • Either form works. A base URL that already ends in an API root (/v1, /openai) is detected, so the path is not doubled into /v1/v1/models.

    • API Key: The authentication token. Optional for custom endpoints: many self-hosted servers (like Ollama) accept no authentication at all, so you can leave the field empty and still use Test Connection to confirm the endpoint is reachable. If your server does require a key but does not actually validate it, any string works (e.g., sk-dummy).

  5. Model Discovery: Backend.AI GO will attempt to list available models from the endpoint. If auto-discovery fails, you may need to manually enter the model names.

Examples

Connecting to Ollama

If you have Ollama running on another machine (or locally on a port):

  • Base URL: http://<ip-address>:11434/v1
  • API Key: ollama (or leave empty if allowed)

Connecting to Groq

For ultra-fast inference using Groq's LPU:

  • Base URL: https://api.groq.com/openai/v1
  • API Key: Your Groq API Key

This flexibility turns Backend.AI GO into a universal frontend for all your AI needs.