Skip to content

OpenAI-Compatible Services

Backend.AI GO's architecture is built around the OpenAI API standard. This means you can connect not just to OpenAI itself, but to any service or local server that speaks the "OpenAI dialect."

What works with this?

You can connect Backend.AI GO to a vast ecosystem of tools and providers, including:

  • Local Inference Servers: Ollama, LocalAI, LM Studio, Text Generation WebUI (Oobabooga).

  • Third-Party APIs: Groq, DeepSeek, Together AI, Perplexity, Anyscale.

  • Enterprise Gateways: MLflow, Azure OpenAI (via proxy), and internal corporate gateways.

How to Connect

  1. Go to the Providers tab.

  2. Click Add Provider.

  3. Select OpenAI Compatible (or "Custom OpenAI").

  4. Enter the details:

    • Base URL: The full URL to the API endpoint (usually ending in /v1).

      • Example (Ollama): http://localhost:11434/v1
      • Example (DeepSeek): https://api.deepseek.com/v1

    • API Key: The authentication token. If running locally (like Ollama), you can often enter any string (e.g., sk-dummy).

  5. Model Discovery: Backend.AI GO will attempt to list available models from the endpoint. If auto-discovery fails, you may need to manually enter the model names.

Examples

Connecting to Ollama

If you have Ollama running on another machine (or locally on a port):

  • Base URL: http://<ip-address>:11434/v1
  • API Key: ollama (or leave empty if allowed)

Connecting to Groq

For ultra-fast inference using Groq's LPU:

  • Base URL: https://api.groq.com/openai/v1
  • API Key: Your Groq API Key

This flexibility turns Backend.AI GO into a universal frontend for all your AI needs.