8.3. Model Hub Mirror (Internal Hugging Face)¶

By default Backend.AI GO searches and downloads models from the public Hugging Face hub (https://huggingface.co). An administrator can instead point the model hub at an internal mirror (a self-hosted, Hugging-Face-compatible server) or switch model search and download off entirely. This is part of the managed service-endpoint configuration (the endpoints.modelHub setting); the egress firewall guarantees that no request reaches the public hub once a custom mirror or the disabled mode is in force.

This page covers:

• The three model-hub modes (default / custom mirror / disabled).
• How to configure them from Settings or a managed policy.
• The mirror-layout contract: the exact URL paths an internal mirror must expose so search, browse, model details, file listing, and download keep working.

When to use this¶

• Air-gapped or restricted networks where huggingface.co is unreachable or blocked, but an internal mirror holds the approved models. (See also Offline-Only Setup for the fully-offline .baimodel workflow.)
• Compliance environments where all model traffic must flow through a vetted internal host.
• Disabling model acquisition so end users can only run models the organization has already provisioned.

Modes¶

The model hub is one of the managed service endpoints. It has three modes:

In an enforcing managed deployment, configuring a custom mirror automatically adds the mirror's host to the egress allowlist, and the public huggingface.co host stays blocked under default mode. You do not need to hand-edit egress.allowedHosts.

Configure from Settings¶

1. Open Settings → Managed Endpoints.
2. Find the Model hub endpoint.
3. Choose a mode:
4. Custom: enter the mirror base URL (for example https://hf.internal.example). Optionally set an authentication method, a per-endpoint CA bundle, or a proxy.
5. Disabled: switch the hub off.
6. Use Test to verify the configured base is reachable through the firewall.

If the model hub is managed by a central policy, the field shows a "managed by your organization" badge and is read-only.

Configure from a managed policy¶

Because the model hub is an AppSettings field, a managed policy drives it through the standard settingsOverrides merge-patch plus lockedPaths. For example, to pin every device to an internal mirror and lock it:

{
  "settingsOverrides": {
    "endpoints": {
      "modelHub": {
        "mode": "custom",
        "baseUrl": "https://hf.internal.example",
        "authMethod": "bearer"
      }
    }
  },
  "lockedPaths": ["endpoints.modelHub"]
}

To switch model acquisition off entirely:

{
  "settingsOverrides": {
    "endpoints": { "modelHub": { "mode": "disabled" } }
  },
  "lockedPaths": ["endpoints.modelHub"]
}

Authentication¶

A mirror can require authentication. The model hub endpoint reuses the standard endpoint auth methods (bearer, api_key, basic, mtls). The existing Hugging Face token (Settings → Hugging Face) maps to a Bearer header and continues to work for gated models on both the public hub and a mirror. The credential value itself is stored only in the OS keychain and is never written to the settings file or sent in any list/get response.

Mirror-layout contract¶

An internal mirror must expose the same URL layout the public Hugging Face hub uses, because Backend.AI GO composes both an API base and a download base from the single configured base URL:

• API base = <baseUrl>/api
• Download base = <baseUrl>

So for baseUrl = https://hf.internal.example, the API base is https://hf.internal.example/api and the download base is https://hf.internal.example.

The mirror must serve the following paths. {id} is a model id of the form publisher/name (for example TheBloke/Llama-2-7B-GGUF).

API endpoints (under <baseUrl>/api)¶

The model and file JSON objects must carry the same field names the public hub returns (the client deserializes id, modelId, author, downloads, likes, tags, siblings/file entries, lfs.size, lfs.oid/sha256, etc.). The simplest way to guarantee compatibility is to mirror the upstream JSON verbatim for the models you serve.

Download endpoints (under <baseUrl>)¶

Scheme and host rules¶

• A non-loopback mirror must use https://. The download-URL allowlist accepts the configured mirror host in addition to huggingface.co / cdn.huggingface.co.
• A mirror on the loopback interface (127.0.0.1, localhost, ::1) may use http://, for an internal air-gapped server without TLS.
• The mirror must support HTTP range requests on the resolve path so interrupted downloads resume correctly.

Minimal mirror checklist¶

• Serve GET /api/models?... returning a Hugging-Face-shaped JSON array for search and browse.
• Serve GET /api/models/{id} returning the model detail object.
• Serve GET /api/models/{id}/tree/main?recursive=true returning the file tree with sizes and LFS hashes.
• Serve GET /{id}/resolve/main/{file} with byte streaming and range support.
• Serve GET /{id}/raw/main/README.md and GET /{id}/raw/main/tokenizer_config.json (or 404).
• Use HTTPS (unless the mirror is loopback-only).

Disabled-mode behavior¶

When the model hub is disabled:

• The Models → Browse tab shows a "model hub is disabled" panel with a link to the endpoint settings, instead of a search box.
• Search, browse, model details, file listing, and download all return a clear "disabled by policy" result without contacting any host.
• Already-downloaded models continue to load and run normally; only acquisition of new models is turned off.

Related¶

• Offline-Only Setup: the fully air-gapped .baimodel export/import workflow.
• External Access: accepting inbound connections (a separate concern from outbound endpoint configuration).