Skip to content

Downloading Models

Backend.AI GO allows you to explore and download the latest open-source models directly from Hugging Face, the world's largest AI model repository.

What is Hugging Face?

Hugging Face is like the "GitHub of AI." It's a hub where researchers and developers share their trained models. Backend.AI GO integrates directly with Hugging Face, allowing you to find the models you need without leaving the app.

Searching for Models

  1. Open the Search (Hugging Face icon) tab in the sidebar.

  2. Use the search bar at the top to enter a specific model or organization name (e.g., Meta-Llama, Qwen).

  3. Filtering: Use tags to find compatible models. Look for:

    • GGUF: The optimized format for various hardware (based on llama.cpp).

    • MLX: Native format for Apple Silicon Macs.

Choosing the Right Variant (Quantization)

Models often come in multiple "Quantization" levels (e.g., Q4_K_M, Q8_0).

  • What is Quantization?: A compression technique that reduces model size and increases speed with minimal loss in quality.

  • Recommendation: For most users, Q4_K_M or Q5_K_M offers the "Golden Balance" between performance and intelligence.

  • RAM Requirements: Ensure the file size of the chosen variant is smaller than your computer's available RAM (or GPU VRAM).

Managing Downloads

  • Queue: You can start multiple downloads at once. They will be queued and processed sequentially.

  • Progress: Check the Downloads tab to see real-time progress, speed, and estimated time remaining.

  • Location: By default, models are saved in the application data directory. You can change this path in Settings.

Importing Local Files

If you already have a .gguf file downloaded elsewhere:

  1. Go to the Models tab.

  2. Click the Import button.

  3. Select the file to add it to your local library.