Skip to content

Running Models & Your First Chat

Once you have downloaded a model, it's time to put it to work. Backend.AI GO provides a modern chat interface that feels familiar and responsive.

Loading a Model

Before you can chat, the model must be "loaded" from your disk into your computer's memory (RAM or VRAM).

  1. Navigate to the Models tab.
  2. Your downloaded models will appear here as cards.
  3. Click the Load button on the model you want to use.
  4. Advanced Settings: Before loading, you can click the settings icon on the model card to adjust parameters like Context Length and GPU Offloading.
  5. Watch the progress bar. Once it turns green and says "Loaded," you are ready!

The Chat Interface

Click the Chat icon in the sidebar to enter the main interface.

Creating Conversations

  • New Chat: Click the "+" button in the sidebar to start a fresh conversation.
  • History: Your previous chats are automatically saved in the sidebar for easy access.
  • Search: Use the search bar in the sidebar to find past conversations by keyword.

Interaction Features

  • Markdown Support: The model can format responses with bold text, lists, and tables.
  • Code Highlighting: Programming code in responses is beautifully highlighted with a "Copy" button.
  • LaTeX Support: Mathematical formulas are rendered cleanly.
  • Thinking Blocks: Some models (like DeepSeek or specialized reasoning models) can show their internal "thinking" process. Backend.AI GO displays these in a dedicated collapsible block.

Understanding Chat Parameters

In the chat interface, you can find a "Parameters" drawer (usually a gear icon on the top right) to fine-tune the model's behavior:

  • Temperature: Controls "creativity." Lower (0.1) is more focused and predictable; higher (0.8+) is more creative and random.
  • Top P: Another way to control randomness.
  • Repeat Penalty: Prevents the model from getting stuck in a loop.
  • System Prompt: Give the model a "personality" or specific instructions (e.g., "You are a helpful coding assistant" or "Speak like a pirate").

Unloading Models

When you are finished, or want to switch to a different model: 1. Go back to the Models tab. 2. Click Unload. 3. This frees up your system RAM/VRAM for other tasks.