Chat Interface¶

The Chat Interface is where you interact with AI models. It supports rich text rendering, code highlighting, and advanced features like reasoning visualization.

Overview¶

The interface consists of three main areas:

1. Message History: The central area displaying the conversation.

2. Input Area: At the bottom, where you type your messages.

3. Control Bar: At the top, showing the current model and providing access to parameters.

Chat and Cowork modes¶

Chat and Cowork share one surface. A single conversation list holds both kinds of conversations, and the mode is a property of the conversation, shown as a badge in the conversation list.

The working folder decides the mode. A conversation without a working folder runs as Chat, a turn-based conversation with the model. A conversation with a working folder runs as Cowork, an autonomous run over that folder with tool use and approvals. The model does not switch the mode on its own.

• Attach folder: Attach folder in the composer's + menu opens a folder picker. On Home or in an empty conversation, picking a folder opens a new Cowork conversation with that folder attached, and the text you typed moves into its composer unsent. In an ongoing chat, you confirm the change and the conversation continues as Cowork.

• Folder suggestion: When the message you are typing, or the one you just sent, looks like work on files or a project and no folder is attached, a one-line suggestion appears above the composer. A path such as ~/Documents or C:\Users, a file name such as report.pdf, or a request to edit, organize, or refactor files counts as file work. The suggestion's Attach folder opens the same picker. The check runs locally with no model call, the suggestion never changes the mode, and × hides it for the rest of the conversation.

A Cowork conversation shows a folder chip in the composer and a collapsible right rail with Progress (run steps and sub-agents), Output (workspace files), and Context (permitted folders and instructions). A conversation in Chat mode shows no rail.

The change from Chat to Cowork is one-way today. The transcript so far carries over as context for the cowork run, and the conversation stays Cowork even if you remove the folder later, because it holds sandbox and approval state that must not change mid-conversation.

Home has no separate Cowork tab, and there is no mode switch or shortcut that preselects Cowork: the working folder is the only entry. The old /cowork route and links that opened Home's former Cowork tab land on the launcher at /chat, where a new conversation starts. Cowork settings such as the profile and permitted folders are changed in the workspace rail once a conversation is running. See Cowork Overview for the cowork capabilities in depth.

Rich Text & Code¶

Backend.AI GO supports full Markdown rendering.

• Code Blocks: Automatically detects programming languages and applies syntax highlighting. You can copy code with a single click.

• Mathematics: Supports LaTeX rendering for mathematical formulas (e.g., \(E = mc^2\)).

• Tables: Renders clean, readable tables from data.

• Remote images: An image in a model reply or a Markdown artifact that points at a web address is not loaded. It appears as a link instead, and clicking the link opens the address in your browser. This stops a reply from sending data to a server of its choosing through an image address. Images embedded in the text itself (data: images) still display. Model cards in Hugging Face search still show images hosted on Hugging Face and show images from other hosts as links.

Artifact Viewer¶

The Artifact Viewer displays AI-generated code blocks, SVG graphics, HTML pages, and Markdown documents in a dedicated side panel or pop-out window with interactive rendering and editing tools.

Opening Artifacts¶

When the AI generates renderable content or a long code block, an Open as Artifact button appears at the top of the code block. Click it to open the Artifact Viewer panel.

You can also press Ctrl+Shift+A (Windows/Linux) or Cmd+Shift+A (macOS) to toggle the panel at any time.

Panel Features¶

• Code tab: Displays the raw source with syntax highlighting, line numbers, and a word-wrap toggle.

• Preview tab: Renders the content visually — SVG graphics with zoom controls, HTML in a sandboxed iframe with viewport presets, or Markdown with full GFM and math support.

• Copy: Copies the artifact content to the clipboard.

• Download: Saves the content as a file with the appropriate extension.

• Pop-out: Opens the artifact in a separate native window for side-by-side viewing.

• History navigation: Use the previous/next arrows to browse artifacts produced earlier in the conversation.

Renderable Types¶

The following code fence languages open directly in Preview mode:

Code blocks in any other language open in Code view. Long code blocks (10 or more lines) also show an Open as Artifact button for convenient viewing.

Saving Artifacts to the Library¶

Click the Save to Library button in the Artifact Viewer toolbar to keep an artifact for later reference. Saved artifacts appear under the Artifacts tab of the Library, which you open from Library in the sidebar.

Security¶

• HTML artifacts are sandboxed (allow-scripts only) and isolated from the main application.

• SVG and Mermaid artifacts have on* event handler attributes and dangerous elements (<script>, <foreignObject>) stripped before rendering.

• All artifact content runs in an isolated origin to prevent cross-origin access.

Advanced Chat Features¶

Message Editing & Forking¶

You can edit any past message (both yours and the AI's) to steer the conversation in a new direction.

1. Hover over a message and click the Edit (Pencil) icon.

2. Modify the text and click Save & Submit.

3. This creates a Fork (or branch) in the conversation tree. The original message is preserved.

Navigating Branches¶

When you edit a message or regenerate a response, you create multiple versions of the conversation at that point.

• Look for navigation arrows (e.g., < 2 / 3 >) below the message bubble.

• Click the arrows to switch between different branches of the conversation history. This allows you to explore "what if" scenarios without losing your previous context.

Regenerate Response¶

If the AI's answer isn't quite right, click the Regenerate (Refresh) icon below the AI's message. This will generate a new response, creating a new branch.

Thinking Blocks (Reasoning Models)¶

When using advanced "Reasoning Models" like DeepSeek-R1 or Qwen3-Thinking, the model outputs its internal thought process before the final answer.

• Collapsible View: By default, the thinking process is collapsed to keep the conversation clean.

• One-Line Summary: When collapsed, a short AI-generated summary of the reasoning is shown next to the "Thinking" label, so you can quickly understand what the model considered without expanding the block.

• Expand to Learn: Click the "Thinking" header (with the brain icon) to expand and read how the model arrived at its conclusion. This is great for debugging logic or understanding complex derivations.

• Streaming: You can watch the thoughts appear in real-time as the model generates them.

Reasoning Effort Control¶

You can control how much "thinking" a reasoning model does using the Reasoning Effort control in the chat input area.

• None (Off): Disables extended thinking. The model responds directly without showing internal reasoning.

• Low / Medium / High / Extra High: Progressively increases the depth of reasoning. Higher values allow the model to think longer and more thoroughly, which can improve answer quality for complex problems.

• Levels the model takes: Only the levels the selected model accepts can be chosen, and the others are shown disabled. Each OpenAI reasoning model takes a fixed set: o3 takes Low to High, and GPT-5 Pro takes only High. A model served by another node, or one from an OpenAI-compatible provider whose levels are not known, is sent at most High. When you switch models, the level moves to the nearest one the new model takes, and your choice comes back when you switch back. If a model refuses a level, the message is sent again without one, and that model gets no level for the rest of the session. For a model that cannot turn reasoning off, such as o3 or GPT-5 Pro, the Off choice reads Default: no level is sent, and the model uses its own default level.

• Default Setting: Configure the default reasoning effort level in Settings > Generation > Default reasoning effort. You can also choose "Last used" to store the most recently used setting and apply it again.

• Numeric thinking budget: Each reasoning effort level is also translated into a thinking_budget_tokens cap on the per-request body so engines that enforce a hard cap on <think> tokens (llama.cpp, mlxcel-server) honor it deterministically. See Model Settings -> Thinking budget (numeric cap) for the exact mapping.

• Preserve thinking across turns (Qwen3.6+): For Qwen3.6+ models, the Model Config Drawer's Advanced tab adds a Preserve thinking across turns toggle that retains <think> blocks from prior assistant turns instead of stripping them. This improves agent KV cache reuse. See Model Settings -> Preserve thinking across turns (Qwen3.6+).

Token Exhaustion Warning¶

When a reasoning model runs out of tokens before it can produce a visible answer, the chat shows a warning banner directly below the affected message. Two cases trigger the banner:

• The model spent all available tokens inside the <think> block and the response content is empty (finish_reason: length with non-zero reasoning_tokens). The banner shows how many tokens were consumed in reasoning and suggests increasing max_tokens or lowering the reasoning effort.

• Reasoning was enabled on the request but the model reported no reasoning_tokens in the usage data while the response is still empty (finish_reason: length). This typically means the engine does not expose per-token reasoning counts but hit the limit inside the thinking block nonetheless.

To resolve either warning, try increasing Max Tokens in the Conversation parameters drawer, shortening the conversation, or reducing the Reasoning Effort level.

Tool Use & Attachments¶

Tool Calling Visualization¶

If you are in Cowork, the AI may use tools (like Web Search).

• Collapsible View: A turn's tool calls, their results, and any related thinking steps collapse into one Agent activity box in the transcript, instead of a separate block for every step.

• Live Status: While the turn is still running, the box stays open and its header names the tool currently in progress.

• Summary When Done: Once the turn finishes, the box collapses and its header reports how many tools ran, how many thinking steps occurred, and how long it took.

• Expand to Inspect: Click the box to open it again. Each row shows what tool was called, with what inputs, and the result (e.g., the search results or file content) that was fed back to the AI.

File Attachments¶

You can attach images to your messages for analysis with multimodal models (like Llama 3.2 Vision, GPT-4o) using any of these methods:

• Drag and drop: Drag image files directly into the chat input area.

• Clipboard paste: Copy an image (e.g., a screenshot) and paste it with Cmd+V (macOS) or Ctrl+V (Windows/Linux). The image appears as an attachment thumbnail.

• Attach button: Click the paperclip icon to browse and select files.

Normal text paste behavior is preserved; only image content from the clipboard triggers the attachment flow.

If the selected model cannot read images, the image is not sent. The chat says so under the reply and lists the models that can read images, local models first. Pick one to switch to it and send your message again; Show all opens the rest of the list.

The same applies when a tool returns an image, such as a screenshot from an MCP tool or an image file the view_image tool opens. A model that reads images receives the image itself, and a model that does not receives a short note instead.

Voice Input¶

Click the microphone icon in the chat input area to start a push-to-talk recording. Speak your message, then click the icon again (or press the same button) to stop. The recording is transcribed locally by whisper-server and the resulting text is inserted into the composer — either waiting for you to review and send, or sent automatically, depending on the Insert mode you configure in Settings > Voice Input.

Voice input requires a running whisper-server with a downloaded model. If the engine or model is not set up yet, the button shows a guide pointing you to the Engines page or Settings.

macOS requires microphone permission. The first time you use voice input, macOS will ask for permission; grant it to continue.

Native audio input¶

When the selected model accepts audio input, you can send the recording (or an attached audio file) straight to the model as audio instead of transcribing it first. The model then hears the audio itself, so intonation, non-speech sound, and multiple speakers are preserved. This is different from dictation, which transcribes locally and sends only text.

The audio affordance appears only for audio-capable models: mlxcel models such as Gemma 4 Unified on Apple Silicon, llama.cpp models running with an audio projector, and cloud audio models (OpenAI input_audio, Gemini). For a text-only model there is no audio affordance and dictation remains available.

The microphone behavior is set in Settings > Voice Input > Microphone behavior: Auto (send audio when the model supports it, otherwise dictate), Always dictate, or Always send audio. When send-audio is active the composer shows the mode and a data-flow hint: Local for a local model (the audio stays on your device) or Upload for a cloud model (the audio is uploaded). You can switch between dictate and send-audio inline from the mic button.

Privacy: dictation keeps audio on your machine because whisper runs locally and only the transcript is sent. Sending audio to a cloud model uploads the recording to that provider. The active mode and data-flow hint make this visible before you record.

Image Generation¶

Backend.AI GO supports AI-powered image generation directly in the chat interface when connected to an image generation model (such as DALL-E or Stable Diffusion via API).

Asking for an Image in a Chat¶

Ask any chat model that supports tool calling for an image in a message, for example 'Draw a cat on a windowsill'. The model writes an image prompt from your request and the conversation so far, and once its reply ends, the chat moves to image generation (the image mode of the chat page) with the prompt filled in. A negative prompt and a shape (square, portrait, or landscape) are filled in too when you asked for them. Nothing is generated until you press Generate, so you can change the prompt, the model, or the settings first.

The request stays in the conversation as a card. Click the card to open image generation with the same prompt again; when the image conversation it opened before still exists, the card returns to it.

A chat model without tool calling cannot open image generation. When your message reads like a request for an image, a line above the composer suggests image generation instead, and Generate image on it opens image generation with your message as the prompt, as the entry of the same name in the + menu does. Click × to hide the line for the rest of the conversation.

When no image model is selected yet, image generation starts on the Image default model set in Settings > Models. An organization that hides the page id /draw hides image generation, and with it the request in a chat, the card, and the suggestion. See Policy Schema Reference.

Generating Images¶

When an image generation model is available, you can request images by describing what you want in your message. The generated images appear inline in the conversation with the following options:

• View Full Size: Click on the image to view it at full resolution.

• Save: Download the image to your local system with embedded metadata (prompt, model, settings).

• Show Details: Expand to see generation parameters including the prompt, model used, size, and timestamp.

Image Variations¶

You can create variations of any generated image:

1. Click the Variations button below a generated image.

2. Configure the variation settings in the popup:

   • Model: Select the model to use for generating variations.
   • Count: Choose how many variations to create (1-10, depending on the model).
   • Size: Select the output size (square, portrait, or landscape).

3. Click Generate Variations to create new images inspired by the original.

Memory Context¶

The Memory system stores persistent facts and context that the AI references across conversations. For a complete guide including memory banks, the Memory Viewer, auto-extraction, and injection settings, see Memory.

Context Menu¶

Each chat message has a context menu providing quick access to common actions. Hover over a message and click the three-dot menu icon that appears in the message header to open the context menu.

Available Actions¶

• Copy: Copy the message content to the clipboard.

• Regenerate (assistant messages): Generate a new response to the same question.

• Branch conversation (assistant messages): Start a new conversation that holds everything up to and including this answer.

• Save to Library (assistant messages): Bookmark the answer in the Library.

• Save to Memory (user and assistant messages): Save the message content to a memory bank for future context injection. Only available when memory is enabled.

• Save to Data (user and assistant messages): Turn the message into a Data Hub document. Only available when the Data Hub is.

• Edit (user messages): Change the question and get a new answer.

Messages cannot be deleted one by one. To replace part of a conversation, edit the question, regenerate the answer, or branch from an earlier answer.

Keyboard Shortcut¶

You can also open the context menu using the keyboard:

1. Focus the message (click on it or use Tab to navigate).

2. Press Shift+F10 to open the context menu.

3. Use Up and Down to navigate menu items, Enter to select, and Esc to close.

Timestamp Display¶

Each message displays a timestamp showing when it was sent or received. You can customize how timestamps appear throughout the application.

Timestamp Formats¶

Go to Settings > General to configure your timestamp display preference:

• Smart (Default): Automatically switches between relative and absolute formats based on age. Recent messages show "2 minutes ago" while older messages show the actual date and time.

• Relative: Always displays time relative to now (e.g., "5 minutes ago", "2 hours ago", "3 days ago").

• Absolute: Always displays the actual time and date (e.g., "10:15 AM", "Yesterday at 2:30 PM", "Jan 15, 2025 at 3:00 PM").

Smart Mode Threshold¶

When using Smart mode, you can configure the threshold (in hours) at which timestamps switch from relative to absolute format. The default is 24 hours, meaning messages older than 24 hours will show absolute timestamps.

Timezone Display¶

Enable Show timezone to include timezone information in absolute timestamps (e.g., "10:15 AM PST").