> For the complete documentation index, see [llms.txt](https://yvechat.gitbook.io/wp/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://yvechat.gitbook.io/wp/architecture/runtimes.md).
# Runtimes
YveChat supports Ollama first and keeps the architecture open to every OpenAI-compatible server.
***
### Position
| Runtime | Status | Statement |
| ------------------------------ | ------------ | ------------------------------------------------------------------- |
| **Ollama** | Available | The supported, full-featured default |
| **OpenAI-compatible endpoint** | Experimental | Configured with a base URL. Chat is served conservatively |
| **vLLM** | Experimental | Reached through the OpenAI-compatible adapter, discovered as `vllm` |
| **ComfyUI** | Available | Local image generation, through a capability-aware registry |
Capabilities are never guessed from a model name. A runtime reports what it supports, and the interface reflects that report.
Ollama leads because it offers the simplest consumer experience: one installer, model discovery, and model pulling. The product is never branded as permanently Ollama-only, because the provider registry and normalized chat service already treat runtimes as interchangeable.
***
### Ollama
The default path for local inference.
| Property | Detail | Source |
| --------------- | ----------------------------------------------------------------------------------------- | --------------------------------------- |
| Default bind | `127.0.0.1:11434` | [FAQ](https://docs.ollama.com/faq) |
| Default context | 4096 tokens, set by `OLLAMA_CONTEXT_LENGTH` or `num_ctx` | [FAQ](https://docs.ollama.com/faq) |
| Concurrency | One parallel request per model by default, three loaded models per GPU, 512-request queue | [FAQ](https://docs.ollama.com/faq) |
| Memory scaling | Scales with parallelism multiplied by context length | [FAQ](https://docs.ollama.com/faq) |
| GPU support | NVIDIA compute capability 5.0 and above, AMD ROCm, Apple Metal, Vulkan | [GPU docs](https://docs.ollama.com/gpu) |
{% hint style="warning" %}
The 4096-token default matters. A local model compared against a frontier model with a large context window will appear to forget things unless the context length is set explicitly. YveChat sets it per request and surfaces the active value.
{% endhint %}
***
### vLLM
The path for users with GPU infrastructure.
vLLM exposes an OpenAI-compatible HTTP server supporting chat, streaming, tools, and multimodal models, which makes it a natural second adapter. Compatibility guarantees the request shape, and feature parity varies: vLLM accepts parameters the OpenAI API omits, such as `top_k` passed through `extra_body`, and omits others, such as `image_url.detail`. Capability is therefore negotiated per backend.
Installation targets CUDA and ROCm environments, which makes vLLM a datacenter-shaped dependency and a power-user path.
Source: [vLLM OpenAI-compatible server](https://docs.vllm.ai/en/latest/serving/openai_compatible_server/)
***
### Model Management
Controls sit directly under the model selector, and every action is provider-aware.
| Control | Ollama | vLLM | Unknown OpenAI-compatible |
| ----------------------- | ------------------------------------------------ | ------------------------ | ------------------------- |
| **Runtime status** | `Connected to Ollama` | Served model list | Endpoint reachability |
| **Manage local models** | Installed models with size | Currently served models | Configured endpoints |
| **Refresh models** | Re-enumerate | Re-query the served list | Re-probe |
| **Get more models** | Browse the Ollama library, then an approved pull | Setup guidance | Endpoint configuration |
Capability filters apply across all runtimes: Chat, Vision, Tools, Embedding.
{% hint style="danger" %}
The hosted browser never executes installation commands. The Engine performs them after explicit user confirmation and reports progress back to the interface.
{% endhint %}
vLLM is presented honestly. It serves models specified at launch, and it carries no Ollama-style pulling workflow, so the interface offers setup guidance in place of a pull button.
***
### Runtime Selection
```mermaid
graph TD
A["Which runtime?"] --> B{"Consumer machine,
simplest path?"}
B -->|Yes| C["Ollama"]
B -->|No| D{"GPU infrastructure,
high throughput?"}
D -->|Yes| E["vLLM"]
D -->|No| F["Any OpenAI-compatible
endpoint by URL"]
```
***
One installer for most people.
An open contract for everyone else.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://yvechat.gitbook.io/wp/architecture/runtimes.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.