Models & Providers

ScalyClaw routes LLM calls through a configurable provider stack. You can register multiple models from different providers and control how they are picked at call time through priority (selection tier) and weight (load-balancing share). All model configuration lives in Redis — no config files on disk — and changes take effect immediately without a restart.

Model Configuration

Models are added and managed in the dashboard under Settings → Models. Each entry in the model list describes a single provider/model combination. You can have as many entries as you need; the orchestrator selects the right one at call time based on priority, weight, and the capabilities required by that particular request.

Model Properties

Model Selection Algorithm

ScalyClaw picks one model per request using a two-step algorithm:

1. Filter out disabled — only models with enabled: true are considered. Capability flags (toolEnabled, imageEnabled, etc.) are not enforced at this step — see the callout below.
2. Top priority group — candidates are sorted by priority (lower number = higher priority). Only the lowest-numbered group is considered. Higher-numbered groups are effectively dormant unless every model in a lower-numbered group is disabled or removed.
3. Weighted random pick — within that priority group, one model is selected probabilistically. Each model's chance equals weight / totalWeight. A model with weight: 75 is picked three times as often as one with weight: 25 in the same group.
4. Retry on transient failure — if the selected provider/model call fails with a retryable error, the same provider/model is retried with exponential backoff (default 3 attempts, 500ms base delay; see shared/src/const/constants.ts). Retries never swap to a different model or priority group. If all retries fail, the error propagates to the caller.

Selection Hierarchy

Different components in ScalyClaw can define their own model lists. Selection follows a fallback chain from most specific to most general:

This means the orchestrator and each agent can run on different models. For example, the orchestrator can use Claude Opus while a lightweight research agent uses GPT-4o-mini — all configurable without code changes.

"auto" Model Selection

Several config fields accept the special value "auto", which means "use weighted-random selection from enabled models instead of a fixed model." This is the recommended default — it enables load balancing and automatic fallback.

Supported Providers

Providers are registered under config.models.providers as a map from a provider key to an object with an optional apiKey and optional baseUrl. Each model entry's provider field references one of these keys. ScalyClaw ships with 13 built-in provider keys. anthropic uses the native Anthropic SDK; every other key routes through a single OpenAI-compatible factory with the provider's own base URL.

A provider is registered at boot and on every config reload. If a provider is marked "key required" above and you haven't supplied apiKey, registration is skipped with a warning log — the provider won't be usable until you add one. There is no azure key in the current build; point a custom entry at your Azure OpenAI endpoint instead.

Example Configuration

A realistic multi-provider setup: Claude Opus 4.7 as the primary, GPT-5.4 as a same-priority peer with lower weight, and a local Qwen model via Ollama for offline resilience. Because there is no automatic cross-priority fallback, put the two cloud models in the same priority tier to share load; the local model sits in a higher-numbered tier and is effectively dormant until you disable the cloud ones.

{
  "models": {
    "providers": {
      "anthropic": { "apiKey": "sk-ant-..." },
      "openai":    { "apiKey": "sk-..." },
      "ollama":    { "baseUrl": "http://localhost:11434/v1" }
    },
    "models": [
      {
        "id": "anthropic:claude-opus-4-7",
        "name": "anthropic:claude-opus-4-7",
        "provider": "anthropic",
        "enabled": true,
        "priority": 1,
        "weight": 75,
        "temperature": 0.7,
        "maxTokens": 8192,
        "contextWindow": 1000000,
        "toolEnabled": true,
        "imageEnabled": true,
        "audioEnabled": false,
        "videoEnabled": false,
        "documentEnabled": true,
        "reasoningEnabled": true,
        "inputPricePerMillion": 5.00,
        "outputPricePerMillion": 25.00
      },
      {
        "id": "openai:gpt-5.4",
        "name": "openai:gpt-5.4",
        "provider": "openai",
        "enabled": true,
        "priority": 1,
        "weight": 25,
        "temperature": 0.7,
        "maxTokens": 8192,
        "contextWindow": 1000000,
        "toolEnabled": true,
        "imageEnabled": true,
        "audioEnabled": false,
        "videoEnabled": false,
        "documentEnabled": false,
        "reasoningEnabled": true,
        "inputPricePerMillion": 2.50,
        "outputPricePerMillion": 15.00
      },
      {
        "id": "ollama:qwen3",
        "name": "ollama:qwen3",
        "provider": "ollama",
        "enabled": true,
        "priority": 2,
        "weight": 100,
        "temperature": 0.7,
        "maxTokens": 2048,
        "contextWindow": 32768,
        "toolEnabled": true,
        "imageEnabled": false,
        "audioEnabled": false,
        "videoEnabled": false,
        "documentEnabled": false,
        "reasoningEnabled": true,
        "inputPricePerMillion": 0,
        "outputPricePerMillion": 0
      }
    ],
    "embeddingModels": []
  }
}

With this config, 75% of priority-1 requests go to Claude and 25% go to GPT-5.4. The local Qwen entry at priority 2 is not consulted while any priority-1 model is enabled — it only comes into play once you disable the cloud models (manually or from the dashboard).

Reasoning Flag

When a model has reasoningEnabled: true, the orchestrator threads the flag through to the provider call. What happens next is provider-specific:

• OpenAI-compatible providers (openai, openrouter, google, groq, xai, deepseek, cohere, mistral, minimax, ollama, lmstudio, custom) translate it to reasoning_effort: "medium" on the request body. Providers that don't implement that field ignore it silently.
• OpenAI reasoning models (o1/o3/o4/gpt-5.x) are auto-detected by name. For these, the provider also switches max_tokens → max_completion_tokens and omits temperature — both required by the API.
• DeepSeek-R1 (deepseek-reasoner) always thinks; the provider captures any reasoning_content in the response for observability but never echoes it back on subsequent turns (echoing it produces a 400).
• Anthropic wraps the flag with a model-name regex check for Claude 4.x. When matched, it sets thinking: { type: "enabled", budget_tokens: … } — Anthropic's extended / adaptive thinking API. Requests against non-4.x models with reasoningEnabled set are not rejected; the flag is simply not forwarded.
• Local models (Qwen, Gemma, DeepSeek-R1 via Ollama/LM Studio) emit <think>…</think> blocks or ChatML-style channel delimiters, which the provider strips before returning content to the orchestrator. Those traces are not persisted anywhere.

Embedding Models

ScalyClaw's memory system stores every saved memory entry alongside a high-dimensional vector embedding. When the orchestrator retrieves relevant context before an LLM call, it runs a cosine-similarity search using sqlite-vec against those stored vectors. The accuracy of that search depends entirely on the quality of the embedding model you choose.

How Embeddings Are Generated

When a memory entry is saved — either automatically by the orchestrator or explicitly via the save_memory tool — ScalyClaw calls the configured embedding model to convert the text into a float32 vector. That vector is stored in the SQLite database alongside the entry. At retrieval time, the query text is embedded on the fly using the same model, and sqlite-vec finds the nearest stored vectors by cosine distance.

Recommended Models

Embedding Configuration

Embedding models live in config.models.embeddingModels, the same config section as chat models but in a separate array. You can use a different provider for embeddings than for chat — for example, use Anthropic for chat but OpenAI's cheaper embedding API for memory. Each entry shares the same providers map as chat models.

{
  "models": {
    "providers": {
      "openai": { "apiKey": "sk-..." }
    },
    "models": [ /* ... chat models ... */ ],
    "embeddingModels": [
      {
        "id": "openai-embed",
        "name": "text-embedding-3-small",
        "provider": "openai",
        "enabled": true,
        "priority": 1,
        "weight": 100,
        "dimensions": 1536,
        "inputPricePerMillion": 0.02,
        "outputPricePerMillion": 0
      }
    ]
  }
}

For a fully local setup with Ollama, add the local provider to the providers map and point the embedding model at it:

{
  "models": {
    "providers": {
      "ollama": { "baseUrl": "http://localhost:11434/v1" }
    },
    "models": [ /* ... chat models ... */ ],
    "embeddingModels": [
      {
        "id": "ollama:nomic-embed-text",
        "name": "ollama:nomic-embed-text",
        "provider": "ollama",
        "enabled": true,
        "priority": 1,
        "weight": 100,
        "dimensions": 768,
        "inputPricePerMillion": 0,
        "outputPricePerMillion": 0
      }
    ]
  }
}

Budget Control

LLM API calls cost money. ScalyClaw tracks token usage per model and per day, accumulates spending estimates based on the inputPricePerMillion and outputPricePerMillion values you configure on each model, and enforces configurable global daily and monthly limits. Budget is a single global config block — there are no per-model budget caps. You can choose between hard enforcement (block all calls when the limit is reached) or soft enforcement (warn but continue).

Enforcement Modes

Per-Model Cost Tracking

Every LLM call records the number of input tokens, output tokens, and the estimated cost in USD using the pricing table ScalyClaw maintains for each known model. Costs are stored in Redis and aggregated by day and by month. The dashboard usage page displays:

• Daily and monthly spend broken down by model
• Token usage histograms per model per day
• Budget consumption as a percentage of configured limits
• A list of the most expensive individual requests

For models with custom or unknown pricing (e.g., local models or new provider releases), set inputPricePerMillion and outputPricePerMillion directly on the model entry in config.models.models. ScalyClaw uses those figures for all cost tracking and budget accounting for that model. Set both to 0 for free local models.

Budget Configuration

{
  "budget": {
    "monthlyLimit": 150,
    "dailyLimit": 10,
    "hardLimit": true,
    "alertThresholds": [50, 80, 90]
  }
}

Budget is a single global block — there are no per-model caps. The fields are:

• monthlyLimit — maximum USD spend per calendar month. Set to 0 for unlimited.
• dailyLimit — maximum USD spend per day (resets at midnight UTC). Set to 0 for unlimited.
• hardLimit — when true, all LLM calls are blocked once a limit is reached. When false, the system continues but emits warnings.
• alertThresholds — array of percentage values (e.g. [50, 80, 90]). A warning is emitted to dashboard logs and any configured alert channels each time cumulative spend crosses one of these thresholds — giving you advance notice before a hard stop occurs.

Custom Pricing

Set inputPricePerMillion and outputPricePerMillion directly on any model entry. ScalyClaw uses those values for all cost tracking for that model:

{
  "id": "custom:my-gateway-gpt4o",
  "name": "custom:my-gateway-gpt4o",
  "provider": "custom",
  "enabled": true,
  "priority": 1,
  "weight": 100,
  "temperature": 0.7,
  "maxTokens": 4096,
  "contextWindow": 128000,
  "toolEnabled": true,
  "imageEnabled": true,
  "audioEnabled": false,
  "videoEnabled": false,
  "documentEnabled": false,
  "reasoningEnabled": false,
  "inputPricePerMillion": 2.50,
  "outputPricePerMillion": 10.00
}