Skip to content
These docs are a preview of Self-Managed Kindo 2026.11, an upcoming release.

Model Configuration

This guide explains how Self-Managed Kindo installations define and maintain global models — the system-wide LLMs (and related models like embeddings or transcription) that power workflows, chat, ingestion, and other core experiences.

Before registering a model, prepare its inference endpoint; see AI models. Register models in Kindo and keep their deployments up to date. Use Unleash feature variants to choose global models for LLM features.

Register embedding and transcription models under their reserved serving names on the superadmin dashboard’s Models page. No Unleash model assignment is needed for either.

Each key below is an Unleash feature whose variant payload contains one or more global model IDs. The backend and frontend read these variants to decide which models to use.

Feature KeyDescription
CRON_EXPRESSION_GENERATIONGenerates cron expressions from natural language schedule descriptions
DEFAULT_WORKFLOW_STEP_MODELDefault model for workflow step execution
INGESTION_WORKERSModels for data ingestion and extraction
INTERNAL_AUTO_GENERATIONInternal model for automatic content generation and internal worker tasks
LOCAL_STORAGE_CHAT_ACTIONS_MODELSFallback default chat model for the model dropdown and chat action selection. Applies when an org admin has not chosen a default chat model in Settings → Models (or the chosen model is unavailable)
TOOL_CALLING_MODELSModels that support tool/function calling (filters global models in chat and agent model pickers)
Feature KeyDescription
API_STEP_GENERATIONModels for generating API steps in workflows
DYNAMIC_API_REQUEST_PARSERParses dynamic API requests

Self-Managed Kindo needs baseline global models to function. Without them, core flows like chat, workflow execution, ingestion, and indexing will fail.

Model TypePurposeRecommendations
Embedding modelSemantic search, retrieval, and indexingOpen source: BAAI/bge-m3. Hosted: Gemini embedding
Strong/large modelComplex reasoning, workflow execution, long-form generationOpen source: gpt-oss 120B. Hosted: Gemini 2.5 Pro, Claude 4.5, GPT 5.2
Audio transcriptionAudio file processing, voice notes, transcription workflowsOpen source: Faster Whisper. Hosted: Deepgram
  • Small model (title generation, quick summaries, low-latency steps): Gemma 3 or Llama 3.2 7B
  • Security-focused: DeepHat V3
  • Multimodal: Gemini 2.5 Pro (image + text)

To add a self-managed model, use Settings → Models → Add model. Enabled self-managed models appear in chat and agent model pickers for users allowed by organization and user-group controls; they do not need a TOOL_CALLING_MODELS entry.

To replace a global model:

  1. Add the new global model from the superadmin dashboard’s Models page.
  2. Update the relevant Unleash variants to reference the new model ID.
  3. Delete the old model with replacement.

A self-managed model is a model you already run or pay for somewhere else (a self-hosted vLLM box, a serverless endpoint such as Together, Fireworks, or Groq, an internal inference gateway, or your own OpenAI account) that you want to use inside Kindo for chat and agents. The endpoint must be OpenAI-compatible (it accepts OpenAI-style chat completion requests), or an Anthropic Messages endpoint whose URL ends in /anthropic, such as Claude on Microsoft Foundry. The Add Model form asks only for the handful of facts any such endpoint needs, in plain English.

FieldRequired?What to enter
Display nameYesThe name you’ll see in Kindo when choosing this model, for example Llama 3.3 70B (self-managed).
Model IDYesThe model’s name exactly as your endpoint expects it in a request, for example Qwen/Qwen3.7-Max, llama-3.3-70b, or gpt-4o. Use exactly what your provider or server calls it. For Azure OpenAI or Microsoft Foundry, use your deployment name.
Endpoint URLYesThe base URL of your model’s API, for example https://api.together.xyz/v1, http://my-vllm:8000/v1, or https://your-resource.openai.azure.com. Paste exactly what your provider gives you — Kindo tidies it up (trims it, drops a trailing slash or a mis-pasted /chat/completions, and adds the version segment if your base URL doesn’t already include one).
API versionNoLeave blank for most providers. Only needed when your endpoint requires an api-version query parameter — for example an Azure OpenAI deployment pinned to a dated version such as 2025-04-01-preview.
API keyYesThe key your endpoint uses to authenticate. Stored encrypted; never shown again.
Context windowYesYour model’s maximum context length in tokens, from your provider’s docs, for example 128000. Kindo uses this to keep long conversations within the model’s limit.
Max output tokensYesThe maximum number of tokens the model should generate per response, from your provider’s docs, for example 4096.
Input price, Output priceYesWhat your provider charges, in US dollars per 1 million input and output tokens. Usage counts toward budgets at this price; enter 0 for a model that costs you nothing. Not asked for System One models.

Paste the base URL your provider hands you. Common shapes already work as-is:

  • OpenAI: https://api.openai.com/v1
  • Together: https://api.together.xyz/v1
  • Groq: https://api.groq.com/openai/v1
  • Fireworks: https://api.fireworks.ai/inference/v1
  • Self-hosted vLLM: http://<host>:8000 (Kindo appends /v1 for you)
  • Azure OpenAI: https://<your-resource>.openai.azure.com (Kindo routes to Azure’s OpenAI-compatible /openai/v1 surface for you)
  • Claude on Microsoft Foundry: https://<your-resource>.services.ai.azure.com/anthropic (see Claude on Microsoft Foundry)

Azure OpenAI endpoints work like any other OpenAI-compatible endpoint:

  • Most deployments: paste your resource URL (for example https://my-resource.openai.azure.com), set Model ID to your deployment name, and leave API version blank. Kindo talks to Azure’s OpenAI-compatible v1 surface, which needs no api-version parameter.
  • Deployments pinned to a dated API version: fill in the API version field (for example 2025-04-01-preview) and Kindo dispatches through Azure’s versioned API instead. Pasting the full target URI from the Azure portal (https://…/openai/deployments/<name>/chat/completions?api-version=…) also works — Kindo picks up the api-version from the URL automatically.

Microsoft Foundry serves Claude only through the Anthropic Messages API, not an OpenAI-compatible route. An Endpoint URL whose path ends in /anthropic, /anthropic/v1, or /anthropic/v1/messages tells Kindo to use that API:

  • Paste https://<your-resource>.services.ai.azure.com/anthropic (the full /anthropic/v1/messages URL from the Foundry portal also works).
  • Set Model ID to your deployment name. Any deployment name works.
  • Leave API version blank. The Messages API does not use one, so Kindo ignores it for these endpoints.
  • Set Max output tokens within the model’s limit. The Messages API requires it on every request.

Kindo authenticates with your key in the x-api-key header and translates chat and tool calls between Kindo and the Messages API for you.

Test & Save (the primary button) sends a real request to your endpoint (a chat completion, or a Messages request for an /anthropic endpoint) before anything is saved, and tries again in the shapes described below when the endpoint asks for one. If the test fails, nothing is created: the form stays filled in and the error points at the field to fix: a rejected key highlights API key, an unknown model name highlights Model ID, an unreachable or timed-out endpoint highlights Endpoint URL, a too-high Max output tokens highlights that field and names the model’s maximum when the provider reports it, and a rejected request parameter is called out by name. Any other failure shows the endpoint’s own error message when it sends one, with your API key and other recognizable credentials removed, or otherwise a description of what the endpoint returned. From the failure banner you can either Edit the form and test again, or Save anyway to store the config without a passing test.

Some models reject the max_tokens request parameter and accept only max_completion_tokens, for example some newer Azure OpenAI deployments. When your endpoint rejects max_tokens by name, Test & Save sends the test once more with max_completion_tokens. If that passes, Kindo saves the model so its requests carry your Max output tokens as max_completion_tokens. Save without testing can’t detect this, so save such a model with Test & Save.

Kindo chat always offers the model tools and streams its replies, so the test does both: it includes a tool, asks for a streamed reply, and passes as soon as the reply starts. Some Azure OpenAI reasoning models reject tools on chat completions and point to the Responses API instead. When the endpoint rejects the tool that way, Test & Save sends the test once more to the Responses API. If that passes, Kindo saves the model so its requests use the Responses API, where chat with tools works. Save without testing can’t detect this either. On an endpoint with an API version, the Responses API needs version 2025-03-01-preview or later.

If the endpoint still doesn’t support tool calling or streaming, or answers the test with another rejection, Test & Save sends the test again without tools or streaming. A rejected key, an unreachable endpoint, or a timeout ends the test instead. If that passes, the model is saved, because it still answers API requests that don’t stream or use tools, and a warning that stays until you close it says chat and agents may not work with it. When you add the endpoint as another provider for an existing model, the warning says chat and agents on that model may not work whenever a request goes to this endpoint. The warning is shown only when you save. For example, a vLLM server needs --enable-auto-tool-choice and --tool-call-parser for tool calling (see AI models), and on an endpoint with an API version, a newer version may add tool calling or streaming.

Save without testing skips the test entirely — useful when the endpoint is temporarily down, a self-hosted model is cold-starting, or you don’t want to spend a request on a quota-limited endpoint.

Organization admins can change a self-managed model and each of its providers after adding them. Kindo-supplied models have no edit action.

  1. Go to Settings → Models.
  2. Select the edit icon for the provider you want to change:
    • For a model with one provider, the edit icon is on the model’s row.
    • For a model with several providers, select the model’s row to show its providers, then select the edit icon on the provider.
  3. Change the fields, then select Save changes. A connection change offers Test & Save and Save without testing instead; see When a change needs a test.

The edit form is the Add Model form, filled in with the saved values. Its top section holds the model’s Display name, which every provider of the model shares, and its Context window, which applies to every provider that doesn’t set its own. Below it are this provider’s connection settings.

Saved changes apply to new requests within about a minute.

  • A change to the display name, context window, or the provider’s Input price and Output price saves without a test: select Save changes. A new price applies to requests from then on; see Budgets for how self-managed prices count toward spend.
  • A change to the connection (Model ID, Endpoint URL, API version, API key, or Max output tokens) offers Test & Save and Save without testing, as when you add a model. Test & Save needs the API key, so select Replace and enter it to test. Save without testing keeps the request settings an earlier test found, such as sending max_completion_tokens or using the Responses API. If Kindo can’t read those saved settings, for example just after the model was added, Save without testing is unavailable: select Replace, enter the API key, and use Test & Save.
  • If Test & Save passes only without tools or streaming, the change is still saved, and a warning that stays until you close it says chat and agents may not work with this endpoint.

The API key field shows the saved key as dots; Kindo never shows a saved key. To change it, select Replace and enter the new key. Cancel keeps the current key, and so does leaving the new key empty, including when you change the Endpoint URL.

Models with Multiple Providers (Model Groups)

Section titled “Models with Multiple Providers (Model Groups)”

A model in Kindo is not tied to one endpoint. You can attach several provider endpoints to the same model — for example the same model served from two regions of your cloud provider, or a self-hosted model behind two vLLM endpoints. To add a provider to a model, select the add icon (Add a fallback provider) on a single-provider model’s row, or Add provider below an expanded model’s providers. The Add Model form opens with that model already picked in the Model field, and the endpoint you configure is added as another provider for it. You can also open Add Model and pick an existing model in the Model field instead of typing a new name. Either way, the provider you add joins the end of the model’s fallback chain. Attaching and reordering a model’s providers is an organization-admin action on the Enterprise tier; the endpoints you attach belong to the model’s own provider.

People always pick the model, never a specific provider. Which of the model’s providers serves a given request is not decided until the request is already on its way, by the model’s fallback chain and its session affinity.

Because of that, a model group behaves as the lowest common denominator of its providers:

ControlHow the group resolves itEffect
Context windowSmallest across the model’s enabled providersA 32k provider under a 200k model makes the whole model 32k.
Tool limitSmallest across the model’s enabled providersAn OpenAI or Azure OpenAI endpoint accepts at most 128 tools per request, so one of them makes 128 tools the model’s effective limit.
DLP filtersStrictest across the model’s providersA filter that redacts for any provider redacts for the whole model.
Provider enablementStrictest across the model’s providersA provider your organization has disabled in its security controls makes the whole model unavailable, even where that provider is attached to the model only through a disabled endpoint. (Turning off one endpoint in the model’s provider list is a different thing — see Fallback chain.)

Two reasons, both about the request being decided before the provider is:

  • Routing is blind and failover is silent. Kindo has to fix the capacity budget for a request before it knows which provider will serve it — and a request that fails over mid-flight can land on a different provider than it started with. The only budget that is valid for every provider the request could reach is the smallest one.
  • Runs have to replay identically. The budget is recorded when the request starts so that a resumed or retried run reproduces exactly the same conversation. If the budget tracked whichever provider happened to answer, a retry that fell back to a smaller provider would overflow it.

Group providers with like-for-like capacity

Section titled “Group providers with like-for-like capacity”

Pick providers that are close to each other in capability, so nothing is stranded at the floor:

  • Match context windows. Attaching a 32k endpoint to a 200k model does not add 32k of capacity — it removes 168k from the model. If a provider is much smaller, give it its own model instead, so people can choose it deliberately.
  • Match tool support. If tool-heavy agents use the model, an OpenAI or Azure OpenAI endpoint accepts at most 128 tools per request — a request carrying more can fail whenever that endpoint serves it, so the model’s effective tool limit becomes 128.
  • Match governance. A provider your organization has disabled, or one with stricter DLP filters, applies its rules to the whole model, not just to the requests it serves.

The Add Model form tells you about this before you save: once you pick a model to attach to, it shows the context window the new endpoint should clear and warns you when the endpoint would newly limit the model’s tool use.

The providers attached to a model form a fallback chain: an ordered list, lowest priority number first, and that is the order its providers show in when you select the model’s row in Settings → Models. Each provider shows its place in the chain, Primary for the first level and Fallback for the rest. To change the chain, use the up and down arrows on a provider, or select Make primary in its menu (⋯) to move it to the top.

Select Turn off in a provider’s menu to take its endpoint out of the chain. The model’s other endpoints keep serving, and the provider’s configuration stays in place so you can select Turn on later. A model’s last active provider can’t be turned off or removed while the model has other providers, and a chat provider without a price can’t be turned back on until it has one (What counts as spend). System One providers take no price and aren’t affected by that rule. A model whose only provider is on has no on/off control; if its only provider is off, select the model’s row, then select Turn on in that provider’s menu.

Two things move a request off the provider that was chosen for it:

  • Retries. A transient failure — a timeout, a rate limit, or a provider-side error — is retried inside the model, up to twice. Failures a retry cannot fix, such as a malformed request, are not retried.
  • Descent to the next priority level. When the retries are used up, the request is re-aimed at the next priority level down the chain. This is also how an interrupted streamed response continues: it resumes on the next level, and text already streamed to the reader is not repeated. Most errors take this path once, not only transient ones — a request whose content the model rejected outright, or one that overflowed the context window, is returned to the caller instead.

Descending the chain needs at least two distinct priority levels. An endpoint you attach in Settings → Models goes to the end of the chain, one level below the model’s last provider. Providers can still share a priority number, for example when they were set to the same priority outside Settings → Models. Active providers that share a priority share traffic rather than one preferring the other: new conversations are split between them, and session affinity keeps each conversation on the provider it started with where it can. A retry can still move a failed request to the other, but there is no lower level to descend to. The provider list shows such providers at the same place, marked shares traffic. To make one of them a deliberate backup, move it down (or the other up) once: the list renumbers into distinct levels and the chain becomes strict.

A provider that fails persistently is put in cooldown: it is dropped from the chain for about half a minute and then becomes eligible again on its own, so a hard-down endpoint is not re-tried by every request in the meantime and recovery needs no administrator action. Cooldown only ever removes a provider that has an alternative — a model with a single provider is never cooled out of its own chain, because there would be nothing left to serve it.

Providers cache the part of a prompt they have already seen, and that cache lives on the endpoint that served the request — it is not shared between two endpoints of the same model. Because a conversation resends its whole history on every turn, a cache hit is the difference between the provider re-reading the entire conversation and re-reading only the newest turn. That drives both latency (how fast a long conversation starts responding) and cost (cached input is billed at a fraction of fresh input). A long conversation bounced between two providers would pay full price for the same history again on every switch.

So once a conversation is under way, Kindo keeps it on the provider that served its earlier turns where it can — session affinity:

  • The pin is per conversation, and it is best effort. It exists to keep a cache warm; it is not a promise about which provider answers you. It covers the conversations Kindo drives — chat and agent runs. Requests sent straight to the Kindo APIs are not pinned.
  • The pin expires. It is upper-bounded by a time-to-live (one hour by default), so a conversation picked back up much later is routed fresh. The lifetime is deliberately longer than the provider cache windows it protects: a pin whose cache has already lapsed simply re-warms the same endpoint.
  • Health beats affinity. If the pinned provider is in cooldown or otherwise out of the healthy set, the conversation falls through to a surviving provider and pins to whichever one serves it. Affinity never holds a conversation on a dead endpoint.
  • Until it is cooled out, though, the pinned provider is still preferred — including over a healthy provider at a lower priority level. A conversation can therefore keep attempting a failing endpoint for the turn or two it takes cooldown to remove it, which is what recovery looks like from the inside rather than a stuck conversation.
  • A single-provider model is unaffected — with one endpoint there is only one choice.

What this means when you plan a model’s providers: adding a provider to a busy model does not cool the caches you already have. A provider you attach starts as the last fallback, so conversations keep going to the providers above it; where active providers share a priority, conversations in flight stay where they are and only later ones spread out. It also means the providers worth grouping are the ones that are close in capability (see above) — a chain whose members are interchangeable is the chain that keeps caches warm, since every fallback lands on a cold cache.

A model has exactly one provider configured unless an administrator attaches another, and that is the state every model starts in. A single-provider model has no chain to descend and no pin worth observing, so everything above describes behavior you opt into per model, one provider at a time — there is nothing to migrate, nothing to re-configure, and nothing required of an administrator who never attaches a second provider.