LLM Router

LLM Router

Know what your AI actually costs

Most teams measure LLM spend by counting the responses they got back. Providers bill for the ones you didn't — retries, schema rejections, truncated generations. The gap shows up on the invoice, not the dashboard.

One request, as your dashboard shows it
Call Model served Attempt Tokens Billed
1 gpt-4o 1 — truncated, unusable 1,610 Yes
2 gpt-4o 2 — schema rejected 1,840 Yes
3 gpt-4o 3 — completed 2,120 Yes
Delivered to the customer: one answer 5,570 All

The customer got one answer. Two of these three calls are invisible in most dashboards — and all three cost real tokens. You can see every one of them in yours.

  • Accounting on every round-trip. Each billed call is recorded with the model the provider actually served, the retry attempt, and the service tier multiplying the price. Attribute real spend by provider, model, and outcome across an entire service.

  • And it still works when you stream. Most gateways quietly degrade the moment you turn streaming on: you get text chunks, but token usage and finish reasons stop coming back, and tool calling often stops working entirely. That's a problem, because customer-facing chat and voice stream by definition — so the visibility disappears exactly where your traffic is.

    Here, streaming gives up nothing. System prompts and multi-turn roles are preserved. Token usage and finish reasons are reported. Tools execute, and tool calls arrive as incremental deltas you can act on mid-response.

    You shouldn't have to choose between a responsive interface and knowing what it cost.

  • Requests shaped for the model receiving them. Explicit model-family rules mean each call carries only the parameters its target accepts — eliminating rejected requests and, more importantly, the silent parameter downgrades that return confident wrong answers instead of errors.

  • Failures you can branch on. Rate limit, authentication, invalid input, unsupported: classified by type, not matched by string. Empty responses report their stop reason and likely cause instead of failing opaquely.

  • One reasoning scale across providers. Control thinking effort from none to max across OpenAI, Anthropic, and Gemini. Levels a model doesn't support clamp inward rather than erroring, so raising effort never breaks a working call.

Runs against what you already use

OpenAI (Chat Completions and Responses, including GPT-5 and o-series), Anthropic Claude (with extended thinking, prompt caching, and web search), Google Gemini, Groq, Mistral, Cohere, DeepSeek, Bedrock, Azure OpenAI, OpenRouter, and local deployments via Ollama, vLLM, and LM Studio.

Why it matters on your invoice

Routing routine turns to cheaper models and eliminating silently-failed calls is how customers stay inside their included usage — and how we can show you exactly where an overage came from.

Most customers run the Router alongside the Agent Platform, where it meters and protects the same allowance. It's also available on its own if you're running your own stack and want the accounting.

Also included: prompt optimizer with custom metrics and rating systems · memory retention across interactions · structured output with JSON schema generation and validation · provider comparison tooling · ChainOfThought and other high-level functions · Mixture of Agents · configuration via environment variables, code, or config files · extensible provider architecture.

© Copyright 2026. Nuestra AI. All rights reserved.