Model providers

A model provider adapter is the object an agent calls to reach a language model. Mayura ships three: OpenAI's Responses API, Anthropic's Messages API, and a Chat Completions adapter for local servers and the many providers that speak the OpenAI dialect. Every adapter has the same shape (a ModelAdapter), so agents, tools, guards and budgets stay the same when you switch provider.

Adapters never look for credentials in the environment, never pick a model or a price for you, never retry, never follow redirects and make no request until an agent calls them. You pass everything explicitly.

import { createRuntime, defineAgent, z } from 'mayura';
import { openAIResponses } from 'mayura/provider-openai';

const Answer = z.object({ answer: z.string() });

const model = openAIResponses({
  apiKey: process.env.OPENAI_API_KEY!,
  model: process.env.OPENAI_MODEL!,
  maxCostMicros: 20_000, // at most $0.02 per model call
  // Use your model's current prices, in micros per million tokens ($0.40 = 400_000).
  pricing: { inputMicrosPerMillionTokens: 400_000, outputMicrosPerMillionTokens: 1_600_000 },
});

const agent = defineAgent({
  id: 'helper', version: '1', instructions: 'Answer the question in one sentence.',
  model, tools: [], input: z.object({ question: z.string() }), output: Answer,
});

const runtime = createRuntime({
  profile: 'ephemeral',
  permissions: { allow: ['model:openai.responses'] },
  limits: { maxCostMicros: 100_000 }, // at most $0.10 per run
});
try {
  const outcome = await runtime.submit(agent, { input: { question: 'What is a haiku?' } }).result();
  console.log(outcome.status === 'succeeded' ? outcome.output.answer : outcome.error);
} finally {
  await runtime.close();
}

#The adapters

A runtime lets an agent call a model only when its permissions include model:<adapter id>, for example model:anthropic.messages. A run without that grant ends blocked with PERMISSION_DENIED before any request is sent. To spread calls over several providers with failover, see Model routing.

openAIResponses sends requests to https://api.openai.com/v1/responses with store: false, and keeps OpenAI's reasoning items between the model calls of one run. anthropicMessages sends them to https://api.anthropic.com/v1/messages; it does not turn on prompt caching, extended thinking or provider-hosted tools. Both destinations are fixed.

import { anthropicMessages } from 'mayura/provider-anthropic';

const claude = anthropicMessages({
  apiKey: process.env.ANTHROPIC_API_KEY!,
  model: process.env.ANTHROPIC_MODEL!,
  maxCostMicros: 30_000,
  pricing: { inputMicrosPerMillionTokens: 3_000_000, outputMicrosPerMillionTokens: 15_000_000 },
});

#Options

All three adapters take these options. openAICompatibleChat adds remote and token, and its apiKey is optional. openAIResponses and anthropicMessages send to the provider's own API unless you give an endpoint, for example a gateway (see Through a gateway).

Cost is computed from the token usage the provider reports and your prices, rounded up to a whole micro. Anthropic cache-creation and cache-read tokens are charged at the input price. This is your own accounting, not the provider's invoice: keep pricing current.

maxCostMicros is a per-call bound. The runtime's run limit (limits.maxCostMicros, default 0) must be at least as large, or the first call cannot start. If a call reports a cost above its bound, the cost is still recorded and the run stops with BUDGET_EXCEEDED. The runtime also sends limits.maxOutputTokens (default 4,096) as the provider's output-token limit, so pick a bound that covers your largest prompt plus that many output tokens. See Costs and budgets.

#JSON Schemas

A provider needs the agent's output, and every tool's input, as JSON Schema. You don't write them: defineAgent and defineTool generate them from your Zod schemas (any validator that implements Standard JSON Schema works; Zod 4.2 and later does), and the runtime sends the agent's output schema with every call. So one adapter can serve many agents. Give outputJsonSchema to the adapter, or to defineAgent, only when your validator cannot describe itself.

Adapters ask the provider for strict structured output and strict tool calls, so the schemas must follow the strict rules:

• the root is type: 'object';
• every object lists all of its properties in required and sets additionalProperties: false (an object with no properties, such as the input of a tool that takes none, may leave required out).

In Zod terms: make a field the model may leave empty .nullable(), not .optional() or .default(), and avoid z.record. Transforms are fine; the schema describes what the model writes. defineAgent checks every tool and the output against these rules with the adapter you give it, and throws INVALID_CONFIG with the field and the fix:

Agent support: The output schema: the object at /properties/tags must set additionalProperties to false; model
providers accept no open objects, records or maps.

The agent's output validator still checks the provider's answer after it arrives, so it may be stricter than the JSON Schema.

Mayura sends each tool under its own id, made safe for provider function names: letters, digits, _ and -, at most 64 characters. So orders.list reaches the model as orders_list, and instructions that name tools by id still match. Two ids that come out the same (orders.list and orders/list) get _2, _3 and so on. Mayura maps the names back, and a call to a name that is not one of the agent's tools ends the run. Write descriptions that say what each tool does and when to use it: the model chooses by name and description.

#OpenAI-compatible providers

Many providers and local model servers accept the OpenAI Chat Completions API. openAICompatibleChat talks to them.

Local servers. Without remote, the endpoint must be plain HTTP on localhost, 127.0.0.1 or [::1], with the exact path /v1/chat/completions. A key is optional.

import { openAICompatibleChat } from 'mayura/provider-openai';

const local = openAICompatibleChat({
  endpoint: 'http://127.0.0.1:11434/v1/chat/completions', // for example, Ollama's default port
  model: 'my-local-model',
  maxCostMicros: 0,
  pricing: { inputMicrosPerMillionTokens: 0, outputMicrosPerMillionTokens: 0 },
});

Remote providers. Sending prompts to another host is a decision you make explicitly with remote. The endpoint must be HTTPS and end in /chat/completions, with no credentials in the URL and no query other than api-version. remote.id (lower-case letters, digits and hyphens) becomes part of the adapter id, so each provider is granted separately: remote: { id: 'groq' } is granted as model:openai-compatible.groq. A remote provider needs apiKey or token.

import { openAICompatibleChat } from 'mayura/provider-openai';

const groq = openAICompatibleChat({
  endpoint: 'https://api.groq.com/openai/v1/chat/completions',
  remote: { id: 'groq' },
  apiKey: process.env.GROQ_API_KEY!,
  model: process.env.GROQ_MODEL!,
  maxCostMicros: 10_000,
  pricing: { inputMicrosPerMillionTokens: 100_000, outputMicrosPerMillionTokens: 300_000 },
});

Commonly used endpoints (check each provider's current documentation):

Check your provider with a small budget before you rely on it (pnpm providers:live-check in the Mayura repository does this through the real runtime). Mayura's CI runs against fake servers; DeepSeek and Cloudflare AI Gateway have also passed the live check. The adapter needs tool calls, structured output and reported token usage (for streaming, stream_options.include_usage). A provider that answers differently, or leaves out usage, fails the call with a clear reason rather than being guessed at. Providers that differ from OpenAI in known ways are configured with these options:

Thinking models. Some providers return the model's reasoning with a tool call and require it back on the next request (DeepSeek answers 400 without it). The adapter keeps each such turn, reasoning included, in the run's private state and sends it back, so thinking models can use tools. The reasoning is never released as output or put in events. anthropicMessages does the same for Claude's signed thinking and redacted_thinking blocks, which Claude models may return before a tool call or an answer.

DeepSeek. It offers JSON mode but not JSON Schema output, and strict tool calls on its beta endpoint:

import { openAICompatibleChat } from 'mayura/provider-openai';

const deepseek = openAICompatibleChat({
  endpoint: 'https://api.deepseek.com/beta/chat/completions',
  remote: { id: 'deepseek' },
  apiKey: process.env.DEEPSEEK_API_KEY!,
  model: 'deepseek-flash',
  output: 'json_object',
  strictTools: true,
  maxCostMicros: 10_000,
  pricing, // check DeepSeek's current prices
});

Grant it as model:openai-compatible.deepseek. To turn thinking off, add body: { thinking: { type: 'disabled' } }.

Cloudflare AI Gateway fronts many providers with one OpenAI-compatible endpoint. Models are named provider/model, such as deepseek/deepseek-flash. An authenticated gateway takes its token in cf-aig-authorization; when the gateway stores the provider's key, no apiKey is needed. Use the dialect of the provider behind it:

import { openAICompatibleChat } from 'mayura/provider-openai';

const gateway = openAICompatibleChat({
  endpoint: `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}/compat/chat/completions`,
  remote: { id: 'cloudflare' },
  headers: { 'cf-aig-authorization': `Bearer ${process.env.CF_AIG_TOKEN}` },
  model: 'deepseek/deepseek-flash',
  output: 'json_object', // DeepSeek behind the gateway
  strictTools: true,
  maxCostMicros: 10_000,
  pricing,
});

For an OpenAI model behind the gateway (openai/...), leave output at its default, set tokenLimitField: 'max_completion_tokens', and check the model's own rules: some of OpenAI's reasoning models accept function tools on Chat Completions only with reasoning turned off, which body: { reasoning_effort: 'none' } does. The provider's 400 names the setting it refused.

mayura init sets up both: pick DeepSeek or Cloudflare AI Gateway in the wizard.

#Through a gateway

openAIResponses and anthropicMessages can send their own protocol through a gateway or proxy, such as Cloudflare AI Gateway's provider endpoints. Give the full endpoint (not a prefix, whose convention differs between SDKs) and the gateway's credential in headers. When the gateway stores the provider key, leave apiKey out:

import { anthropicMessages } from 'mayura/provider-anthropic';
import { openAIResponses } from 'mayura/provider-openai';

const base = `https://gateway.ai.cloudflare.com/v1/${accountId}/${gatewayId}`;
const headers = { 'cf-aig-authorization': `Bearer ${process.env.CF_AIG_TOKEN}` };

const gpt = openAIResponses({ endpoint: `${base}/openai/responses`, headers, model: 'gpt-6-luna', maxCostMicros: 10_000, pricing });
const claude = anthropicMessages({ endpoint: `${base}/anthropic/v1/messages`, headers, model: 'claude-sonnet-5', maxCostMicros: 10_000, pricing });

The adapters' ids, and so their grants (model:openai.responses, model:anthropic.messages), stay the same. A gateway is where your prompts and outputs go: choose it as deliberately as the provider. Both adapters pass every pnpm providers:live-check check through Cloudflare AI Gateway this way.

Azure OpenAI sends the key in the api-key header instead of Authorization: Bearer; set auth: 'api-key'.

Vertex AI uses short-lived Google access tokens. Pass token instead of apiKey: it is called before every request, so each call gets a fresh token.

const vertex = openAICompatibleChat({
  endpoint: vertexChatCompletionsUrl, // Vertex AI's OpenAI-compatible endpoint for your project and region
  remote: { id: 'vertex' },
  token: () => getAccessToken(), // for example from google-auth-library
  model: process.env.VERTEX_MODEL!,
  maxCostMicros: 20_000,
  pricing,
});

The compatible adapter sends the whole conversation on every call. For OpenAI itself, use openAIResponses, which keeps OpenAI's reasoning items between calls.

#Errors

A failed call ends the run with MODEL_FAILED and a message that says why (see Outcomes):

The messages are Mayura's own. Provider error bodies and your key never appear in outcomes or events. Cancelling a run aborts the request in flight, but cannot prove the provider did no work, so an unconfirmed call keeps its full reservation.

#Writing your own adapter

Any object with this shape is a model adapter. Implement one for a provider Mayura does not cover, or to wrap an SDK you already use:

import { MayuraError, type JsonValue, type ModelAdapter, type ModelRequest, type ModelResponse } from 'mayura';

export const myModel: ModelAdapter = {
  id: 'my-provider.chat', // granted as model:my-provider.chat
  capabilities: { tools: true, structuredOutput: true },
  maxCostMicros: 20_000,
  async generate(request: ModelRequest): Promise<ModelResponse> {
    // request.instructions, request.messages and request.tools describe the call; honour request.signal.
    const answer = await callMyProvider(request, { signal: request.signal }) as {
      readonly toolCalls: { readonly id: string; readonly toolId: string; readonly input: JsonValue }[];
      readonly output: JsonValue;
      readonly costMicros: number;
    };
    if (!answer) throw new MayuraError('MODEL_FAILED', 'The provider returned no answer.');
    if (answer.toolCalls.length > 0) {
      return { type: 'tool_calls', calls: answer.toolCalls, usage: { costMicros: answer.costMicros } };
    }
    return { type: 'final', output: answer.output, usage: { costMicros: answer.costMicros } };
  },
};

• generate returns either tool calls (each with its own id, the Mayura toolId and the input) or a final output, with the call's cost in micros. Mayura validates both before using them.
• To say why a call failed, throw new ModelProviderError(reason, { httpStatus, costMicros }) with one of the reasons above: the outcome then carries Mayura's message for that reason. Pass costMicros when the provider reported usage before failing, so the known cost is charged; otherwise the call keeps its full maxCostMicros reservation. Any other error is reported with a generic message, since an adapter's own text never reaches an outcome.
• capabilities.media, { types, urls }, declares the images and PDFs the model can see. Messages then carry media (bytes as data, or a url); bytesToBase64, mediaDataUrl and encodedMediaBytes from mayura/core/host help encode them. Leave it out for a model that cannot see. See Vision.
• An optional checkDefinition(definition) method lets defineAgent check the agent's tools and output schema once, so a problem shows when the agent is defined. checkStrictDefinition from mayura/core/host implements the strict rules above.
• An optional stream(request) method yields { type: 'output.delta', text } fragments of the final output's JSON text, then exactly one { type: 'response', response }. See Streaming.
• continuation on a response is opaque state the runtime hands back on the next call of the same run.

For tests, scriptedModel from mayura/testing is a ready-made adapter; see Testing.

#Embeddings

openAIEmbeddings({ apiKey, model, dimensions }) from mayura/provider-openai is an embedding adapter for native memory, not a chat model. See Memory and context.

#Related

• Model routing
• Streaming
• Costs and budgets
• Permissions
• Testing