Groq models

Groq serves open models, such as OpenAI's gpt-oss and Qwen, on its own inference hardware. @mayurajs/provider-groq is a provider package for the model registry: every Groq model gets an id such as groq/openai/gpt-oss-120b, which you price, bound and grant like any other registry model. It calls Groq's Chat Completions API through the official groq-sdk, pinned at 1.6.0, which is its only dependency (mayura is a peer). The SDK's retries and logging are turned off. The package itself uses no Node built-in modules and sends every request through fetch; its engines field names Node.js 22 (22.12 or later) and 24 (24.14.1 or later), and its tests run on Node.

#Install

npm install mayura @mayurajs/provider-groq

#Credentials

Create an API key in the Groq console. It is sent as Authorization: Bearer.

The package reads nothing from the environment: not GROQ_API_KEY, GROQ_BASE_URL, GROQ_LOG, nor GROQ_CUSTOM_HEADERS, which the SDK would otherwise add to every request. Pass the key in yourself, for example from GROQ_API_KEY.

#Use it

import { createModels, createRuntime, defineAgent, defineTool, z } from 'mayura';
import { groq } from '@mayurajs/provider-groq';

const models = createModels({
  providers: [groq({ apiKey: process.env.GROQ_API_KEY ?? '' })],
  prices: 'catalog',         // Groq's on-demand list prices on the catalog's date
  maxCallCostMicros: 20_000, // at most $0.02 per model call
});

const lookup = defineTool({
  id: 'orders.lookup', version: '1', description: 'Look up an order by its id.', effects: 'none', capabilities: [],
  input: z.object({ orderId: z.string() }), output: z.object({ status: z.string() }),
  execute: async ({ orderId }) => ({ status: orderId === 'ord-1' ? 'shipped' : 'unknown' }),
});

const agent = defineAgent({
  id: 'support', version: '1', instructions: 'Use orders.lookup, then answer in one sentence.',
  model: models.model('groq/openai/gpt-oss-120b', { retry: { attempts: 2 } }), tools: [lookup],
  input: z.object({ question: z.string() }), output: z.object({ answer: z.string() }),
});

const runtime = createRuntime({
  profile: 'ephemeral',
  permissions: { allow: ['model:groq/openai/gpt-oss-120b', 'tool:orders.lookup'] },
  limits: { maxCostMicros: 200_000 }, // at most $0.20 per run
});
try {
  const outcome = await runtime.submit(agent, { input: { question: 'Where is order ord-1?' } }).result();
  console.log(outcome.status === 'succeeded' ? outcome.output.answer : outcome.error);
} finally {
  await runtime.close();
}

The adapter's id is the model id, so the run needs model:groq/openai/gpt-oss-120b, and that grant admits no other model. A run without it ends blocked with PERMISSION_DENIED before any request is sent. See Permissions.

#Model ids

A model id is groq/<model>, where <model> is Groq's own model name (up to 128 characters). Groq's names often contain a / themselves, so the id has two: groq/openai/gpt-oss-120b. Everything after the first / is sent to Groq. Any model name works once it has a price; the catalog prices the models below.

#Catalog prices

prices: 'catalog' uses Groq's on-demand list prices as of 2026-09-29, from Groq's models page. In US dollars per million tokens:

Cached input is charged at the full input rate. Models Groq sells only through its sales team (Llama 3.3 70B Versatile and Llama 3.1 8B Instant, on that date) are left out, and Batch and Flex tier prices differ: give those yourself.

Prices change: models.list() shows each model's catalogAsOf, context and output limits, and the catalog is exported as catalog. A price you give, in prices or as pricing on one model, always wins:

const models = createModels({
  providers: [groq({ apiKey: process.env.GROQ_API_KEY ?? '' })],
  prices: { 'groq/openai/gpt-oss-20b': { inputMicrosPerMillionTokens: 75_000, outputMicrosPerMillionTokens: 300_000 } },
  maxCallCostMicros: 10_000,
});

#Options

The registry's timeoutMs (on createModels or on one model, 100 to 3,600,000 ms) wins over the provider's.

#What it sends and supports

• Structured output. A call without tools asks for the agent's output as strict JSON Schema (response_format of type json_schema, strict: true), so choose a model with strict structured outputs. The schemas must follow the strict rules; defineAgent checks them when the agent is defined.
• Tools. Tools are sent as strict functions with an auto tool choice, and parallel tool calls are on. Groq refuses a response format beside tools, so a call with tools offers the tools alone. When the model calls no more tools, the package makes one more request in the same model call, with the format and without the tools, and that answer is the output. Both requests are charged to the call.
• Streaming. With a stream policy on the agent, output text is reported as Groq writes it. With tools, the model's free reply is never streamed, only the answer in the format.
• Reasoning. Reasoning a model returns with a tool call is kept in the run's private state and sent back on the next call. It is never released as output or put in events. That state names the provider and model, so it never moves to another provider or model.
• Narration. Text a model writes before a tool call is not an answer and is dropped.
• Images. Models see nothing unless you say so. For a vision model, pass for example media: { types: ['image/png', 'image/jpeg'], urls: true }: images go inline as data URLs, or as URLs Groq fetches. Images a tool returns follow the tool results in one user message. See Vision.
• Output tokens. max_completion_tokens is the run's limits.maxOutputTokens (4,096 unless you set it).

#Costs

Each request's cost is Groq's reported prompt and completion tokens at your prices, rounded up to a whole micro. Reasoning tokens are part of the completion tokens; cached prompt tokens are charged at the full rate. A model call with tools that ends in an answer makes two requests, and its cost is the sum of both.

maxCallCostMicros (or a model's maxCostMicros) is reserved before every call, and a call that doesn't fit in the run's budget never starts. With tools, choose it to cover two requests of your largest prompt plus maxOutputTokens: on openai/gpt-oss-120b, two requests of 20,000 input tokens and 4,096 output tokens cost at most 2 × (3,000 + 2,458) = 10,916 micros, so 20,000 covers them. A call whose cost is unknown, such as one that timed out after it was sent, is charged its full bound. See Costs and budgets.

#Retries and fallback

The SDK's own retries are off: each request is sent once. Retries belong to the registry:

const model = models.model('groq/openai/gpt-oss-120b', { retry: { attempts: 3, backoffMs: 500 } });

• retry calls the model again after rate_limited, unavailable (including Groq's 498 when a Flex tier model is over capacity) or timeout; never after the caller cancels, a failure that would repeat, or once streamed text was released.
• attempts is 1 to 5, and the model's bound becomes attempts times its bound. Every attempt is charged.
• Waits double from backoffMs (default 500 ms).

To fall back to another model, use a chain. It is a model router granted by the name you choose, and granting it grants every model in it:

const support = models.chain('support', ['groq/openai/gpt-oss-120b', 'groq/openai/gpt-oss-20b']);
// Grant model:support.

#Errors

A failed call ends the run with MODEL_FAILED and a reason (see Errors for the messages):

Groq's error text, the generation it failed to parse, your key and your headers never appear in outcomes or events. Options the package cannot use, such as an empty key, an http:// baseURL, a header that would replace Authorization, or media with a type other than the four image types, throw INVALID_CONFIG when you call groq(). So does a version of the SDK whose environment headers the package cannot remove: it refuses to start rather than send them. A model name over 128 characters throws INVALID_CONFIG when you call models.model().

#How it is tested

• Unit tests run the package through the Groq SDK against a fake Chat Completions API, given through fetch. It passes the model adapter conformance suite from mayura/testing: response shapes, costs, tool calls, streaming, the reason for each failure, no leaked provider text, cancellation, timeouts and strict schemas.
• They also check that nothing comes from the environment (GROQ_CUSTOM_HEADERS included) and the SDK logs nothing, that baseURL and headers are honoured, that 429, 498 and 503 are one request each, the two-request answer with tools and its charge, that the free reply is never streamed, Groq's parse failures, that reasoning is sent back and never released, images, and the catalog through a registry run.
• This package passed live runs against the real API through a Cloudflare AI Gateway on September 30, 2026 with GPT-OSS 20B and 120B: structured output, tools and streaming.
• scripts/live/provider-groq.mjs, a structured answer and a tool call through a registry priced from the catalog, passed directly against Groq on October 12, 2026.

#Related

• Model registry: ids, prices, the per-call bound, retries and chains.
• Model routing: failover, circuit breakers and accounting.
• Ollama: the same gpt-oss models on your own machine.
• Costs and budgets: how a call's cost is reserved and charged.