Datadog APM and LLM Observability

Datadog is a monitoring platform; its APM shows distributed traces and its LLM Observability shows agent, LLM and tool spans with token usage. @mayurajs/observability-datadog presets Mayura's OTLP trace exporter (mayura/exporter-otlp) for Datadog's OTLP intake on your Datadog site, for APM or for LLM Observability. No Datadog Agent is needed. It sends OTLP/HTTP JSON with fetch: no Datadog SDK, no dependencies besides mayura, and it runs on every runtime Mayura runs on.

#Install

npm install mayura @mayurajs/observability-datadog

#Credentials

The intake authenticates with a Datadog API key (not an application key). Create one in Datadog under Organization Settings, API Keys, on the site your account lives on. The package reads nothing from the environment: pass the key in yourself, for example from DD_API_KEY, with the site from DD_SITE and the LLM Observability application name from DD_LLMOBS_ML_APP.

#Use it

Follow a run with an observer, then turn its events into spans with agentRunTraceSpans and give them to the exporter's sink:

import { createObserver } from 'mayura/observability';
import { agentRunTraceSpans } from 'mayura/exporter-otlp';
import { datadogTraceExporter } from '@mayurajs/observability-datadog';

const datadog = datadogTraceExporter({
  apiKey: process.env.DD_API_KEY ?? '',
  site: 'datadoghq.eu',
  llmObservability: { mlApp: 'support-bot' },
  serviceName: 'support-agent',
});
// Keep enough events per run to pair every call's start and completion.
const observer = createObserver({ maxRecentEventsPerRun: 256 });

const handle = runtime.submit(agent, { input: { question: 'Where is order 42?' } });
const observation = observer.observe(handle);
await handle.result();
await observation.done();

const spans = agentRunTraceSpans(observer.inspect(handle.id)?.recent ?? []);
if (spans.length > 0) await datadog.sink(spans, { signal: AbortSignal.timeout(10_000) });
console.log(datadog.inspect().metrics.recordsAccepted);

Leave out llmObservability to send the same spans to APM instead. The exporter is not a tool, so a run needs no permission string for it: it runs in your code after the run settles.

For durable workflows, give a second exporter's sink to createWorkflowTraceExport; each export needs its own instance, because an exporter sends one request at a time. LLM Observability keeps only spans with GenAI attributes, and workflow and step spans have none, so send workflow traces to APM:

import { createWorkflowTraceExport } from 'mayura/workflows';
import { datadogTraceExporter } from '@mayurajs/observability-datadog';

const workflowTraces = createWorkflowTraceExport({
  source: host.runtime,
  store,
  scope,
  exportId: 'datadog-apm',
  definitions,
  sink: datadogTraceExporter({ apiKey, site: 'datadoghq.eu', serviceName: 'orders' }).sink,
});

See Trace durable workflow runs for the worker unit and nesting an agent's spans under the step that ran it.

#Options

#Sites and authentication

The endpoint is https://otlp.<site>/v1/traces, where site is one of:

Every request carries the key as dd-api-key. With llmObservability, requests also carry dd-otlp-source: llmobs, and dd-ml-app: <mlApp> when you give one.

#What is sent

Each span is metadata from the run's events: names, ids, timing, statuses and integers from a fixed attribute catalog:

• agent:<agent id> for the run, with gen_ai.operation.name: invoke_agent, gen_ai.agent.id and gen_ai.agent.name.
• model.call for each model call, with gen_ai.provider.name, gen_ai.request.model, gen_ai.usage.input_tokens and gen_ai.usage.output_tokens (when the provider reports them), and the call's mayura.cost.micros.
• tool:<tool id> for each tool call, with gen_ai.tool.name and gen_ai.tool.call.id.

In APM, a run is a trace with its model and tool calls. In LLM Observability, Datadog reads the GenAI attributes and shows agent, LLM and tool spans with models and token usage. No prompt, message, tool input, tool output or error text is sent: the catalog has no field that could carry them.

The package sends traces only. Datadog's OTLP metric intake takes delta sums, and Mayura's metric exporter sends cumulative ones.

#Delivery and limits

• One sink call is one HTTP request of at most maxBatchSize spans. Split a larger run into several calls.
• The exporter sends one request at a time; a second call while one is in flight is refused with CONFLICT.
• It never retries and follows no redirects. A failed batch is dropped and counted.
• inspect() returns { state, inFlight, metrics }. The metrics count batchesAttempted, recordsAttempted, recordsAccepted, recordsDropped, partialResponses, failedRequests, timedOutRequests, cancelledRequests, requestBytes and responseBytes. Headers and the key are never shown.
• close() aborts the request in flight and refuses later calls.

#With your OpenTelemetry SDK

If your application already runs an OpenTelemetry SDK that exports to Datadog, you can send Mayura's spans through it with createOpenTelemetryTraceBridge instead of this package. The spans carry the same GenAI attributes either way.

#Errors

Failures reach you as a MayuraError with a code. None of them repeats your key or Datadog's reply text.

A reply that reports some spans rejected (OTLP partial success) resolves normally and counts them in recordsDropped and partialResponses.

#How it is tested

• Unit tests run the exporter against a fake fetch that checks the site's endpoint, the dd-api-key header, the LLM Observability headers with and without an ML app, and that the body carries the GenAI attributes.
• They also check refused keys, sites, ML app names and LLM Observability on ddog-gov.com, and a rejected request, and that no error or inspect() result contains the key.
• scripts/live/observability-datadog.mjs sends one agent run's spans to APM and to LLM Observability: Datadog accepted both on the ap1.datadoghq.com site on October 12, 2026.

#Related

• Observability: observers, the OTLP exporters and trace projections.
• Sentry tracing and Better Stack tracing: other monitoring backends.
• Durable workflows: runs whose traces createWorkflowTraceExport sends.