OpenTelemetry for GenAI Observability Trial

Overview

OpenTelemetry for GenAI observability applies vendor-neutral telemetry to model calls, agent steps, tool invocations, token usage, exceptions, latency, and system behavior. The OpenTelemetry GenAI semantic conventions define signals for GenAI inputs and outputs as events, exceptions, operation metrics, model-operation spans, and agent-operation spans (OpenTelemetry GenAI semantic conventions).

This matters because AI incidents rarely live only inside the LLM layer. Teams need to correlate prompts, retrieval, tool calls, downstream services, infrastructure metrics, and application errors in the same observability workflow.

This entry is rated Trial. OpenTelemetry is the right portability layer for connecting AI traces to existing observability pipelines, and specialized LLMOps tools already build on it, but the GenAI conventions themselves are not stable yet: every GenAI convention is still marked Development, and in June 2026 (semantic conventions v1.42.0) the gen_ai.*, OpenAI, and MCP definitions were deprecated in the core repository and moved to a dedicated repository that has no tagged release yet (OpenTelemetry semantic conventions v1.42.0, OpenTelemetry GenAI semantic conventions).

Adoption Signals

• OpenTelemetry has dedicated GenAI semantic conventions for events, exceptions, metrics, inference token metrics, model spans, and agent spans (OpenTelemetry GenAI semantic conventions).
• The conventions include technology-specific guidance for Anthropic, Azure AI Inference, AWS Bedrock, OpenAI, and Model Context Protocol, which reflects broad provider coverage (OpenTelemetry GenAI semantic conventions).
• The conventions are under active development: releases in 2026 added retrieval spans, cache-token and reasoning-token attributes, tool-definition capture, and streaming attributes and metrics, and the GenAI work moved to its own repository in June 2026 (OpenTelemetry semantic conventions releases, OpenTelemetry semantic conventions v1.42.0).
• LangSmith supports native tracing for popular agent frameworks and OpenTelemetry, and can send LangSmith trace data to existing tools or ingest OTel data into LangSmith (LangSmith).
• Langfuse states that its tracing is based on OpenTelemetry to increase compatibility and reduce vendor lock-in, and supports traces across LLM calls, retrieval, embedding, API calls, sessions, and agent graphs (Langfuse).
• Production dashboards increasingly track token usage, latency, error rates, cost breakdowns, feedback scores, and online evals as first-class AI operations metrics (LangSmith, Langfuse).

Risks

The GenAI conventions are still marked Development and now live in a separate repository without tagged releases, so teams should expect naming, attribute, and stability changes and should pin the convention version their instrumentation emits (OpenTelemetry GenAI semantic conventions, OpenTelemetry semantic conventions v1.42.0).

Prompt and response telemetry can contain sensitive data. Instrumentation must include redaction, sampling, retention policies, access controls, and clear rules for whether full content, hashes, metadata, or externalized payload references are stored.

Telemetry cost can grow quickly. Agent traces can include deeply nested runs, large payloads, many tool calls, and repeated model invocations, so teams should set sampling and retention policies early (LangSmith).

Observability does not replace evaluation. Traces show what happened; teams still need quality checks, safety checks, and regression suites to decide whether behavior is acceptable.

Recommendation

Trial OTel-compatible instrumentation on production AI services and agents that already run on an OpenTelemetry pipeline, with the convention version pinned and a budget for attribute renames when the GenAI conventions change. Capture model spans, tool spans, retrieval spans, token metrics, latency, errors, cache events, and structured correlation IDs, while redacting sensitive content by default.

Use LLMOps tools for prompt/version debugging, online evals, and trace review, but keep the telemetry foundation portable. Prefer platforms that can export to or ingest from OpenTelemetry so AI operations do not become a disconnected dashboard. Move from Trial to Adopt once the GenAI conventions reach a stable release and your collectors, backends, and SDK instrumentations support it.

Sources

• OpenTelemetry: Semantic conventions for generative AI systems
• OpenTelemetry GenAI semantic conventions repository
• OpenTelemetry semantic conventions v1.42.0 release notes
• OpenTelemetry semantic conventions releases
• LangSmith Observability
• Langfuse Documentation