Integrations & TraceQL

Traces to Metrics (Metrics Generator)

The Metrics Generator extracts metrics from trace data, providing:

Span Metrics

• Automatic RED metrics per service and operation
• Latency histograms: traces_spanmetrics_latency_bucket
• Request counts: traces_spanmetrics_calls_total
• Dimensions tracked: service.name, span.name, span.kind, status.code

Service Graphs

• Automatic service dependency discovery
• Metrics: traces_service_graph_request_total, traces_service_graph_request_server_seconds
• Visualized in Grafana Node Graph panel

Local Blocks (TraceQL Metrics)

• Enables TraceQL Drilldown view in Grafana
• Real-time metrics from TraceQL queries
• Required for the new Grafana Explore Traces UI

Traces to Logs Integration

Configured in Grafana to link traces with Loki logs:

• Filter logs by TraceID
• Span time shift: ±1 hour
• Tag mappings: service.name → service
• Additional tags: job, instance, pod, namespace

Traces to Profiles Integration

Configured in Grafana to link traces with Pyroscope profiles:

• Map service_name tag to profile data
• Enables performance profiling from trace spans

TraceQL

TraceQL is Tempo’s query language for searching and analyzing traces.

Examples:

// Find all traces from a specific service
{ resource.service.name = "frontend" }

// Find traces with errors
{ status = error }

// Find slow spans (>1s)
{ span.duration > 1s }

// Find traces by attribute
{ span.http.status_code = 500 }

// Aggregate metrics
{ resource.service.name = "frontend" } | rate()

MCP Server

Tempo includes an experimental MCP (Model Context Protocol) server that provides AI assistants and LLMs with direct access to distributed tracing data.

Status: ✅ Configured

What is MCP?

Model Context Protocol is an open standard developed by Anthropic that defines how LLMs and AI assistants (Claude Code, Cursor, etc.) can securely connect to external data sources.

Endpoint

After deployment, the MCP server is available at:

http://tempo-query-frontend.monitoring.svc.cluster.local:3200/api/mcp

Connecting Claude Code

To add Tempo as an MCP server in Claude Code:

# For local port-forward access
kubectl port-forward svc/tempo-query-frontend -n monitoring 3200:3200

# Then register with Claude Code
claude mcp add --transport=http tempo http://localhost:3200/api/mcp

Security Warning

Using this feature will cause tracing data to be passed to an LLM provider. Review your tracing data for sensitive information before enabling in production.

Configuration (Helm values)

queryFrontend:
  mcp_server:
    enabled: true

Location: tempo.values.yaml:230-232