Configuration & Deployment
Configuration Files
| File | Purpose |
|---|---|
| tempo.values.yaml | Main Helm values for Tempo Distributed |
| deploy-tempo-module.sh | Deployment script |
| alloy.values.yaml | Alloy trace forwarding config (lines 205-217) |
| kp-stack.values.yaml | Grafana with Tempo datasource |
Key Configuration Highlights
Storage (Azure Blob)
storage:
trace:
backend: azure
azure:
container_name: tempo-traces
storage_account_name: lokistorepato846
block:
version: vParquet4 # Columnar format for fast queries
Location: tempo.values.yaml:13-36
Metrics Generator
metricsGenerator:
enabled: true
config:
processor:
local_blocks: {} # TraceQL metrics
span_metrics: {} # RED metrics
service_graphs: {} # Dependency graphs
registry:
external_labels:
source: tempo
cluster: tempo-distributed
storage:
remote_write:
- url: http://mimir-nginx.monitoring.svc.cluster.local:80/api/v1/push
Location: tempo.values.yaml:118-153
TraceQL Features
query_frontend:
search:
query_backend_after: 15m
metrics:
max_duration: 0s # No limit
Location: tempo.values.yaml:86-99
Alloy to Tempo Connection
otelcol.exporter.otlp "tempo" {
client {
endpoint = "http://tempo-distributor.monitoring.svc.cluster.local:4317"
tls { insecure = true }
}
}
Location: alloy.values.yaml:213-217
Grafana Integration
Datasource URL
- HTTP:
http://tempo-query-frontend.monitoring.svc.cluster.local:3200 - gRPC:
tempo-query-frontend.monitoring.svc.cluster.local:9095
Enabled Features
- Node Graph (service dependencies)
- Search (global trace search)
- Traces to Logs (Loki)
- Traces to Metrics (Mimir)
- Traces to Profiles (Pyroscope)
- TraceQL Drilldown view
Deployment
# Deploy Tempo
./run_deploy_scripts/deploy-tempo-module.sh
# Or manually:
helm upgrade --install tempo grafana/tempo-distributed \
--namespace monitoring \
-f helm_values/tempo/tempo.values.yaml \
--version 1.48.1
Required Secret:
kubectl create secret generic tempo-traces-stg-key \
--namespace monitoring \
--from-literal=tempo-traces-key="$STORAGE_KEY"
Retention & Compaction
- Block retention: 24 hours
- Compacted block retention: 1 hour
- Compactor runs automatically to merge and clean up blocks
Location: tempo.values.yaml:101-115
Resources
| Component | CPU Request | CPU Limit | Memory Request | Memory Limit |
|---|---|---|---|---|
| Distributor | 100m | 500m | 256Mi | 512Mi |
| Ingester | 200m | 1000m | 512Mi | 1Gi |
| Querier | 100m | 500m | 256Mi | 512Mi |
| Query Frontend | 100m | 500m | 256Mi | 512Mi |
| Gateway | 50m | 200m | 128Mi | 256Mi |