Tracking LLM Latency & Cost: How I Instrumented an AI Agent Pipeline Using OpenTelemetry and SigNoz

Short summary

A practical guide to instrumenting an AI agent pipeline with OpenTelemetry and SigNoz to trace LLM latency, vector DB lookups, and token costs. Covers tracer initialization, wrapping agent logic with nested spans for DB search and LLM calls, analyzing flame graphs, and building a custom token-cost dashboard. Includes gotchas like over-tracing async streams.

• •Instrument AI agent pipelines with OpenTelemetry spans for LLM calls and vector DB lookups
• •Track token consumption and latency per request using semantic span attributes
• •Build custom cost dashboards in SigNoz using ClickHouse queries on span data
• •Key gotcha: avoid over-tracing async token streams to prevent inflated latency metrics

Generated with AI, which can make mistakes.