last9-genai
OpenTelemetry SDK for Python that tracks LLM cost, conversations, and agent workflows — with one-call setup for OpenAI and AutoGen apps.
Features
install() — one call wires TracerProvider, LoggerProvider, processors, and OpenAI instrumentation
- Conversation tracking —
gen_ai.conversation.id across multi-turn sessions
- Workflow cost aggregation — group LLM calls by workflow, track total spend
- Agent identity —
gen_ai.agent.* attributes per OTel GenAI semantic conventions
- Cost calculation — automatic for 20+ models; bring-your-own pricing for the rest
- Log-to-span bridge — promotes
opentelemetry-instrumentation-openai-v2 log events onto spans so the Last9 LLM dashboard renders prompts, completions, and tool calls
@observe decorator — manual span creation with tags, metadata, and category
- Full OTel GenAI v1.28.0 compliance
Quick Start
pip install last9-genai opentelemetry-exporter-otlp-proto-grpc
from last9_genai import install
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
handle = install()
handle.tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
That's it. All providers, processors, and OpenAI instrumentation are wired automatically.
Set these environment variables before running:
OTEL_SERVICE_NAME=my-llm-app
OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp.last9.io
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <your-token>"
Python 3.14 + openai-v2: pin wrapt<2. A kwarg rename in wrapt 2.0 breaks opentelemetry-instrumentation-openai-v2 instrumentation silently.
Conversation & Workflow Tracking
from last9_genai import install, conversation_context, workflow_context
handle = install()
# ... wire OTLP exporter ...
with conversation_context(conversation_id="thread-123", user_id="user-456"):
with workflow_context(workflow_id="support-flow", workflow_type="chat"):
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}]
)
# Span has gen_ai.conversation.id, workflow.id, user.id automatically
Contexts nest — conversation_context wraps multiple workflow_context calls, all spans in scope get tagged.
Agent Identity
from last9_genai import agent_context
with conversation_context(conversation_id="thread-1"):
with agent_context(agent_name="support-bot", agent_id="bot-001"):
# All spans: gen_ai.agent.name, gen_ai.agent.id
response = client.chat.completions.create(...)
agent_context composes with conversation_context and workflow_context. Use it for multi-agent handoffs — each agent sets its own identity on its spans.
Manual Instrumentation
When install() isn't enough — bring your own providers:
from opentelemetry import trace, _logs
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk._logs import LoggerProvider
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
from last9_genai import Last9SpanProcessor, Last9LogToSpanProcessor
import os
os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "true"
log_bridge = Last9LogToSpanProcessor()
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(Last9SpanProcessor(log_processor=log_bridge))
trace.set_tracer_provider(tracer_provider)
logger_provider = LoggerProvider()
logger_provider.add_log_record_processor(log_bridge)
_logs.set_logger_provider(logger_provider)
OpenAIInstrumentor().instrument(logger_provider=logger_provider)
@observe Decorator
from last9_genai import observe
from openai import OpenAI
client = OpenAI()
@observe(
tags=["production"],
metadata={"category": "customer_support"}
)
def handle_query(query: str):
return client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": query}]
)
category appears in the Last9 LLM dashboard Category column and filter dropdown. Use underscores for multi-word categories (data_analysis → "data analysis").
Configuration
Environment variables
| Variable |
Description |
Default |
OTEL_SERVICE_NAME |
Service name in traces |
unknown-service |
OTEL_EXPORTER_OTLP_ENDPOINT |
OTLP endpoint URL |
required |
OTEL_EXPORTER_OTLP_HEADERS |
Auth headers (key=value) |
required |
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT |
Capture prompt/completion bodies |
false |
OTEL_RESOURCE_ATTRIBUTES |
Additional resource attributes |
— |
install() kwargs
| Kwarg |
Description |
Default |
capture_content |
Sets OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true |
True |
instrument_openai |
Call OpenAIInstrumentor().instrument(logger_provider=...) |
True |
set_global |
Register providers as OTel globals |
True |
tracer_provider |
Provide an existing TracerProvider |
None |
logger_provider |
Provide an existing LoggerProvider |
None |
**span_processor_kwargs |
Forwarded to Last9SpanProcessor (e.g. custom_pricing) |
— |
install() returns an InstallHandle with .tracer_provider, .logger_provider, and .shutdown().
Cost Tracking
from last9_genai import install, ModelPricing
handle = install(
capture_content=True,
custom_pricing={
"gpt-4o": ModelPricing(input=2.50, output=10.0),
"gpt-4o-mini": ModelPricing(input=0.15, output=0.60),
"claude-sonnet-4-5": ModelPricing(input=3.0, output=15.0),
}
)
Prices are USD per million tokens. Conversion: per-token $0.000003 → 3.0; per-1K $0.003 → 3.0.
Common models
| Provider |
Model |
Input $/1M |
Output $/1M |
| Anthropic |
claude-opus-4-6 |
15.0 |
75.0 |
| Anthropic |
claude-sonnet-4-5 |
3.0 |
15.0 |
| Anthropic |
claude-haiku-4-5 |
0.8 |
4.0 |
| OpenAI |
gpt-4o |
2.50 |
10.0 |
| OpenAI |
gpt-4o-mini |
0.15 |
0.60 |
| OpenAI |
o1 |
15.0 |
60.0 |
| Google |
gemini-1.5-pro |
1.25 |
10.0 |
| Google |
gemini-2.0-flash |
0.075 |
0.30 |
For current pricing: Anthropic · OpenAI · Google · llm-prices.com
Use "azure/gpt-4o" for Azure, "ollama/llama3.1" with input=0.0, output=0.0 for self-hosted.
Architecture
your app
└── install()
├── TracerProvider
│ └── Last9SpanProcessor ← enriches spans with cost, conversation,
│ │ workflow, agent attrs
│ └── (your OTLP exporter)
└── LoggerProvider
└── Last9LogToSpanProcessor ← bridges openai-v2 log events
onto the active span
How Last9LogToSpanProcessor works: opentelemetry-instrumentation-openai-v2 emits prompt/completion content as OTel log records (new GenAI semconv), not span attributes. The bridge listens to those log records and writes gen_ai.prompt, gen_ai.completion, span events, and indexed gen_ai.prompt.{i}.* / gen_ai.completion.{i}.* onto the active span — so the Last9 LLM dashboard can render them.
Troubleshooting
gen_ai.prompt / gen_ai.completion missing on spans
Two likely causes:
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT is not true. install(capture_content=True) sets this automatically.
OpenAIInstrumentor().instrument() was called without logger_provider=. The bridge only receives events if openai-v2 routes logs to the same LoggerProvider. install() handles this automatically.
No traces appearing in Last9
install() does not add an exporter — you must wire one:
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
handle.tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
OTLPSpanExporter reads OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS at instantiation time.
Python 3.14 + wrapt error
TypeError: wrap_function_wrapper() got an unexpected keyword argument 'module'
Pin wrapt<2 — wrapt 2.0 renamed the kwarg and opentelemetry-instrumentation-openai-v2 2.3b0 hasn't caught up yet.
Tool call spans missing message content
execute_tool span content capture (tool arguments and results) is Phase 2 work — not yet implemented. Tracked in the project issues.
Span Attributes Reference
| Attribute |
Description |
gen_ai.conversation.id |
Thread / session identifier |
gen_ai.prompt |
JSON array of prompt messages |
gen_ai.completion |
JSON array of completion choices |
gen_ai.prompt.{i}.role / .content |
Indexed prompt messages |
gen_ai.completion.{i}.role / .content |
Indexed completion choices |
workflow.id |
Workflow identifier |
workflow.type |
Workflow type |
user.id |
User identifier |
gen_ai.agent.id |
Agent identifier |
gen_ai.agent.name |
Agent name |
gen_ai.usage.cost |
Computed cost in USD |
gen_ai.l9.span.kind |
llm / tool / prompt |
Examples
See examples/ directory:
Contributing
- Fork the repository
- Create a feature branch
- Run tests:
uv run pytest
- Submit a pull request
License
MIT
Support