llm-observability

Open repos and models their makers tagged llm-observability, most starred first. The same for everyone.

AI on my own machineRepo

langfuse

🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.

  • 38k
  • TypeScript
  • 9 h ago
langfuseTake it
AI on my own machineRepo

opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

  • 23k
  • Java
  • Apache-2.0
  • 20 h ago
comet-mlTake it
AI agentsRepo

bifrost

Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.

  • 9.8k
  • Go
  • Apache-2.0
  • 2 d ago
maximhqTake it
AI agentsRepo

coze-loop

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.

  • 5.8k
  • Go
  • Apache-2.0
  • 5 d ago
coze-devTake it
AI agentsRepo

future-agi

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

  • 2.3k
  • Python
  • Apache-2.0
  • 1 d ago
future-agiTake it
AI agentsRepo

Tracely-ai

Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.

  • 1.1k
  • Python
  • MIT
  • 2 d ago
JwuthriTake it
AI agentsRepo

clawmetry

See your agent think. Zero-config observability & governance for 30 AI agent runtimes: Claude Code, OpenAI Codex, Hermes, OpenClaw & 26 more. Live token costs, sessions, tool calls, crons.

  • 437
  • Python
  • MIT
  • 3 h ago
vivekchandTake it
Developer toolsRepo

loongsuite-pilot

Local-first telemetry collector for AI coding agents — unified OpenTelemetry events for Claude Code, Codex, Cursor and more. Token usage, cost, traces and security audit, exported anywhere.

  • 265
  • TypeScript
  • Apache-2.0
  • 2 d ago
alibabaTake it
AI on my own machineRepo

awesome-ai-tokenomics

A curated list on AI token economics: what tokens cost, where they get wasted, and how to cut the bill. Tools, benchmarks, papers, and copy-paste configs for the token economy of LLMs and coding agents.

  • 221
  • Python
  • CC0-1.0
  • 9 d ago
QuesmaOrgTake it
AI on my own machineRepo

oss-llmops-stack

Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability, prompt versioning, and performance evaluations.

  • 155
  • MIT
  • 4 mo ago
langfuseTake it
Search GitHub and Hugging Face for llm-observability

Get , open it once and sign in. Then come back here and press the button again.