Observability & Evals

ClickHouseObservability

A column-oriented engine for append-heavy, high-cardinality data, which is the shape agent telemetry takes: one row per model call, filtered later by model, cost or error. Langfuse moved its traces here from Postgres in December 2024 and ClickHouse acquired the project in January 2026, so a self-hosted observability stack increasingly means running this underneath. It is the wrong choice for anything transactional — updates and deletes rewrite whole column parts rather than edit rows — so treat it as where agent runs land, not as the database your application writes state to.

LangfuseObservability

Records what an agent actually did — every call, tool invocation and cost — and turns those traces into evaluation datasets. The managed service runs the same codebase you can self-host with Docker Compose, so migrating in either direction stays cheap. Some enterprise features sit outside the MIT core; check which tier you need before committing.

skillsTools & Integrations

Over a hundred skill directories of Google's own written guidance, installed selectively with `npx skills add google/skills`. The bulk sits under `skills/cloud` — GKE, BigQuery, Agent Platform, the six Well-Architected pillars — while `skills/ads` and `skills/analytics` cover the Google Ads API, the Mobile Ads and IMA SDKs and the two Google Analytics APIs. Depth is uneven: `skills/cloud/gke-*` accounts for twenty-nine directories on its own, Cloud Run and Firebase get one each, and the README labels the repository as under active development. A stack on neither Google Cloud nor Google's ads and analytics products gets nothing here, and app-side coverage stops at the Mobile Ads SDK — Android, Flutter, Dart, Genkit and ADK skills are separate repositories the README only links to.

PhoenixObservability

Built on OpenTelemetry, so traces are portable rather than locked to one vendor, and it will run locally next to the code you are debugging. Licensed under Elastic 2.0 — fine for internal use, restrictive if you intend to offer it as a managed service.

agentgatewayGuardrails

A Rust data plane that sits between agents and everything they call — model providers, MCP servers, other agents — so authentication, RBAC, rate limits and OpenTelemetry export are configured once instead of reimplemented in every agent. Solo.io donated it to the Linux Foundation in August 2025 and it moved under the Agentic AI Foundation in June 2026, alongside MCP and goose. It is young for something that sits in the request path: per-user identity onto downstream MCP servers is still an open issue, so confirm the authentication model you need before putting it in front of production traffic.

numbatGuardrails

numbat is a single binary that watches supported desktop, CLI, IDE and gateway agents through hooks, plugins and OTLP/HTTP log exporters, normalizes their activity into one event model, and evaluates it with a local CEL rule engine that writes versioned NDJSON events, findings and enforcement decisions. It also reconstructs past sessions from on-disk artifacts via `numbat scan`, so you can investigate an agent that was never instrumented. Blocking is deliberately conservative: enforce mode is off by default and every shipped rule is monitor-only, so denying an action means copying the rule YAML into your own `--rules-dir`, keeping its id, adding `enforce: true` and bumping the version. It is a poor fit if you expect coverage of arbitrary agents or a definitive account of what ran — the coverage matrix decides what is observable per host and surface, at-rest reconstruction cannot recover activity an agent never persisted, and findings are rule matches rather than proof that an action completed.