agentgateway
Proxy that puts auth, policy and tracing in front of MCP, A2A and LLM traffic
Overview
A Rust data plane that sits between agents and everything they call — model providers, MCP servers, other agents — so authentication, RBAC, rate limits and OpenTelemetry export are configured once instead of reimplemented in every agent. Solo.io donated it to the Linux Foundation in August 2025 and it moved under the Agentic AI Foundation in June 2026, alongside MCP and goose. It is young for something that sits in the request path: per-user identity onto downstream MCP servers is still an open issue, so confirm the authentication model you need before putting it in front of production traffic.
What can you do with agentgateway?
- Put several MCP servers behind one port — One
mcpbackend takes a list of named targets — the repository's multiplex example launchesmcp-server-timewithuvxandserver-everythingwithnpx— and exposes them together on a single bind. stdio, SSE and streamable HTTP all terminate at the gateway, so the client sees one endpoint. - Authorise individual tool calls with CEL —
mcpAuthorizationrules are CEL expressions over the verified JWT and the call itself:mcp.tool.name == "echo"leaves one tool open to anyone whilejwt.sub == "test-user" && mcp.tool.name == "get-sum"restricts another to a single subject. Tool-level policy is what the MCP specification explicitly declines to enforce itself. - Route by cost rather than by model name — A virtual model dispatches to real backends through conditional rules — the cost-routing example sends requests whose
max_tokensis 1024 or less togpt-4o-miniand larger ones up the tiers — with the last target acting as the required fallback. Budget and spend controls, prompt enrichment and failover sit on the same OpenAI-compatible surface. - Run it standalone or as a Gateway API controller — A single binary reading flat YAML covers local and non-Kubernetes deployments; the built-in controller implements Gateway API plus the Inference Gateway extensions, routing to self-hosted models on GPU utilisation, KV cache state, LoRA adapters and queue depth. Guardrails hook into regex filters, OpenAI moderation, Bedrock Guardrails, Model Armor or a custom webhook.
- Check the identity story before committing — Enterprise SSO with per-user credentials to downstream MCP services, OAuth Identity Assertion for cross-app access and custom OAuth scopes are all still open issues, and another reports that the current versioning scheme breaks automated patch updates. The proxy is production-shaped; the delegated-identity half of it is not finished.
Documentation
Reproduced from the agentgateway/agentgateway README, published under Apache-2.0. Read the original ↗
Agentgateway is an open source proxy built on AI-native protocols (MCP & A2A) that provides drop-in security, observability, and governance for agent-to-LLM, agent-to-tool, and agent-to-agent communication across any framework and environment.
Intro to Agentgateway Video
Key Features
-
LLM Gateway Route traffic to major LLM providers (OpenAI, Anthropic, Gemini, Bedrock, and more) through a unified OpenAI-compatible API with budget and spend controls, prompt enrichment, load balancing, and failover.
-
MCP Gateway Connect LLMs to tools and external data sources via MCP with tool federation, stdio/HTTP/SSE/Streamable HTTP transports, OpenAPI integration, and OAuth authentication.
-
A2A Gateway Enable secure agent-to-agent communication using A2A, with capability discovery, modality negotiation, and task collaboration.
-
Inference Routing Intelligent routing to self-hosted models using Kubernetes Inference Gateway extensions, with decisions based on GPU utilization, KV cache, LoRA adapters, and queue depth.
-
Guardrails Multi-layered content filtering with regex, OpenAI moderation, AWS Bedrock Guardrails, Google Model Armor, and custom webhooks.
-
Security & Observability Auth (JWT, API keys, OAuth), fine-grained RBAC with CEL policy engine, rate limiting, TLS, and OpenTelemetry metrics/logs/tracing.
Getting Started
- Standalone Quickstart — Get started with agentgateway in minutes.
- Kubernetes Quickstart — Deploy on Kubernetes using the built-in controller and Gateway API.
Documentation
Depending on your deployment environment, check out the following docs:
- agentgateway.dev/docs: For deployments without the full Kubernetes controller, driven by flat yaml config.
- agentgateway.dev/docs/kubernetes/latest: For Kubernetes-based deployments using the built-in Kubernetes controller and Gateway API support.
Agentgateway has a built-in UI for you to explore agentgateway connecting agent-to-agent or agent-to-tool:
Community Meetings
To join a community meeting, add the agentgateway calendar to your Google account. Then, you can find event details on the calendar.
Recordings of the community meetings will be published on our google drive.
Roadmap
agentgateway is currently in active development. If you’d like a feature that’s missing, open an issue in our GitHub repo.