← Back to all projects

Portkey AI Gateway

One endpoint in front of every provider, with the retries, fallbacks and cost controls agents make necessary

MIT
Stars
13.3k
Forks
1.4k
Open issues
323
Last commit
25 May 2026

What is Portkey AI Gateway?

Agents call models far more often than a chat application does, which turns a provider's occasional rate limit into a daily outage. Portkey's gateway sits between your code and the providers and absorbs that: a failed call retries, a rate-limited key gives way to another, an unavailable model falls back to a second choice, and repeated prompts can be served from a cache instead of being paid for twice. Routing rules can send different requests to different models — cheap for classification, expensive for the hard step — without that decision spreading through your code. The project states it adds under a millisecond of latency in a footprint of about 122KB. One thing to check before planning around a feature: the documentation describes the open gateway and the company's hosted platform together.

What can you do with Portkey AI Gateway?

  • Change provider without changing code — One OpenAI-shaped endpoint stands in front of a very large number of models, so switching is a configuration entry rather than a client rewrite.
  • Survive a provider having a bad hour — Automatic retries, fallback to a second model and load balancing across several keys keep a run alive through rate limits and outages.
  • Stop paying twice for the same prompt — A cache serves repeated requests, which matters most in agent loops where the same context is sent again on every turn.
  • Send each request to the model it deserves — Conditional routing picks the destination from the request itself, so a cheap model handles the routine steps and an expensive one handles the rest.
  • Check requests and responses against rules — Guardrails run on the way in and the way out, so a policy about what may be sent or returned is enforced in one place rather than in every service.
  • Put MCP servers behind the same door — Remote MCP servers can be reached through the gateway, which gives tool access the same authentication and logging as model calls.

Before you choose Portkey AI Gateway

  • The documentation covers the open gateway and Portkey's hosted platform together without marking which is which, so confirm a feature ships in the build you run before designing around it.
  • The maintainers describe a 2.0 pre-release that merges their enterprise gateway into the open one, which means the open build and the product they document are converging but are not yet the same thing.

Star history

27 Aug to 30 Oct · +455

12.8k13.3k

Frequently asked questions

Is Portkey AI Gateway free for commercial use?

Portkey AI Gateway is released under the MIT licence — OSI-approved open source, which permits commercial use.

How can Portkey AI Gateway be deployed?

Portkey AI Gateway is available as Self-hosted / Runs locally / Managed cloud.

Portkey AI Gateway
Ask AI
GitHub