← Back to all projects

LiteLLM

One OpenAI-shaped interface in front of a hundred model providers — as a library or a gateway

OfficialMIT
Stars
62.4k
Forks
12.9k
Open issues
5.6k
Last commit
30 Oct 2026

What is LiteLLM?

The library half lets one function call reach any supported provider in the OpenAI request and response shape. The gateway half is the reason most teams arrive: a self-hosted server that issues virtual keys with their own budgets and rate limits, falls back when a provider fails, spreads load across deployments, and records cost per key. The price is an extra component in the request path that you now operate, and code under the enterprise directory is not covered by the MIT grant.

What can you do with LiteLLM?

  • Call any provider with one function — A single completion call takes a model name and returns the OpenAI response shape, so switching between OpenAI, Anthropic, Vertex AI, Bedrock and the rest is a string change rather than a rewrite of the call site.
  • Put a gateway in front of everything — The self-hosted proxy exposes an OpenAI-compatible endpoint, which means existing clients, SDKs and tools that already speak OpenAI point at it unchanged and gain routing they never had to know about.
  • Hand out keys with limits attached — Virtual keys carry their own budgets and rate limits per key, per team and per user, so an experiment cannot spend the department's quota and the provider's real credential never leaves the gateway.
  • Keep serving when a provider does not — Fallbacks move a failing request to another deployment automatically, and load can be spread across several deployments of the same model rather than pinned to one.
  • See the spend and filter the traffic — Logging, cost tracking and an admin dashboard cover what went through and what it cost, and guardrails at the same layer can apply content filtering and mask personal data before a request leaves.

Before you choose LiteLLM

  • The MIT grant stops at the enterprise directory, which the licence file places under separate terms — so the open part and the commercial part live in the same repository and the boundary is a path.
  • Adopting the gateway puts a component you operate into the path of every model call, so its availability and its upgrades become your concern on top of the providers' own.

Star history

21 Aug to 30 Oct · +5.5k

56.9k62.4k

Frequently asked questions

Is LiteLLM free for commercial use?

LiteLLM is released under the MIT licence — OSI-approved open source, which permits commercial use.

How can LiteLLM be deployed?

LiteLLM is available as Self-hosted / Runs locally.