
NeMo Guardrails
Programmable rails around an LLM application, checking input, retrieval, dialogue, tool calls and output
What is NeMo Guardrails?
Most guardrail tools check one thing in one place. This defines five interception points instead — before the model sees a request, over what retrieval pulled in, over the shape of the conversation, around tool calls, and before a reply reaches the user — and it sits between the application and the model, so it can be added to a system already in production without changing either side. The costs are latency and vocabulary: several of the stronger rails are model calls of their own, and dialogue rules are written in Colang, a purpose-built language.
What can you do with NeMo Guardrails?
- Check at five points, not one — Input rails screen a request before the model sees it, retrieval rails inspect what a RAG pipeline pulled in, dialog rails govern conversational flow, execution rails wrap tool calls, and output rails screen the reply before it is returned.
- Add it without touching the app or the model — It sits between the application and the LLM, so guardrails can be retrofitted onto a system that is already running rather than requiring the application to be rebuilt around them.
- Deploy it as a library or as a service — A Python SDK embeds it directly, or a FastAPI server exposes an OpenAI-compatible chat completions endpoint so clients in any language can sit behind it; LangChain and LangGraph are supported natively.
- Write the policy declaratively — Colang expresses conversational flows and guardrail logic alongside YAML configuration, which makes a policy a reusable artefact that moves between deployments instead of code embedded in one application.
- Combine built-in and third-party checks — Self-checking with the application's own model and jailbreak detection using heuristics and NVIDIA's guard models sit alongside connectors for LlamaGuard, Presidio, ActiveFence, Cisco AI Defense and several personal-data detectors.
Before you choose NeMo Guardrails
- Several of the stronger rails are themselves model calls that run on every turn, so guarding an application adds latency and per-request cost to the traffic it is protecting.
- Dialogue rules are written in Colang, a purpose-built language: policies expressed in it move cleanly between deployments, but it is one more thing for the team to learn and to maintain.
Star history
21 Aug to 30 Oct · +410
Frequently asked questions
Is NeMo Guardrails free for commercial use?
NeMo Guardrails is released under the Apache-2.0 licence — OSI-approved open source, which permits commercial use.
How can NeMo Guardrails be deployed?
NeMo Guardrails is available as Self-hosted / Runs locally.