haystack
Document-grounded answers and agents, assembled from components you wire yourself — a Python library, not a server.
What is haystack?
Haystack builds document-grounded answering and agents out of components — file converters, splitters, embedders, retrievers, chat generators, tools — that you register in a Pipeline and connect output-name to input-name, with a mismatch raising an error at connection time rather than partway through a run. Branches and capped loops live in the same graph, and an Agent is itself a component, so a whole tool-calling loop drops into a pipeline or becomes a tool for another agent. The cost is that the wiring is yours: there is no one-call path from a folder of PDFs to a working assistant, and the library runs inside your own process — serving a pipeline over HTTP or as an MCP server means adding Hayhooks, a separate deepset project, or writing that wrapper yourself.
What can you do with haystack?
- Wire components by name, and hear about a mismatch at once — You register each step in a Pipeline and join one component's named output to another's named input; if the declared types cannot meet, the connection itself raises there and then. The same graph branches — PDFs down one path, spreadsheets down another — and loops back, with a per-component run cap that stops a check-and-rewrite cycle spinning forever.
- Build the retrieval side out of parts you can replace — Going in, file converters and splitters feed an embedder that writes into a document store; coming out, a retriever fetches candidates by keyword, by vector or both, and a joiner merges the two rankings, with reciprocal rank fusion among the options. Retrievers come paired with their store, so moving off the in-memory one to Qdrant, Elasticsearch or pgvector changes those two pieces and leaves the rest of the graph alone.
- An agent is itself a component, so agents nest — The Agent runs the whole loop — call the chat model, run the tools it asks for, update a typed state the tools share, stop on an exit condition or a step budget — streaming tokens and running several tool calls at once. Being an ordinary component, it drops into a pipeline, and a wrapper hands one agent to another as a tool.
- Turn a function, a component or an entire pipeline into a tool — A decorator makes a tool out of a Python function and its type hints; wrappers expose an existing component, or a whole pipeline, as one. When the catalogue grows past a threshold, a searchable toolset shows the model a single search call and loads the rest on demand, and a skill toolset does the same for folders of written instructions.
- Step into the agent's loop at six named points — Hooks fire before the run, before the model call, before and after a tool, on exit and after the run. Two are shipped ready-made: a confirmation step that asks a person before a tool executes — always, once per tool and arguments, or never — and one that writes a bulky tool result out to a file instead, leaving a short preview and a pointer in its place, for tools you nominate or once a result passes a size you set.
- Score retrieval and answers before you trust them — Evaluators grade the retrieval half — whether the right documents came back and how near the top — and the answer half, including whether an answer stays faithful to those documents, plus a model-as-judge whose criteria you write. Ragas and DeepEval plug in the same way if you already use them.
Before you choose haystack
- Version 3.0 dropped the legacy generators and moved thirty components into separately installed integration packages; deepset states 2.31 gets only security patches and critical bug fixes until the end of October 2026.
- Pipeline templates, a Helm chart and deployment guides, direct support and early access to security work such as prompt-injection defence are sold as Haystack Enterprise rather than shipped in the open framework.
Star history
17 Aug to 28 Aug · +116
Frequently asked questions
Is haystack free for commercial use?
haystack is released under the Apache-2.0 licence — OSI-approved open source, which permits commercial use.
How can haystack be deployed?
haystack is available as Self-hosted / Runs locally.
Documentation
Reproduced from the deepset-ai/haystack README, published under Apache-2.0. Read the original ↗
| CI/CD |   |
| Docs | |
| Package |    |
| Meta |
🎉🎊✨ Haystack 3.0 is out! ✨🎊🎉
Read the announcement here!
🥳 🎈 🎆 🪅 🎇 🍾 🥂 🎁 🌈
Haystack is an open-source AI orchestration framework for building production-ready LLM applications in Python.
Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Build scalable RAG systems, multimodal applications, semantic search, question answering, and autonomous agents, all in a transparent architecture that lets you experiment, customize deeply, and deploy with confidence.
Table of Contents
- Installation
- Documentation
- Features
- Haystack Enterprise: Support & Platform
- Telemetry
- 🖖 Community
- Contributing to Haystack
- Organizations using Haystack
Installation
The simplest way to get Haystack is via pip:
pip install haystack-aiInstall nightly pre-releases to try the newest features:
pip install --pre haystack-aiHaystack supports multiple installation methods, including Docker images. For a comprehensive guide, please refer to the documentation.
Documentation
If you’re new to the project, check out “What is Haystack?” then go through the “Get Started Guide” and build your first LLM application in a matter of minutes. Keep learning with the tutorials. For more advanced use cases, or just to get some inspiration, you can browse our Haystack recipes in the Cookbook.
At any given point, hit the documentation to learn more about Haystack, what it can do for you, and the technology behind.
Features
Agents built for production
Extend agent behavior with lifecycle hooks (before_llm, before_tool, on_exit, …) for guardrails and custom logic, and track step_count, token_usage, and tool calls out of the box for monitoring and cost control. Get started fast with ready-made agents from Agent Pack (e.g., a deep research agent, or an advanced RAG agent) or give your own agents progressive skill discovery via SkillToolset, so skill descriptions only enter context when needed.
Built for context engineering
Design flexible systems with explicit control over how information is retrieved, ranked, filtered, combined, structured, and routed before it reaches the model. Define pipelines and agent workflows where retrieval, memory, tools, and generation are transparent and traceable.
Native Async Support
One Pipeline runs synchronously or asynchronously and streams token by token. Agent can run concurrent tool calls.
Modular and customizable
Use built-in components for retrieval, indexing, tool calling, memory, and evaluation, or create your own. Add loops, branches, and conditional logic to precisely control how context moves through your pipelines and agent workflows.
Model- and vendor-agnostic
Integrate with OpenAI, Mistral, Anthropic, Cohere, Hugging Face, Google, Azure OpenAI, AWS Bedrock, local models, and many others. Swap models or infrastructure components without rewriting your system.
Extensible ecosystem
Build and share custom components through a consistent interface that makes it easy for the community and third parties to extend Haystack and contribute to an open ecosystem.
[!TIP]
Would you like to deploy and serve Haystack pipelines as REST APIs or MCP servers? Hayhooks provides a simple way for you to wrap pipelines and agents with custom logic and expose them through HTTP endpoints or MCP. It also supports OpenAI-compatible chat completion endpoints and works with chat UIs like open-webui.
Haystack Enterprise: Support & Platform
Get expert support from the Haystack team, build faster with enterprise-grade templates, and scale securely with deployment guides for cloud and on-prem environments with Haystack Enterprise Starter. Read more about it in the announcement post.
👉 Get Haystack Enterprise Starter
Need a managed production setup for Haystack? The Haystack Enterprise Platform helps you build, test, deploy and operate Haystack pipelines with built-in observability, collaboration, governance, and access controls. It’s available as a managed cloud service or as a self-hosted solution.
👉 Learn more about Haystack Enterprise Platform or try it free
Telemetry
Haystack collects anonymous usage statistics of pipeline components. We receive an event every time these components are initialized. This way, we know which components are most relevant to our community.
Read more about telemetry in Haystack or how you can opt out in Haystack docs.
🖖 Community
If you have a feature request or a bug report, feel free to open an issue in GitHub. We regularly check these, so you can expect a quick response. If you’d like to discuss a topic or get more general advice on how to make Haystack work for your project, you can start a thread in Github Discussions or our Discord channel. We also check 𝕏 (Twitter) and Stack Overflow.
Organizations using Haystack
Haystack is used by thousands of teams building production AI systems across industries, including:
- Technology & AI Infrastructure: Apple, Meta, Databricks, NVIDIA, Intel
- Public Sector AI Initiatives: European Commission, German Federal Ministry of Research, Technology, and Space (BMFTR), PD, Baden-Württemberg State
- Enterprise & Industrial AI Applications: Airbus, Lufthansa Industry Solutions, Infineon, LEGO, Comcast, Accenture, TELUS Agriculture & Consumer Goods
- Knowledge & Content Platforms: Netflix, ZEIT Online, Rakuten, Oxford University Press, Manz, YPulse
Are you also using Haystack? Open a PR or tell us your story
Other options in Agent Frameworks
deepseek-harnessFrameworks`dsh` is the agent harness DeepSeek AI develops itself. `npx @deepseek-ai/dsh web` brings up a Web UI on `http://127.0.0.1:3080`, and the architecture behind it is one where everything is a plugin, powered by the Cordis runtime, so it suits people who intend to extend a harness rather than only operate one. The caveat comes from the project itself: it is labelled a developer preview and warns in capitals that there will be compatibility-breaking changes, so plugins written today should budget for rework. It is also a poor fit if you want a documented scripted or embedded entry point right now, since the README covers only the `web` command and a source checkout, and links out to a development guide and architecture documentation without saying what either contains.
AutoGPTWorkflowAn agent here is a graph of blocks — one block calls a service, transforms data, prompts a model or branches — assembled on a canvas rather than written in code, then run on demand, on a schedule or from a trigger. One repository holds both the free self-hosted platform and the code behind the paid hosted one, and that is the catch: everything under the platform directory is PolyForm Shield, which permits use but forbids offering it as a competing product.
DifyWorkflowA complete platform — prompt orchestration, retrieval, agent tools and an admin UI — that non-engineers can operate once a team has set it up. The licence is a modified Apache 2.0 with additional conditions on multi-tenant hosting and branding, so it is source-available rather than open source. Read it before building a product on top.