Choosing an agent framework: five questions that eliminate options

Start with elimination, not ranking. Eight catalogue entries could host your agent loop, and a feature table will not choose between them: ticks where projects document a thing, blanks where nobody does, no column for what hurts in month three.

The five whose primary category is frameworks: LangChain (MIT, 144,288 stars), deepseek-harness (MIT, 116,370), LangGraph (MIT, 39,751), the OpenAI Agents SDK (MIT, 28,662) and Pydantic AI (MIT, 19,318). Filed elsewhere but competing for the same slot: CrewAI (MIT, 57,126), AutoGen (60,436) and LlamaIndex (MIT, 51,662). Five questions cut that list down, each removing candidates rather than ordering them.

1. Does a run have to survive a crash or a human?

The process dies at minute forty of a fifty-minute run. Or step seven needs human approval before step eight writes to production. Does the framework resume, or start over?

Most start over. LangGraph is built around not doing that: durable execution persists state as the graph advances, so an agent that dies mid-run resumes from exactly where it left off, and interrupts stop execution at a chosen point so a person can inspect and modify the agent state. The verbosity everyone complains about is the price of that machinery.

Pydantic AI arrives from the other direction. TemporalDurability() turns every model and tool call into a Temporal activity, so a run on a background queue survives restarts, failures and long waits; DBOS and Prefect attach the same way, and human-in-the-loop tool approval is documented rather than invented by you. The durability lives in an engine you operate, not in the library.

Conversation persistence is different: the OpenAI Agents SDK’s Sessions manages history across runs, resuming a conversation rather than an in-flight run.

Which I would pick: LangGraph if you do not already run Temporal, because one system beats two; Pydantic AI if your platform team operates Temporal, DBOS or Prefect already, because durability then sits on their surface. And if a failed run is cheap to repeat and your tools are idempotent, skip this axis: unneeded durability is the largest single source of unnecessary structure in agent codebases.

2. Is the hard part the reasoning or the corpus?

When the system answers wrongly, is that because the model reasoned badly, or because it never saw the right document?

If it is the second, your framework choice barely matters and retrieval design is the whole problem. LlamaIndex starts from the data rather than the agent loop: ingestion, indexing strategies, and the retrieval patterns that decide whether answers are grounded. Letting it own retrieval while another framework owns control flow is my default.

Budget for the corpus itself, where these projects actually fail. Firecrawl (AGPL-3.0, 167,800 stars) renders JavaScript and strips the navigation and boilerplate that poison a corpus. Crawl4AI (Apache-2.0, 78,234 stars) is the permissively licensed alternative, which matters if you will expose the result as a service — read the AGPL terms before building on Firecrawl in that shape.

3. Which language does the service that will own this agent already speak?

Under-weighted, and the most expensive to get wrong: the fix is a deployment change, not a refactor.

Python only: CrewAI and Pydantic AI. The OpenAI Agents SDK is listed as Python too; its README points JS/TS users at a separate repository, openai-agents-js — a second codebase with its own release cadence, not a binding.

Python and TypeScript from one design: LangGraph, via LangGraph.js and the @langchain/langgraph npm package; LangChain, via LangChain.js and the langchain package; LlamaIndex, via llamaindex. AutoGen pairs Python with C#, reachable from .NET.

TypeScript only: deepseek-harness. It is the one candidate a Node service could adopt with no Python anywhere, and it still misses the shortlist below, for a reason unrelated to language. Its README documents a harness you run — npx @deepseek-ai/dsh web, serving a Web UI on http://127.0.0.1:3080, plus a source checkout — and no scripted or embedded entry point; the development guide and architecture documentation are linked without the README saying what either contains. Embedding an agent in an existing TypeScript service is what that surface does not cover.

The position: if a Node or TypeScript service has to hold the agent inside it, the shortlist is LangGraph.js, LangChain.js and LlamaIndex’s TS package. A Python-only library means a sidecar — a second deployment, a second on-call surface, a second dependency graph — an infrastructure decision wearing a library decision’s clothes. deepseek-harness is the exception worth naming: if you want a plugin host to extend rather than a library to call, it is TypeScript-native and no sidecar arises. Read question 4 first.

4. Will the project still want you in a year?

AutoGen carries a maintenance-mode banner in its own README: no new features, community managed, new users sent to Microsoft Agent Framework, contributions limited to bug fixes, security patches and documentation. GitHub also reports its licence as CC-BY-4.0, a content licence rather than an OSI-approved software one. Of the catalogue’s 45 entries, 41 are OSI-approved; AutoGen is one of the four that are not.

deepseek-harness is the opposite failure of the same signal: enormous attention, and a project that calls itself a developer preview and warns in capitals about compatibility-breaking changes. That is accurate self-description, not a flaw. Budget rework for any plugin written today.

Letta (Apache-2.0, 24,259 stars) is a third shape: those stars belong to the legacy server behind the V1 API, while active development moved to letta-ai/letta-code. Read each README’s status banner before its feature list; all three declare themselves in the first screen.

5. Will you have to explain a wrong answer to someone?

Debuggability rarely appears on a feature list and is a real criterion. When a customer, an auditor or your PM asks why the agent did that, you have a trace or an apology.

Where each puts it decides things. LangGraph’s README hands debugging to LangSmith, a separate LangChain product. The OpenAI Agents SDK counts tracing among its primitives. Pydantic AI is OpenTelemetry-native, so any OTel backend works. CrewAI’s own README places tracing and observability inside the commercial AMP Suite.

Prefer OTel-native instrumentation: it makes the trace store replaceable rather than a second vendor commitment. Langfuse (MIT, 33,155 stars) runs the same codebase managed or self-hosted, so migrating either direction stays cheap — check which features sit outside the MIT core. Phoenix (Elastic-2.0, 11,065 stars) is OpenTelemetry-based and runs locally beside the code you debug: fine internally, restrictive if you resell it as a service.

What not to decide on

Star counts. deepseek-harness has 116,370 to LangGraph’s 39,751, nearly three times as many, and is a self-declared developer preview. Stars measure attention drawn, not whether you can operate something next year. This site publishes them as a fact about a repository, not a ranking — how we curate says where they come from.

Benchmark leaderboards. An agent benchmark scores a harness, a model and a prompt together. Swap the model and the ordering moves. The framework is the smallest contribution, and the number came from someone else’s prompt on someone else’s task.

Integration counts. LlamaHub lists more connectors than anyone will read, and LangChain’s coverage is explicitly its selling point. You will use three to five, and the question is whether those three are maintained. MCP drained the metric further: AutoGen’s McpWorkbench and the MCP capability in Pydantic AI’s core reach any MCP server, and published servers are listed on the MCP Registry. The reference MCP servers repository (MIT) is not where that breadth lives — its entry here notes it holds only the seven servers the steering group maintains, called educational examples rather than production-ready. An integration is now a server you point at.

Three situations, and one aside

An internal workflow that touches production and needs human sign-off. LangGraph. Accept the verbosity; the interrupt and the checkpoint are what you are buying. Against its siblings, LangChain vs LangGraph and LangGraph vs OpenAI Agents SDK are the pages that matter.

One agent inside an existing, fully typed Python service. Pydantic AI. Malformed output fails where you can see it, and if durability becomes a requirement later you attach an engine instead of rewriting.

Question answering over your own documents. LlamaIndex for retrieval, whatever the owning service uses for the loop, and more calendar on ingestion than on the agent.

And if someone asked for multi-agent. Measure one well-prompted agent first. The CrewAI entry says it plainly: whether several role-playing agents beat one well-prompted agent is worth measuring, and the answer is often no. If you still want a crew, AutoGen vs CrewAI is the page; the conclusion is CrewAI or neither, since AutoGen’s README sends new projects elsewhere.