← Back to all projects

Firecrawl

Turns whole sites into clean markdown for model consumption

OfficialAGPL-3.0
Stars
197k
Forks
10.4k
Open issues
716
Last commit
30 Oct 2026

What is Firecrawl?

Renders JavaScript, follows links and strips the navigation and boilerplate that otherwise poison a retrieval corpus. Solves the unglamorous half of RAG properly. AGPL-licensed, so check the terms if you plan to expose it as a service.

What can you do with Firecrawl?

  • Turn any URL into markdown — app.scrape('firecrawl.dev') returns LLM-ready data from a page as markdown, JSON or a screenshot; the README's CLI equivalents are firecrawl scrape https://firecrawl.dev and firecrawl https://firecrawl.dev --only-main-content.
  • Crawl a whole site as one job — POST to /v2/crawl with a limit and scrapeOptions.formats returns a job ID you poll for status, creditsUsed and per-page markdown; the SDKs run that polling for you.
  • Search the web and get the pages — app.search("query", limit=5) returns results that already carry each page's markdown, so there is no second fetch step between finding a source and reading it.
  • Describe the data instead of URLs — /v2/agent takes a prompt, finds and navigates the pages itself, and returns sources alongside the result — pass a Pydantic model as schema= for structured output, or model="spark-1-pro" for complex research.
  • Connect it to an MCP client — firecrawl-mcp runs as an MCP server through npx with FIRECRAWL_API_KEY in its env block, and npx -y firecrawl-cli@latest init --all --browser installs Firecrawl as a skill for Claude Code, OpenCode and similar agents.

Before you choose Firecrawl

  • Being considerate to the sites you crawl is the operator's job here. A site owner has asked for a way to recognise and refuse this crawler, writing that at present there is none.Reported in#1169

Star history

15 Aug to 30 Oct · +29.3k

168k197k

Frequently asked questions

Is Firecrawl free for commercial use?

Firecrawl is released under the AGPL-3.0 licence — OSI-approved open source, which permits commercial use.

How can Firecrawl be deployed?

Firecrawl is available as Self-hosted / Managed cloud.