← Back to all projects

Firecrawl
Turns whole sites into clean markdown for model consumption
OfficialAGPL-3.0
What is Firecrawl?
Renders JavaScript, follows links and strips the navigation and boilerplate that otherwise poison a retrieval corpus. Solves the unglamorous half of RAG properly. AGPL-licensed, so check the terms if you plan to expose it as a service.
What can you do with Firecrawl?
- Turn any URL into markdown —
app.scrape('firecrawl.dev')returns LLM-ready data from a page as markdown, JSON or a screenshot; the README's CLI equivalents arefirecrawl scrape https://firecrawl.devandfirecrawl https://firecrawl.dev --only-main-content. - Crawl a whole site as one job — POST to
/v2/crawlwith alimitandscrapeOptions.formatsreturns a job ID you poll forstatus,creditsUsedand per-page markdown; the SDKs run that polling for you. - Search the web and get the pages —
app.search("query", limit=5)returns results that already carry each page's markdown, so there is no second fetch step between finding a source and reading it. - Describe the data instead of URLs —
/v2/agenttakes a prompt, finds and navigates the pages itself, and returnssourcesalongside the result — pass a Pydantic model asschema=for structured output, ormodel="spark-1-pro"for complex research. - Connect it to an MCP client —
firecrawl-mcpruns as an MCP server throughnpxwithFIRECRAWL_API_KEYin its env block, andnpx -y firecrawl-cli@latest init --all --browserinstalls Firecrawl as a skill for Claude Code, OpenCode and similar agents.
Before you choose Firecrawl
- Being considerate to the sites you crawl is the operator's job here. A site owner has asked for a way to recognise and refuse this crawler, writing that at present there is none.Reported in#1169
Star history
15 Aug to 30 Oct · +29.3k
168k197k
Frequently asked questions
Is Firecrawl free for commercial use?
Firecrawl is released under the AGPL-3.0 licence — OSI-approved open source, which permits commercial use.
How can Firecrawl be deployed?
Firecrawl is available as Self-hosted / Managed cloud.