Description
Firecrawl crawl is an agent skill for bulk extracting content from a website or a section of one. It follows links from a starting page up to a set depth and page limit, and you can include or exclude paths so the crawl stays within a section such as /docs. The result is saved to a file, and you can wait for the job to finish, watch progress, or poll a running job by its ID. A full crawl is limited by a maximum depth, and a page limit keeps it bounded. Crawling needs authentication, because the Firecrawl CLI has no keyless free tier for it. Without credentials, the CLI prompts for an interactive login. The skill is aimed at tasks like extracting everything under a documentation path, or collecting content that spans many linked pages, rather than one page at a time. For a single page, the scrape skill is the better fit. The skill is part of the Firecrawl CLI, which is listed on skills.sh under the Firecrawl CLI repository with about 651 stars and about 80.3K installs. The skills.sh listing shows Gen Agent Trust Hub: Pass, Socket: Pass and Snyk: Warn.…
Highlights
- Bulk extraction from a website or section
- Depth and page limits
- Include and exclude path filters
- Wait for completion or poll a running job
- Requires login for crawling
Ratings & Reviews
No ratings yet
Be the first to rate firecrawl-crawl.
People also visit
firecrawl-crawl FAQ
What does it crawl?
Pages reached by following links from a starting URL, limited by depth, page count and the path filters you set.
Can I limit it to one section?
Yes. Use the include-paths option, for example /docs, to stay within that section.
Does it need an account?
Yes. Crawling requires authentication, and without credentials the CLI prompts an interactive login.
When should I use scrape instead?
For a single known page. The crawl skill is for many linked pages.
How do I install it?
Run npx skills add https://github.com/firecrawl/cli --skill firecrawl-crawl.
What security audits has it passed?
skills.sh lists three audits: Gen Agent Trust Hub Pass, Socket Pass and Snyk Warn. A Warn or Fail is worth reading before you install.
What are the best firecrawl-crawl alternatives?
Popular alternatives to firecrawl-crawl include firecrawl-scrape, firecrawl-agent and firecrawl-map. See all firecrawl-crawl alternatives