200 apps like Webcrawler API, closest match first.
Gaffa
An API that makes scraping and automating data extraction
pragmar/mcp-server-webcrawl
Search and retrieve web crawler data from WARC, wget, Katana
Scrapeless
A platform with web scraping toolkit to extracts data
MinerU-HTML
An SLM-powered HTML main content extractor
NBot
Track and summarize topic-based web content
firecrawl-download
Download entire websites as organized local files in multiple formats
agent-browser
Browser automation CLI for AI agents
pure.md
Global cache between LLMs and the web
tavily-best-practices
Web search API for LLMs with real-time data access, content extraction
Tavily AI
Tavily is the real‑time search engine for AI agents
SocialKit
Scrape any social media platform with one API
Greenflare
Open Source SEO Web Crawler
Sriram-PR/doc-scraper
Crawl documentation sites into local Markdown or JSONL corpora
mcp-duckgo
Skills for web search and content scraping via DuckDuckGo MCP Server
FreeCrawl SEO Tool
A high-performance website crawler built for serious SEO audits
Scrapper
Web scraper with a simple REST API living in Docker
Sosse
Selenium Open Source Search Engine & crawler
Forage AI
Your AI-powered data extraction and automation partner
clone-website
Reverse-engineer and clone one or more websites in one shot
Piloterr
Web scraping API handles headless browsers, rotates proxies for you
Deep Dive Duck
Monitor Website Changes for Competitor Insights
Agenty
Web Scraping with AI
Sitemap AI
Crawl websites and generate XML sitemaps, llms.txt & markdown
web-crawler
Web scraping plus social data: YouTube, TikTok, Instagram (incl
tavily-cli
Set up, authenticate, update, troubleshoot
SiteSnapper
Archive webpages with high-quality screenshots, metadata, and full text content
Ask
Get precise answers, refine search results
modelcontextprotocol/server-fetch
Web content fetching and processing for AI consumption
Marreta
Tool for analyzing, cleaning, and tidying up web URLs
AICrawlable
Free llms.txt generator for any website
web-scraping
Expert in web scraping and data extraction with Python tools
brave/llm-context
Return pre-extracted web content (text, tables, code) from Brave Search
Retio-ai/pagemap
Compress web page HTML into compact structured maps
ShowUpInAI
Get Cited in ChatGPT and more!
ScrapingBot
The API you need for efficient scraping!
tavily-dynamic-search
Programmatic Tavily search with context isolation
lfnovo/content-core
Extract content from web pages, PDFs, Word documents, YouTube transcripts
mini-browser
Browser automation skill for AI agents using the mb CLI
playwright-scraper
Playwright web scraping: dynamic content, auth flows, pagination
Locus
Find information on web pages, refer to sections in long documents
Dataminer
Easy to use web scraping
Apify
Thousands of tools to automate your business
profullstack/mcp-server
Developer utilities: SEO optimization, document conversion, domain lookup
Capture
Automate the Web, One Pixel at a Time
Content Snare
Automate content gathering, pre-approve answers
ProxyShare
Provide intelligent web crawling and IP proxy services
ScrapX
Monitor your competitors
ChatWebpage
Ask questions, request info, instruct actions, engage deeply
Browserbase
Give your agents access to the whole web
UScraper
Design no-code web scraping visually