Rating
Votes
0
score
Downloads
0
total
Price
Free
No login needed
Works With
About
Website | GitHub | Docs | PyPi
mcp-server-webcrawl
Advanced search and retrieval for web crawler data. With mcp-server-webcrawl, your AI client filters and analyzes web content under your direction or autonomously. The server includes a fulltext search interface with boolean support, and resource filtering by type, HTTP status, and more.
mcp-server-webcrawl provides the LLM a complete menu with which to search, and works with a variety of web crawlers:
| Crawler/Format | Description | Platforms | Setup Guide |
|---|---|---|---|
| [ArchiveBox][1] | Web archiving tool | macOS/Linux | [Setup Guide][8] |
| [HTTrack][2] | GUI mirroring tool | macOS/Windows/Linux | [Setup Guide][9] |
| [InterroBot][3] | GUI crawler and analyzer | macOS/Windows/Linux | [Setup Guide][10] |
| [Katana][4] | CLI security-focused crawler | macOS/Windows/Linux | [Setup Guide][11] |
| [SiteOne][5] | GUI crawler and analyzer | macOS/Windows/Linux | [Setup Guide][12] |
| [WARC][6] | Standard web archive format | varies by client | [Setup Guide][13] |
| [wget][7] | CLI website mirroring tool | macOS/Linux | [Setup Guide][14] |
[1]: https://archivebox.io [2]: https://github.com/xroche/httrack [3]: https://interro.bot [4]: https://github.com/projectdiscovery/katana [5]: https://crawler.siteone.io [6]: https://en.wikipedia.org/wiki/WARC_(file_format) [7]: https://en.wikipedia.org/wiki/Wget [8]: https://pragmar.github.io/mcp-server-webcr
Works with Claude (desktop and mobile), Cursor, Windsurf, VS Code, and any MCP-compatible AI app.
Category: Developer Tools
Don't lose this
Three weeks from now, you'll want pragmar/mcp-server-webcrawl again. Will you remember where to find it?
Save it to your library and the next time you need pragmar/mcp-server-webcrawl, it’s one tap away — from any AI app you use. Group it into a bench with the rest of the team for that kind of task and you can pull the whole stack at once.
⚡ Pro tip for geeks: add a-gnt 🤵🏻♂️ as a custom connector in Claude or a custom GPT in ChatGPT — one click and your library is right there in the chat. Or, if you’re in an editor, install the a-gnt MCP server and say “use my [bench name]” in Claude Code, Cursor, VS Code, or Windsurf.
a-gnt's Take
Our honest review
This plugs directly into your AI and gives it new abilities it didn't have before. pragmar/mcp-server-webcrawl. Once connected, just ask your AI to use it. It's completely free and works across most major AI apps. This one just landed in the catalog — worth trying while it's fresh.
Tips for getting started
Tap "Get" above, pick your AI app, and follow the steps. Most installs take under 30 seconds.
What's New
Imported from awesome:punkpeye/awesome-mcp-servers
Ratings & Reviews
0.0
out of 5
0 ratings
No reviews yet. Be the first to share your experience.