WebScraper Toolkit
Complete web scraping suite: Chrome extension, backend with REST API, headless scraper, MCP server for AI agents, and dashboard with sitemap visualization. Captures DOM, styles, assets, network requests and metadata from any site.
Interactive Demo
Interact with a live version of the product. No screenshots — real code.
Features
16 features included
Core Features
- Full page capture
DOM, computed styles, images, scripts and stylesheets
- Multi-page crawling
BFS engine with configurable depth, page limit and inter-request delay
- URL filtering
Glob/regex patterns to include or exclude URLs during crawl
- Pagination engine
Auto-detect next buttons, load-more, numbered pagination and infinite scroll
- Network capture
XHR/fetch interception with request body capture (10KB)
Stealth & Anti-detection
- Stealth mode
User-Agent rotation, viewport jitter, proxy support and randomized delays
Automation
- Scheduled scraping
Recurring jobs with cron expressions via REST API
Security
- Auth profiles
Cookies and headers encrypted with AES-256-GCM per domain
Monitoring
- Change detection
SHA-256 hashing, line-by-line diffs, snapshot history
- RSS/Atom feeds
Automatic feed discovery and item polling
Integrations
- YouTube integration
Channel video listing, details and description link extraction
- MCP server
11 tools for AI agents (Claude, etc.) via MCP protocol
- REST API
21 endpoints with Bearer token auth, OpenAPI 3.0 spec
Data Intelligence
- Contact extraction
Emails, phones, social profiles, physical addresses and contact forms
- SEO analysis
Per-page scoring and site-wide aggregation with detailed metrics
Export
- Multi-format export
.wst.json (AI-optimized), CSV, PDF (executive summary)