crawlie
starsforks
How to install crawlie?
spronta/crawlie
$git clone https://github.com/spronta/crawlieFast, free, open-source technical SEO + GEO crawler: built for humans and agents.
People who liked this also viewed
6 picksType
- WayforthThe API runtime for AI agents. One tool call. Any API. No setup. Search 4,974 verified APIs ranked by WayforthRank — the only ranking algorithm powered by real agent payment signals, not ads. Execute via 18 managed services with zero API keMCP Serversrecently
- meta-ads-mcpMeta Ads (Facebook/Instagram) MCP server for Claude, ChatGPT, Perplexity & Cursor — the Meta node of Pipeboard’s 5-platform family (+ Google, TikTok, Snap, Reddit). Hosted remote MCP, badged Meta Business Partner, free plan — no self-hosting required.MCP ServersJul 2026
- SEO Web Analysis MCP — Site Audit & Tech StackSEO audit toolkit: feed it a URL, get back rendered content, the technology stack (CMS, CDN, frameworks), DNS records, SSL cert status, and WHOIS. Useful for competitive analysis ('what's our competitor running?'), security checks, or due-dMCP Serversrecently
- RedditReddit is a social news platform with user-driven communities (subreddits), offering content sharing, discussions, and viral marketing opportunities for brandsMCP Serversrecently
- Keyword Research — Google Suggest, Intent & Long-TailSEO keyword research API for AI agents. Generate keyword ideas from Google Suggest with search intent classification (informational/transactional/navigational), long-tail variations, related queries, and content planning data. Tools: seo_rMCP Serversrecently
- BudgetFitterBudgetFitter is a free UK deal discovery platform with a public MCP server. Search verified discount codes, look up brand intelligence, and navigate deals — no auth required.MCP Serversrecently
