seo-firecrawl
installsforks
How to install seo-firecrawl?
agricidaniel/claude-seo/seo-firecrawl
$npx -y skills add AgriciDaniel/claude-seo --skill seo-firecrawlFull-site crawling, scraping, and site mapping via Firecrawl MCP. Use when user says "crawl site", "map site", "full crawl", "find all pages", "broken links", "site structure", "discover pages", "JS rendering", or needs site-wide analysis.
SKILL.md
| 1 | # Firecrawl Extension for Claude SEO |
| 2 | |
| 3 | This skill requires the Firecrawl extension to be installed: |
| 4 | ```bash |
| 5 | ./extensions/firecrawl/install.sh |
| 6 | ``` |
| 7 | |
| 8 | **Check availability:** Before using any Firecrawl tool, verify the MCP server |
| 9 | is connected by checking if `firecrawl_scrape` or any Firecrawl tool |
| 10 | is available. If tools are not available, inform the user the extension is not |
| 11 | installed and provide install instructions. |
| 12 | |
| 13 | ## Quick Reference |
| 14 | |
| 15 | | Command | Purpose | |
| 16 | |---------|---------| |
| 17 | | `/seo firecrawl crawl <url>` | Full-site crawl with content extraction | |
| 18 | | `/seo firecrawl map <url>` | Discover site structure (URLs only, fast) | |
| 19 | | `/seo firecrawl scrape <url>` | Single-page scrape with JS rendering | |
| 20 | | `/seo firecrawl search <query> <url>` | Search within a crawled site | |
| 21 | |
| 22 | ## Commands |
| 23 | |
| 24 | ### crawl -- Full-Site Crawl |
| 25 | |
| 26 | Crawl an entire website starting from the given URL. Returns page content, |
| 27 | metadata, and links for all discovered pages. |
| 28 | |
| 29 | **MCP Tool:** `firecrawl_crawl` |
| 30 | |
| 31 | **Parameters:** |
| 32 | - `url` (required): Starting URL to crawl |
| 33 | - `limit`: Max pages to crawl (default: 100, max: 500) |
| 34 | - `maxDepth`: Max link depth from start URL (default: 3) |
| 35 | - `includePaths`: Array of glob patterns to include (e.g., `["/blog/*"]`) |
| 36 | - `excludePaths`: Array of glob patterns to exclude (e.g., `["/admin/*", "/api/*"]`) |
| 37 | - `scrapeOptions.formats`: Output formats -- `["markdown", "html", "links"]` |
| 38 | |
| 39 | **SEO Usage Patterns:** |
| 40 | 1. **Comprehensive audit crawl**: Crawl full site, extract all pages for subagent analysis |
| 41 | 2. **Section-focused crawl**: Use `includePaths` to audit only `/blog/*` or `/products/*` |
| 42 | 3. **Broken link detection**: Crawl with `["links"]` format, check all hrefs for 404s |
| 43 | 4. **Content inventory**: Extract all page titles, meta descriptions, H1s at scale |
| 44 | 5. **SPA/JS-rendered sites**: Firecrawl renders JavaScript, solving the Issue #11 problem |
| 45 | |
| 46 | **Example orchestration for `/seo audit`:** |
| 47 | ``` |
| 48 | 1. firecrawl_map(url) -> get all URLs (fast, no content) |
| 49 | 2. Filter to top 50 most important pages (homepage, key sections) |
| 50 | 3. firecrawl_crawl(url, limit=50) -> get full content |
| 51 | 4. Feed content to seo-technical, seo-content, seo-schema agents |
| 52 | ``` |
| 53 | |
| 54 | **Cost awareness:** |
| 55 | - Free tier: 500 credits/month |
| 56 | - 1 credit = 1 page crawled or scraped |
| 57 | - Map operations are cheaper (0.5 credits per URL discovered) |
| 58 | - Always inform user of estimated credit usage before large crawls |
| 59 | |
| 60 | ### map -- Site Structure Discovery |
| 61 | |
| 62 | Discover all URLs on a website without fetching content. Fast and credit-efficient. |
| 63 | |
| 64 | **MCP Tool:** `firecrawl_map` |
| 65 | |
| 66 | **Parameters:** |
| 67 | - `url` (required): Website URL to map |
| 68 | - `limit`: Max URLs to discover (default: 5000) |
| 69 | - `search`: Optional search term to filter URLs |
| 70 | |
| 71 | **SEO Usage Patterns:** |
| 72 | 1. **Sitemap comparison**: Map site, compare discovered URLs vs XML sitemap |
| 73 | 2. **Orphan page detection**: URLs in sitemap but not linked from any page |
| 74 | 3. **Crawl budget analysis**: Total indexable pages vs pages linked from homepage |
| 75 | 4. **URL pattern analysis**: Identify URL structure patterns, duplicates, parameter bloat |
| 76 | 5. **Pre-audit discovery**: Run map first, then targeted crawl on key sections |
| 77 | |
| 78 | **Output:** Array of URLs. Present as: |
| 79 | ``` |
| 80 | Site: example.com |
| 81 | Pages discovered: 342 |
| 82 | |
| 83 | URL Pattern Breakdown: |
| 84 | /blog/* - 128 pages (37%) |
| 85 | /products/* - 89 pages (26%) |
| 86 | /category/* - 45 pages (13%) |
| 87 | /pages/* - 32 pages (9%) |
| 88 | / (root pages) - 48 pages (14%) |
| 89 | ``` |
| 90 | |
| 91 | ### scrape -- Single-Page Deep Scrape |
| 92 | |
| 93 | Scrape a single page with full JavaScript rendering. More thorough than |
| 94 | `fetch_page.py` because it executes JS and waits for dynamic content. |
| 95 | |
| 96 | **MCP Tool:** `firecrawl_scrape` |
| 97 | |
| 98 | **Parameters:** |
| 99 | - `url` (required): Page URL to scrape |
| 100 | - `formats`: Output formats -- `["markdown", "html", "links", "screenshot"]` |
| 101 | - `onlyMainContent`: Strip nav/footer/sidebar (default: true) |
| 102 | - `waitFor`: CSS selector or milliseconds to wait for content |
| 103 | - `timeout`: Request timeout in ms (default: 30000) |
| 104 | - `actions`: Browser actions before scraping (click, scroll, wait) |
| 105 | |
| 106 | **SEO Usage Patterns:** |
| 107 | 1. **SPA content extraction**: Scrape JS-rendered React/Vue/Angular pages |
| 108 | 2. **Dynamic content audit**: Pages with lazy-loaded content below the fold |
| 109 | 3. **Paywall/login detection**: Identify content behind authentication walls |
| 110 | 4. **Main content extraction**: Use `onlyMainContent` for clean E-E-A-T analysis |
| 111 | 5. **Screenshot capture**: Use `screenshot` format for visual analysis |
| 112 | |
| 113 | **When to use scrape vs fetch_page.py:** |
| 114 | | Scenario | Use | |
| 115 | |----------|-----| |
| 116 | | Static HTML page | `fetch_page.py` (no API cost) | |
| 117 | | JS-rende |
People who liked this also viewed
6 picksType
- seo-auditWhen the user wants to audit, review, or diagnose SEO issues on their site. Also use when the user mentions "SEO audit," "technical SEO," "why am I not ranking," "SEO issues," "on-page SEO," "meta tagSkillsJul 2026
- copywritingWhen the user wants to write, rewrite, or improve marketing copy for any page — including homepage, landing pages, pricing pages, feature pages, about pages, or product pages. Also use when the user sSkillsJul 2026
- marketing-psychologyWhen the user wants to apply psychological principles, mental models, or behavioral science to marketing. Also use when the user mentions 'psychology,' 'mental models,' 'cognitive bias,' 'persuasion,'SkillsJul 2026
- content-strategyWhen the user wants to plan a content strategy, decide what content to create, or figure out what topics to cover. Also use when the user mentions "content strategy," "what should I write about," "conSkillsJul 2026
- programmatic-seoWhen the user wants to create SEO-driven pages at scale using templates and data. Also use when the user mentions "programmatic SEO," "template pages," "pages at scale," "directory pages," "location pSkillsJul 2026
- marketing-ideasWhen the user needs marketing ideas, inspiration, or strategies for their SaaS or software product. Also use when the user asks for 'marketing ideas,' 'growth ideas,' 'how to market,' 'marketing stratSkillsJul 2026
