| 1 | <div align="center"> |
| 2 | <img src="https://raw.githubusercontent.com/jae-jae/fetcher-mcp/refs/heads/main/icon.svg" width="100" height="100" alt="Fetcher MCP Icon" /> |
| 3 | </div> |
| 4 | |
| 5 | [中文](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=zh) | |
| 6 | [Deutsch](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=de) | |
| 7 | [Español](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=es) | |
| 8 | [français](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=fr) | |
| 9 | [日本語](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=ja) | |
| 10 | [한국어](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=ko) | |
| 11 | [Português](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=pt) | |
| 12 | [Русский](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=ru) |
| 13 | |
| 14 | # Fetcher MCP |
| 15 | |
| 16 | MCP server for fetch web page content using Playwright headless browser. |
| 17 | |
| 18 | > 🌟 **Recommended**: [OllaMan](https://ollaman.com/) - Powerful Ollama AI Model Manager. |
| 19 | |
| 20 | ## Advantages |
| 21 | |
| 22 | - **JavaScript Support**: Unlike traditional web scrapers, Fetcher MCP uses Playwright to execute JavaScript, making it capable of handling dynamic web content and modern web applications. |
| 23 | |
| 24 | - **Intelligent Content Extraction**: Built-in Readability algorithm automatically extracts the main content from web pages, removing ads, navigation, and other non-essential elements. |
| 25 | |
| 26 | - **Flexible Output Format**: Supports both HTML and Markdown output formats, making it easy to integrate with various downstream applications. |
| 27 | |
| 28 | - **Parallel Processing**: The `fetch_urls` tool enables concurrent fetching of multiple URLs, significantly improving efficiency for batch operations. |
| 29 | |
| 30 | - **Resource Optimization**: Automatically blocks unnecessary resources (images, stylesheets, fonts, media) to reduce bandwidth usage and improve performance. |
| 31 | |
| 32 | - **Robust Error Handling**: Comprehensive error handling and logging ensure reliable operation even when dealing with problematic web pages. |
| 33 | |
| 34 | - **Configurable Parameters**: Fine-grained control over timeouts, content extraction, and output formatting to suit different use cases. |
| 35 | |
| 36 | ## Quick Start |
| 37 | |
| 38 | Run directly with npx: |
| 39 | |
| 40 | ```bash |
| 41 | npx -y fetcher-mcp |
| 42 | ``` |
| 43 | |
| 44 | First time setup - install the required browser by running the following command in your terminal: |
| 45 | |
| 46 | ```bash |
| 47 | npx playwright install chromium |
| 48 | ``` |
| 49 | |
| 50 | ### HTTP and SSE Transport |
| 51 | |
| 52 | Use the `--transport=http` parameter to start both Streamable HTTP endpoint and SSE endpoint services simultaneously: |
| 53 | |
| 54 | ```bash |
| 55 | npx -y fetcher-mcp --log --transport=http --host=0.0.0.0 --port=3000 |
| 56 | ``` |
| 57 | |
| 58 | After startup, the server provides the following endpoints: |
| 59 | |
| 60 | - `/mcp` - Streamable HTTP endpoint (modern MCP protocol) |
| 61 | - `/sse` - SSE endpoint (legacy MCP protocol) |
| 62 | |
| 63 | Clients can choose which method to connect based on their needs. |
| 64 | |
| 65 | ### Debug Mode |
| 66 | |
| 67 | Run with the `--debug` option to show the browser window for debugging: |
| 68 | |
| 69 | ```bash |
| 70 | npx -y fetcher-mcp --debug |
| 71 | ``` |
| 72 | |
| 73 | ## Configuration MCP |
| 74 | |
| 75 | Configure this MCP server in Claude Desktop: |
| 76 | |
| 77 | On MacOS: `~/Library/Application Support/Claude/claude_desktop_config.json` |
| 78 | |
| 79 | On Windows: `%APPDATA%/Claude/claude_desktop_config.json` |
| 80 | |
| 81 | ```json |
| 82 | { |
| 83 | "mcpServers": { |
| 84 | "fetcher": { |
| 85 | "command": "npx", |
| 86 | "args": ["-y", "fetcher-mcp"] |
| 87 | } |
| 88 | } |
| 89 | } |
| 90 | ``` |
| 91 | |
| 92 | ## Docker Deployment |
| 93 | |
| 94 | ### Running with Docker |
| 95 | |
| 96 | ```bash |
| 97 | docker run -p 3000:3000 ghcr.io/jae-jae/fetcher-mcp:latest |
| 98 | ``` |
| 99 | |
| 100 | ### Deploying with Docker Compose |
| 101 | |
| 102 | Create a `docker-compose.yml` file: |
| 103 | |
| 104 | ```yaml |
| 105 | version: "3.8" |
| 106 | |
| 107 | services: |
| 108 | fetcher-mcp: |
| 109 | image: ghcr.io/jae-jae/fetcher-mcp:latest |
| 110 | container_name: fetcher-mcp |
| 111 | restart: unless-stopped |
| 112 | ports: |
| 113 | - "3000:3000" |
| 114 | environment: |
| 115 | - NODE_ENV=production |
| 116 | # Using host network mode on Linux hosts can improve browser access efficiency |
| 117 | # network_mode: "host" |
| 118 | volumes: |
| 119 | # For Playwright, may need to share certain system paths |
| 120 | - /tmp:/tmp |
| 121 | # Health check |
| 122 | healthcheck: |
| 123 | test: ["CMD", "wget", "--spider", "-q", "http://localhost:3000"] |
| 124 | interval: 30s |
| 125 | timeout: 10s |
| 126 | retries: 3 |
| 127 | ``` |
| 128 | |
| 129 | Then run: |
| 130 | |
| 131 | ```bash |
| 132 | docker-compose up -d |
| 133 | ``` |
| 134 | |
| 135 | ## F |