crw
starsforks
How to install crw?
us/crw
$git clone https://github.com/us/crw---
README.md
| 1 | <p align="center"> |
| 2 | <img src="docs/logo-animation.gif" alt="fastCRW" width="220" /> |
| 3 | </p> |
| 4 | |
| 5 | <h1 align="center">fastCRW</h1> |
| 6 | |
| 7 | <p align="center"> |
| 8 | The web data API for AI agents — search, scrape, map, and crawl any site into |
| 9 | clean <strong>markdown</strong> or <strong>JSON</strong>. |
| 10 | </p> |
| 11 | |
| 12 | <p align="center"> |
| 13 | Use the managed cloud for zero infra, or self-host the same open-source engine. |
| 14 | Start with Python, TypeScript, cURL, or MCP — you never touch Rust. |
| 15 | </p> |
| 16 | |
| 17 | <p align="center"> |
| 18 | <b>Beats Firecrawl and Crawl4AI on truth-recall — measured on Firecrawl's own public dataset.</b> |
| 19 | </p> |
| 20 | |
| 21 | <p align="center"> |
| 22 | <a href="https://fastcrw.com/register"><img src="https://img.shields.io/badge/Start%20free-500%20credits%2C%20no%20card-7c3aed?style=for-the-badge" alt="Start free"></a> |
| 23 | </p> |
| 24 | <p align="center"> |
| 25 | <a href="https://docs.fastcrw.com/quickstart/">Quickstart</a> · |
| 26 | <a href="https://docs.fastcrw.com">Docs</a> · |
| 27 | <a href="https://fastcrw.com/pricing">Pricing</a> |
| 28 | </p> |
| 29 | |
| 30 | <p align="center"> |
| 31 | <a href="https://crates.io/crates/crw-server"><img src="https://img.shields.io/crates/v/crw-server.svg" alt="crates.io"></a> |
| 32 | <a href="https://pypi.org/project/crw/"><img src="https://img.shields.io/pypi/v/crw.svg?label=pypi" alt="PyPI"></a> |
| 33 | <a href="https://www.npmjs.com/package/crw-mcp"><img src="https://img.shields.io/npm/v/crw-mcp.svg?label=npm%20mcp" alt="npm crw-mcp"></a> |
| 34 | <a href="https://github.com/us/crw/actions/workflows/ci.yml"><img src="https://github.com/us/crw/actions/workflows/ci.yml/badge.svg?branch=main&event=push" alt="CI"></a> |
| 35 | <a href="https://www.bestpractices.dev/projects/13533"><img src="https://www.bestpractices.dev/projects/13533/badge" alt="OpenSSF Best Practices"></a> |
| 36 | <a href="https://scorecard.dev/viewer/?uri=github.com/us/crw"><img src="https://api.scorecard.dev/projects/github.com/us/crw/badge" alt="OpenSSF Scorecard"></a> |
| 37 | <a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue.svg" alt="License"></a> |
| 38 | <a href="https://github.com/us/crw/stargazers"><img src="https://img.shields.io/github/stars/us/crw?style=social" alt="GitHub Stars"></a> |
| 39 | </p> |
| 40 | |
| 41 | <p align="center"> |
| 42 | Works with |
| 43 | <a href="https://docs.fastcrw.com/mcp-clients/#claude-code">Claude Code</a> · |
| 44 | <a href="https://docs.fastcrw.com/mcp-clients/#cursor">Cursor</a> · |
| 45 | <a href="https://docs.fastcrw.com/mcp-clients/#windsurf">Windsurf</a> · |
| 46 | <a href="https://docs.fastcrw.com/mcp-clients/#cline">Cline</a> · |
| 47 | <a href="https://docs.fastcrw.com/mcp-clients/#openai-codex-cli">Codex</a> · |
| 48 | <a href="https://docs.fastcrw.com/mcp-clients/#gemini-cli">Gemini CLI</a> |
| 49 | </p> |
| 50 | |
| 51 | --- |
| 52 | |
| 53 | ## Quickstart — your first scrape in 30 seconds |
| 54 | |
| 55 | **[Get a free API key → fastcrw.com/register](https://fastcrw.com/register)** — 500 free credits (1 credit ≈ 1 page), no card. |
| 56 | |
| 57 | ```bash |
| 58 | export CRW_API_KEY="crw_live_..." |
| 59 | ``` |
| 60 | |
| 61 | ```bash |
| 62 | # cURL — works anywhere, no SDK |
| 63 | curl -X POST https://api.fastcrw.com/v1/scrape \ |
| 64 | -H "Authorization: Bearer $CRW_API_KEY" -H "Content-Type: application/json" \ |
| 65 | -d '{"url":"https://example.com","formats":["markdown"]}' |
| 66 | ``` |
| 67 | |
| 68 | <table> |
| 69 | <tr><th>Python</th><th>Node.js</th></tr> |
| 70 | <tr valign="top"><td> |
| 71 | |
| 72 | ```bash |
| 73 | pip install crw |
| 74 | ``` |
| 75 | ```python |
| 76 | from crw import CrwClient |
| 77 | |
| 78 | crw = CrwClient() # reads CRW_API_KEY from your env |
| 79 | page = crw.scrape("https://example.com", |
| 80 | formats=["markdown"]) |
| 81 | print(page["markdown"]) |
| 82 | ``` |
| 83 | |
| 84 | </td><td> |
| 85 | |
| 86 | ```bash |
| 87 | npm install crw-sdk |
| 88 | ``` |
| 89 | ```javascript |
| 90 | import { CrwClient } from "crw-sdk"; |
| 91 | |
| 92 | const crw = new CrwClient(); // reads CRW_API_KEY from your env |
| 93 | const page = await crw.scrape("https://example.com", |
| 94 | { formats: ["markdown"] }); |
| 95 | console.log(page.markdown); |
| 96 | ``` |
| 97 | |
| 98 | </td></tr> |
| 99 | </table> |
| 100 | |
| 101 | In both SDKs `page` is a plain object (`markdown`, `metadata`, `contentType`, …), so `page["markdown"]` / `page.markdown` is clean content: |
| 102 | |
| 103 | ```markdown |
| 104 | # Example Domain |
| 105 | |
| 106 | This domain is for use in documentation examples without needing permission. Avoid use in operations. |
| 107 | |
| 108 | [Learn more](https://iana.org/ |
People who liked this also viewed
6 picksType
- ddgsA metasearch library that aggregates results from diverse web search servicesMCP ServersMay 2026
- firecrawl-mcp-server🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.MCP ServersJul 2026
- fliGoogle Flights MCP, CLI and Python LibraryMCP ServersJul 2026
- sembleFast and Accurate Code Search for Agents. Uses ~98% fewer tokens than grep+readMCP ServersJul 2026
- duckduckgo-mcp-serverA Model Context Protocol (MCP) server that provides web search capabilities through DuckDuckGo, with additional features for content fetching and parsing.MCP ServersJul 2026
- wikipedia-mcpA Model Context Protocol (MCP) server that retrieves information from Wikipedia to provide context to LLMs.MCP ServersMay 2026
