bybesoeasy· 40 skills
Scrape websites at scale using Scrapy, a Python web crawling and scraping framework. Use when: (1) Crawling multiple pages or entire sites, (2) Extracting structured data from HTML/XML, or (3) Building automated data pipelines from web sources.
$npx -y skills add besoeasy/open-skills --skill crawl-websites-at-scaleInstalls into the current project.
Run `npx skills use "https://github.com/besoeasy/open-skills" --skill "besoeasy/open-skills/crawl-websites-at-scale"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.
Use the skills in "https://github.com/besoeasy/open-skills" that are relevant to the current task. Run `npx skills add "https://github.com/besoeasy/open-skills"` and select the relevant skills, then follow their instructions.