baoyu-url-to-markdown

Name: baoyu-url-to-markdown
Author: jimliu

byjimliu· 68 skills

Installs

28k

Stars

24k

Forks

2.7k

How to install baoyu-url-to-markdown?

jimliu/baoyu-skills/baoyu-url-to-markdown

$npx -y skills add jimliu/baoyu-skills --skill baoyu-url-to-markdown

Installs into the current project.

›Prefer a prompt? Paste this to your agent

Use this skill

Run `npx skills use "https://github.com/jimliu/baoyu-skills" --skill "jimliu/baoyu-skills/baoyu-url-to-markdown"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.

Use the whole pack

Use the skills in "https://github.com/jimliu/baoyu-skills" that are relevant to the current task. Run `npx skills add "https://github.com/jimliu/baoyu-skills"` and select the relevant skills, then follow their instructions.

Files · 1

View on GitHub

SKILL.md

1	# URL to Markdown
2
3	Fetches any URL via `baoyu-fetch` CLI (Chrome CDP + site-specific adapters) and converts it to clean markdown.
4
5	## User Input Tools
6
7	When this skill prompts the user, follow this tool-selection rule (priority order):
8
9	1. Prefer built-in user-input tools exposed by the current agent runtime — e.g., `AskUserQuestion`, `request_user_input`, `clarify`, `ask_user`, or any equivalent.
10	2. Fallback: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.
11	3. Batching: if the tool supports multiple questions per call, combine all applicable questions into a single call; if only single-question, ask them one at a time in priority order.
12
13	Concrete `AskUserQuestion` references below are examples — substitute the local equivalent in other runtimes.
14
15	## CLI Setup
16
17	Important: The CLI source is vendored in `{baseDir}/scripts/lib`. `scripts/package.json` installs only third-party runtime dependencies.
18
19	Agent Execution Instructions:
20	1. Determine this SKILL.md file's directory path as `{baseDir}`
21	2. Resolve `${BUN}` runtime: if `bun` installed → `bun`; else suggest installing Bun
22	3. If `{baseDir}/scripts/node_modules` does not exist, run `${BUN} install --cwd {baseDir}/scripts`
23	4. `${READER}` = `{baseDir}/scripts/baoyu-fetch`
24	5. Replace all `${READER}` in this document with the resolved value
25
26	## Preferences (EXTEND.md)
27
28	Check EXTEND.md in priority order — the first one found wins:
29
30	\| Priority \| Path \| Scope \|
31	\|----------\|------\|-------\|
32	\| 1 \| `.baoyu-skills/baoyu-url-to-markdown/EXTEND.md` \| Project \|
33	\| 2 \| `${XDG_CONFIG_HOME:-$HOME/.config}/baoyu-skills/baoyu-url-to-markdown/EXTEND.md` \| XDG \|
34	\| 3 \| `$HOME/.baoyu-skills/baoyu-url-to-markdown/EXTEND.md` \| User home \|
35
36	\| Result \| Action \|
37	\|--------\|--------\|
38	\| Found \| Read, parse, apply settings \|
39	\| Not found \| MUST run first-time setup (see below) — do NOT silently create defaults \|
40
41	EXTEND.md supports: download media by default, default output directory.
42
43	### First-Time Setup ⛔ BLOCKING
44
45	When EXTEND.md is not found, you MUST use `AskUserQuestion` to gather preferences before creating EXTEND.md. NEVER create EXTEND.md with silent defaults. Generation is BLOCKED until setup completes. Batch all three questions into a single call:
46
47	- Q1 — Media (header "Media"): "How to handle images and videos in pages?"
48	- "Ask each time (Recommended)" — Prompt after each save
49	- "Always download" — Download to local `imgs/` and `videos/`
50	- "Never download" — Keep remote URLs
51	- Q2 — Output (header "Output"): "Default output directory?"
52	- "url-to-markdown (Recommended)" — Save to `./url-to-markdown/{domain}/{slug}.md`
53	- User may pick "Other" and type a custom path
54	- Q3 — Save (header "Save"): "Where to save preferences?"
55	- "User (Recommended)" — `~/.baoyu-skills/` (all projects)
56	- "Project" — `.baoyu-skills/` (this project only)
57
58	After answers, write EXTEND.md, confirm "Preferences saved to [path]", then continue.
59
60	Full template: [references/config/first-time-setup.md](references/config/first-time-setup.md).
61
62	### Supported Keys
63
64	\| Key \| Default \| Values \| Description \|
65	\|-----\|---------\|--------\|-------------\|
66	\| `download_media` \| `ask` \| `ask` / `1` / `0` \| `ask` = prompt each time, `1` = always, `0` = never \|
67	\| `default_output_dir` \| empty \| path or empty \| Default output directory (empty = `./url-to-markdown/`) \|
68
69	EXTEND.md → CLI mapping:
70
71	\| EXTEND.md key \| CLI argument \| Notes \|
72	\|---------------\|-------------\|-------\|
73	\| `download_media: 1` \| `--download-media` \| Requires `--output` to be set \|
74	\| `default_output_dir: ./posts/` \| Agent constructs `--output ./posts/{domain}/{slug}.md` \| Agent generates path, not a direct flag \|
75
76	Value priority: CLI arguments → EXTEND.md → skill defaults.
77
78	## Usage
79
80	```bash
81	# Default: headless capture, markdown to stdout
82	${READER} <url>
83
84	# Save to file
85	${READER} <url> --output article.md
86
87	# Save with media download
88	${READER} <url> --output article.md --download-media
89
90	# Wait for interaction (login/CAPTCHA) — auto-detect and continue
91	${READER} <url> --wait-for interaction --output article.md
92
93	# Wait for interaction — manual control (Enter to continue)
94	${READER} <url> --wait-for force --output article.md
95
96	# JSON output
97	${READER} <url> --format json --output article.json
98
99	# Force specific adapter
100	${READER} <url> --adapter youtube --output transcript.md
101	```
102
103	## Options
104
105	\| Option \| Description \|
106	\|--------\|-------------\|
107	\| `<url>` \| URL to fetch \|
108	\| `--output <path>` \| Output file path (default: stdout) \|
109	\| `--format <type>` \| Output format: `markdown` (default) or `json` \|
110	\| `--json` \| Shorthand for `--format json` \|
111	\| `--adapter <name>` \| Force adapter: `x`, `youtube`, `hn`, or `generic` (default: auto-detect) \|
112	\| `--headless` \| Force headless Chrome (no visible window) \|
113	\| `--wait-for <mode>` \| Interaction wait mode: `none` (default), `interaction`, or `force` \|
114	\| `--wait-for-interaction` \| Alias for `--wait-for interaction` \|
115	\| `--wait-for-login` \| Alias for `--wait-for interaction` \|
116	\| `--timeout <ms>` \| Page load timeout (default: 30000) \|
117	\| `--interaction-timeout <ms>` \| Login/CAPTCHA wait timeout (default: 600000 = 10 min) \|
118	\| `--interaction-poll-interval <ms>` \| Poll interval for interaction checks (default: 1500) \|
119	\| `--download-media` \| Download images/videos to local `imgs/` and `videos/`, rewrite markdown links. Requires `--output` \|
120	\| `--media-dir <dir>` \| Base directory for downloaded media (default: same as `--output` directory) \|
121	\| `--cdp-url <url>` \| Reuse existing Chrome DevTools Protocol endpoint \|
122	\| `--browser-path <path>` \| Custom Chrome/Chromium binary path \|
123	\| `--chrome-profile-dir <path>` \| Chrome user data directory (default: `BAOYU_CHROME_PROFILE_DIR` env or `./baoyu-skills/chrome-profile`) \|
124	\| `--debug-dir <dir>` \| Write debug artifacts (document.json, markdown.md, page.html, network.json) \|
125
126	## Agent Quality Gate
127
128	CRITICAL: treat default headless capture as provisional. Some sites render differently in headless mode and can silently return low-quality content without failing the CLI.
129
130	After every headless run, inspect the saved markdown. See [references/quality-gate.md](references/quality-gate.md) for the full checklist, recovery workflow, and capture-mode table. Read it whenever a run looks suspicious or the user asks about login/CAPTCHA handling.
131
132	## Output Path Generation
133
134	The agent must construct the output file path — `baoyu-fetch` does not auto-generate paths.
135
136	Algorithm:
137	1. Determine base directory from EXTEND.md `default_output_dir` or default `./url-to-markdown/`
138	2. Extract domain from URL (e.g., `example.com`)
139	3. Generate slug from URL path or page title (kebab-case, 2-6 words)
140	4. Construct: `{base_dir}/{domain}/{slug}/{slug}.md` — each URL gets its own directory so media files stay isolated
141	5. Conflict resolution: append timestamp `{slug}-YYYYMMDD-HHMMSS/{slug}-YYYYMMDD-HHMMSS.md`
142
143	Pass the constructed path to `--output`. Media files (`--download-media`) are saved into subdirectories next to the markdown file, keeping each URL's assets self-contained.
144
145	## Adapters & Media
146
147	See [references/adapters.md](references/adapters.md) for the adapter catalog (X, YouTube, Hacker News, generic), per-adapter notes, the media download flow (`ask` / always / never), and the JSON output schema. Read it before answering adapter-specific questions or handling media prompts.
148
149	## Environment Variables
150
151	\| Variable \| Description \|
152	\|----------\|-------------\|
153	\| `BAOYU_CHROME_PROFILE_DIR` \| Chrome user data directory (can also use `--chrome-profile-dir`) \|
154
155	Troubleshooting: Chrome not found → use `--browser-path`. Timeout → increase `--timeout`. Login/CAPTCHA → `--wait-for interaction`. Debug → `--debug-dir` to inspect captured HTML and network logs.
156
157	## Extension Support
158
159	Custom configurations via EXTEND.md. See Preferences section above for paths and supported keys.

6 picks

Type

Run `npx skills use "https://github.com/jimliu/baoyu-skills" --skill "jimliu/baoyu-skills/baoyu-url-to-markdown"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.

Use the skills in "https://github.com/jimliu/baoyu-skills" that are relevant to the current task. Run `npx skills add "https://github.com/jimliu/baoyu-skills"` and select the relevant skills, then follow their instructions.

baoyu-url-to-markdown

How to install baoyu-url-to-markdown?

Related

baoyu-url-to-markdown

How to install baoyu-url-to-markdown?

Related