$npx -y skills add genli-ai/market-research-skills --skill local-vaultBuild and query a local Markdown knowledge base ("vault"). TWO functions — (1) CONVERT raw files (PDF, Word/docx, PowerPoint/pptx, Excel/xlsx, csv/tsv, images, html, md/txt, json/yaml/code, audio/video) into clean Markdown with retrieval-friendly frontmatter; local-first (pandoc
| 1 | # local-vault |
| 2 | |
| 3 | Turn a folder of raw files into a **Markdown vault** that an LLM can grep, and |
| 4 | then answer questions over that vault responsibly. |
| 5 | |
| 6 | **Mental model:** `SOURCE` = raw files (source of truth). `VAULT` = one `.md` per |
| 7 | source file, carrying retrieval frontmatter (abstract / tags / synonyms) + a |
| 8 | `source` backlink. The vault is the layer the LLM reads; the raw files are where |
| 9 | the user goes to verify. |
| 10 | |
| 11 | There are **two distinct jobs** — figure out which the user wants: |
| 12 | |
| 13 | - **A. Convert / sync** — they dropped files in and want them in the vault → run |
| 14 | the pipeline (`scripts/sync.py`). |
| 15 | - **B. Retrieve / answer** — they want answers from an existing vault → follow |
| 16 | the *Retrieval & feedback protocol* below. Do **not** run the pipeline for this. |
| 17 | |
| 18 | --- |
| 19 | |
| 20 | ## A. Convert / sync |
| 21 | |
| 22 | ### One-time setup (do this for the user if not already done) |
| 23 | |
| 24 | 1. **Python deps** (user-level, no venv): |
| 25 | ``` |
| 26 | python3 -m pip install --user requests python-dotenv pypdf pymupdf4llm openpyxl python-pptx |
| 27 | ``` |
| 28 | 2. **pandoc** (for docx/rtf/odt/epub): `brew install pandoc` (macOS) / distro pkg. |
| 29 | 3. **ffmpeg** (only for audio/video transcription): `brew install ffmpeg` (macOS) / |
| 30 | distro pkg. The **whisper engine is auto-selected by platform** — `mlx-whisper` |
| 31 | on Apple Silicon (GPU), `faster-whisper` elsewhere (cross-platform CPU/CUDA) — and |
| 32 | **auto-installed after the user consents** at the first-run prompt (no manual pip |
| 33 | needed). On that first run with audio/video present, the tool shows the model-size |
| 34 | options (tiny ~75 MB / small ~480 MB / turbo ~1.6 GB / large-v3 ~3 GB) and lets the |
| 35 | user pick or skip; the choice is saved to `.env` (`KB_WHISPER_MODEL`) so it never |
| 36 | re-asks. Fully local — no token/quota; the model downloads once, then offline. |
| 37 | 4. **`claude` CLI on PATH** — the pipeline shells out to `claude -p` for frontmatter |
| 38 | enrichment and PPT-image OCR. If absent, those steps are skipped (not fatal). |
| 39 | 5. **Configure paths** — two ways: |
| 40 | - **Guided (recommended for the user):** just run `python3 scripts/sync.py` in a |
| 41 | terminal. On first run (when paths aren't configured yet) it launches an |
| 42 | interactive wizard: it asks for the raw-files folder + the vault folder |
| 43 | (+ optional MinerU token), creates them, writes `scripts/.env`, and prints how |
| 44 | to use the tool. Then they re-run to convert. |
| 45 | - **Manual:** copy `scripts/.env.example` → `scripts/.env` and set |
| 46 | `KB_SOURCE_DIR` (raw files) and `KB_TARGET_DIR` (the Markdown vault), both |
| 47 | absolute. `MINERU_TOKEN` is optional (only for legacy .doc/.ppt, .html, scanned |
| 48 | PDFs, images — get one at https://mineru.net). |
| 49 | - When **you (Claude)** run the setup for the user, prefer the manual path: ask |
| 50 | them for the two folders, then write `scripts/.env` directly (the wizard only |
| 51 | fires on an interactive TTY, which a `claude -p` subprocess is not). |
| 52 | |
| 53 | ### Run it |
| 54 | |
| 55 | ``` |
| 56 | python3 scripts/sync.py |
| 57 | ``` |
| 58 | |
| 59 | On macOS, the first run (wizard or any normal run) also drops a clickable |
| 60 | `sync.command` **into the knowledge-base root — the parent of the SOURCE folder**, |
| 61 | with the absolute path to `sync.py` baked in (tool and data live apart — under |
| 62 | `/plugin install` the script sits in `~/.claude/plugins/cache/…`, far from the data |
| 63 | folders, so a relative launcher can't work). After that the daily loop is: drop |
| 64 | files into SOURCE → double-click `sync.command` → read the `.md` in VAULT. The |
| 65 | launcher is idempotent; a stale auto-generated copy left in the SOURCE folder by an |
| 66 | older version is removed automatically (a user-written one is never touched). If a |
| 67 | *different* `sync.command` already exists at the root, an interactive terminal |
| 68 | prompts **update / skip**; non-interactively, our own out-of-date launcher self-heals |
| 69 | silently while a user-customized one is left alone. |
| 70 | |
| 71 | First terminal run with no config → the setup wizard (above). Once `.env` exists: |
| 72 | |
| 73 | - **Incremental**: only files in SOURCE without a matching `.md` in VAULT are |
| 74 | processed. To force a re-convert, delete that `.md` first, then re-run. |
| 75 | - **No MinerU token needed** for the local paths (xlsx/csv/docx/pptx/md/txt/code + |
| 76 | digital PDF). Token is validated lazily, only |