pdf-ingestion
starsforks
How to install pdf-ingestion?
archive228/loopkit/pdf-ingestion
$npx -y skills add Archive228/loopkit --skill pdf-ingestionGet a PDF into the model without blowing the context window or losing structure. Native PDF beats OCR-then-text for most cases; extract-then-summarize beats native for very long docs.
SKILL.md
| 1 | # PDF Ingestion |
| 2 | |
| 3 | Three ways to feed a PDF to the model, in increasing order of preprocessing: |
| 4 | |
| 5 | 1. **Native PDF input** — pass the file directly. Model sees pages as images + extracted text. Best for docs under ~100 pages with meaningful layout (tables, figures, forms). Preserves structure. |
| 6 | |
| 7 | 2. **Text extraction then send** — `pdftotext` / `pypdf` / equivalent, then send the text. Loses layout but cheap. Fine for prose-heavy docs where tables don't matter. |
| 8 | |
| 9 | 3. **Extract → chunk → summarize → send** — for docs >100 pages or when you'll query the same doc many times. Preprocess once, cache the summary. |
| 10 | |
| 11 | ## Deciding which path |
| 12 | |
| 13 | | Doc shape | Path | |
| 14 | |---|---| |
| 15 | | <20 pages, layout matters (report, form, invoice) | Native | |
| 16 | | <20 pages, pure prose (article, memo) | Text extraction | |
| 17 | | 20-100 pages, mixed | Native, but chunk if context tight | |
| 18 | | >100 pages | Extract → chunk → summarize | |
| 19 | | Scanned PDF (no text layer) | OCR first (Tesseract or vision model), then treat as extracted text | |
| 20 | | Tables are the point | Native — text extractors mangle tables | |
| 21 | | Figures/diagrams are the point | Native + explicit "describe the figure on page N" prompt | |
| 22 | |
| 23 | ## Native PDF — the good defaults |
| 24 | |
| 25 | - Cache the PDF at a prompt-caching breakpoint (see `prompt-caching`). Native PDFs are large — every uncached turn costs full input price on the whole doc. |
| 26 | - Ask about **specific pages** ("summarize section 3.2 on page 14") rather than the whole doc. The model handles targeted queries better than "summarize this 80-page report". |
| 27 | - Follow up with **page-cited claims** — "on which page does the doc say X?" — as a sanity check the model isn't hallucinating. |
| 28 | |
| 29 | ## Extract-then-send — the traps |
| 30 | |
| 31 | - **`pdftotext` reading order.** Multi-column PDFs come out as interleaved lines. Use `pdftotext -layout` for column preservation, or `pdftotext -raw` for straight reading order — pick per doc, don't guess. |
| 32 | - **Tables become word soup.** If tables are load-bearing, native or per-table image extraction. Not text. |
| 33 | - **Headers/footers repeat on every page.** Strip them before sending, or the model will treat them as content. |
| 34 | - **Footnotes drift** to random positions in the extracted stream. Filter or accept the noise. |
| 35 | |
| 36 | ## Extract → chunk → summarize (long docs) |
| 37 | |
| 38 | - Chunk by section, not by token count. A section-aware split respects the doc's logic; a naive 4K-token split cuts sentences and tables. |
| 39 | - Summarize per section into a "map" — 1-2 paragraphs each. Keep the map short enough to fit in context whole (~2-4K tokens for a 200-page doc). |
| 40 | - Store the full section text alongside the map (paths in a manifest). Fetch on demand when a question needs detail beyond the summary. |
| 41 | - Cache the map at a prompt-caching breakpoint so multi-turn Q&A over the doc doesn't reprocess. |
| 42 | |
| 43 | ## Red flags |
| 44 | |
| 45 | - **Sending a 200-page PDF native to answer one question.** Extract the relevant page range first. |
| 46 | - **Trusting the text extractor on a form or invoice.** Layout carries meaning. Use native. |
| 47 | - **OCR'ing a PDF that already has a text layer.** Check `pdftotext -q file.pdf -` first — if text comes out, no OCR needed. |
| 48 | - **No page citations in output.** Model can hallucinate confidently across long PDFs. Force page numbers into the response format. |
| 49 | - **Re-uploading the same PDF every turn without caching.** Cost climbs linearly; a 5-minute cache fixes it. |
| 50 | |
| 51 | ## Loopkit-adjacent |
| 52 | |
| 53 | If the PDF is a spec, extract it into `PROMPT.md` via `spec-first` — the agent should re-read prose, not re-scan the PDF, on every turn. |
People who liked this also viewed
6 picksType
- azure-kustoQuery and analyze data in Azure Data Explorer (Kusto/ADX) using KQL for log analytics, telemetry, and time series analysis. WHEN: KQL queries, Kusto database queries, Azure Data Explorer, ADX clustersSkillsJul 2026
- lark-base飞书多维表格（Base）操作：建表、字段、记录、视图、统计、公式/lookup、表单、仪表盘、workflow、角色权限；遇到 Base/多维表格/bitable 或 /base/ 链接时使用。文件导入转 lark-drive，认证/授权转 lark-shared。SkillsJul 2026
- lark-calendar飞书日历：管理日历日程和会议室。查看/搜索日程、创建/更新日程、管理参会人、查询忙闲和推荐时段、预定会议室。当用户需要查看日程安排、创建/修改会议、查询/预定会议室时使用。不负责：查询过去的视频会议记录（走 lark-vc）、待办任务（走 lark-task）。SkillsJul 2026
- shadcnManages shadcn components and projects — adding, searching, fixing, debugging, styling, and composing UI, including chat interfaces. Provides project context, component docs, and usage examples. AppliSkillsJul 2026
- xlsxUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.SkillsJul 2026
- firebase-data-connectAgent Skills for FirebaseSkillsJul 2026
