document-workflows

Name: document-workflows
Author: landing-ai

58stars14forks

How to install document-workflows?

landing-ai/ade-document-processing-skills/document-workflows

$npx -y skills add landing-ai/ade-document-processing-skills --skill document-workflows

Installs into the current project.

Summary

Builds end-to-end document processing pipelines using LandingAI ADE. Covers batch and async processing, classify-then-extract workflows for mixed document types, RAG pipelines with vector DB ingestion, database integration (Snowflake, CSV, DataFrames), visualization (bounding box

Files · 1

View on GitHub

SKILL.md

1	# Document Workflows — ADE Pipeline Patterns
2
3	## Overview
4
5	This skill provides reusable building blocks for composing LandingAI ADE
6	primitives (parse, extract, split) into production-ready document processing
7	pipelines. It complements the `document-extraction` skill:
8
9	\| Concern \| `document-extraction` \| `document-workflows` \|
10	\|---------\|----------------------\|---------------------\|
11	\| Scope \| ADE SDK API: parse, extract, split, grounding \| End-to-end pipelines: batch, RAG, DB, classify-route \|
12	\| When \| Need to call a single ADE operation \| Need to compose operations into a workflow \|
13	\| Code \| SDK method calls with parameters \| Complete functions with error handling, parallelism \|
14	\| Deps \| `landingai-ade` only \| + workflow-specific libs (pandas, chromadb, etc.) \|
15
16	Philosophy: Organize by workflow pattern (batch, RAG, DB insertion),
17	not by document type. The same pattern applies whether documents are invoices,
18	utility bills, or medical forms.
19
20	---
21
22	## Step 0 (mandatory) — Pre-Flight Document Exploration {#pre-flight}
23
24	Run this before writing any pipeline code whenever working with documents
25	whose internal structure has not already been inspected in this session.
26
27	> **Rule: never write section-detection, heading-matching, or text-search code
28	> without first running Tool 2 (diagnostic parse) on the sample document.
29	> Heading format is document-specific and cannot be inferred from the task
30	> description or document type alone — the only reliable way to know it is to
31	> look at the actual ADE output.**
32	>
33	> Common surprises: a paper's "Introduction" heading may appear as
34	> `1. Introduction` (plain text, no `#`), `## Introduction`, `INTRODUCTION`
35	> (all-caps), or embedded inside a text chunk with body copy. Getting this
36	> wrong means a silent failure (zero chunks matched) that requires a full
37	> re-parse to debug.
38
39	Run Tool 1 (visual render) and Tool 2 (diagnostic parse) on 1–3 representative
40	sample documents before writing any code. This takes under a minute and
41	prevents debugging iterations that a pre-flight would have avoided.
42
43	### Tool 1 — Visual page render
44
45	Render 1–2 pages as PNG and read them as visual context. No ADE credits used,
46	but each PNG consumes context tokens. Use when layout is ambiguous or document
47	origin is unknown (handwriting? scan? form?).
48
49	```bash
50	.venv/bin/python - << 'EOF'
51	import pymupdf
52	from pathlib import Path
53	from PIL import Image
54
55	pdf = Path('path/to/sample.pdf')
56	out_dir = Path('/tmp/ade_preflight'); out_dir.mkdir(exist_ok=True)
57	doc = pymupdf.open(pdf)
58	for pg in range(min(2, len(doc))): # first 2 pages only
59	pix = doc[pg].get_pixmap(matrix=pymupdf.Matrix(1.5, 1.5)) # 108 DPI
60	img = Image.frombytes("RGB", [pix.width, pix.height], pix.samples)
61	out = out_dir / f"{pdf.stem}_page{pg + 1}.png"
62	img.save(out)
63	print(out)
64	doc.close()
65	EOF
66	```
67
68	Then read the saved PNGs. Immediately answers:
69	- Are headings bold text (→ ADE may output plain-text heading, not `# Heading`)
70	- Is the document handwritten or scanned? → Tesseract OCR needed, not PyMuPDF
71	- Single-column or two-column layout?
72	- Any noise: running headers, page numbers, watermarks, stamps?
73
74	### Tool 2 — ADE diagnostic parse
75
76	Parses 1 sample and prints markdown structure + chunk inventory. Uses ADE
77	credits — keep to 1–3 samples only, never the full corpus.
78
79	```bash
80	.venv/bin/python - << 'EOF'
81	import os
82	from pathlib import Path
83	from collections import Counter
84	from dotenv import load_dotenv
85
86	# Load API key: prefer existing env var, then .env file lookup
87	load_dotenv() # Load API key from .env. Add a path to the .env if needed.
88
89	from landingai_ade import LandingAIADE
90	client = LandingAIADE()
91	pr = client.parse(document=Path('path/to/sample.pdf'))
92
93	print("=== MARKDOWN (first 80 lines) ===")
94	for i, ln in enumerate(pr.markdown.splitlines()[:80], 1):
95	print(f"{i:3}: {ln}")
96
97	print("\n=== CHUNKS ===")
98	for ch in pr.chunks:
99	txt = (ch.markdown or '').replace('\n', ' ')[:70]
100	b = ch.grounding.box
101	print(f"p{ch.grounding.page} {ch.type:12} "
102	f"l={b.left:.2f} t={b.top:.2f} r={b.right:.2f} b={b.bottom:.2f} \| {txt}")
103
104	print(f"\nPages: {pr.metadata.page_count} "
105	f"

People who liked this also viewed

6 picks

Type

document-workflows

58stars14forks

View on GitHub

How to install document-workflows?

landing-ai/ade-document-processing-skills/document-workflows

$npx -y skills add landing-ai/ade-document-processing-skills --skill document-workflows

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Document Workflows — ADE Pipeline Patterns
2
3	## Overview
4
5	This skill provides reusable building blocks for composing LandingAI ADE
6	primitives (parse, extract, split) into production-ready document processing
7	pipelines. It complements the `document-extraction` skill:
8
9	\| Concern \| `document-extraction` \| `document-workflows` \|
10	\|---------\|----------------------\|---------------------\|
11	\| Scope \| ADE SDK API: parse, extract, split, grounding \| End-to-end pipelines: batch, RAG, DB, classify-route \|
12	\| When \| Need to call a single ADE operation \| Need to compose operations into a workflow \|
13	\| Code \| SDK method calls with parameters \| Complete functions with error handling, parallelism \|
14	\| Deps \| `landingai-ade` only \| + workflow-specific libs (pandas, chromadb, etc.) \|
15
16	Philosophy: Organize by workflow pattern (batch, RAG, DB insertion),
17	not by document type. The same pattern applies whether documents are invoices,
18	utility bills, or medical forms.
19
20	---
21
22	## Step 0 (mandatory) — Pre-Flight Document Exploration {#pre-flight}
23
24	Run this before writing any pipeline code whenever working with documents
25	whose internal structure has not already been inspected in this session.
26
27	> **Rule: never write section-detection, heading-matching, or text-search code
28	> without first running Tool 2 (diagnostic parse) on the sample document.
29	> Heading format is document-specific and cannot be inferred from the task
30	> description or document type alone — the only reliable way to know it is to
31	> look at the actual ADE output.**
32	>
33	> Common surprises: a paper's "Introduction" heading may appear as
34	> `1. Introduction` (plain text, no `#`), `## Introduction`, `INTRODUCTION`
35	> (all-caps), or embedded inside a text chunk with body copy. Getting this
36	> wrong means a silent failure (zero chunks matched) that requires a full
37	> re-parse to debug.
38
39	Run Tool 1 (visual render) and Tool 2 (diagnostic parse) on 1–3 representative
40	sample documents before writing any code. This takes under a minute and
41	prevents debugging iterations that a pre-flight would have avoided.
42
43	### Tool 1 — Visual page render
44
45	Render 1–2 pages as PNG and read them as visual context. No ADE credits used,
46	but each PNG consumes context tokens. Use when layout is ambiguous or document
47	origin is unknown (handwriting? scan? form?).
48
49	```bash
50	.venv/bin/python - << 'EOF'
51	import pymupdf
52	from pathlib import Path
53	from PIL import Image
54
55	pdf = Path('path/to/sample.pdf')
56	out_dir = Path('/tmp/ade_preflight'); out_dir.mkdir(exist_ok=True)
57	doc = pymupdf.open(pdf)
58	for pg in range(min(2, len(doc))): # first 2 pages only
59	pix = doc[pg].get_pixmap(matrix=pymupdf.Matrix(1.5, 1.5)) # 108 DPI
60	img = Image.frombytes("RGB", [pix.width, pix.height], pix.samples)
61	out = out_dir / f"{pdf.stem}_page{pg + 1}.png"
62	img.save(out)
63	print(out)
64	doc.close()
65	EOF
66	```
67
68	Then read the saved PNGs. Immediately answers:
69	- Are headings bold text (→ ADE may output plain-text heading, not `# Heading`)
70	- Is the document handwritten or scanned? → Tesseract OCR needed, not PyMuPDF
71	- Single-column or two-column layout?
72	- Any noise: running headers, page numbers, watermarks, stamps?
73
74	### Tool 2 — ADE diagnostic parse
75
76	Parses 1 sample and prints markdown structure + chunk inventory. Uses ADE
77	credits — keep to 1–3 samples only, never the full corpus.
78
79	```bash
80	.venv/bin/python - << 'EOF'
81	import os
82	from pathlib import Path
83	from collections import Counter
84	from dotenv import load_dotenv
85
86	# Load API key: prefer existing env var, then .env file lookup
87	load_dotenv() # Load API key from .env. Add a path to the .env if needed.
88
89	from landingai_ade import LandingAIADE
90	client = LandingAIADE()
91	pr = client.parse(document=Path('path/to/sample.pdf'))
92
93	print("=== MARKDOWN (first 80 lines) ===")
94	for i, ln in enumerate(pr.markdown.splitlines()[:80], 1):
95	print(f"{i:3}: {ln}")
96
97	print("\n=== CHUNKS ===")
98	for ch in pr.chunks:
99	txt = (ch.markdown or '').replace('\n', ' ')[:70]
100	b = ch.grounding.box
101	print(f"p{ch.grounding.page} {ch.type:12} "
102	f"l={b.left:.2f} t={b.top:.2f} r={b.right:.2f} b={b.bottom:.2f} \| {txt}")
103
104	print(f"\nPages: {pr.metadata.page_count} "
105	f"

People who liked this also viewed

6 picks

Type