fact-check

Name: fact-check
Author: scdenney

34stars0forks

How to install fact-check?

scdenney/open-science-skills/fact-check

$npx -y skills add scdenney/open-science-skills --skill fact-check

Installs into the current project.

Summary

Fact-check manuscript claims against cited sources in a per-source Markdown knowledge base. Use to audit claim support, overclaiming, direction, scope, and misattribution after source intake is complete.

Files · 1

View on GitHub

SKILL.md

1	# Claim–Source Fact Checker
2
3	## Heritage and scope
4
5	This is an original Open Science Skills workflow. It extends `citation-check` (which verifies a citation exists, resolves, and is formatted) to the next question: does the cited source actually support the claim it is attached to? It checks claims against the project's per-source Markdown knowledge base — the files produced by `process-source`, where each source is converted or summarized to `sources/md/<author>-<year>-<slug>.md`. This is local-source verification, not open-web fact-checking: it audits whether a manuscript's sentences are backed by the documents the author has actually read and filed. Run once a draft has citations and a populated knowledge base.
6
7	## Instructions
8
9	### 1. Pre-flight gate — orient, then refuse if the knowledge base is not ready
10
11	Orient. Identify the inputs:
12
13	- Manuscript source: LaTeX, Markdown, Word/PDF text, or pasted prose with citations or footnotes.
14	- Bibliography: `.bib`, CSL JSON, or reference list — used to resolve cite keys to author-year-title so source files can be found.
15	- Knowledge base: the project's per-source Markdown directory. Search in order: `sources/md/`, `knowledge_base/md/`, `sources/`, `knowledge_base/`, then any chapter-local `/sources/`. Also read any index or crosswalk file (`sources/inventory.md`, `notes/source_map.md`, `sourcecrosswalk`, `sourcematrix*`) that maps bib keys or titles to files.
16	- Scope: whole manuscript, one section, the literature review/theory sections, or a supplied list of claims.
17
18	Pre-flight gate (hard stop). This skill verifies claims against local Markdown source files; it does not fact-check from memory or the open web. A fact-check run against an absent or half-built knowledge base produces false reassurance, which is worse than no check. So before doing anything else, decide readiness and refuse to run — return a `Pre-flight: NOT READY` notice instead of a report — when any of the following holds:
19
20	1. No knowledge base. No per-source Markdown corpus exists: `sources/md/`, `knowledge_base/md/`, or an equivalent is missing or empty.
21	2. Unconverted sources (not cleaned up). Raw documents — `.pdf`, `.docx`, `.epub`, or scanned images — sit in the sources area (a drop-zone, `sources/`, `sources/pdf/`, `*/source_pdfs/`) without a corresponding `.md`. The base is not clean: every source must be converted to Markdown first (or, if long, to a Markdown summary).
22	3. Coverage too low. The available Markdown does not cover the cited works in scope. Compute coverage = matched source files ÷ cited non-background works; below roughly two-thirds, treat as not ready.
23
24	Do not partially fact-check around the gap, and do not fall back to memory or web lookups for missing sources. Stop and report. The user may rerun after fixing it, or explicitly rescope the run to the already-covered subset; only then proceed with that subset.
25
26	Remediation to print on refusal:
27
28	- Convert each raw source to Markdown with the project's `process-source` command or equivalent intake workflow. It should turn each PDF into `sources/md/<author>-<year>-<slug>.md`, summarizing long sources.
29	- Populate any cited work that has no file yet: locate the PDF, then run `process-source`.
30	- Re-run `fact-check` once `sources/md/` covers the cited set and no raw files remain unconverted.
31	- Best practices: the per-source Markdown knowledge base and intake conventions are documented in the Open Science Skills repo (`github.com/scdenney/open-science-skills`) and the `process-source` skill; Anthropic's skill-authoring guidance is linked from that README.
32
33	Only when the gate passes — knowledge base present, clean, and covering the cited set in scope — continue to step 2.
34
35	### 2. Run citation-check first (always)
36
37	Before judging support, run the `citation-check` skill on the same inputs and consume its report. A claim cannot be evaluated against a source that is missing, fabricated, or mis-keyed. Carry its findings forward: any `LIKELY FABRICATED`, `DOI RESOLVES TO DIFFERENT WORK`, or in-text/reference parity break makes the dependent claim `UNVERIFIABLE — citation problem`, not a clean support check. Do not silently repeat citation-check's work; cite its result and build on it.
38
39	### 3. Build the claim inventory
40
41	For each in-text citation, parenthetical, narrative cite, or footnote that attaches a source to a substantive statement, record:
42
43	- the claim — the exact sentence or clause the citation backs;
44	- the cite key(s) / author-year;
45	- the claim type — empirical result, descriptive fact, definition/concept, method precedent, theoretical position, or direct quotation;
46	- the asserted strength — the verb does work:

People who liked this also viewed

6 picks

Type

fact-check

34stars0forks

View on GitHub

How to install fact-check?

scdenney/open-science-skills/fact-check

$npx -y skills add scdenney/open-science-skills --skill fact-check

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Claim–Source Fact Checker
2
3	## Heritage and scope
4
5	This is an original Open Science Skills workflow. It extends `citation-check` (which verifies a citation exists, resolves, and is formatted) to the next question: does the cited source actually support the claim it is attached to? It checks claims against the project's per-source Markdown knowledge base — the files produced by `process-source`, where each source is converted or summarized to `sources/md/<author>-<year>-<slug>.md`. This is local-source verification, not open-web fact-checking: it audits whether a manuscript's sentences are backed by the documents the author has actually read and filed. Run once a draft has citations and a populated knowledge base.
6
7	## Instructions
8
9	### 1. Pre-flight gate — orient, then refuse if the knowledge base is not ready
10
11	Orient. Identify the inputs:
12
13	- Manuscript source: LaTeX, Markdown, Word/PDF text, or pasted prose with citations or footnotes.
14	- Bibliography: `.bib`, CSL JSON, or reference list — used to resolve cite keys to author-year-title so source files can be found.
15	- Knowledge base: the project's per-source Markdown directory. Search in order: `sources/md/`, `knowledge_base/md/`, `sources/`, `knowledge_base/`, then any chapter-local `/sources/`. Also read any index or crosswalk file (`sources/inventory.md`, `notes/source_map.md`, `sourcecrosswalk`, `sourcematrix*`) that maps bib keys or titles to files.
16	- Scope: whole manuscript, one section, the literature review/theory sections, or a supplied list of claims.
17
18	Pre-flight gate (hard stop). This skill verifies claims against local Markdown source files; it does not fact-check from memory or the open web. A fact-check run against an absent or half-built knowledge base produces false reassurance, which is worse than no check. So before doing anything else, decide readiness and refuse to run — return a `Pre-flight: NOT READY` notice instead of a report — when any of the following holds:
19
20	1. No knowledge base. No per-source Markdown corpus exists: `sources/md/`, `knowledge_base/md/`, or an equivalent is missing or empty.
21	2. Unconverted sources (not cleaned up). Raw documents — `.pdf`, `.docx`, `.epub`, or scanned images — sit in the sources area (a drop-zone, `sources/`, `sources/pdf/`, `*/source_pdfs/`) without a corresponding `.md`. The base is not clean: every source must be converted to Markdown first (or, if long, to a Markdown summary).
22	3. Coverage too low. The available Markdown does not cover the cited works in scope. Compute coverage = matched source files ÷ cited non-background works; below roughly two-thirds, treat as not ready.
23
24	Do not partially fact-check around the gap, and do not fall back to memory or web lookups for missing sources. Stop and report. The user may rerun after fixing it, or explicitly rescope the run to the already-covered subset; only then proceed with that subset.
25
26	Remediation to print on refusal:
27
28	- Convert each raw source to Markdown with the project's `process-source` command or equivalent intake workflow. It should turn each PDF into `sources/md/<author>-<year>-<slug>.md`, summarizing long sources.
29	- Populate any cited work that has no file yet: locate the PDF, then run `process-source`.
30	- Re-run `fact-check` once `sources/md/` covers the cited set and no raw files remain unconverted.
31	- Best practices: the per-source Markdown knowledge base and intake conventions are documented in the Open Science Skills repo (`github.com/scdenney/open-science-skills`) and the `process-source` skill; Anthropic's skill-authoring guidance is linked from that README.
32
33	Only when the gate passes — knowledge base present, clean, and covering the cited set in scope — continue to step 2.
34
35	### 2. Run citation-check first (always)
36
37	Before judging support, run the `citation-check` skill on the same inputs and consume its report. A claim cannot be evaluated against a source that is missing, fabricated, or mis-keyed. Carry its findings forward: any `LIKELY FABRICATED`, `DOI RESOLVES TO DIFFERENT WORK`, or in-text/reference parity break makes the dependent claim `UNVERIFIABLE — citation problem`, not a clean support check. Do not silently repeat citation-check's work; cite its result and build on it.
38
39	### 3. Build the claim inventory
40
41	For each in-text citation, parenthetical, narrative cite, or footnote that attaches a source to a substantive statement, record:
42
43	- the claim — the exact sentence or clause the citation backs;
44	- the cite key(s) / author-year;
45	- the claim type — empirical result, descriptive fact, definition/concept, method precedent, theoretical position, or direct quotation;
46	- the asserted strength — the verb does work:

People who liked this also viewed

6 picks

Type