dedupe-rank

Name: dedupe-rank
Author: WILLOSCAR

489stars39forks

How to install dedupe-rank?

willoscar/research-units-pipeline-skills/dedupe-rank

$npx -y skills add WILLOSCAR/research-units-pipeline-skills --skill dedupe-rank

Installs into the current project.

Summary

Use when a broad paper candidate pool needs deterministic deduplication and a stable core set. Trigger: dedupe, rank, core set, 去重, 排序, 精选论文, 核心集合. Use when: 检索后需要把广覆盖集合收敛成可管理的 core set（用于 taxonomy/outline/mapping）。 Skip if: 已经有人手工整理了稳定的 papers/core_set.csv（无需再次 chu

Files · 1

View on GitHub

SKILL.md

1	# Dedupe + Rank
2
3	Turns a raw candidate pool into a deduped pool and a stable core set.
4
5	## Input
6
7	- `papers/papers_raw.jsonl`
8
9	## Outputs
10
11	- `papers/papers_dedup.jsonl`
12	- `papers/core_set.csv`
13
14	## Script boundary
15
16	`scripts/run.py` should own only:
17	- title/year deduplication
18	- deterministic ranking
19	- stable `paper_id` generation
20
21	Use shared domain packs or pipeline contract metadata for topic-specific or product-specific behavior.
22
23	## Contract-driven behavior
24
25	The script should prefer pipeline contract metadata over profile-name branching.
26
27	Current important field:
28	- `quality_contract.candidate_pool_policy.keep_full_deduped_pool`
29	- `quality_contract.candidate_pool_policy.include_domain_pins`
30	- `quality_contract.candidate_pool_policy.minimum_domain_surveys`
31	- `quality_contract.candidate_pool_policy.survey_title_bonus`
32
33	If true, the script keeps the full deduped pool in `papers/core_set.csv` unless the user explicitly overrides core size.
34	Focused briefing Workflows can disable unconditional domain pins, reserve a
35	smaller survey floor, and remove the generic survey-title bonus without
36	changing broad Survey defaults.
37
38	## Acceptance
39
40	- deduped JSONL exists
41	- core-set CSV exists
42	- reruns are stable for the same inputs
43
44	## Non-goals
45
46	- retrieval
47	- screening
48	- manual topic authoring inside the script

People who liked this also viewed

6 picks

Type

dedupe-rank

489stars39forks

View on GitHub

How to install dedupe-rank?

willoscar/research-units-pipeline-skills/dedupe-rank

$npx -y skills add WILLOSCAR/research-units-pipeline-skills --skill dedupe-rank

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Dedupe + Rank
2
3	Turns a raw candidate pool into a deduped pool and a stable core set.
4
5	## Input
6
7	- `papers/papers_raw.jsonl`
8
9	## Outputs
10
11	- `papers/papers_dedup.jsonl`
12	- `papers/core_set.csv`
13
14	## Script boundary
15
16	`scripts/run.py` should own only:
17	- title/year deduplication
18	- deterministic ranking
19	- stable `paper_id` generation
20
21	Use shared domain packs or pipeline contract metadata for topic-specific or product-specific behavior.
22
23	## Contract-driven behavior
24
25	The script should prefer pipeline contract metadata over profile-name branching.
26
27	Current important field:
28	- `quality_contract.candidate_pool_policy.keep_full_deduped_pool`
29	- `quality_contract.candidate_pool_policy.include_domain_pins`
30	- `quality_contract.candidate_pool_policy.minimum_domain_surveys`
31	- `quality_contract.candidate_pool_policy.survey_title_bonus`
32
33	If true, the script keeps the full deduped pool in `papers/core_set.csv` unless the user explicitly overrides core size.
34	Focused briefing Workflows can disable unconditional domain pins, reserve a
35	smaller survey floor, and remove the generic survey-title bonus without
36	changing broad Survey defaults.
37
38	## Acceptance
39
40	- deduped JSONL exists
41	- core-set CSV exists
42	- reruns are stable for the same inputs
43
44	## Non-goals
45
46	- retrieval
47	- screening
48	- manual topic authoring inside the script

People who liked this also viewed

6 picks

Type