customize

Name: customize
Author: anthropics

6.7kstars543forks

How to install customize?

anthropics/defending-code-reference-harness/customize

$npx -y skills add anthropics/defending-code-reference-harness --skill customize

Installs into the current project.

Summary

Adapt this C/C++ ASAN vulnerability pipeline to a different vulnerability class, target shape, language, or detection mechanism. Use when the user wants to port, migrate, retarget, customize, or fork the pipeline for something other than C/C++ memory-safety bugs — web apps, smart

Files · 1

View on GitHub

SKILL.md

1	# Customize the vuln-pipeline
2
3	This pipeline ships as an opinionated C/C++ + AddressSanitizer demo. Its real shape is more general: an agent crafts an input, runs a target in a sandbox, a detector fires, a second agent verifies, a third agent analyzes exploitability. Every noun in that sentence can be swapped. Your job is to interview the user, figure out which nouns they want to swap, and rewrite the relevant files.
4
5	The existing C/C++ code is the worked example. You don't need a playbook for each domain — read what's there, understand what's generic vs. ASAN-specific, and adapt.
6
7	## STEP 1 — Read the pipeline (do this BEFORE asking anything)
8
9	Skim these files so your questions are grounded:
10
11	- `README.md` — pipeline overview (recon → find → grade → judge → report)
12	- `harness/cli.py` — orchestration; shows how stages wire together and what lands on disk
13	- `harness/find.py`, `harness/grade.py`, `harness/report.py` — the three container-agent loops; mostly generic plumbing
14	- `harness/prompts/find_prompt.py`, `harness/prompts/grade_prompt.py` — the C/C++-specific parts; bug taxonomy, quality tiers, grading rubric
15	- `harness/prompts/report_prompt.py`, `harness/prompts/report_grader_prompt.py` — also C/C++-specific; exploitability sections (primitive, heap layout, escalation path) and the rubric that scores them
16	- `harness/prompts/judge_prompt.py` — triage prompt; keys on ASAN excerpts and memory-safety crash classes
17	- `harness/prompts/system_prompt.py` — authorization block; hard-codes "C/C++ target" and "sanitizer output"
18	- `harness/asan.py` — stack-trace parser for dedup/judge signatures; ASAN-specific regex
19	- `harness/artifacts.py` — `CrashArtifact`, `GraderVerdict`, `JudgeVerdict`, `ReportVerdict` data contracts
20	- `harness/config.py`, `targets/drlibs/config.yaml` — target config schema
21	- `targets/README.md` — how a target directory is structured (Dockerfile + config.yaml + entry wrapper)
22
23	You don't need `agent.py`, `docker_ops.py`, `recon.py`, `judge.py`, or `novelty.py` in detail — they're generic plumbing (judge/novelty domain-specificity lives in the prompts and the asan parser, not the flow). One caution: part of `harness/` doubles as shared infrastructure — `dnr_harness/` (the detection-and-response pipeline) imports `agent.py`, `sandbox.py`, `docker_ops.py`, `agent_image.py`'s `build`/`ensure_base`, `auth.py`, `cli.py`'s `resolve_target_dir`/`terminate_subprocesses`, and `prompts/system_prompt.py`'s `load_engagement_context`. A domain port should touch only the domain-specific files (the prompts, `asan.py`, `artifacts.py`, target Dockerfiles); treat any edit to the shared files as a change to both pipelines.
24
25	## STEP 2 — Interview the user
26
27	Use AskUserQuestion to gather requirements. Start with broad context, then narrow to technical specifics based on what they say.
28
29	### Round 1 — Context (always ask these first, together)
30
31	Two open-ended questions to understand who you're talking to and what they're after. Expect most answers to come via Other as free text — the options are there to prompt thinking, not to constrain.
32
33	Question A — Operating context
34	- header: `Context`
35	- question: `What's your operating environment? Who will run this pipeline and why?`
36	- options: a few archetypes as inspiration — e.g. "Pentesting firm — client engagements, need reportable findings", "Internal appsec team — scan our own services in CI", "Smart-contract auditor — pre-deployment reviews", "Security researcher — hunting novel bug classes". These tell you what output format, grading rigor, and workflow integration matter.
37
38	Question B — Goal
39	- header: `Goal`
40	- question: `Describe in your own words what you want this pipeline to find. What kind of target, what kind of bugs?`
41	- options: 2–3 concrete examples (e.g. "Web vulnerabilities like SQLi/XSS in HTTP services", "Reentrancy and access-control bugs in Solidity contracts", "Deserialization RCE in Java microservices").
42
43	The context answer calibrates your follow-ups: a pentesting firm probably cares about CVSS scoring and SARIF output; a researcher may want differential testing and novel detection signals; an internal team likely wants CI integration and low false-positive rates.
44
45	### Round 2 — Technical follow-ups (adaptive — derive from round-1 answers)
46
47	Parse their round-1 answers against the axes of variation below. For each axis left ambiguous, ask a targeted follow-up. Batch up to 4 questions per AskUserQuestion call. Common follow-ups:
48
49	- Detection signal — "How will the pipeline know it found something?" (crash, exception, canary file appears, DNS callback,

People who liked this also viewed

6 picks

Type

customize

6.7kstars543forks

View on GitHub

How to install customize?

anthropics/defending-code-reference-harness/customize

$npx -y skills add anthropics/defending-code-reference-harness --skill customize

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Customize the vuln-pipeline
2
3	This pipeline ships as an opinionated C/C++ + AddressSanitizer demo. Its real shape is more general: an agent crafts an input, runs a target in a sandbox, a detector fires, a second agent verifies, a third agent analyzes exploitability. Every noun in that sentence can be swapped. Your job is to interview the user, figure out which nouns they want to swap, and rewrite the relevant files.
4
5	The existing C/C++ code is the worked example. You don't need a playbook for each domain — read what's there, understand what's generic vs. ASAN-specific, and adapt.
6
7	## STEP 1 — Read the pipeline (do this BEFORE asking anything)
8
9	Skim these files so your questions are grounded:
10
11	- `README.md` — pipeline overview (recon → find → grade → judge → report)
12	- `harness/cli.py` — orchestration; shows how stages wire together and what lands on disk
13	- `harness/find.py`, `harness/grade.py`, `harness/report.py` — the three container-agent loops; mostly generic plumbing
14	- `harness/prompts/find_prompt.py`, `harness/prompts/grade_prompt.py` — the C/C++-specific parts; bug taxonomy, quality tiers, grading rubric
15	- `harness/prompts/report_prompt.py`, `harness/prompts/report_grader_prompt.py` — also C/C++-specific; exploitability sections (primitive, heap layout, escalation path) and the rubric that scores them
16	- `harness/prompts/judge_prompt.py` — triage prompt; keys on ASAN excerpts and memory-safety crash classes
17	- `harness/prompts/system_prompt.py` — authorization block; hard-codes "C/C++ target" and "sanitizer output"
18	- `harness/asan.py` — stack-trace parser for dedup/judge signatures; ASAN-specific regex
19	- `harness/artifacts.py` — `CrashArtifact`, `GraderVerdict`, `JudgeVerdict`, `ReportVerdict` data contracts
20	- `harness/config.py`, `targets/drlibs/config.yaml` — target config schema
21	- `targets/README.md` — how a target directory is structured (Dockerfile + config.yaml + entry wrapper)
22
23	You don't need `agent.py`, `docker_ops.py`, `recon.py`, `judge.py`, or `novelty.py` in detail — they're generic plumbing (judge/novelty domain-specificity lives in the prompts and the asan parser, not the flow). One caution: part of `harness/` doubles as shared infrastructure — `dnr_harness/` (the detection-and-response pipeline) imports `agent.py`, `sandbox.py`, `docker_ops.py`, `agent_image.py`'s `build`/`ensure_base`, `auth.py`, `cli.py`'s `resolve_target_dir`/`terminate_subprocesses`, and `prompts/system_prompt.py`'s `load_engagement_context`. A domain port should touch only the domain-specific files (the prompts, `asan.py`, `artifacts.py`, target Dockerfiles); treat any edit to the shared files as a change to both pipelines.
24
25	## STEP 2 — Interview the user
26
27	Use AskUserQuestion to gather requirements. Start with broad context, then narrow to technical specifics based on what they say.
28
29	### Round 1 — Context (always ask these first, together)
30
31	Two open-ended questions to understand who you're talking to and what they're after. Expect most answers to come via Other as free text — the options are there to prompt thinking, not to constrain.
32
33	Question A — Operating context
34	- header: `Context`
35	- question: `What's your operating environment? Who will run this pipeline and why?`
36	- options: a few archetypes as inspiration — e.g. "Pentesting firm — client engagements, need reportable findings", "Internal appsec team — scan our own services in CI", "Smart-contract auditor — pre-deployment reviews", "Security researcher — hunting novel bug classes". These tell you what output format, grading rigor, and workflow integration matter.
37
38	Question B — Goal
39	- header: `Goal`
40	- question: `Describe in your own words what you want this pipeline to find. What kind of target, what kind of bugs?`
41	- options: 2–3 concrete examples (e.g. "Web vulnerabilities like SQLi/XSS in HTTP services", "Reentrancy and access-control bugs in Solidity contracts", "Deserialization RCE in Java microservices").
42
43	The context answer calibrates your follow-ups: a pentesting firm probably cares about CVSS scoring and SARIF output; a researcher may want differential testing and novel detection signals; an internal team likely wants CI integration and low false-positive rates.
44
45	### Round 2 — Technical follow-ups (adaptive — derive from round-1 answers)
46
47	Parse their round-1 answers against the axes of variation below. For each axis left ambiguous, ask a targeted follow-up. Batch up to 4 questions per AskUserQuestion call. Common follow-ups:
48
49	- Detection signal — "How will the pipeline know it found something?" (crash, exception, canary file appears, DNS callback,

People who liked this also viewed

6 picks

Type