characterization-tests

Name: characterization-tests
Author: AdamBien

40stars10forks

How to install characterization-tests?

adambien/airails/characterization-tests

$npx -y skills add AdamBien/airails --skill characterization-tests

Installs into the current project.

Summary

Pin the observed behavior of a running legacy system as replayable golden masters — record normalized stimulus/observation pairs into migration/characterization/, replay them against the lifted or re-architected system, report behavior diffs. Owns the record/replay/reconcile disc

Files · 1

View on GitHub

SKILL.md

1	# Characterization Tests
2
3	A regular test fails when behavior is wrong; a characterization test fails when behavior is different. Record what the system does — bugs included — and pin it as the expected value. Correctness is the information the legacy system lost; observed behavior is the only surviving specification, and downstream consumers may depend on the bugs.
4
5	Characterization tests are system tests: black-box, against the running system, from the outside — the system-tests skill owns that contract. The only difference from ordinary system tests is the origin of the expectations — recorded, not specified.
6
7	## Surfaces
8
9	The observable surface is a property of the system, not of this skill. Inventory all of them:
10
11	\| Surface \| Stimulus → Observation \| Mechanics \|
12	\|---\|---\|---\|
13	\| HTTP (REST, server-rendered pages) \| request → response \| [references/http.md](references/http.md) \|
14	\| Messaging (JMS, queues) \| message in → message(s) out \| same discipline; document per system \|
15	\| Batch / file exchange \| input file → output file, report \| same discipline; document per system \|
16	\| CLI \| invocation → exit code, output \| same discipline; document per system \|
17	\| Database side effects \| any stimulus → state delta \| record only when no outer surface shows the effect \|
18
19	HTTP mechanics are bundled; for other surfaces apply the identical record/normalize/replay discipline and document the surface mechanics next to the recordings. Prefer the outermost surface that shows the behavior — assert database state only when nothing above it does.
20
21	## Record
22
23	1. Inventory the system's surfaces and their entry points (JAX-RS resources, `web.xml` mappings, queue names, batch jobs, and the external contracts in `migration/CONCEPTS.md` when present).
24	2. Ask the user: target coordinates (base URL, broker, job trigger — never guess), environment, credentials, and which business flows matter most. Confirm the target is not an unprotected production system.
25	3. Observe-only by default: record stimuli without side effects. Record mutating stimuli (writes, consumed messages, batch runs) only after explicit confirmation, only against a disposable environment with resettable seed data.
26	4. Execute each stimulus, normalize the observation per [references/recording-format.md](references/recording-format.md) and the surface reference, store one file per interaction under `migration/characterization/recordings/`.
27	5. Each recording must replay in isolation — note seed-data assumptions in its header.
28
29	## Replay
30
31	1. Ask for the target coordinates of the migrated system.
32	2. Re-execute every recording; normalize the observation with the same rules; compare against the stored expectation.
33	3. Write `migration/characterization/REPORT.md`: one verdict per recording (`same` \| `different` with diff excerpt \| `unreachable`), plus summary counts. Every recording appears in the report — skipped is a verdict, not an omission.
34	4. For repeated replays (CI, per-carving checks), generate a replay suite following the composed stack skill's system-test conventions — an `-st` module with JUnit 5 for microprofile-server targets, zunit for java-cli-app, a zero-dependency single-file Java 25 runner when no stack skill is composed.
35
36	## Reconcile
37
38	A diff is a question, not a verdict: behavior changed — intended?
39
40	- Regression → the migrated system is fixed; the recording stays.
41	- Accepted change (e.g. a bug deliberately fixed) → update the recording and append a decision line to its header: `accepted: <name>, <YYYY-MM-DD> — <reason>`.
42
43	Never update a recording silently, and never decide yourself — acceptance is a human call, same rule as the clarifier.
44
45	## Rules
46
47	- Record actual observations, including apparently wrong ones — correctness judgments happen at reconcile time, by a human.
48	- Normalize before storing and before comparing; never diff raw observations.
49	- The suite encodes what is, not what should be — it is scaffolding with an expiry: retire recordings BC by BC as sbce specs and ears-tests encode intent, delete the rest with `migration/`.
50	- Recording exercises a real system: respect rate limits, stop on unexpected error storms, and never

People who liked this also viewed

6 picks

Type

characterization-tests

40stars10forks

View on GitHub

How to install characterization-tests?

adambien/airails/characterization-tests

$npx -y skills add AdamBien/airails --skill characterization-tests

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Characterization Tests
2
3	A regular test fails when behavior is wrong; a characterization test fails when behavior is different. Record what the system does — bugs included — and pin it as the expected value. Correctness is the information the legacy system lost; observed behavior is the only surviving specification, and downstream consumers may depend on the bugs.
4
5	Characterization tests are system tests: black-box, against the running system, from the outside — the system-tests skill owns that contract. The only difference from ordinary system tests is the origin of the expectations — recorded, not specified.
6
7	## Surfaces
8
9	The observable surface is a property of the system, not of this skill. Inventory all of them:
10
11	\| Surface \| Stimulus → Observation \| Mechanics \|
12	\|---\|---\|---\|
13	\| HTTP (REST, server-rendered pages) \| request → response \| [references/http.md](references/http.md) \|
14	\| Messaging (JMS, queues) \| message in → message(s) out \| same discipline; document per system \|
15	\| Batch / file exchange \| input file → output file, report \| same discipline; document per system \|
16	\| CLI \| invocation → exit code, output \| same discipline; document per system \|
17	\| Database side effects \| any stimulus → state delta \| record only when no outer surface shows the effect \|
18
19	HTTP mechanics are bundled; for other surfaces apply the identical record/normalize/replay discipline and document the surface mechanics next to the recordings. Prefer the outermost surface that shows the behavior — assert database state only when nothing above it does.
20
21	## Record
22
23	1. Inventory the system's surfaces and their entry points (JAX-RS resources, `web.xml` mappings, queue names, batch jobs, and the external contracts in `migration/CONCEPTS.md` when present).
24	2. Ask the user: target coordinates (base URL, broker, job trigger — never guess), environment, credentials, and which business flows matter most. Confirm the target is not an unprotected production system.
25	3. Observe-only by default: record stimuli without side effects. Record mutating stimuli (writes, consumed messages, batch runs) only after explicit confirmation, only against a disposable environment with resettable seed data.
26	4. Execute each stimulus, normalize the observation per [references/recording-format.md](references/recording-format.md) and the surface reference, store one file per interaction under `migration/characterization/recordings/`.
27	5. Each recording must replay in isolation — note seed-data assumptions in its header.
28
29	## Replay
30
31	1. Ask for the target coordinates of the migrated system.
32	2. Re-execute every recording; normalize the observation with the same rules; compare against the stored expectation.
33	3. Write `migration/characterization/REPORT.md`: one verdict per recording (`same` \| `different` with diff excerpt \| `unreachable`), plus summary counts. Every recording appears in the report — skipped is a verdict, not an omission.
34	4. For repeated replays (CI, per-carving checks), generate a replay suite following the composed stack skill's system-test conventions — an `-st` module with JUnit 5 for microprofile-server targets, zunit for java-cli-app, a zero-dependency single-file Java 25 runner when no stack skill is composed.
35
36	## Reconcile
37
38	A diff is a question, not a verdict: behavior changed — intended?
39
40	- Regression → the migrated system is fixed; the recording stays.
41	- Accepted change (e.g. a bug deliberately fixed) → update the recording and append a decision line to its header: `accepted: <name>, <YYYY-MM-DD> — <reason>`.
42
43	Never update a recording silently, and never decide yourself — acceptance is a human call, same rule as the clarifier.
44
45	## Rules
46
47	- Record actual observations, including apparently wrong ones — correctness judgments happen at reconcile time, by a human.
48	- Normalize before storing and before comparing; never diff raw observations.
49	- The suite encodes what is, not what should be — it is scaffolding with an expiry: retire recordings BC by BC as sbce specs and ears-tests encode intent, delete the rest with `migration/`.
50	- Recording exercises a real system: respect rate limits, stop on unexpected error storms, and never

People who liked this also viewed

6 picks

Type