journey-test-harness

Name: journey-test-harness
Author: DanWahlin

36stars10forks

How to install journey-test-harness?

danwahlin/github-azure-agentic-journeys/journey-test-harness

$npx -y skills add DanWahlin/github-azure-agentic-journeys --skill journey-test-harness

Installs into the current project.

Summary

Run multiple journeys as a cross-platform test suite. Discover journeys, invoke journey-runner in isolated workspaces, deploy, verify, capture screenshots, clean up only owned Azure resources, and produce a consolidated report. USE FOR: test all journeys, regression test, CI jour

Files · 1

View on GitHub

SKILL.md

1	# Journey Test Harness
2
3	Orchestrate `journey-runner` across the selected journey directories. The harness must work on Windows, macOS, and Linux and must never delete an Azure environment merely because it appears in `azd env list`.
4
5	## Inputs
6
7	1. Journeys, default `all`
8	2. Stack, default Node.js/TypeScript for multi-stack journeys
9	3. Deploy, default `true`
10	4. Location, default `westus`
11	5. Concurrency, default `1`; increase only when quota, local ports, and API limits allow it
12
13	Each deployed journey uses `cleanup: after-verification` unless the user explicitly requests otherwise.
14
15	## Step 1: Discover Journeys
16
17	Use repository file-search APIs or Node.js `fs.readdir()` to find `journeys/*/README.md`. Do not use `ls \| sed`, shell globs, or platform-specific path parsing.
18
19	For each journey, record:
20
21	- Folder name and absolute source path
22	- Journey type
23	- `PLAN.md` presence
24	- Required and optional tools from the journey prerequisite section
25	- Estimated cost and time
26	- Screenshot requirement
27
28	Filter only against the discovered folder names. Fail early for an unknown requested journey.
29
30	## Step 2: Suite Preflight
31
32	Load `journey-runner` and run its cross-platform preflight before creating any workspace or Azure resource.
33
34	The union of selected-journey requirements may include:
35
36	- Node.js 24 LTS or later
37	- Azure CLI and valid authentication
38	- `azd` 1.28.0 or later with `auth.useAzCliAuth=true`
39	- GitHub Copilot CLI
40	- Node.js 24 LTS or later for hooks, tests, and verifiers
41	- Azure Functions Core Tools v4, Azurite, and Go-based `sqlcmd` for SmartTodo
42	- The pinned Playwright package and bundled Chromium for web screenshots
43	- Xcode 16+ only when SmartTodo iOS execution is requested on macOS
44
45	Use `.github/skills/journey-runner/scripts/check-prerequisites.mjs` with the union of required tools. Missing required tools stop the whole suite before provider registration. Do not install system tools during the suite.
46
47	## Step 3: Azure Preparation
48
49	Check Azure CLI and `azd` authentication separately. Confirm the intended subscription and location.
50
51	Register only providers required by selected journeys. Run provider commands as individual processes with argument arrays, not shell loops. Wait for required registrations before starting the first deployment.
52
53	Record the subscription ID and each provider's state in the suite report.
54
55	## Step 4: Create Suite and Journey Workspaces
56
57	Create directories through Node.js filesystem APIs or the active agent's file tools:
58
59	```text
60	<journey-runs-root>/test-suite-<UTC timestamp>/
61	├── test-report.md
62	├── screenshots/
63	└── runs/
64	└── <journey>-<UTC timestamp>/
65	```
66
67	Use `path.join()` and absolute paths. Don't embed `~/`, `date`, `mkdir`, or shell-specific separators in executable instructions.
68
69	Copy `PLAN.md` with `fs.copyFile()` when needed. Generated application code, infrastructure, logs, and secrets stay inside the isolated run directory, never the source repository.
70
71	## Step 5: Run Each Journey
72
73	Invoke `journey-runner` with:
74
75	```text
76	Journey: <journey-source-path>
77	Stack: <stack>
78	Location: <location>
79	Working directory: <absolute-run-directory>
80	Cleanup: after-verification
81	```
82
83	The runner owns per-journey preflight, prompts, local ports, deployment, verification, screenshots, and scoped cleanup.
84
85	Run serially by default. Parallel runs require:
86
87	- Distinct local ports
88	- Distinct `azd` environment names
89	- Distinct resource groups
90	- Sufficient regional quota
91	- Independent log and screenshot paths
92
93	Never share an `azd` environment across journeys.
94
95	## Step 6: Preserve Evidence
96
97	Before removing a run directory, copy these artifacts with Node.js filesystem APIs:
98
99	- `run-report.md`
100	- `issues.md`
101	- `screenshot-*.png`
102	- Sanitized deployment and verification logs
103
104	Use the mapping below:
105
106	\| Journey \| Web output \| Screenshot \|
107	\|---\|---\|---\|
108	\| Grafana \| `GRAFANA_URL` \| Login page \|
109	\| n8n \| `N8N_URL` \| Owner-setup or login page \|
110	\| Superset \| `SUPERSET_URL` \| Authenticated welcome page when credentials are available \|
111	\| AIMarket \| `WEB_URL` \| Storefront with all images loaded \|
112	\| SmartTodo \| None \| No screenshot on Windows/Linux; simulator screenshot only on a suitable macOS/Xcode host \|
113
114	Use journey-runner's pinned Playwright Chromium helper. Never use a branded Chrome channel.
115
116	## Step 7: Guaranteed Scoped Cleanup
117
118	Cleanup runs in a `finally` path after each deployment attempt, including build, deploym

People who liked this also viewed

6 picks

Type

journey-test-harness

36stars10forks

View on GitHub

How to install journey-test-harness?

danwahlin/github-azure-agentic-journeys/journey-test-harness

$npx -y skills add DanWahlin/github-azure-agentic-journeys --skill journey-test-harness

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Journey Test Harness
2
3	Orchestrate `journey-runner` across the selected journey directories. The harness must work on Windows, macOS, and Linux and must never delete an Azure environment merely because it appears in `azd env list`.
4
5	## Inputs
6
7	1. Journeys, default `all`
8	2. Stack, default Node.js/TypeScript for multi-stack journeys
9	3. Deploy, default `true`
10	4. Location, default `westus`
11	5. Concurrency, default `1`; increase only when quota, local ports, and API limits allow it
12
13	Each deployed journey uses `cleanup: after-verification` unless the user explicitly requests otherwise.
14
15	## Step 1: Discover Journeys
16
17	Use repository file-search APIs or Node.js `fs.readdir()` to find `journeys/*/README.md`. Do not use `ls \| sed`, shell globs, or platform-specific path parsing.
18
19	For each journey, record:
20
21	- Folder name and absolute source path
22	- Journey type
23	- `PLAN.md` presence
24	- Required and optional tools from the journey prerequisite section
25	- Estimated cost and time
26	- Screenshot requirement
27
28	Filter only against the discovered folder names. Fail early for an unknown requested journey.
29
30	## Step 2: Suite Preflight
31
32	Load `journey-runner` and run its cross-platform preflight before creating any workspace or Azure resource.
33
34	The union of selected-journey requirements may include:
35
36	- Node.js 24 LTS or later
37	- Azure CLI and valid authentication
38	- `azd` 1.28.0 or later with `auth.useAzCliAuth=true`
39	- GitHub Copilot CLI
40	- Node.js 24 LTS or later for hooks, tests, and verifiers
41	- Azure Functions Core Tools v4, Azurite, and Go-based `sqlcmd` for SmartTodo
42	- The pinned Playwright package and bundled Chromium for web screenshots
43	- Xcode 16+ only when SmartTodo iOS execution is requested on macOS
44
45	Use `.github/skills/journey-runner/scripts/check-prerequisites.mjs` with the union of required tools. Missing required tools stop the whole suite before provider registration. Do not install system tools during the suite.
46
47	## Step 3: Azure Preparation
48
49	Check Azure CLI and `azd` authentication separately. Confirm the intended subscription and location.
50
51	Register only providers required by selected journeys. Run provider commands as individual processes with argument arrays, not shell loops. Wait for required registrations before starting the first deployment.
52
53	Record the subscription ID and each provider's state in the suite report.
54
55	## Step 4: Create Suite and Journey Workspaces
56
57	Create directories through Node.js filesystem APIs or the active agent's file tools:
58
59	```text
60	<journey-runs-root>/test-suite-<UTC timestamp>/
61	├── test-report.md
62	├── screenshots/
63	└── runs/
64	└── <journey>-<UTC timestamp>/
65	```
66
67	Use `path.join()` and absolute paths. Don't embed `~/`, `date`, `mkdir`, or shell-specific separators in executable instructions.
68
69	Copy `PLAN.md` with `fs.copyFile()` when needed. Generated application code, infrastructure, logs, and secrets stay inside the isolated run directory, never the source repository.
70
71	## Step 5: Run Each Journey
72
73	Invoke `journey-runner` with:
74
75	```text
76	Journey: <journey-source-path>
77	Stack: <stack>
78	Location: <location>
79	Working directory: <absolute-run-directory>
80	Cleanup: after-verification
81	```
82
83	The runner owns per-journey preflight, prompts, local ports, deployment, verification, screenshots, and scoped cleanup.
84
85	Run serially by default. Parallel runs require:
86
87	- Distinct local ports
88	- Distinct `azd` environment names
89	- Distinct resource groups
90	- Sufficient regional quota
91	- Independent log and screenshot paths
92
93	Never share an `azd` environment across journeys.
94
95	## Step 6: Preserve Evidence
96
97	Before removing a run directory, copy these artifacts with Node.js filesystem APIs:
98
99	- `run-report.md`
100	- `issues.md`
101	- `screenshot-*.png`
102	- Sanitized deployment and verification logs
103
104	Use the mapping below:
105
106	\| Journey \| Web output \| Screenshot \|
107	\|---\|---\|---\|
108	\| Grafana \| `GRAFANA_URL` \| Login page \|
109	\| n8n \| `N8N_URL` \| Owner-setup or login page \|
110	\| Superset \| `SUPERSET_URL` \| Authenticated welcome page when credentials are available \|
111	\| AIMarket \| `WEB_URL` \| Storefront with all images loaded \|
112	\| SmartTodo \| None \| No screenshot on Windows/Linux; simulator screenshot only on a suitable macOS/Xcode host \|
113
114	Use journey-runner's pinned Playwright Chromium helper. Never use a branded Chrome channel.
115
116	## Step 7: Guaranteed Scoped Cleanup
117
118	Cleanup runs in a `finally` path after each deployment attempt, including build, deploym

People who liked this also viewed

6 picks

Type