newman-report-analyzer

Name: newman-report-analyzer
Author: LambdaTest

335stars67forks

How to install newman-report-analyzer?

lambdatest/agent-skills/newman-report-analyzer

$npx -y skills add LambdaTest/agent-skills --skill newman-report-analyzer

Installs into the current project.

Summary

Analyze Newman test run output, JSON reports, and JUnit XML results to summarize failures, diagnose root causes, identify slow requests, and suggest fixes. Use this skill whenever the user shares Newman output (terminal logs, JSON export, JUnit XML, or HTMLextra report data), wan

Files · 1

View on GitHub

SKILL.md

1	# Newman Report Analyzer
2
3	Analyze Newman run output in any format — terminal logs, JSON exports, JUnit XML — to give clear summaries, diagnose failures, spot slow requests, and recommend fixes.
4
5	---
6
7	## Supported Input Formats
8
9	\| Format \| How to get it \|
10	\|--------\|--------------\|
11	\| Terminal (CLI) output \| Copy/paste from terminal \|
12	\| JSON export \| `--reporter-json-export results.json` \|
13	\| JUnit XML \| `--reporter-junit-export results.xml` \|
14	\| HTMLextra data \| User describes or pastes key sections \|
15
16	Accept whichever format the user provides. Parse all available information.
17
18	---
19
20	## What to Extract and Report
21
22	### 1. Run Summary
23	Always lead with the high-level outcome:
24	- Total requests run
25	- Total assertions (tests) run
26	- ✅ Passed / ❌ Failed counts
27	- Total run duration
28	- Any skipped requests
29
30	### 2. Failed Tests
31	For each failure, report:
32	- Request name and folder path
33	- The `pm.test()` description that failed
34	- Expected vs. actual (if available in output)
35	- HTTP status code received
36	- Response time
37
38	### 3. Request-Level Errors (non-test failures)
39	Newman can fail at the request level before tests even run:
40	- Connection errors — ECONNREFUSED, ENOTFOUND, ETIMEDOUT
41	- SSL errors — cert validation failures
42	- Redirect issues
43	- Script errors — syntax errors in pre-request or test scripts
44
45	Distinguish these from test assertion failures.
46
47	### 4. Slow Requests
48	Flag any request exceeding a reasonable threshold (default: 2000ms unless user specifies). Report:
49	- Request name
50	- Response time
51	- Whether a `--timeout-request` would have killed it
52
53	### 5. Variable/Chaining Issues
54	Common symptom: a later request fails because a previous one didn't set an environment variable. Look for:
55	- `undefined` values in request URLs or bodies
56	- `{{variable}}` not replaced (appears literally in URLs)
57	- Tests that set variables followed by tests that fail with unexpected nulls
58
59	---
60
61	## Common Newman Error Patterns & Diagnoses
62
63	### `Error: ECONNREFUSED`
64	Cause: The server isn't running or the wrong port/host is configured.
65	Fix: Check `baseUrl` environment variable. Confirm the server is up.
66
67	### `Error: ENOTFOUND`
68	Cause: DNS resolution failed — hostname doesn't exist.
69	Fix: Check the URL for typos. Confirm environment is set correctly.
70
71	### `Error: connect ETIMEDOUT` / `Request timeout`
72	Cause: Server took too long to respond.
73	Fix: Increase `--timeout-request`, or investigate server-side latency.
74
75	### `AssertionError: expected 401 to equal 200`
76	Cause: Auth failed — token missing, expired, or wrong.
77	Fix: Check that the login/auth request ran first and set `{{authToken}}`. Check token expiry.
78
79	### `TypeError: Cannot read property 'X' of undefined`
80	Cause: `pm.response.json()` returned something unexpected — often an HTML error page or empty body.
81	Fix: Add a status code check before accessing body properties. Log `pm.response.text()` to see raw response.
82
83	### `Error: Request Failed: self signed certificate`
84	Cause: SSL cert is self-signed (common in staging/local).
85	Fix: Add `-k` / `--insecure` flag to Newman command.
86
87	### `{{variable}}` appearing literally in URL
88	Cause: The environment variable was never set or the wrong environment file was loaded.
89	Fix: Check `-e` flag points to correct environment. Verify the variable name matches exactly.
90
91	### `Script execution error`
92	Cause: Syntax error in a test or pre-request script.
93	Fix: Check the script in the failing request. Common issues: missing semicolons, unclosed brackets, referencing `pm.response.json()` when response has no body.
94
95	---
96
97	## JSON Report Structure (for reference)
98
99	Key fields in a Newman JSON export:
100	```json
101	{
102	"run": {
103	"stats": {
104	"requests": { "total": 10, "failed": 2 },
105	"assertions": { "total": 25, "failed": 3 }
106	},
107	"timings": {
108	"started": "...",
109	"completed": "...",
110	"responseAverage": 432
111	},
112	"executions": [
113	{
114	"item": { "name": "Get User" },
115	"response": { "responseTime": 312,

People who liked this also viewed

6 picks

Type

newman-report-analyzer

335stars67forks

View on GitHub

How to install newman-report-analyzer?

lambdatest/agent-skills/newman-report-analyzer

$npx -y skills add LambdaTest/agent-skills --skill newman-report-analyzer

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Newman Report Analyzer
2
3	Analyze Newman run output in any format — terminal logs, JSON exports, JUnit XML — to give clear summaries, diagnose failures, spot slow requests, and recommend fixes.
4
5	---
6
7	## Supported Input Formats
8
9	\| Format \| How to get it \|
10	\|--------\|--------------\|
11	\| Terminal (CLI) output \| Copy/paste from terminal \|
12	\| JSON export \| `--reporter-json-export results.json` \|
13	\| JUnit XML \| `--reporter-junit-export results.xml` \|
14	\| HTMLextra data \| User describes or pastes key sections \|
15
16	Accept whichever format the user provides. Parse all available information.
17
18	---
19
20	## What to Extract and Report
21
22	### 1. Run Summary
23	Always lead with the high-level outcome:
24	- Total requests run
25	- Total assertions (tests) run
26	- ✅ Passed / ❌ Failed counts
27	- Total run duration
28	- Any skipped requests
29
30	### 2. Failed Tests
31	For each failure, report:
32	- Request name and folder path
33	- The `pm.test()` description that failed
34	- Expected vs. actual (if available in output)
35	- HTTP status code received
36	- Response time
37
38	### 3. Request-Level Errors (non-test failures)
39	Newman can fail at the request level before tests even run:
40	- Connection errors — ECONNREFUSED, ENOTFOUND, ETIMEDOUT
41	- SSL errors — cert validation failures
42	- Redirect issues
43	- Script errors — syntax errors in pre-request or test scripts
44
45	Distinguish these from test assertion failures.
46
47	### 4. Slow Requests
48	Flag any request exceeding a reasonable threshold (default: 2000ms unless user specifies). Report:
49	- Request name
50	- Response time
51	- Whether a `--timeout-request` would have killed it
52
53	### 5. Variable/Chaining Issues
54	Common symptom: a later request fails because a previous one didn't set an environment variable. Look for:
55	- `undefined` values in request URLs or bodies
56	- `{{variable}}` not replaced (appears literally in URLs)
57	- Tests that set variables followed by tests that fail with unexpected nulls
58
59	---
60
61	## Common Newman Error Patterns & Diagnoses
62
63	### `Error: ECONNREFUSED`
64	Cause: The server isn't running or the wrong port/host is configured.
65	Fix: Check `baseUrl` environment variable. Confirm the server is up.
66
67	### `Error: ENOTFOUND`
68	Cause: DNS resolution failed — hostname doesn't exist.
69	Fix: Check the URL for typos. Confirm environment is set correctly.
70
71	### `Error: connect ETIMEDOUT` / `Request timeout`
72	Cause: Server took too long to respond.
73	Fix: Increase `--timeout-request`, or investigate server-side latency.
74
75	### `AssertionError: expected 401 to equal 200`
76	Cause: Auth failed — token missing, expired, or wrong.
77	Fix: Check that the login/auth request ran first and set `{{authToken}}`. Check token expiry.
78
79	### `TypeError: Cannot read property 'X' of undefined`
80	Cause: `pm.response.json()` returned something unexpected — often an HTML error page or empty body.
81	Fix: Add a status code check before accessing body properties. Log `pm.response.text()` to see raw response.
82
83	### `Error: Request Failed: self signed certificate`
84	Cause: SSL cert is self-signed (common in staging/local).
85	Fix: Add `-k` / `--insecure` flag to Newman command.
86
87	### `{{variable}}` appearing literally in URL
88	Cause: The environment variable was never set or the wrong environment file was loaded.
89	Fix: Check `-e` flag points to correct environment. Verify the variable name matches exactly.
90
91	### `Script execution error`
92	Cause: Syntax error in a test or pre-request script.
93	Fix: Check the script in the failing request. Common issues: missing semicolons, unclosed brackets, referencing `pm.response.json()` when response has no body.
94
95	---
96
97	## JSON Report Structure (for reference)
98
99	Key fields in a Newman JSON export:
100	```json
101	{
102	"run": {
103	"stats": {
104	"requests": { "total": 10, "failed": 2 },
105	"assertions": { "total": 25, "failed": 3 }
106	},
107	"timings": {
108	"started": "...",
109	"completed": "...",
110	"responseAverage": 432
111	},
112	"executions": [
113	{
114	"item": { "name": "Get User" },
115	"response": { "responseTime": 312,

People who liked this also viewed

6 picks

Type