dotnet-ci-benchmarking

Name: dotnet-ci-benchmarking
Author: wshaddix

65stars10forks

How to install dotnet-ci-benchmarking?

wshaddix/dotnet-skills/dotnet-ci-benchmarking

$npx -y skills add wshaddix/dotnet-skills --skill dotnet-ci-benchmarking

Installs into the current project.

Summary

Gating CI on perf regressions. Automated threshold alerts, baseline tracking, trend reports.

Files · 1

View on GitHub

SKILL.md

1	# dotnet-ci-benchmarking
2
3	Continuous benchmarking guidance for detecting performance regressions in CI pipelines. Covers baseline file management with BenchmarkDotNet JSON exporters, GitHub Actions workflows for artifact-based baseline comparison, regression detection patterns with configurable thresholds, and alerting strategies for performance degradation.
4
5	Version assumptions: BenchmarkDotNet v0.14+ for JSON export, GitHub Actions runner environment. Examples use `actions/upload-artifact@v4` and `actions/download-artifact@v4`.
6
7	Out of scope: BenchmarkDotNet setup, benchmark class design, memory diagnosers, and common pitfalls are owned by this epic's companion skill -- see [skill:dotnet-benchmarkdotnet]. Performance-oriented architecture patterns are owned by [skill:dotnet-performance-patterns]. Profiling tools (dotnet-counters, dotnet-trace, dotnet-dump) are covered by `dotnet-profiling`. OpenTelemetry metrics collection and distributed tracing -- see [skill:dotnet-observability]. Composable CI/CD workflow design and matrix build strategies -- see [skill:dotnet-gha-patterns]. Architecture patterns (caching, resilience) -- see [skill:dotnet-architecture-patterns].
8
9	Cross-references: [skill:dotnet-benchmarkdotnet] for benchmark class setup and JSON exporter configuration, [skill:dotnet-observability] for correlating benchmark regressions with runtime metrics changes, [skill:dotnet-gha-patterns] for composable workflow patterns (reusable workflows, composite actions, matrix builds).
10
11	---
12
13	## Baseline File Management
14
15	### BenchmarkDotNet JSON Export
16
17	BenchmarkDotNet's JSON exporter produces machine-readable results for automated comparison. Configure the exporter in benchmark classes:
18
19	```csharp
20	using BenchmarkDotNet.Attributes;
21	using BenchmarkDotNet.Exporters.Json;
22
23	[JsonExporterAttribute.Full]
24	[MemoryDiagnoser]
25	public class CriticalPathBenchmarks
26	{
27	[Benchmark(Baseline = true)]
28	public void ProcessOrder() { /* ... */ }
29
30	[Benchmark]
31	public void ProcessOrderOptimized() { /* ... */ }
32	}
33	```
34
35	Or configure via custom config for all benchmark classes:
36
37	```csharp
38	using BenchmarkDotNet.Configs;
39	using BenchmarkDotNet.Exporters.Json;
40	using BenchmarkDotNet.Jobs;
41	using BenchmarkDotNet.Running;
42
43	var config = ManualConfig.Create(DefaultConfig.Instance)
44	.AddJob(Job.ShortRun) // fewer iterations for CI speed
45	.AddExporter(JsonExporter.Full)
46	.WithArtifactsPath("./benchmark-results");
47
48	BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args, config);
49	```
50
51	### JSON Export Structure
52
53	The exported JSON file (`*-report-full.json`) contains structured benchmark results:
54
55	```json
56	{
57	"Title": "CriticalPathBenchmarks",
58	"Benchmarks": [
59	{
60	"FullName": "MyApp.Benchmarks.CriticalPathBenchmarks.ProcessOrder",
61	"Statistics": {
62	"Mean": 1234.5678,
63	"Median": 1230.1234,
64	"StandardDeviation": 15.234,
65	"StandardError": 4.812
66	},
67	"Memory": {
68	"BytesAllocatedPerOperation": 1024,
69	"Gen0Collections": 0.0012,
70	"Gen1Collections": 0,
71	"Gen2Collections": 0
72	}
73	}
74	]
75	}
76	```
77
78	Key fields for regression comparison:
79
80	\| Field \| Purpose \|
81	\|-------\|---------\|
82	\| `Statistics.Mean` \| Average execution time (nanoseconds) \|
83	\| `Statistics.Median` \| Middle execution time (more robust to outliers) \|
84	\| `Statistics.StandardDeviation` \| Measurement variability \|
85	\| `Memory.BytesAllocatedPerOperation` \| GC allocation per operation \|
86
87	### Baseline Storage Strategies
88
89	\| Strategy \| Pros \| Cons \| Best For \|
90	\|----------\|------\|------\|----------\|
91	\| Git-committed baseline file \| Versioned, auditable, no external deps \| Repo size grows; must update deliberately \| Small benchmark suites, stable hardware \|
92	\| GitHub Actions artifacts \| No repo bloat; automatic retention \| 90-day default retention; cross-workflow access requires tokens \| Large benchmark suites, shared runners \|
93	\| External storage (S3/Azure Blob) \| Unlimited history; cross-repo sharing \| Extra infrastructure; credential management \| Multi-repo benchmark comparison \|
94
95	This skill focuses on the GitHub Actions artifact strategy as the default. For composable workflow patterns and reusable actions, see [skill:dotnet-gha-patterns].
96
97	---
98
99	## GitHub Actions Benchmark Workflow
100
101	### Basic Benchmark Workflow
102
103	```yaml
104	name: Benchmarks
105
106	on:
107	pull_request:
108	paths:
109	- 'src/**'
110	- 'benchmarks/**'
111	workflow_dispatch:
112
113	permissions:
114	contents: read
115	actions: read # required for artifact download
116
117	jobs:
118	benchmark:
119	runs-on: ubuntu-latest
120	steps:
121	- uses: actions/checkout@v4
122
123	- name: Setup .NET
124	uses: actions/setup-dotnet@v4
125	with:
126	dotnet-version: '8.0.x'
127
128	- name: Run benchmarks
129	run: dotnet run -c Release --project benchmarks/MyBenchmarks.csproj -- --exporters json
130
131	- name: U

wshaddixPublisher · 77 listings

Preview

wshaddix/dotnet-skills

# dotnet-ci-benchmarking

Continuous benchmarking guidance for detecting performance regressions in CI pipelines. Covers baseline file management with BenchmarkDotNet JSON exporters, Git

**Version assumptions:** BenchmarkDotNet v0.14+ for JSON export, GitHub Actions runner environment. Examples use `actions/upload-artifact@v4` and `actions/downl

**Out of scope:** BenchmarkDotNet setup, benchmark class design, memory diagnosers, and common pitfalls are owned by this epic's companion skill -- see [skill:d

Repowshaddix/dotnet-skills

License—

TypeSkills

CategoryTesting

UpdatedFeb 2026

Repo stars65

First seenJul 22, 2026

People who liked this also viewed

6 picks

Type

dotnet-ci-benchmarking

65stars10forks

View on GitHub

How to install dotnet-ci-benchmarking?

wshaddix/dotnet-skills/dotnet-ci-benchmarking

$npx -y skills add wshaddix/dotnet-skills --skill dotnet-ci-benchmarking

Installs into the current project.

Summary

Gating CI on perf regressions. Automated threshold alerts, baseline tracking, trend reports.

Files · 1

View on GitHub

SKILL.md

1	# dotnet-ci-benchmarking
2
3	Continuous benchmarking guidance for detecting performance regressions in CI pipelines. Covers baseline file management with BenchmarkDotNet JSON exporters, GitHub Actions workflows for artifact-based baseline comparison, regression detection patterns with configurable thresholds, and alerting strategies for performance degradation.
4
5	Version assumptions: BenchmarkDotNet v0.14+ for JSON export, GitHub Actions runner environment. Examples use `actions/upload-artifact@v4` and `actions/download-artifact@v4`.
6
7	Out of scope: BenchmarkDotNet setup, benchmark class design, memory diagnosers, and common pitfalls are owned by this epic's companion skill -- see [skill:dotnet-benchmarkdotnet]. Performance-oriented architecture patterns are owned by [skill:dotnet-performance-patterns]. Profiling tools (dotnet-counters, dotnet-trace, dotnet-dump) are covered by `dotnet-profiling`. OpenTelemetry metrics collection and distributed tracing -- see [skill:dotnet-observability]. Composable CI/CD workflow design and matrix build strategies -- see [skill:dotnet-gha-patterns]. Architecture patterns (caching, resilience) -- see [skill:dotnet-architecture-patterns].
8
9	Cross-references: [skill:dotnet-benchmarkdotnet] for benchmark class setup and JSON exporter configuration, [skill:dotnet-observability] for correlating benchmark regressions with runtime metrics changes, [skill:dotnet-gha-patterns] for composable workflow patterns (reusable workflows, composite actions, matrix builds).
10
11	---
12
13	## Baseline File Management
14
15	### BenchmarkDotNet JSON Export
16
17	BenchmarkDotNet's JSON exporter produces machine-readable results for automated comparison. Configure the exporter in benchmark classes:
18
19	```csharp
20	using BenchmarkDotNet.Attributes;
21	using BenchmarkDotNet.Exporters.Json;
22
23	[JsonExporterAttribute.Full]
24	[MemoryDiagnoser]
25	public class CriticalPathBenchmarks
26	{
27	[Benchmark(Baseline = true)]
28	public void ProcessOrder() { /* ... */ }
29
30	[Benchmark]
31	public void ProcessOrderOptimized() { /* ... */ }
32	}
33	```
34
35	Or configure via custom config for all benchmark classes:
36
37	```csharp
38	using BenchmarkDotNet.Configs;
39	using BenchmarkDotNet.Exporters.Json;
40	using BenchmarkDotNet.Jobs;
41	using BenchmarkDotNet.Running;
42
43	var config = ManualConfig.Create(DefaultConfig.Instance)
44	.AddJob(Job.ShortRun) // fewer iterations for CI speed
45	.AddExporter(JsonExporter.Full)
46	.WithArtifactsPath("./benchmark-results");
47
48	BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args, config);
49	```
50
51	### JSON Export Structure
52
53	The exported JSON file (`*-report-full.json`) contains structured benchmark results:
54
55	```json
56	{
57	"Title": "CriticalPathBenchmarks",
58	"Benchmarks": [
59	{
60	"FullName": "MyApp.Benchmarks.CriticalPathBenchmarks.ProcessOrder",
61	"Statistics": {
62	"Mean": 1234.5678,
63	"Median": 1230.1234,
64	"StandardDeviation": 15.234,
65	"StandardError": 4.812
66	},
67	"Memory": {
68	"BytesAllocatedPerOperation": 1024,
69	"Gen0Collections": 0.0012,
70	"Gen1Collections": 0,
71	"Gen2Collections": 0
72	}
73	}
74	]
75	}
76	```
77
78	Key fields for regression comparison:
79
80	\| Field \| Purpose \|
81	\|-------\|---------\|
82	\| `Statistics.Mean` \| Average execution time (nanoseconds) \|
83	\| `Statistics.Median` \| Middle execution time (more robust to outliers) \|
84	\| `Statistics.StandardDeviation` \| Measurement variability \|
85	\| `Memory.BytesAllocatedPerOperation` \| GC allocation per operation \|
86
87	### Baseline Storage Strategies
88
89	\| Strategy \| Pros \| Cons \| Best For \|
90	\|----------\|------\|------\|----------\|
91	\| Git-committed baseline file \| Versioned, auditable, no external deps \| Repo size grows; must update deliberately \| Small benchmark suites, stable hardware \|
92	\| GitHub Actions artifacts \| No repo bloat; automatic retention \| 90-day default retention; cross-workflow access requires tokens \| Large benchmark suites, shared runners \|
93	\| External storage (S3/Azure Blob) \| Unlimited history; cross-repo sharing \| Extra infrastructure; credential management \| Multi-repo benchmark comparison \|
94
95	This skill focuses on the GitHub Actions artifact strategy as the default. For composable workflow patterns and reusable actions, see [skill:dotnet-gha-patterns].
96
97	---
98
99	## GitHub Actions Benchmark Workflow
100
101	### Basic Benchmark Workflow
102
103	```yaml
104	name: Benchmarks
105
106	on:
107	pull_request:
108	paths:
109	- 'src/**'
110	- 'benchmarks/**'
111	workflow_dispatch:
112
113	permissions:
114	contents: read
115	actions: read # required for artifact download
116
117	jobs:
118	benchmark:
119	runs-on: ubuntu-latest
120	steps:
121	- uses: actions/checkout@v4
122
123	- name: Setup .NET
124	uses: actions/setup-dotnet@v4
125	with:
126	dotnet-version: '8.0.x'
127
128	- name: Run benchmarks
129	run: dotnet run -c Release --project benchmarks/MyBenchmarks.csproj -- --exporters json
130
131	- name: U

wshaddixPublisher · 77 listings

Preview

wshaddix/dotnet-skills

# dotnet-ci-benchmarking

Continuous benchmarking guidance for detecting performance regressions in CI pipelines. Covers baseline file management with BenchmarkDotNet JSON exporters, Git

**Version assumptions:** BenchmarkDotNet v0.14+ for JSON export, GitHub Actions runner environment. Examples use `actions/upload-artifact@v4` and `actions/downl

**Out of scope:** BenchmarkDotNet setup, benchmark class design, memory diagnosers, and common pitfalls are owned by this epic's companion skill -- see [skill:d

Repowshaddix/dotnet-skills

License—

TypeSkills

CategoryTesting

UpdatedFeb 2026

Repo stars65

First seenJul 22, 2026

People who liked this also viewed

6 picks

Type