fabric-performance-monitoring

Name: fabric-performance-monitoring
Author: PatrickGallucci

16stars1forks

How to install fabric-performance-monitoring?

patrickgallucci/fabric-skills/fabric-performance-monitoring

$npx -y skills add PatrickGallucci/fabric-skills --skill fabric-performance-monitoring

Installs into the current project.

Summary

Monitor and optimize Microsoft Fabric capacity, Spark compute, and workload performance. Use when asked to check capacity utilization, diagnose throttling (HTTP 430), monitor Spark VCore consumption, analyze CU usage, review Monitoring Hub jobs, query Fabric REST APIs for capacit

Files · 1

View on GitHub

SKILL.md

1	# Microsoft Fabric Performance Monitoring
2
3	Toolkit for monitoring, diagnosing, and optimizing Microsoft Fabric capacity and workload performance across Spark, Data Warehouse, Lakehouse, and Pipeline workloads.
4
5	## When to Use This Skill
6
7	- Checking Fabric capacity utilization or CU consumption
8	- Diagnosing throttling errors (HTTP 430 / TooManyRequestsForCapacity)
9	- Monitoring Spark VCore usage and concurrency limits
10	- Querying Fabric REST APIs for capacity and workspace health
11	- Generating capacity performance reports
12	- Tuning Spark resource profiles (readHeavy, writeHeavy, balanced)
13	- Investigating job failures in the Monitoring Hub
14	- Analyzing autoscale billing vs capacity-based billing
15	- Reviewing background vs interactive operation patterns
16	- Planning capacity SKU sizing or rightsizing
17
18	## Prerequisites
19
20	- PowerShell 7+ with Az.Fabric module installed
21	- Microsoft Entra ID app registration with Fabric API permissions
22	- Fabric Capacity Admin or Workspace Admin role
23	- Fabric Capacity Metrics app installed (for visual monitoring)
24
25	## Core Concepts
26
27	### Capacity Units and Spark VCores
28
29	One Capacity Unit (CU) equals two Apache Spark VCores. Fabric capacity is shared across all workspaces assigned to it, and Spark VCores are shared among notebooks, Spark job definitions, and lakehouses within those workspaces.
30
31	### Operation Types
32
33	Fabric classifies operations as interactive (on-demand, like DAX queries) or background (scheduled, like refreshes and Spark jobs). Background operations are smoothed over a 24-hour period. All Spark operations are background operations.
34
35	### Throttling Behavior
36
37	When capacity is fully utilized, new Spark jobs receive HTTP 430 with `TooManyRequestsForCapacity`. With queueing enabled, pipeline-triggered and scheduled jobs enter a FIFO queue and retry automatically when capacity becomes available.
38
39	### Capacity SKU Limits
40
41	\| SKU \| Spark VCores \| Queue Limit \|
42	\|------\|-------------\|-------------\|
43	\| F2 \| 4 \| 4 \|
44	\| F4 \| 8 \| 4 \|
45	\| F8 \| 16 \| 8 \|
46	\| F16 \| 32 \| 16 \|
47	\| F32 \| 64 \| 32 \|
48	\| F64 \| 128 \| 64 \|
49	\| F128 \| 256 \| 128 \|
50	\| F256 \| 512 \| 256 \|
51	\| F512 \| 1024 \| 512 \|
52
53	### Spark Resource Profiles
54
55	Fabric supports predefined Spark resource profiles for workload optimization. New workspaces default to `writeHeavy`. Available profiles: `readHeavy`, `writeHeavy`, `balanced`. When `writeHeavy` is used, VOrder is disabled by default and must be manually enabled.
56
57	## Step-by-Step Workflows
58
59	### Workflow 1: Capacity Health Check
60
61	Run the [capacity health check script](./scripts/Get-FabricCapacityHealth.ps1) to retrieve current capacity status, SKU details, and state.
62
63	```powershell
64	./scripts/Get-FabricCapacityHealth.ps1 -SubscriptionId "<sub-id>" -ResourceGroupName "<rg>" -CapacityName "<name>"
65	```
66
67	See [capacity-health-reference.md](./references/capacity-health-reference.md) for detailed API response schemas and interpretation guidance.
68
69	### Workflow 2: Spark Concurrency Analysis
70
71	Run the [Spark concurrency analyzer](./scripts/Get-FabricSparkConcurrency.ps1) to check active sessions, queued jobs, and throttling status.
72
73	```powershell
74	./scripts/Get-FabricSparkConcurrency.ps1 -WorkspaceId "<workspace-id>"
75	```
76
77	### Workflow 3: Monitoring Hub Job Audit
78
79	Run the [job audit script](./scripts/Get-FabricJobHistory.ps1) to retrieve recent job executions, durations, and failure details.
80
81	```powershell
82	./scripts/Get-FabricJobHistory.ps1 -WorkspaceId "<workspace-id>" -HoursBack 24
83	```
84
85	### Workflow 4: Generate Performance Report
86
87	Use the [performance report template](./templates/performance-report.sql) to query the SQL analytics endpoint for Lakehouse operation metrics, then generate a summary with the [report generator](./scripts/New-FabricPerformanceReport.ps1).
88
89	### Workflow 5: Autoscale vs Capacity Cost Analysis
90
91	See [cost-analysis-reference.md](./references/cost-analysis-reference.md) for guidance on comparing autoscale billing vs capacity-based models using Azure Cost Management.
92
93	## remediate
94
95	\| Symptom \| Likely Cause \| Resolution \|
96	\|---------\|-------------\|------------\|
97	\| HTTP 430 errors \| Capacity fully utilized \| Scale SKU, cancel idle sessions, enable queueing \|
98	\| Jobs stuck in queue \| All VCores consumed \| Check Monitoring Hub, stop idle notebooks \|
99	\| Slow Spark startup \| Using custom

People who liked this also viewed

6 picks

Type

fabric-performance-monitoring

16stars1forks

View on GitHub

How to install fabric-performance-monitoring?

patrickgallucci/fabric-skills/fabric-performance-monitoring

$npx -y skills add PatrickGallucci/fabric-skills --skill fabric-performance-monitoring

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Microsoft Fabric Performance Monitoring
2
3	Toolkit for monitoring, diagnosing, and optimizing Microsoft Fabric capacity and workload performance across Spark, Data Warehouse, Lakehouse, and Pipeline workloads.
4
5	## When to Use This Skill
6
7	- Checking Fabric capacity utilization or CU consumption
8	- Diagnosing throttling errors (HTTP 430 / TooManyRequestsForCapacity)
9	- Monitoring Spark VCore usage and concurrency limits
10	- Querying Fabric REST APIs for capacity and workspace health
11	- Generating capacity performance reports
12	- Tuning Spark resource profiles (readHeavy, writeHeavy, balanced)
13	- Investigating job failures in the Monitoring Hub
14	- Analyzing autoscale billing vs capacity-based billing
15	- Reviewing background vs interactive operation patterns
16	- Planning capacity SKU sizing or rightsizing
17
18	## Prerequisites
19
20	- PowerShell 7+ with Az.Fabric module installed
21	- Microsoft Entra ID app registration with Fabric API permissions
22	- Fabric Capacity Admin or Workspace Admin role
23	- Fabric Capacity Metrics app installed (for visual monitoring)
24
25	## Core Concepts
26
27	### Capacity Units and Spark VCores
28
29	One Capacity Unit (CU) equals two Apache Spark VCores. Fabric capacity is shared across all workspaces assigned to it, and Spark VCores are shared among notebooks, Spark job definitions, and lakehouses within those workspaces.
30
31	### Operation Types
32
33	Fabric classifies operations as interactive (on-demand, like DAX queries) or background (scheduled, like refreshes and Spark jobs). Background operations are smoothed over a 24-hour period. All Spark operations are background operations.
34
35	### Throttling Behavior
36
37	When capacity is fully utilized, new Spark jobs receive HTTP 430 with `TooManyRequestsForCapacity`. With queueing enabled, pipeline-triggered and scheduled jobs enter a FIFO queue and retry automatically when capacity becomes available.
38
39	### Capacity SKU Limits
40
41	\| SKU \| Spark VCores \| Queue Limit \|
42	\|------\|-------------\|-------------\|
43	\| F2 \| 4 \| 4 \|
44	\| F4 \| 8 \| 4 \|
45	\| F8 \| 16 \| 8 \|
46	\| F16 \| 32 \| 16 \|
47	\| F32 \| 64 \| 32 \|
48	\| F64 \| 128 \| 64 \|
49	\| F128 \| 256 \| 128 \|
50	\| F256 \| 512 \| 256 \|
51	\| F512 \| 1024 \| 512 \|
52
53	### Spark Resource Profiles
54
55	Fabric supports predefined Spark resource profiles for workload optimization. New workspaces default to `writeHeavy`. Available profiles: `readHeavy`, `writeHeavy`, `balanced`. When `writeHeavy` is used, VOrder is disabled by default and must be manually enabled.
56
57	## Step-by-Step Workflows
58
59	### Workflow 1: Capacity Health Check
60
61	Run the [capacity health check script](./scripts/Get-FabricCapacityHealth.ps1) to retrieve current capacity status, SKU details, and state.
62
63	```powershell
64	./scripts/Get-FabricCapacityHealth.ps1 -SubscriptionId "<sub-id>" -ResourceGroupName "<rg>" -CapacityName "<name>"
65	```
66
67	See [capacity-health-reference.md](./references/capacity-health-reference.md) for detailed API response schemas and interpretation guidance.
68
69	### Workflow 2: Spark Concurrency Analysis
70
71	Run the [Spark concurrency analyzer](./scripts/Get-FabricSparkConcurrency.ps1) to check active sessions, queued jobs, and throttling status.
72
73	```powershell
74	./scripts/Get-FabricSparkConcurrency.ps1 -WorkspaceId "<workspace-id>"
75	```
76
77	### Workflow 3: Monitoring Hub Job Audit
78
79	Run the [job audit script](./scripts/Get-FabricJobHistory.ps1) to retrieve recent job executions, durations, and failure details.
80
81	```powershell
82	./scripts/Get-FabricJobHistory.ps1 -WorkspaceId "<workspace-id>" -HoursBack 24
83	```
84
85	### Workflow 4: Generate Performance Report
86
87	Use the [performance report template](./templates/performance-report.sql) to query the SQL analytics endpoint for Lakehouse operation metrics, then generate a summary with the [report generator](./scripts/New-FabricPerformanceReport.ps1).
88
89	### Workflow 5: Autoscale vs Capacity Cost Analysis
90
91	See [cost-analysis-reference.md](./references/cost-analysis-reference.md) for guidance on comparing autoscale billing vs capacity-based models using Azure Cost Management.
92
93	## remediate
94
95	\| Symptom \| Likely Cause \| Resolution \|
96	\|---------\|-------------\|------------\|
97	\| HTTP 430 errors \| Capacity fully utilized \| Scale SKU, cancel idle sessions, enable queueing \|
98	\| Jobs stuck in queue \| All VCores consumed \| Check Monitoring Hub, stop idle notebooks \|
99	\| Slow Spark startup \| Using custom

People who liked this also viewed

6 picks

Type