fabric-udf-perf-remediate

Name: fabric-udf-perf-remediate
Author: PatrickGallucci

16stars1forks

How to install fabric-udf-perf-remediate?

patrickgallucci/fabric-skills/fabric-udf-perf-remediate

$npx -y skills add PatrickGallucci/fabric-skills --skill fabric-udf-perf-remediate

Installs into the current project.

Summary

Diagnose and resolve performance issues with Microsoft Fabric User Data Functions. Use when functions are slow, timing out, returning errors, consuming excessive capacity units, or exhibiting cold start latency. Covers execution timeouts, response size limits, connection bottlene

Files · 1

View on GitHub

SKILL.md

1	# Microsoft Fabric User Data Functions Performance remediate
2
3	Systematic guide for diagnosing and resolving performance issues with Fabric User Data Functions (UDFs). Covers cold starts, execution timeouts, capacity consumption, connection bottlenecks, and Python code optimization.
4
5	## When to Use This Skill
6
7	- Function invocations are slow or intermittently timing out
8	- Capacity metrics show unexpected CU consumption from UDF operations
9	- Functions fail with timeout, response size, or connection errors
10	- Cold start latency is impacting downstream consumers (Pipelines, Notebooks, Power BI)
11	- Historical logs show increasing duration trends
12	- Need to optimize UDF code for better performance within service limits
13
14	## Prerequisites
15
16	- Access to the Fabric portal with permissions on the User Data Functions item
17	- Microsoft Fabric Capacity Metrics app installed (for CU analysis)
18	- Python 3.11+ locally (for code profiling outside Fabric)
19	- PowerShell 7+ (for running diagnostic scripts)
20
21	## Service Limits Quick Reference
22
23	\| Limit \| Value \| Impact \|
24	\| --------------------- \| ----------- \| -------------------------------- \|
25	\| Request payload \| 4 MB \| All input parameters combined \|
26	\| Execution timeout \| 240 seconds \| Maximum function runtime \|
27	\| Response size \| 30 MB \| Maximum return value size \|
28	\| Log retention \| 30 days \| Historical invocation log window \|
29	\| Private library max \| 28.6 MB \| Per `.whl` file upload \|
30	\| Test session timeout \| 15 minutes \| Idle timeout in Develop mode \|
31	\| Daily log ingestion \| 250 MB \| Logs may be sampled beyond this \|
32	\| Python version (Run) \| 3.11 \| Published functions runtime \|
33	\| Python version (Test) \| 3.12 \| Develop mode test runtime \|
34
35	## Step-by-Step remediate Workflow
36
37	### Step 1: Identify the Symptom
38
39	Determine which category your issue falls into:
40
41	\| Symptom \| Likely Root Cause \| Go To \|
42	\| -------------------------------------- \| ----------------------------------------- \| ------ \|
43	\| First invocation slow, subsequent fast \| Cold start / initialization \| Step 2 \|
44	\| All invocations consistently slow \| Code inefficiency or data volume \| Step 3 \|
45	\| Intermittent timeouts \| Connection issues or capacity throttling \| Step 4 \|
46	\| Response too large error \| Unbounded query results \| Step 5 \|
47	\| High CU consumption in Metrics app \| Excessive execution frequency or duration \| Step 6 \|
48	\| Function fails with import errors \| Library loading overhead \| Step 7 \|
49
50	### Step 2: Diagnose Cold Start Latency
51
52	Fabric User Data Functions run in a serverless environment. The first invocation after a period of inactivity incurs initialization overhead.
53
54	Check historical logs for the pattern:
55
56	1. Switch to Run only mode in the Functions portal
57	2. Open View historical log for the target function
58	3. Compare Duration(ms) of the first invocation vs. subsequent ones
59	4. A 3-10x difference confirms cold start behavior
60
61	Mitigations:
62
63	- Implement a health-check or warm-up invocation on a schedule via Pipeline
64	- Minimize top-level imports; use lazy imports for heavy libraries
65	- Reduce private library count and size (each `.whl` adds init time)
66	- Keep PyPI dependency list minimal in `definition.json`
67
68	### Step 3: Profile Slow Function Code
69
70	For consistently slow functions, instrument your code with timing:
71
72	```python
73	import logging
74	import time
75
76	@udf.function()
77	def my_function(param: str) -> str:
78	start = time.perf_counter()
79
80	# Phase 1: Data retrieval
81	t1 = time.perf_counter()
82	data = fetch_data(param)
83	logging.info(f"Data retrieval: {time.perf_counter() - t1:.3f}s")
84
85	# Phase 2: Processing
86	t2 = time.perf_counter()
87	result = process(data)
88	logging.info(f"Processing: {time.perf_counter() - t2:.3f}s")
89
90	logging.info(f"Total execution: {time.perf_counter() - start:.3f}s")
91	return result
92	```
93
94	Review logs in the Invocation details pane to identify the slowest phase.
95
96	Common bottlenecks and fixes:
97
98	- Data source queries: Add WHERE clauses, limit columns, use parameterized queries
99	- DataFrame operations: Filter early, avoid iterrows(), use vectorized operations
100	- Serialization: Return only required fields, use compact formats
101	- **External API ca

People who liked this also viewed

6 picks

Type

fabric-udf-perf-remediate

16stars1forks

View on GitHub

How to install fabric-udf-perf-remediate?

patrickgallucci/fabric-skills/fabric-udf-perf-remediate

$npx -y skills add PatrickGallucci/fabric-skills --skill fabric-udf-perf-remediate

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Microsoft Fabric User Data Functions Performance remediate
2
3	Systematic guide for diagnosing and resolving performance issues with Fabric User Data Functions (UDFs). Covers cold starts, execution timeouts, capacity consumption, connection bottlenecks, and Python code optimization.
4
5	## When to Use This Skill
6
7	- Function invocations are slow or intermittently timing out
8	- Capacity metrics show unexpected CU consumption from UDF operations
9	- Functions fail with timeout, response size, or connection errors
10	- Cold start latency is impacting downstream consumers (Pipelines, Notebooks, Power BI)
11	- Historical logs show increasing duration trends
12	- Need to optimize UDF code for better performance within service limits
13
14	## Prerequisites
15
16	- Access to the Fabric portal with permissions on the User Data Functions item
17	- Microsoft Fabric Capacity Metrics app installed (for CU analysis)
18	- Python 3.11+ locally (for code profiling outside Fabric)
19	- PowerShell 7+ (for running diagnostic scripts)
20
21	## Service Limits Quick Reference
22
23	\| Limit \| Value \| Impact \|
24	\| --------------------- \| ----------- \| -------------------------------- \|
25	\| Request payload \| 4 MB \| All input parameters combined \|
26	\| Execution timeout \| 240 seconds \| Maximum function runtime \|
27	\| Response size \| 30 MB \| Maximum return value size \|
28	\| Log retention \| 30 days \| Historical invocation log window \|
29	\| Private library max \| 28.6 MB \| Per `.whl` file upload \|
30	\| Test session timeout \| 15 minutes \| Idle timeout in Develop mode \|
31	\| Daily log ingestion \| 250 MB \| Logs may be sampled beyond this \|
32	\| Python version (Run) \| 3.11 \| Published functions runtime \|
33	\| Python version (Test) \| 3.12 \| Develop mode test runtime \|
34
35	## Step-by-Step remediate Workflow
36
37	### Step 1: Identify the Symptom
38
39	Determine which category your issue falls into:
40
41	\| Symptom \| Likely Root Cause \| Go To \|
42	\| -------------------------------------- \| ----------------------------------------- \| ------ \|
43	\| First invocation slow, subsequent fast \| Cold start / initialization \| Step 2 \|
44	\| All invocations consistently slow \| Code inefficiency or data volume \| Step 3 \|
45	\| Intermittent timeouts \| Connection issues or capacity throttling \| Step 4 \|
46	\| Response too large error \| Unbounded query results \| Step 5 \|
47	\| High CU consumption in Metrics app \| Excessive execution frequency or duration \| Step 6 \|
48	\| Function fails with import errors \| Library loading overhead \| Step 7 \|
49
50	### Step 2: Diagnose Cold Start Latency
51
52	Fabric User Data Functions run in a serverless environment. The first invocation after a period of inactivity incurs initialization overhead.
53
54	Check historical logs for the pattern:
55
56	1. Switch to Run only mode in the Functions portal
57	2. Open View historical log for the target function
58	3. Compare Duration(ms) of the first invocation vs. subsequent ones
59	4. A 3-10x difference confirms cold start behavior
60
61	Mitigations:
62
63	- Implement a health-check or warm-up invocation on a schedule via Pipeline
64	- Minimize top-level imports; use lazy imports for heavy libraries
65	- Reduce private library count and size (each `.whl` adds init time)
66	- Keep PyPI dependency list minimal in `definition.json`
67
68	### Step 3: Profile Slow Function Code
69
70	For consistently slow functions, instrument your code with timing:
71
72	```python
73	import logging
74	import time
75
76	@udf.function()
77	def my_function(param: str) -> str:
78	start = time.perf_counter()
79
80	# Phase 1: Data retrieval
81	t1 = time.perf_counter()
82	data = fetch_data(param)
83	logging.info(f"Data retrieval: {time.perf_counter() - t1:.3f}s")
84
85	# Phase 2: Processing
86	t2 = time.perf_counter()
87	result = process(data)
88	logging.info(f"Processing: {time.perf_counter() - t2:.3f}s")
89
90	logging.info(f"Total execution: {time.perf_counter() - start:.3f}s")
91	return result
92	```
93
94	Review logs in the Invocation details pane to identify the slowest phase.
95
96	Common bottlenecks and fixes:
97
98	- Data source queries: Add WHERE clauses, limit columns, use parameterized queries
99	- DataFrame operations: Filter early, avoid iterrows(), use vectorized operations
100	- Serialization: Return only required fields, use compact formats
101	- **External API ca

People who liked this also viewed

6 picks

Type