notebook-to-production-hardening

Name: notebook-to-production-hardening
Author: vaquarkhan

19stars4forks

How to install notebook-to-production-hardening?

vaquarkhan/data-engineering-agent-skills/notebook-to-production-hardening

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill notebook-to-production-hardening

Installs into the current project.

Summary

Guides agents through converting exploratory notebooks into production-ready data jobs. Use when operationalizing notebooks from Databricks, Jupyter, or similar environments into tested, packaged, repeatable workflows.

Files · 1

View on GitHub

SKILL.md

1	# Notebook To Production Hardening
2
3	## Overview
4
5	Use this skill when a notebook has outgrown exploration and needs to become a maintainable delivery artifact. It helps agents separate experimentation from production packaging, testing, configuration, and orchestration.
6
7	## When to Use
8
9	- moving notebook logic into scheduled jobs
10	- hardening `Databricks` or `Jupyter` notebooks for repeated use
11	- extracting reusable logic from cells into modules or packages
12	- improving testability and deployment discipline
13
14	Do not treat a manually rerun notebook as production just because it worked once.
15
16	## Workflow
17
18	1. Separate exploratory work from production logic.
19	Identify:
20	- reusable transformation code
21	- parameters
22	- environment assumptions
23	- manual steps
24
25	2. Extract logic into versioned, testable units.
26
27	3. Replace hidden state with explicit inputs and configuration.
28
29	4. Add validation and operational hooks.
30	Include:
31	- contracts
32	- logging
33	- error handling
34	- retry-safe outputs
35
36	5. Define how the job is deployed and monitored.
37
38	## Common Rationalizations
39
40	\| Rationalization \| Reality \|
41	\| --- \| --- \|
42	\| "The notebook already works." \| Interactive success does not mean repeatable, testable, or observable production behavior. \|
43	\| "We can keep using widgets and manual edits." \| Hidden runtime state makes failures and reproducibility much worse. \|
44	\| "We will modularize later." \| Notebook sprawl grows quickly once other teams depend on it. \|
45
46	## Red Flags
47
48	- business logic depends on cell order
49	- configuration is hard-coded in notebook cells
50	- outputs are written with no validation or idempotency plan
51	- the deployment path is undefined
52
53	## Verification
54
55	- [ ] Reusable logic is extracted from the notebook flow
56	- [ ] Inputs, configuration, and outputs are explicit
57	- [ ] Validation, logging, and retry-safe behavior exist
58	- [ ] The production deployment and monitoring model are defined

People who liked this also viewed

6 picks

Type

notebook-to-production-hardening

19stars4forks

View on GitHub

How to install notebook-to-production-hardening?

vaquarkhan/data-engineering-agent-skills/notebook-to-production-hardening

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill notebook-to-production-hardening

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Notebook To Production Hardening
2
3	## Overview
4
5	Use this skill when a notebook has outgrown exploration and needs to become a maintainable delivery artifact. It helps agents separate experimentation from production packaging, testing, configuration, and orchestration.
6
7	## When to Use
8
9	- moving notebook logic into scheduled jobs
10	- hardening `Databricks` or `Jupyter` notebooks for repeated use
11	- extracting reusable logic from cells into modules or packages
12	- improving testability and deployment discipline
13
14	Do not treat a manually rerun notebook as production just because it worked once.
15
16	## Workflow
17
18	1. Separate exploratory work from production logic.
19	Identify:
20	- reusable transformation code
21	- parameters
22	- environment assumptions
23	- manual steps
24
25	2. Extract logic into versioned, testable units.
26
27	3. Replace hidden state with explicit inputs and configuration.
28
29	4. Add validation and operational hooks.
30	Include:
31	- contracts
32	- logging
33	- error handling
34	- retry-safe outputs
35
36	5. Define how the job is deployed and monitored.
37
38	## Common Rationalizations
39
40	\| Rationalization \| Reality \|
41	\| --- \| --- \|
42	\| "The notebook already works." \| Interactive success does not mean repeatable, testable, or observable production behavior. \|
43	\| "We can keep using widgets and manual edits." \| Hidden runtime state makes failures and reproducibility much worse. \|
44	\| "We will modularize later." \| Notebook sprawl grows quickly once other teams depend on it. \|
45
46	## Red Flags
47
48	- business logic depends on cell order
49	- configuration is hard-coded in notebook cells
50	- outputs are written with no validation or idempotency plan
51	- the deployment path is undefined
52
53	## Verification
54
55	- [ ] Reusable logic is extracted from the notebook flow
56	- [ ] Inputs, configuration, and outputs are explicit
57	- [ ] Validation, logging, and retry-safe behavior exist
58	- [ ] The production deployment and monitoring model are defined

People who liked this also viewed

6 picks

Type