mlops-industrialization

Name: mlops-industrialization
Author: MLOps-Courses

19stars3forks

How to install mlops-industrialization?

mlops-courses/mlops-coding-skills/mlops-industrialization

$npx -y skills add MLOps-Courses/mlops-coding-skills --skill mlops-industrialization

Installs into the current project.

Summary

Guide to transform prototypes into robust, distributable Python packages using the src layout, hybrid paradigm, and strict configuration management.

Files · 1

View on GitHub

SKILL.md

1	# MLOps Coding - Productionizing Skill
2
3	## Goal
4
5	To convert experimental code (notebooks/scripts) into a high-quality, distributable Python package. This skill enforces the src/ layout, a Hybrid Paradigm (OOP structure + Functional purity), and Strict Configuration to ensure scalability, security, and maintainability.
6
7	## Prerequisites
8
9	- Language: Python
10	- Manager: `uv`
11	- Context: Moving from `notebooks/` to `src/`.
12
13	## Instructions
14
15	### 1. Packaging Structure (`src` Layout)
16
17	Adopt the `src` layout to prevent import errors and separate source from tooling.
18
19	1. Directory Tree:
20
21	```text
22	my-project/
23	├── pyproject.toml # Dependencies & Metadata
24	├── uv.lock
25	├── README.md
26	└── src/
27	└── my_package/ # Main package directory
28	├── __init__.py
29	├── io/ # Side-effects (Datasets, APIs)
30	├── domain/ # Pure business logic (Models, Features)
31	└── application/ # Orchestration (Training loops, Inference)
32	```
33
34	2. Configuration: Use `pyproject.toml` for all build metadata and dependencies.
35
36	### 2. Modularity & Paradigm (Hybrid Style)
37
38	Balance structure with predictability.
39
40	1. Domain Layer (Pure):
41	- Rule: Code here must be deterministic and free of side effects (no I/O).
42	- Use Case: Feature transformations, Model architecture definitions.
43	- Style: Functional (pure functions) or Immutable Objects (dataclasses).
44	2. I/O Layer (Impure):
45	- Rule: Isolate external interactions here.
46	- Use Case: Loading data from S3, saving models to disk, logging to MLflow.
47	- Style: OOP (Classes to manage connections/state).
48	3. Application Layer (Orchestration):
49	- Rule: Wire Domain and I/O together.
50	- Use Case: Tuning, Training, Inference, Evaluation, etc.
51
52	### 3. Application Entrypoints
53
54	Create standard, installable CLI tools.
55
56	1. Define Script: Create `src/my_package/scripts.py` with a `main()` function.
57	2. Register: Add to `pyproject.toml`:
58
59	```toml
60	[project.scripts]
61	my-tool = "my_package.scripts:main"
62	```
63
64	3. CLI Execution:
65	- Dev: `uv run my-tool` (No install needed).
66	- Prod: `pip install .` -> `my-tool` (Installed on PATH).
67	4. Guard: Always use `if __name__ == "__main__":` in scripts to prevent execution on import.
68
69	### 4. Configuration Management
70
71	Decouple settings from code using OmegaConf (Parsing) and Pydantic (Validation).
72
73	1. Define Schema (Pydantic):
74	- Create a class that defines expected types and defaults.
75
76	```python
77	from pydantic import BaseModel
78
79	class TrainingConfig(BaseModel):
80	batch_size: int = 32
81	learning_rate: float = 0.001
82	use_gpu: bool = False
83	```
84
85	2. Parse & Validate (OmegaConf):
86	- Load YAML, merge with CLI args, and validate against the schema.
87
88	```python
89	import omegaconf
90
91	# 1. Load YAML
92	conf = omegaconf.OmegaConf.load("config.yaml")
93	# 2. Merge with CLI (optional)
94	cli_conf = omegaconf.OmegaConf.from_cli()
95	merged = omegaconf.OmegaConf.merge(conf, cli_conf)
96	# 3. Validate -> Returns a validated Pydantic object
97	cfg: TrainingConfig = TrainingConfig(**omegaconf.OmegaConf.to_container(merged))
98	```
99
100	3. Secrets: Use Environment Variables (`os.getenv`), never commit them.
101
102	### 5. Documentation & Quality
103
104	Make code usable and maintainable.
105
106	1. Docstrings: Use Google Style docstrings for all modules, classes, and functions.
107
108	```python
109	def calculate_metric(y_true: np.ndarray, y_pred: np.ndarray) -> float:
110	"""Calculates the accuracy score.
111
112	Args:
113	y_true: Ground truth labels.
114	y_pred: Predicted labels.
115
116	Returns:
117	The accuracy as a float between 0 and 1.
118	"""
119	```
120
121	2. Type Hints: Use standard python typing (`typing`, `list[str]`) everywhere.
122
123	### 6. Best Practices Summary
124
125	- Config != Code: Never hardcode paths or hyperparams; use the `Pydantic + OmegaConf` pattern.
126	- Entrypoints are APIs: Design your CLI (`[project.scripts]`) as the public interface for your automation tools.
127	- Immutable Core: Keep your domain logic side-effect free; push I/O to the edges.
128
129	## Self-Correction Checklist
130
131	- [ ] No Side Effects on Import: Does `import my_package` run any code? (It shouldn't).
132	- [ ] Src Layout: Is code inside `src/`?
133	- [ ] Config Safety: Are secrets excluded from `pyproject.toml` and YAML?
134	- [ ] Typing: Are function signatures fully type-hinted?
135	- [ ] Entrypoints: Is the CLI registered in `pyproject.toml`?

People who liked this also viewed

6 picks

Type

mlops-industrialization

19stars3forks

View on GitHub

How to install mlops-industrialization?

mlops-courses/mlops-coding-skills/mlops-industrialization

$npx -y skills add MLOps-Courses/mlops-coding-skills --skill mlops-industrialization

Installs into the current project.

Summary

Guide to transform prototypes into robust, distributable Python packages using the src layout, hybrid paradigm, and strict configuration management.

Files · 1

View on GitHub

SKILL.md

1	# MLOps Coding - Productionizing Skill
2
3	## Goal
4
5	To convert experimental code (notebooks/scripts) into a high-quality, distributable Python package. This skill enforces the src/ layout, a Hybrid Paradigm (OOP structure + Functional purity), and Strict Configuration to ensure scalability, security, and maintainability.
6
7	## Prerequisites
8
9	- Language: Python
10	- Manager: `uv`
11	- Context: Moving from `notebooks/` to `src/`.
12
13	## Instructions
14
15	### 1. Packaging Structure (`src` Layout)
16
17	Adopt the `src` layout to prevent import errors and separate source from tooling.
18
19	1. Directory Tree:
20
21	```text
22	my-project/
23	├── pyproject.toml # Dependencies & Metadata
24	├── uv.lock
25	├── README.md
26	└── src/
27	└── my_package/ # Main package directory
28	├── __init__.py
29	├── io/ # Side-effects (Datasets, APIs)
30	├── domain/ # Pure business logic (Models, Features)
31	└── application/ # Orchestration (Training loops, Inference)
32	```
33
34	2. Configuration: Use `pyproject.toml` for all build metadata and dependencies.
35
36	### 2. Modularity & Paradigm (Hybrid Style)
37
38	Balance structure with predictability.
39
40	1. Domain Layer (Pure):
41	- Rule: Code here must be deterministic and free of side effects (no I/O).
42	- Use Case: Feature transformations, Model architecture definitions.
43	- Style: Functional (pure functions) or Immutable Objects (dataclasses).
44	2. I/O Layer (Impure):
45	- Rule: Isolate external interactions here.
46	- Use Case: Loading data from S3, saving models to disk, logging to MLflow.
47	- Style: OOP (Classes to manage connections/state).
48	3. Application Layer (Orchestration):
49	- Rule: Wire Domain and I/O together.
50	- Use Case: Tuning, Training, Inference, Evaluation, etc.
51
52	### 3. Application Entrypoints
53
54	Create standard, installable CLI tools.
55
56	1. Define Script: Create `src/my_package/scripts.py` with a `main()` function.
57	2. Register: Add to `pyproject.toml`:
58
59	```toml
60	[project.scripts]
61	my-tool = "my_package.scripts:main"
62	```
63
64	3. CLI Execution:
65	- Dev: `uv run my-tool` (No install needed).
66	- Prod: `pip install .` -> `my-tool` (Installed on PATH).
67	4. Guard: Always use `if __name__ == "__main__":` in scripts to prevent execution on import.
68
69	### 4. Configuration Management
70
71	Decouple settings from code using OmegaConf (Parsing) and Pydantic (Validation).
72
73	1. Define Schema (Pydantic):
74	- Create a class that defines expected types and defaults.
75
76	```python
77	from pydantic import BaseModel
78
79	class TrainingConfig(BaseModel):
80	batch_size: int = 32
81	learning_rate: float = 0.001
82	use_gpu: bool = False
83	```
84
85	2. Parse & Validate (OmegaConf):
86	- Load YAML, merge with CLI args, and validate against the schema.
87
88	```python
89	import omegaconf
90
91	# 1. Load YAML
92	conf = omegaconf.OmegaConf.load("config.yaml")
93	# 2. Merge with CLI (optional)
94	cli_conf = omegaconf.OmegaConf.from_cli()
95	merged = omegaconf.OmegaConf.merge(conf, cli_conf)
96	# 3. Validate -> Returns a validated Pydantic object
97	cfg: TrainingConfig = TrainingConfig(**omegaconf.OmegaConf.to_container(merged))
98	```
99
100	3. Secrets: Use Environment Variables (`os.getenv`), never commit them.
101
102	### 5. Documentation & Quality
103
104	Make code usable and maintainable.
105
106	1. Docstrings: Use Google Style docstrings for all modules, classes, and functions.
107
108	```python
109	def calculate_metric(y_true: np.ndarray, y_pred: np.ndarray) -> float:
110	"""Calculates the accuracy score.
111
112	Args:
113	y_true: Ground truth labels.
114	y_pred: Predicted labels.
115
116	Returns:
117	The accuracy as a float between 0 and 1.
118	"""
119	```
120
121	2. Type Hints: Use standard python typing (`typing`, `list[str]`) everywhere.
122
123	### 6. Best Practices Summary
124
125	- Config != Code: Never hardcode paths or hyperparams; use the `Pydantic + OmegaConf` pattern.
126	- Entrypoints are APIs: Design your CLI (`[project.scripts]`) as the public interface for your automation tools.
127	- Immutable Core: Keep your domain logic side-effect free; push I/O to the edges.
128
129	## Self-Correction Checklist
130
131	- [ ] No Side Effects on Import: Does `import my_package` run any code? (It shouldn't).
132	- [ ] Src Layout: Is code inside `src/`?
133	- [ ] Config Safety: Are secrets excluded from `pyproject.toml` and YAML?
134	- [ ] Typing: Are function signatures fully type-hinted?
135	- [ ] Entrypoints: Is the CLI registered in `pyproject.toml`?

People who liked this also viewed

6 picks

Type