finetuning

Name: finetuning
Author: awslabs

843stars132forks

How to install finetuning?

awslabs/agent-plugins/finetuning

$npx -y skills add awslabs/agent-plugins --skill finetuning

Installs into the current project.

Summary

Generates code that fine-tunes a base model using SageMaker serverless training jobs. Use when the user says "start training", "fine-tune my model", "I'm ready to train", or when the plan reaches the finetuning step. Supports SFT, DPO, RLVR, and RLAIF trainers, including RLVR Lam

Files · 1

View on GitHub

SKILL.md

1	# Prerequisites
2
3	Before starting this workflow, verify:
4
5	1. A `use_case_spec.md` file exists
6	- If missing: Activate the `use-case-specification` skill first, then resume
7	- DON'T EVER offer to create a use case spec without activating the use-case-specification skill.
8
9	2. A fine-tuning technique (SFT, DPO, RLVR, RLAIF, or CPT/RFT (for Nova)) and base model have already been selected
10	- If missing: Activate the `model-selection` and/or `finetuning-technique` skills to collect what's missing, then resume
11	- Don't make recommendations on the spot. You MUST activate the appropriate skill.
12
13	3. A base model name available on SageMakerHub has been identified
14	- If missing: Activate the `model-selection` skill to get it
15	- Important: Only use the model name that `model-selection` retrieves, as it may differ from other commonly used names for the same model
16
17	4. The SDK environment has been verified (SDK version, region, execution role)
18	- If not done: Activate the `sdk-getting-started` skill first, then resume
19
20	5. A training dataset uploaded to a bucket in the environment's default region.
21	- If not met: Help the user upload the dataset to the correct S3
22
23	---
24
25	# Critical Rules
26
27	## Code Generation Rules
28
29	- ✅ Use EXACTLY the imports shown in each code template
30	- ❌ Do NOT add additional imports even if they seem helpful
31	- ❌ Do NOT create variables before they're needed in that section
32	- 📋 Copy the code structure precisely - no improvisation
33	- 🎯 Follow the minimal code principle strictly
34	- ✅ When writing code, make sure the indentation and f strings are correct
35
36	## User Communication Rules
37
38	- ❌ NEVER offer to move on to a downstream skill while training is in progress (logically impossible)
39	- ❌ NEVER set ACCEPT_EULA to True without explicit user confirmation in the conversation
40	- ✅ Always mention both the number AND title of sections you reference
41	- ✅ If user asks how to run (notebook): If `run_cell` is available, offer to run it. Otherwise, tell them to run cells one by one (mention ipykernel requirement).
42	- ✅ If user asks how to run (script): Tell them to run with `python3 <script>.py`
43
44	---
45
46	# Workflow
47
48	## 1. Code Generation Setup
49
50	### 1.1 Directory Setup
51
52	1. Identify project directory from conversation context
53	- If unclear (multiple relevant directories exist) → Ask user which folder to use
54	- If no project directory exists → activate the directory-management skill to set one up
55
56	⏸ Wait for user.
57
58	### 1.2 Select Code Template
59
60	Read `references/code_output_guide.md` for output format rules, then read the code template matching the finetuning strategy:
61
62	- SFT → `code_templates/sft.py`
63	- DPO → `code_templates/dpo.py`
64	- RLVR → `code_templates/rlvr.py`
65	- RLAIF with built-in rewards → `code_templates/rlaif_builtin.py`
66	- RLAIF with custom prompt → `code_templates/rlaif_custom_prompt.py`
67
68	The template is a Python file where each `# Cell N: Label` comment marks the start of a new section. Split on these markers — everything between one marker and the next becomes one unit of output.
69
70	### 1.3 Generate Code
71
72	1. Write the code from the template following the rules in `code_output_guide.md`
73	2. Use same order, dependencies, and imports as the template
74	3. DO NOT improvise or add extra code
75	4. If the model is NOT a Meta/Llama model (model ID does NOT start with `meta-`):
76	- Omit the `ACCEPT_EULA = False` line from the config cell
77	- Omit the `accept_eula=ACCEPT_EULA,` line from the trainer call
78	5. If the model is from the Nova family, omit any code containing `max_epochs` or `lr_warmup_steps_ratio` from the Configure Trainer section and the Hyperparameter Overrides section
79
80	### 1.4 Auto-Generate Configuration Values
81
82	In the 'Setup & Credentials' cell, populate:
83
84	1. BASE_MODEL
85	- Use the exact SageMakerHub model name from context
86
87	2. MODEL_PACKAGE_GROUP_NAME
88	- Generate from use case (read `use_case_spec.md` if needed)
89	- Format rules:
90	- Lowercase, alphanumeric with hyphens only
91	- 1-63 characters
92	- Pattern: `[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}`
93	- Example: "Customer Support Chatbot" → `customer-support-chatbot-v1`
94
95	3. Save notebook
96
97	## 2. RLVR Reward Function (for RLVR only, skip this section if technique is SFT or DPO)
98
99	### 2.1 Check Reward Function Status
100
101	- Ask if user has a reward function already, or would like help creating one.
102	- If user says they have one → Ask for the SageMaker Hub Evaluator ARN. Only proceed to Section 2.3 once the user provides a valid Evaluator ARN. If they don't have it registered as a SageMaker Hub Evaluator, continue to 2.2.
103	- If u

People who liked this also viewed

6 picks

Type

finetuning

843stars132forks

View on GitHub

How to install finetuning?

awslabs/agent-plugins/finetuning

$npx -y skills add awslabs/agent-plugins --skill finetuning

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Prerequisites
2
3	Before starting this workflow, verify:
4
5	1. A `use_case_spec.md` file exists
6	- If missing: Activate the `use-case-specification` skill first, then resume
7	- DON'T EVER offer to create a use case spec without activating the use-case-specification skill.
8
9	2. A fine-tuning technique (SFT, DPO, RLVR, RLAIF, or CPT/RFT (for Nova)) and base model have already been selected
10	- If missing: Activate the `model-selection` and/or `finetuning-technique` skills to collect what's missing, then resume
11	- Don't make recommendations on the spot. You MUST activate the appropriate skill.
12
13	3. A base model name available on SageMakerHub has been identified
14	- If missing: Activate the `model-selection` skill to get it
15	- Important: Only use the model name that `model-selection` retrieves, as it may differ from other commonly used names for the same model
16
17	4. The SDK environment has been verified (SDK version, region, execution role)
18	- If not done: Activate the `sdk-getting-started` skill first, then resume
19
20	5. A training dataset uploaded to a bucket in the environment's default region.
21	- If not met: Help the user upload the dataset to the correct S3
22
23	---
24
25	# Critical Rules
26
27	## Code Generation Rules
28
29	- ✅ Use EXACTLY the imports shown in each code template
30	- ❌ Do NOT add additional imports even if they seem helpful
31	- ❌ Do NOT create variables before they're needed in that section
32	- 📋 Copy the code structure precisely - no improvisation
33	- 🎯 Follow the minimal code principle strictly
34	- ✅ When writing code, make sure the indentation and f strings are correct
35
36	## User Communication Rules
37
38	- ❌ NEVER offer to move on to a downstream skill while training is in progress (logically impossible)
39	- ❌ NEVER set ACCEPT_EULA to True without explicit user confirmation in the conversation
40	- ✅ Always mention both the number AND title of sections you reference
41	- ✅ If user asks how to run (notebook): If `run_cell` is available, offer to run it. Otherwise, tell them to run cells one by one (mention ipykernel requirement).
42	- ✅ If user asks how to run (script): Tell them to run with `python3 <script>.py`
43
44	---
45
46	# Workflow
47
48	## 1. Code Generation Setup
49
50	### 1.1 Directory Setup
51
52	1. Identify project directory from conversation context
53	- If unclear (multiple relevant directories exist) → Ask user which folder to use
54	- If no project directory exists → activate the directory-management skill to set one up
55
56	⏸ Wait for user.
57
58	### 1.2 Select Code Template
59
60	Read `references/code_output_guide.md` for output format rules, then read the code template matching the finetuning strategy:
61
62	- SFT → `code_templates/sft.py`
63	- DPO → `code_templates/dpo.py`
64	- RLVR → `code_templates/rlvr.py`
65	- RLAIF with built-in rewards → `code_templates/rlaif_builtin.py`
66	- RLAIF with custom prompt → `code_templates/rlaif_custom_prompt.py`
67
68	The template is a Python file where each `# Cell N: Label` comment marks the start of a new section. Split on these markers — everything between one marker and the next becomes one unit of output.
69
70	### 1.3 Generate Code
71
72	1. Write the code from the template following the rules in `code_output_guide.md`
73	2. Use same order, dependencies, and imports as the template
74	3. DO NOT improvise or add extra code
75	4. If the model is NOT a Meta/Llama model (model ID does NOT start with `meta-`):
76	- Omit the `ACCEPT_EULA = False` line from the config cell
77	- Omit the `accept_eula=ACCEPT_EULA,` line from the trainer call
78	5. If the model is from the Nova family, omit any code containing `max_epochs` or `lr_warmup_steps_ratio` from the Configure Trainer section and the Hyperparameter Overrides section
79
80	### 1.4 Auto-Generate Configuration Values
81
82	In the 'Setup & Credentials' cell, populate:
83
84	1. BASE_MODEL
85	- Use the exact SageMakerHub model name from context
86
87	2. MODEL_PACKAGE_GROUP_NAME
88	- Generate from use case (read `use_case_spec.md` if needed)
89	- Format rules:
90	- Lowercase, alphanumeric with hyphens only
91	- 1-63 characters
92	- Pattern: `[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}`
93	- Example: "Customer Support Chatbot" → `customer-support-chatbot-v1`
94
95	3. Save notebook
96
97	## 2. RLVR Reward Function (for RLVR only, skip this section if technique is SFT or DPO)
98
99	### 2.1 Check Reward Function Status
100
101	- Ask if user has a reward function already, or would like help creating one.
102	- If user says they have one → Ask for the SageMaker Hub Evaluator ARN. Only proceed to Section 2.3 once the user provides a valid Evaluator ARN. If they don't have it registered as a SageMaker Hub Evaluator, continue to 2.2.
103	- If u

People who liked this also viewed

6 picks

Type