deepmd-finetune-dpa3

Name: deepmd-finetune-dpa3
Author: jinzhezenggroup

113stars23forks

How to install deepmd-finetune-dpa3?

jinzhezenggroup/computational-chemistry-agent-skills/deepmd-finetune-dpa3

$npx -y skills add jinzhezenggroup/computational-chemistry-agent-skills --skill deepmd-finetune-dpa3

Installs into the current project.

Summary

Fine-tune a DPA3 model in DeePMD-kit using the PyTorch backend. Use when the user wants to adapt a pre-trained DPA3 model to a new downstream dataset. Supports fine-tuning from a self-trained DPA3 model (.pt checkpoint), from a multi-task pre-trained model, or from a built-in pre

Files · 1

View on GitHub

SKILL.md

1	# DeePMD-kit Fine-tuning: DPA3
2
3	Fine-tune a pre-trained DPA3 model on a downstream dataset. This skill covers three scenarios:
4
5	1. Fine-tuning from a self-trained single-task DPA3 model
6	1. Fine-tuning from a multi-task pre-trained DPA3 model
7	1. Fine-tuning from a built-in pretrained model (e.g., DPA-3.1-3M, DPA-3.2-5M, DPA-3.3-1M) downloaded via `dp pretrained download`
8
9	## Quick Start
10
11	```bash
12	# Fine-tune from a self-trained model
13	dp --pt train input.json --finetune pretrained.pt --use-pretrain-script
14
15	# Fine-tune from a built-in pretrained model
16	dp pretrained download DPA-3.2-5M
17	dp --pt train input.json --finetune /path/to/DPA-3.2-5M.pt --use-pretrain-script --model-branch OMat24
18	```
19
20	## Agent Responsibilities
21
22	1. Determine the fine-tuning scenario:
23	- Does the user have a self-trained `.pt` model?
24	- Does the user want to use a built-in pretrained model (DPA-3.1-3M, DPA-3.2-5M, DPA-3.3-1M, etc.)?
25	- Is the pre-trained model single-task or multi-task?
26	1. If using a built-in pretrained model, download it first with `dp pretrained download`.
27	1. Collect the downstream training data paths and element types.
28	1. Generate the fine-tuning `input.json`.
29	1. Run fine-tuning and monitor the learning curve.
30	1. Freeze and test the fine-tuned model.
31
32	## Scenario 1: Fine-tune from a Self-trained Single-task Model
33
34	When you have trained a DPA3 model yourself and want to adapt it to new data.
35
36	### Step 1: Prepare input.json
37
38	When using `--use-pretrain-script`, the model architecture is inherited from the pre-trained model. You only need to specify `type_map`, data paths, and training parameters:
39
40	```json
41	{
42	"model": {
43	"type_map": [
44	"O",
45	"H"
46	],
47	"descriptor": {},
48	"fitting_net": {}
49	},
50	"learning_rate": {
51	"type": "exp",
52	"decay_steps": 5000,
53	"start_lr": 0.0001,
54	"stop_lr": 3e-06
55	},
56	"loss": {
57	"type": "ener",
58	"start_pref_e": 0.2,
59	"limit_pref_e": 20,
60	"start_pref_f": 100,
61	"limit_pref_f": 60,
62	"start_pref_v": 0.02,
63	"limit_pref_v": 1
64	},
65	"optimizer": {
66	"type": "AdamW",
67	"weight_decay": 0.001
68	},
69	"training": {
70	"training_data": {
71	"systems": [
72	"./downstream_data/train_0",
73	"./downstream_data/train_1"
74	],
75	"batch_size": 1
76	},
77	"validation_data": {
78	"systems": [
79	"./downstream_data/valid_0"
80	],
81	"batch_size": 1
82	},
83	"numb_steps": 200000,
84	"gradient_max_norm": 5.0,
85	"seed": 10,
86	"disp_file": "lcurve.out",
87	"disp_freq": 100,
88	"save_freq": 2000
89	}
90	}
91	```
92
93	Fine-tuning tips:
94
95	- Use a smaller `start_lr` (e.g., 1e-4) than training from scratch (1e-3).
96	- Use fewer `numb_steps` since the model is already pre-trained.
97	- The elements in the downstream data must be a subset of the pre-trained model's `type_map`.
98
99	### Step 2: Run Fine-tuning
100
101	```bash
102	dp --pt train input.json --finetune pretrained.pt --use-pretrain-script
103	```
104
105	The `--use-pretrain-script` flag tells DeePMD-kit to inherit the model architecture from the pre-trained model, so the `descriptor` and `fitting_net` sections in `input.json` can be empty.
106
107	Without `--use-pretrain-script`, the model section in `input.json` must exactly match the pre-trained model's architecture.
108
109	## Scenario 2: Fine-tune from a Multi-task Pre-trained Model
110
111	When the pre-trained model was trained with multiple datasets (multi-task training), you can select a specific branch to fine-tune from.
112
113	### Check Available Branches
114
115	```bash
116	dp --pt show multitask_pretrained.pt model-branch
117	```
118
119	### Run Fine-tuning from a Specific Branch
120
121	```bash
122	dp --pt train input.json --finetune multitask_pretrained.pt --model-branch CHOSEN_BRANCH --use-pretrain-script
123	```
124
125	If `--model-branch` is not set or set to `RANDOM`, a randomly initialized fitting net will be used.
126
127	### Multi-task Fine-tuning (Prevent Forgetting)
128
129	To retain knowledge from the pre-trained datasets during fine-tuning, use multi-task fine-tuning. Prepare a multi-task input script:
130
131	```json
132	{
133	"model": {
134	"shared_dict": {
135	"type_map_all": [
136	"O",
137	"H",
138	"C",
139	"N"
140	],
141	"dpa3_desc": {
142	"type": "dpa3",
143	"repflow": {}
144	}
145	},
146	"model_dict": {
147	"pre_data_1": {
148	"type_map": "type_map_all",
149	"descriptor": "dpa3_desc",

People who liked this also viewed

6 picks

Type

deepmd-finetune-dpa3

113stars23forks

View on GitHub

How to install deepmd-finetune-dpa3?

jinzhezenggroup/computational-chemistry-agent-skills/deepmd-finetune-dpa3

$npx -y skills add jinzhezenggroup/computational-chemistry-agent-skills --skill deepmd-finetune-dpa3

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# DeePMD-kit Fine-tuning: DPA3
2
3	Fine-tune a pre-trained DPA3 model on a downstream dataset. This skill covers three scenarios:
4
5	1. Fine-tuning from a self-trained single-task DPA3 model
6	1. Fine-tuning from a multi-task pre-trained DPA3 model
7	1. Fine-tuning from a built-in pretrained model (e.g., DPA-3.1-3M, DPA-3.2-5M, DPA-3.3-1M) downloaded via `dp pretrained download`
8
9	## Quick Start
10
11	```bash
12	# Fine-tune from a self-trained model
13	dp --pt train input.json --finetune pretrained.pt --use-pretrain-script
14
15	# Fine-tune from a built-in pretrained model
16	dp pretrained download DPA-3.2-5M
17	dp --pt train input.json --finetune /path/to/DPA-3.2-5M.pt --use-pretrain-script --model-branch OMat24
18	```
19
20	## Agent Responsibilities
21
22	1. Determine the fine-tuning scenario:
23	- Does the user have a self-trained `.pt` model?
24	- Does the user want to use a built-in pretrained model (DPA-3.1-3M, DPA-3.2-5M, DPA-3.3-1M, etc.)?
25	- Is the pre-trained model single-task or multi-task?
26	1. If using a built-in pretrained model, download it first with `dp pretrained download`.
27	1. Collect the downstream training data paths and element types.
28	1. Generate the fine-tuning `input.json`.
29	1. Run fine-tuning and monitor the learning curve.
30	1. Freeze and test the fine-tuned model.
31
32	## Scenario 1: Fine-tune from a Self-trained Single-task Model
33
34	When you have trained a DPA3 model yourself and want to adapt it to new data.
35
36	### Step 1: Prepare input.json
37
38	When using `--use-pretrain-script`, the model architecture is inherited from the pre-trained model. You only need to specify `type_map`, data paths, and training parameters:
39
40	```json
41	{
42	"model": {
43	"type_map": [
44	"O",
45	"H"
46	],
47	"descriptor": {},
48	"fitting_net": {}
49	},
50	"learning_rate": {
51	"type": "exp",
52	"decay_steps": 5000,
53	"start_lr": 0.0001,
54	"stop_lr": 3e-06
55	},
56	"loss": {
57	"type": "ener",
58	"start_pref_e": 0.2,
59	"limit_pref_e": 20,
60	"start_pref_f": 100,
61	"limit_pref_f": 60,
62	"start_pref_v": 0.02,
63	"limit_pref_v": 1
64	},
65	"optimizer": {
66	"type": "AdamW",
67	"weight_decay": 0.001
68	},
69	"training": {
70	"training_data": {
71	"systems": [
72	"./downstream_data/train_0",
73	"./downstream_data/train_1"
74	],
75	"batch_size": 1
76	},
77	"validation_data": {
78	"systems": [
79	"./downstream_data/valid_0"
80	],
81	"batch_size": 1
82	},
83	"numb_steps": 200000,
84	"gradient_max_norm": 5.0,
85	"seed": 10,
86	"disp_file": "lcurve.out",
87	"disp_freq": 100,
88	"save_freq": 2000
89	}
90	}
91	```
92
93	Fine-tuning tips:
94
95	- Use a smaller `start_lr` (e.g., 1e-4) than training from scratch (1e-3).
96	- Use fewer `numb_steps` since the model is already pre-trained.
97	- The elements in the downstream data must be a subset of the pre-trained model's `type_map`.
98
99	### Step 2: Run Fine-tuning
100
101	```bash
102	dp --pt train input.json --finetune pretrained.pt --use-pretrain-script
103	```
104
105	The `--use-pretrain-script` flag tells DeePMD-kit to inherit the model architecture from the pre-trained model, so the `descriptor` and `fitting_net` sections in `input.json` can be empty.
106
107	Without `--use-pretrain-script`, the model section in `input.json` must exactly match the pre-trained model's architecture.
108
109	## Scenario 2: Fine-tune from a Multi-task Pre-trained Model
110
111	When the pre-trained model was trained with multiple datasets (multi-task training), you can select a specific branch to fine-tune from.
112
113	### Check Available Branches
114
115	```bash
116	dp --pt show multitask_pretrained.pt model-branch
117	```
118
119	### Run Fine-tuning from a Specific Branch
120
121	```bash
122	dp --pt train input.json --finetune multitask_pretrained.pt --model-branch CHOSEN_BRANCH --use-pretrain-script
123	```
124
125	If `--model-branch` is not set or set to `RANDOM`, a randomly initialized fitting net will be used.
126
127	### Multi-task Fine-tuning (Prevent Forgetting)
128
129	To retain knowledge from the pre-trained datasets during fine-tuning, use multi-task fine-tuning. Prepare a multi-task input script:
130
131	```json
132	{
133	"model": {
134	"shared_dict": {
135	"type_map_all": [
136	"O",
137	"H",
138	"C",
139	"N"
140	],
141	"dpa3_desc": {
142	"type": "dpa3",
143	"repflow": {}
144	}
145	},
146	"model_dict": {
147	"pre_data_1": {
148	"type_map": "type_map_all",
149	"descriptor": "dpa3_desc",

People who liked this also viewed

6 picks

Type