pytorch

Name: pytorch
Author: itsmostafa

22installs0forks

How to install pytorch?

itsmostafa/llm-engineering-skills/pytorch

$npx -y skills add itsmostafa/llm-engineering-skills --skill pytorch

Installs into the current project.

Summary

Building and training neural networks with PyTorch. Use when implementing deep learning models, training loops, data pipelines, model optimization with torch.compile, distributed training, or deploying PyTorch models.

Files · 1

View on GitHub

SKILL.md

1	# Using PyTorch
2
3	PyTorch is a deep learning framework with dynamic computation graphs, strong GPU acceleration, and Pythonic design. This skill covers practical patterns for building production-quality neural networks.
4
5	## Table of Contents
6
7	- [Core Concepts](#core-concepts)
8	- [Model Architecture](#model-architecture)
9	- [Training Loop](#training-loop)
10	- [Data Loading](#data-loading)
11	- [Performance Optimization](#performance-optimization)
12	- [Distributed Training](#distributed-training)
13	- [Saving and Loading](#saving-and-loading)
14	- [Best Practices](#best-practices)
15	- [References](#references)
16
17	## Core Concepts
18
19	### Tensors
20
21	```python
22	import torch
23
24	# Create tensors
25	x = torch.tensor([[1, 2], [3, 4]], dtype=torch.float32)
26	x = torch.zeros(3, 4)
27	x = torch.randn(3, 4) # Normal distribution
28
29	# Device management
30	device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
31	x = x.to(device)
32
33	# Operations (all return new tensors)
34	y = x + 1
35	y = x @ x.T # Matrix multiplication
36	y = x.view(2, 6) # Reshape
37	```
38
39	### Autograd
40
41	```python
42	# Enable gradient tracking
43	x = torch.randn(3, requires_grad=True)
44	y = x ** 2
45	loss = y.sum()
46
47	# Compute gradients
48	loss.backward()
49	print(x.grad) # dy/dx
50
51	# Disable gradients for inference
52	with torch.no_grad():
53	pred = model(x)
54
55	# Or use inference mode (more efficient)
56	with torch.inference_mode():
57	pred = model(x)
58	```
59
60	## Model Architecture
61
62	### nn.Module Pattern
63
64	```python
65	import torch.nn as nn
66	import torch.nn.functional as F
67
68	class Model(nn.Module):
69	def __init__(self, input_dim: int, hidden_dim: int, output_dim: int):
70	super().__init__()
71	self.fc1 = nn.Linear(input_dim, hidden_dim)
72	self.fc2 = nn.Linear(hidden_dim, output_dim)
73	self.dropout = nn.Dropout(0.1)
74
75	def forward(self, x: torch.Tensor) -> torch.Tensor:
76	x = F.relu(self.fc1(x))
77	x = self.dropout(x)
78	return self.fc2(x)
79	```
80
81	### Common Layers
82
83	```python
84	# Convolution
85	nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1)
86
87	# Normalization
88	nn.BatchNorm2d(num_features)
89	nn.LayerNorm(normalized_shape)
90
91	# Attention
92	nn.MultiheadAttention(embed_dim, num_heads)
93
94	# Recurrent
95	nn.LSTM(input_size, hidden_size, num_layers, batch_first=True)
96	nn.GRU(input_size, hidden_size, num_layers, batch_first=True)
97	```
98
99	### Weight Initialization
100
101	```python
102	def init_weights(module):
103	if isinstance(module, nn.Linear):
104	nn.init.xavier_uniform_(module.weight)
105	if module.bias is not None:
106	nn.init.zeros_(module.bias)
107	elif isinstance(module, nn.Embedding):
108	nn.init.normal_(module.weight, std=0.02)
109
110	model.apply(init_weights)
111	```
112
113	## Training Loop
114
115	### Standard Pattern
116
117	```python
118	model = Model(input_dim, hidden_dim, output_dim).to(device)
119	optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=0.01)
120	criterion = nn.CrossEntropyLoss()
121	scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=num_epochs)
122
123	for epoch in range(num_epochs):
124	model.train()
125	for batch in train_loader:
126	inputs, targets = batch
127	inputs, targets = inputs.to(device), targets.to(device)
128
129	optimizer.zero_grad()
130	outputs = model(inputs)
131	loss = criterion(outputs, targets)
132	loss.backward()
133
134	# Optional: gradient clipping
135	torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
136
137	optimizer.step()
138
139	scheduler.step()
140
141	# Validation
142	model.eval()
143	with torch.no_grad():
144	for batch in val_loader:
145	# ... validation logic
146	```
147
148	### Mixed Precision Training
149
150	```python
151	from torch.amp import autocast, GradScaler
152
153	scaler = GradScaler("cuda")
154
155	for batch in train_loader:
156	inputs, targets = batch
157	inputs, targets = inputs.to(device), targets.to(device)
158
159	optimizer.zero_grad()
160
161	with autocast("cuda", dtype=torch.bfloat16):
162	outputs = model(inputs)
163	loss = criterion(outputs, targets)
164
165	scaler.scale(loss).backward()
166	scaler.step(optimizer)
167	scaler.update()
168	```
169
170	### Gradient Accumulation
171
172	```python
173	# Requires setup from Mixed Precision Training above:
174	# scaler = GradScaler(), model, criterion, optimizer, device
175
176	accumulation_steps = 4
177
178	for i, batch in enumerate(train_loader):
179	inputs, targets = batch
180	inputs, targets = inputs.to(device), targets.to(device)
181
182	with autocast("cuda", dtype=torch.bfloat16):
183	outputs = model(inputs)
184	loss = criterion(outputs, targets) / accumulation_steps
185
186	scaler.scale(loss).backward()
187
188	if (i + 1) % accumulation_steps == 0:
189	scaler.step(optimizer)
190	scaler.update()
191	optimizer.zero_grad()
192	```
193
194	## Data Loading
195
196	### Dataset and DataLoader
197
198	```python
199	from torch.utils.data import Dataset, DataLoader
200
201	class CustomData

People who liked this also viewed

6 picks

Type

pytorch

22installs0forks

View on GitHub

How to install pytorch?

itsmostafa/llm-engineering-skills/pytorch

$npx -y skills add itsmostafa/llm-engineering-skills --skill pytorch

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Using PyTorch
2
3	PyTorch is a deep learning framework with dynamic computation graphs, strong GPU acceleration, and Pythonic design. This skill covers practical patterns for building production-quality neural networks.
4
5	## Table of Contents
6
7	- [Core Concepts](#core-concepts)
8	- [Model Architecture](#model-architecture)
9	- [Training Loop](#training-loop)
10	- [Data Loading](#data-loading)
11	- [Performance Optimization](#performance-optimization)
12	- [Distributed Training](#distributed-training)
13	- [Saving and Loading](#saving-and-loading)
14	- [Best Practices](#best-practices)
15	- [References](#references)
16
17	## Core Concepts
18
19	### Tensors
20
21	```python
22	import torch
23
24	# Create tensors
25	x = torch.tensor([[1, 2], [3, 4]], dtype=torch.float32)
26	x = torch.zeros(3, 4)
27	x = torch.randn(3, 4) # Normal distribution
28
29	# Device management
30	device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
31	x = x.to(device)
32
33	# Operations (all return new tensors)
34	y = x + 1
35	y = x @ x.T # Matrix multiplication
36	y = x.view(2, 6) # Reshape
37	```
38
39	### Autograd
40
41	```python
42	# Enable gradient tracking
43	x = torch.randn(3, requires_grad=True)
44	y = x ** 2
45	loss = y.sum()
46
47	# Compute gradients
48	loss.backward()
49	print(x.grad) # dy/dx
50
51	# Disable gradients for inference
52	with torch.no_grad():
53	pred = model(x)
54
55	# Or use inference mode (more efficient)
56	with torch.inference_mode():
57	pred = model(x)
58	```
59
60	## Model Architecture
61
62	### nn.Module Pattern
63
64	```python
65	import torch.nn as nn
66	import torch.nn.functional as F
67
68	class Model(nn.Module):
69	def __init__(self, input_dim: int, hidden_dim: int, output_dim: int):
70	super().__init__()
71	self.fc1 = nn.Linear(input_dim, hidden_dim)
72	self.fc2 = nn.Linear(hidden_dim, output_dim)
73	self.dropout = nn.Dropout(0.1)
74
75	def forward(self, x: torch.Tensor) -> torch.Tensor:
76	x = F.relu(self.fc1(x))
77	x = self.dropout(x)
78	return self.fc2(x)
79	```
80
81	### Common Layers
82
83	```python
84	# Convolution
85	nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1)
86
87	# Normalization
88	nn.BatchNorm2d(num_features)
89	nn.LayerNorm(normalized_shape)
90
91	# Attention
92	nn.MultiheadAttention(embed_dim, num_heads)
93
94	# Recurrent
95	nn.LSTM(input_size, hidden_size, num_layers, batch_first=True)
96	nn.GRU(input_size, hidden_size, num_layers, batch_first=True)
97	```
98
99	### Weight Initialization
100
101	```python
102	def init_weights(module):
103	if isinstance(module, nn.Linear):
104	nn.init.xavier_uniform_(module.weight)
105	if module.bias is not None:
106	nn.init.zeros_(module.bias)
107	elif isinstance(module, nn.Embedding):
108	nn.init.normal_(module.weight, std=0.02)
109
110	model.apply(init_weights)
111	```
112
113	## Training Loop
114
115	### Standard Pattern
116
117	```python
118	model = Model(input_dim, hidden_dim, output_dim).to(device)
119	optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=0.01)
120	criterion = nn.CrossEntropyLoss()
121	scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=num_epochs)
122
123	for epoch in range(num_epochs):
124	model.train()
125	for batch in train_loader:
126	inputs, targets = batch
127	inputs, targets = inputs.to(device), targets.to(device)
128
129	optimizer.zero_grad()
130	outputs = model(inputs)
131	loss = criterion(outputs, targets)
132	loss.backward()
133
134	# Optional: gradient clipping
135	torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
136
137	optimizer.step()
138
139	scheduler.step()
140
141	# Validation
142	model.eval()
143	with torch.no_grad():
144	for batch in val_loader:
145	# ... validation logic
146	```
147
148	### Mixed Precision Training
149
150	```python
151	from torch.amp import autocast, GradScaler
152
153	scaler = GradScaler("cuda")
154
155	for batch in train_loader:
156	inputs, targets = batch
157	inputs, targets = inputs.to(device), targets.to(device)
158
159	optimizer.zero_grad()
160
161	with autocast("cuda", dtype=torch.bfloat16):
162	outputs = model(inputs)
163	loss = criterion(outputs, targets)
164
165	scaler.scale(loss).backward()
166	scaler.step(optimizer)
167	scaler.update()
168	```
169
170	### Gradient Accumulation
171
172	```python
173	# Requires setup from Mixed Precision Training above:
174	# scaler = GradScaler(), model, criterion, optimizer, device
175
176	accumulation_steps = 4
177
178	for i, batch in enumerate(train_loader):
179	inputs, targets = batch
180	inputs, targets = inputs.to(device), targets.to(device)
181
182	with autocast("cuda", dtype=torch.bfloat16):
183	outputs = model(inputs)
184	loss = criterion(outputs, targets) / accumulation_steps
185
186	scaler.scale(loss).backward()
187
188	if (i + 1) % accumulation_steps == 0:
189	scaler.step(optimizer)
190	scaler.update()
191	optimizer.zero_grad()
192	```
193
194	## Data Loading
195
196	### Dataset and DataLoader
197
198	```python
199	from torch.utils.data import Dataset, DataLoader
200
201	class CustomData

People who liked this also viewed

6 picks

Type