huggingface-vision-trainer

Name: huggingface-vision-trainer
Author: huggingface

1.2kinstalls722forks

How to install huggingface-vision-trainer?

huggingface/skills/huggingface-vision-trainer

$npx -y skills add huggingface/skills --skill huggingface-vision-trainer

Installs into the current project.

Summary

Trains and fine-tunes vision models for object detection (D-FINE, RT-DETR v2, DETR, YOLOS), image classification (timm models — MobileNetV3, MobileViT, ResNet, ViT/DINOv3 — plus any Transformers classifier), and SAM/SAM2 segmentation using Hugging Face Transformers on Hugging Fac

Files · 1

View on GitHub

SKILL.md

1	# Vision Model Training on Hugging Face Jobs
2
3	Train object detection, image classification, and SAM/SAM2 segmentation models on managed cloud GPUs. No local GPU setup required—results are automatically saved to the Hugging Face Hub.
4
5	## When to Use This Skill
6
7	Use this skill when users want to:
8	- Fine-tune object detection models (D-FINE, RT-DETR v2, DETR, YOLOS) on cloud GPUs or local
9	- Fine-tune image classification models (timm: MobileNetV3, MobileViT, ResNet, ViT/DINOv3, or any Transformers classifier) on cloud GPUs or local
10	- Fine-tune SAM or SAM2 models for segmentation / image matting using bbox or point prompts
11	- Train bounding-box detectors on custom datasets
12	- Train image classifiers on custom datasets
13	- Train segmentation models on custom mask datasets with prompts
14	- Run vision training jobs on Hugging Face Jobs infrastructure
15	- Ensure trained vision models are permanently saved to the Hub
16
17	## Related Skills
18
19	- `hugging-face-jobs` — General HF Jobs infrastructure: token authentication, hardware flavors, timeout management, cost estimation, secrets, environment variables, scheduled jobs, and result persistence. Refer to the Jobs skill for any non-training-specific Jobs questions (e.g., "how do secrets work?", "what hardware is available?", "how do I pass tokens?").
20	- `hugging-face-model-trainer` — TRL-based language model training (SFT, DPO, GRPO). Use that skill for text/language model fine-tuning.
21
22	## Local Script Execution
23
24	Helper scripts use PEP 723 inline dependencies. Run them with `uv run`:
25	```bash
26	uv run scripts/dataset_inspector.py --dataset username/dataset-name --split train
27	uv run scripts/estimate_cost.py --help
28	```
29
30	## Prerequisites Checklist
31
32	Before starting any training job, verify:
33
34	### Account & Authentication
35	- Hugging Face Account with [Pro](https://hf.co/pro), [Team](https://hf.co/enterprise), or [Enterprise](https://hf.co/enterprise) plan (Jobs require paid plan)
36	- Authenticated login: Check with `hf_whoami()` (tool) or `hf auth whoami` (terminal)
37	- Token has write permissions
38	- MUST pass token in job secrets — see directive #3 below for syntax (MCP tool vs Python API)
39
40	### Dataset Requirements — Object Detection
41	- Dataset must exist on Hub
42	- Annotations must use the `objects` column with `bbox`, `category` (and optionally `area`) sub-fields
43	- Bboxes can be in xywh (COCO) or xyxy (Pascal VOC) format — auto-detected and converted
44	- Categories can be integers or strings — strings are auto-remapped to integer IDs
45	- `image_id` column is optional — generated automatically if missing
46	- ALWAYS validate unknown datasets before GPU training (see Dataset Validation section)
47
48	### Dataset Requirements — Image Classification
49	- Dataset must exist on Hub
50	- Must have an `image` column (PIL images) and a `label` column (integer class IDs or strings)
51	- The label column can be `ClassLabel` type (with names) or plain integers/strings — strings are auto-remapped
52	- Common column names auto-detected: `label`, `labels`, `class`, `fine_label`
53	- ALWAYS validate unknown datasets before GPU training (see Dataset Validation section)
54
55	### Dataset Requirements — SAM/SAM2 Segmentation
56	- Dataset must exist on Hub
57	- Must have an `image` column (PIL images) and a `mask` column (binary ground-truth segmentation mask)
58	- Must have a prompt — either:
59	- A `prompt` column with JSON containing `{"bbox": [x0,y0,x1,y1]}` or `{"point": [x,y]}`
60	- OR a dedicated `bbox` column with `[x0,y0,x1,y1]` values
61	- OR a dedicated `point` column with `[x,y]` or `[[x,y],...]` values
62	- Bboxes should be in xyxy format (absolute pixel coordinates)
63	- Example dataset: `merve/MicroMat-mini` (image matting with bbox prompts)
64	- ALWAYS validate unknown datasets before GPU training (see Dataset Validation section)
65
66	### Critical Settings
67	- Timeout must exceed expected training time — Default 30min is TOO SHORT. See directive #6 for recommended values.
68	- Hub push must be enabled — `push_to_hub=True`, `hub_model_id="username/model-name"`, token in `secrets`
69
70	## Dataset Validation
71
72	**Validate dataset format BEFORE launching GPU training

People who liked this also viewed

6 picks

Type

huggingface-vision-trainer

1.2kinstalls722forks

View on GitHub

How to install huggingface-vision-trainer?

huggingface/skills/huggingface-vision-trainer

$npx -y skills add huggingface/skills --skill huggingface-vision-trainer

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Vision Model Training on Hugging Face Jobs
2
3	Train object detection, image classification, and SAM/SAM2 segmentation models on managed cloud GPUs. No local GPU setup required—results are automatically saved to the Hugging Face Hub.
4
5	## When to Use This Skill
6
7	Use this skill when users want to:
8	- Fine-tune object detection models (D-FINE, RT-DETR v2, DETR, YOLOS) on cloud GPUs or local
9	- Fine-tune image classification models (timm: MobileNetV3, MobileViT, ResNet, ViT/DINOv3, or any Transformers classifier) on cloud GPUs or local
10	- Fine-tune SAM or SAM2 models for segmentation / image matting using bbox or point prompts
11	- Train bounding-box detectors on custom datasets
12	- Train image classifiers on custom datasets
13	- Train segmentation models on custom mask datasets with prompts
14	- Run vision training jobs on Hugging Face Jobs infrastructure
15	- Ensure trained vision models are permanently saved to the Hub
16
17	## Related Skills
18
19	- `hugging-face-jobs` — General HF Jobs infrastructure: token authentication, hardware flavors, timeout management, cost estimation, secrets, environment variables, scheduled jobs, and result persistence. Refer to the Jobs skill for any non-training-specific Jobs questions (e.g., "how do secrets work?", "what hardware is available?", "how do I pass tokens?").
20	- `hugging-face-model-trainer` — TRL-based language model training (SFT, DPO, GRPO). Use that skill for text/language model fine-tuning.
21
22	## Local Script Execution
23
24	Helper scripts use PEP 723 inline dependencies. Run them with `uv run`:
25	```bash
26	uv run scripts/dataset_inspector.py --dataset username/dataset-name --split train
27	uv run scripts/estimate_cost.py --help
28	```
29
30	## Prerequisites Checklist
31
32	Before starting any training job, verify:
33
34	### Account & Authentication
35	- Hugging Face Account with [Pro](https://hf.co/pro), [Team](https://hf.co/enterprise), or [Enterprise](https://hf.co/enterprise) plan (Jobs require paid plan)
36	- Authenticated login: Check with `hf_whoami()` (tool) or `hf auth whoami` (terminal)
37	- Token has write permissions
38	- MUST pass token in job secrets — see directive #3 below for syntax (MCP tool vs Python API)
39
40	### Dataset Requirements — Object Detection
41	- Dataset must exist on Hub
42	- Annotations must use the `objects` column with `bbox`, `category` (and optionally `area`) sub-fields
43	- Bboxes can be in xywh (COCO) or xyxy (Pascal VOC) format — auto-detected and converted
44	- Categories can be integers or strings — strings are auto-remapped to integer IDs
45	- `image_id` column is optional — generated automatically if missing
46	- ALWAYS validate unknown datasets before GPU training (see Dataset Validation section)
47
48	### Dataset Requirements — Image Classification
49	- Dataset must exist on Hub
50	- Must have an `image` column (PIL images) and a `label` column (integer class IDs or strings)
51	- The label column can be `ClassLabel` type (with names) or plain integers/strings — strings are auto-remapped
52	- Common column names auto-detected: `label`, `labels`, `class`, `fine_label`
53	- ALWAYS validate unknown datasets before GPU training (see Dataset Validation section)
54
55	### Dataset Requirements — SAM/SAM2 Segmentation
56	- Dataset must exist on Hub
57	- Must have an `image` column (PIL images) and a `mask` column (binary ground-truth segmentation mask)
58	- Must have a prompt — either:
59	- A `prompt` column with JSON containing `{"bbox": [x0,y0,x1,y1]}` or `{"point": [x,y]}`
60	- OR a dedicated `bbox` column with `[x0,y0,x1,y1]` values
61	- OR a dedicated `point` column with `[x,y]` or `[[x,y],...]` values
62	- Bboxes should be in xyxy format (absolute pixel coordinates)
63	- Example dataset: `merve/MicroMat-mini` (image matting with bbox prompts)
64	- ALWAYS validate unknown datasets before GPU training (see Dataset Validation section)
65
66	### Critical Settings
67	- Timeout must exceed expected training time — Default 30min is TOO SHORT. See directive #6 for recommended values.
68	- Hub push must be enabled — `push_to_hub=True`, `hub_model_id="username/model-name"`, token in `secrets`
69
70	## Dataset Validation
71
72	**Validate dataset format BEFORE launching GPU training

People who liked this also viewed

6 picks

Type