huggingface-zerogpu

Name: huggingface-zerogpu
Author: huggingface

722installs722forks

How to install huggingface-zerogpu?

huggingface/skills/huggingface-zerogpu

$npx -y skills add huggingface/skills --skill huggingface-zerogpu

Installs into the current project.

Summary

AI demos and GPU compute with Gradio Spaces and Hugging Face Spaces ZeroGPU. Use when writing or reviewing code that uses @spaces.GPU, configuring python_version or requirements.txt for a ZeroGPU Space, or handling ZeroGPU-specific code constraints — pickle-based process is

Files · 1

View on GitHub

SKILL.md

1	# Hugging Face ZeroGPU
2
3	Rules and patterns for ML demos on Hugging Face Spaces with ZeroGPU hardware. Covers `@spaces.GPU`, duration and quota tuning, process isolation, the CUDA availability model, concurrency safety, and CUDA build constraints.
4
5	## Scope
6
7	This skill is for Gradio SDK Spaces using ZeroGPU hardware. Docker and Static Spaces cannot schedule onto ZeroGPU, and Streamlit apps now run as Docker Spaces — so this skill applies only to Gradio. For general Gradio coding (components, layouts, event listeners), see the `huggingface-gradio` skill in this repo. The authoritative ZeroGPU docs live at https://huggingface.co/docs/hub/spaces-zerogpu — refer to them for the current backing GPU, runtime version lists, and tier thresholds, all of which change over time.
8
9	## Reference Files
10
11	\| Reference \| When to read \|
12	\|-----------\|--------------\|
13	\| `references/concurrency.md` \| Always read alongside SKILL.md when writing ZeroGPU code — handlers run in parallel by default \|
14	\| `references/how-zerogpu-works.md` \| When reasoning about cold-starts, worker reuse, why module-scope warmup does not carry to requests, or why returning CUDA tensors hangs \|
15	\| `references/how-quota-works.md` \| When choosing `duration` values, debugging `illegal duration` vs `quota exceeded` errors, or explaining why default 60s blocks short tasks \|
16	\| `references/cuda-and-deps.md` \| When installing CUDA-dependent packages (e.g. `flash-attn`), pinning torch side-cars, or reading wheel filename tags \|
17
18	## Hardware
19
20	ZeroGPU exposes two GPU sizes that map to a fraction of the backing card:
21
22	\| `size` \| Slice of backing GPU \| Quota cost \|
23	\|--------\|----------------------\|------------\|
24	\| `large` (default) \| Half \| 1x \|
25	\| `xlarge` \| Full \| 2x \|
26
27	Default `large` gives half a physical GPU, so memory bandwidth and compute are significantly lower than the full card's specs. Use `xlarge` only when the workload genuinely needs the extra memory or compute.
28
29	> Backing GPU changes without notice. ZeroGPU has already migrated across GPU generations several times; older write-ups may name A100 or H200, but those are outdated. For the current backing GPU and exact per-size VRAM, always check the [ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu) before sizing workloads.
30
31	## Basic Pattern
32
33	```python
34	import spaces
35	import torch
36	from transformers import pipeline
37
38	pipe = pipeline("text-generation", model="...", device="cuda")
39
40	@spaces.GPU
41	def generate(prompt: str) -> str:
42	return pipe(prompt, max_new_tokens=100)[0]["generated_text"]
43	```
44
45	Key rules:
46
47	1. Instantiate models at module scope and call `.to("cuda")` eagerly. ZeroGPU handles the actual device mapping transparently (see CUDA availability model below).
48	2. Decorate GPU functions with `@spaces.GPU`. The decorator is a no-op outside ZeroGPU, so it is safe to keep in all environments.
49	3. Set `duration` to match the realistic worst-case workload (default 60s). The platform pre-checks `requested duration` against the user's `remaining quota` — not against the actual run time — so a 10-second task left at the 60s default fails with `quota exceeded` as soon as the user's remaining quota drops below 60s. Smaller declared `duration` also ranks higher in the node-level queue. See "Duration and Quota" below.
50	4. `torch.compile` is NOT supported. Use PyTorch [ahead-of-time compilation (AoTI)](https://huggingface.co/blog/zerogpu-aoti) (torch 2.8+) instead.
51	5. Use `size="xlarge"` sparingly. It allocates the full backing GPU, but costs 2x quota and tends to queue longer.
52
53	```python
54	@spaces.GPU(duration=120)
55	def generate_image(prompt: str):
56	return pipe(prompt).images[0]
57	```
58
59	## CUDA Availability Model
60
61	Real GPU access is only available inside `@spaces.GPU`-decorated functions. Outside those functions, the GPU is not attached to the process.
62
63	However, `import spaces` monkey-patches `torch` so that:
64
65	- `torch.cuda.is_available()` returns `True` globally.
66	- `.to("cuda")` / `device="cuda"` calls at module scope succeed without error.
67
68	This is intentional. Module-scope

People who liked this also viewed

6 picks

Type

huggingface-zerogpu

722installs722forks

View on GitHub

How to install huggingface-zerogpu?

huggingface/skills/huggingface-zerogpu

$npx -y skills add huggingface/skills --skill huggingface-zerogpu

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Hugging Face ZeroGPU
2
3	Rules and patterns for ML demos on Hugging Face Spaces with ZeroGPU hardware. Covers `@spaces.GPU`, duration and quota tuning, process isolation, the CUDA availability model, concurrency safety, and CUDA build constraints.
4
5	## Scope
6
7	This skill is for Gradio SDK Spaces using ZeroGPU hardware. Docker and Static Spaces cannot schedule onto ZeroGPU, and Streamlit apps now run as Docker Spaces — so this skill applies only to Gradio. For general Gradio coding (components, layouts, event listeners), see the `huggingface-gradio` skill in this repo. The authoritative ZeroGPU docs live at https://huggingface.co/docs/hub/spaces-zerogpu — refer to them for the current backing GPU, runtime version lists, and tier thresholds, all of which change over time.
8
9	## Reference Files
10
11	\| Reference \| When to read \|
12	\|-----------\|--------------\|
13	\| `references/concurrency.md` \| Always read alongside SKILL.md when writing ZeroGPU code — handlers run in parallel by default \|
14	\| `references/how-zerogpu-works.md` \| When reasoning about cold-starts, worker reuse, why module-scope warmup does not carry to requests, or why returning CUDA tensors hangs \|
15	\| `references/how-quota-works.md` \| When choosing `duration` values, debugging `illegal duration` vs `quota exceeded` errors, or explaining why default 60s blocks short tasks \|
16	\| `references/cuda-and-deps.md` \| When installing CUDA-dependent packages (e.g. `flash-attn`), pinning torch side-cars, or reading wheel filename tags \|
17
18	## Hardware
19
20	ZeroGPU exposes two GPU sizes that map to a fraction of the backing card:
21
22	\| `size` \| Slice of backing GPU \| Quota cost \|
23	\|--------\|----------------------\|------------\|
24	\| `large` (default) \| Half \| 1x \|
25	\| `xlarge` \| Full \| 2x \|
26
27	Default `large` gives half a physical GPU, so memory bandwidth and compute are significantly lower than the full card's specs. Use `xlarge` only when the workload genuinely needs the extra memory or compute.
28
29	> Backing GPU changes without notice. ZeroGPU has already migrated across GPU generations several times; older write-ups may name A100 or H200, but those are outdated. For the current backing GPU and exact per-size VRAM, always check the [ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu) before sizing workloads.
30
31	## Basic Pattern
32
33	```python
34	import spaces
35	import torch
36	from transformers import pipeline
37
38	pipe = pipeline("text-generation", model="...", device="cuda")
39
40	@spaces.GPU
41	def generate(prompt: str) -> str:
42	return pipe(prompt, max_new_tokens=100)[0]["generated_text"]
43	```
44
45	Key rules:
46
47	1. Instantiate models at module scope and call `.to("cuda")` eagerly. ZeroGPU handles the actual device mapping transparently (see CUDA availability model below).
48	2. Decorate GPU functions with `@spaces.GPU`. The decorator is a no-op outside ZeroGPU, so it is safe to keep in all environments.
49	3. Set `duration` to match the realistic worst-case workload (default 60s). The platform pre-checks `requested duration` against the user's `remaining quota` — not against the actual run time — so a 10-second task left at the 60s default fails with `quota exceeded` as soon as the user's remaining quota drops below 60s. Smaller declared `duration` also ranks higher in the node-level queue. See "Duration and Quota" below.
50	4. `torch.compile` is NOT supported. Use PyTorch [ahead-of-time compilation (AoTI)](https://huggingface.co/blog/zerogpu-aoti) (torch 2.8+) instead.
51	5. Use `size="xlarge"` sparingly. It allocates the full backing GPU, but costs 2x quota and tends to queue longer.
52
53	```python
54	@spaces.GPU(duration=120)
55	def generate_image(prompt: str):
56	return pipe(prompt).images[0]
57	```
58
59	## CUDA Availability Model
60
61	Real GPU access is only available inside `@spaces.GPU`-decorated functions. Outside those functions, the GPU is not attached to the process.
62
63	However, `import spaces` monkey-patches `torch` so that:
64
65	- `torch.cuda.is_available()` returns `True` globally.
66	- `.to("cuda")` / `device="cuda"` calls at module scope succeed without error.
67
68	This is intentional. Module-scope

People who liked this also viewed

6 picks

Type