vllm-deploy-simple

Name: vllm-deploy-simple
Author: vllm-project

89stars23forks

How to install vllm-deploy-simple?

vllm-project/vllm-skills/vllm-deploy-simple

$npx -y skills add vllm-project/vllm-skills --skill vllm-deploy-simple

Installs into the current project.

Summary

Quick install and deploy vLLM, start serving with a simple LLM, and test OpenAI API.

Files · 1

View on GitHub

SKILL.md

1	# vLLM Simple Deployment
2
3	A simple skill to quickly install vLLM, start a server, and validate the OpenAI-compatible API.
4
5	## What this skill does
6
7	This skill provides a streamlined workflow to:
8	- Detect hardware backend (NVIDIA CUDA, AMD ROCm, Google TPU, or CPU)
9	- Install vLLM with appropriate backend support
10	- Start the vLLM server with configurable model and port
11	- Test the OpenAI-compatible API endpoint
12	- Validate the deployment is working correctly
13	- Support virtual environment isolation
14
15	## Prerequisites
16
17	- Python 3.10+
18	- GPU (NVIDIA CUDA, AMD ROCm) (recommended) or TPU or CPU
19	- pip or uv package manager
20	- curl (for API testing)
21	- Virtual environment (optional but recommended)
22
23	## Usage
24
25	### Create a venv
26
27	If user did not specify the venv path or asked to deploy in the current environment, create a venv using uv with python 3.12 in the current folder. If uv not found, make a folder in this path and use python to create a virtual environment.
28
29	### Run the complete workflow (suggested)
30
31	If user did not specify the venv path, model, or port, use default options:
32
33	```bash
34	# Default deployment options (--venv "." --model "Qwen/Qwen2.5-1.5B-Instruct" --port 8000 --gpu_memory_utilization 0.8)
35	scripts/quickstart.sh
36	```
37
38	Or with custom options:
39
40	```bash
41	# Use custom virtual environment
42	scripts/quickstart.sh --venv /path/to/venv
43
44	# Use custom model and port
45	scripts/quickstart.sh --model "Qwen/Qwen2.5-1.5B-Instruct" --port 8000
46
47	# Use custom GPU memory utilization
48	scripts/quickstart.sh --gpu_memory_utilization 0.6
49
50	# Combine all options
51	scripts/quickstart.sh --venv /path/to/venv --model "Qwen/Qwen2.5-1.5B-Instruct" --port 8000 --gpu_memory_utilization 0.8
52	```
53
54	This will:
55	1. Activate the virtual environment (if specified)
56	2. Detect hardware backend (CUDA/ROCm/TPU/CPU)
57	3. Install vLLM with appropriate backend support
58	4. Start the vLLM server in the background
59	5. Wait for the server to be ready
60	6. Test the API with a sample request
61	7. Display the server status
62
63	### Run individual commands (for step-by-step usage or troubleshooting)
64
65	Install vLLM:
66	```bash
67	scripts/quickstart.sh install
68	# Or with virtual environment
69	scripts/quickstart.sh install --venv /path/to/venv
70	```
71
72	Start the server:
73	```bash
74	scripts/quickstart.sh start
75	# Or with custom options
76	scripts/quickstart.sh start --venv /path/to/venv --model "Qwen/Qwen2.5-1.5B-Instruct" --port 8000 --gpu_memory_utilization 0.8
77	```
78
79	Test the API:
80	```bash
81	scripts/quickstart.sh test
82	# Or with custom port
83	scripts/quickstart.sh test --port 8000
84	```
85
86	Stop the server:
87	```bash
88	scripts/quickstart.sh stop
89	# Or with virtual environment
90	scripts/quickstart.sh stop --venv /path/to/venv
91	```
92
93	Check server status:
94	```bash
95	scripts/quickstart.sh status
96	```
97
98	Restart the server:
99	```bash
100	scripts/quickstart.sh restart
101	# Or with custom options
102	scripts/quickstart.sh restart --venv /path/to/venv --port 8000 --gpu_memory_utilization 0.8
103	```
104
105	## Configuration
106
107	The script supports the following command-line options:
108
109	```bash
110	scripts/quickstart.sh [command] [OPTIONS]
111
112	Commands:
113	install - Install vLLM and dependencies
114	start - Start the vLLM server
115	stop - Stop the vLLM server
116	test - Test the OpenAI-compatible API
117	status - Show server status
118	restart - Restart the server
119	all - Run complete workflow (default)
120
121	Options:
122	--model MODEL Model to use (default: Qwen/Qwen2.5-1.5B-Instruct)
123	--port PORT Port to run server on (default: 8000)
124	--venv VENV_PATH Virtual environment path (default: .)
125	--gpu_memory_utilization VRAM GPU memory utilization (default: 0.8)
126	```
127
128	### Hardware Backend Detection
129
130	The script automatically detects your hardware and installs the appropriate vLLM version:
131
132	- NVIDIA CUDA: Detected via `nvidia-smi` command
133	- AMD ROCm: Detected via `/dev/kfd` and `/dev/dri` devices
134	- Google TPU: Detected via `TPU_NAME` environment variable or `gcloud` command
135	- CPU: Fallback if no GPU/TPU detected
136
137	For Google TPU, the script installs `vllm-tpu` instead of the standard `vllm` package.
138
139	## API Testing
140
141	The test script sends a simple chat completion request:
142
143	```bash
144	curl http://localhost:8000/v1/chat/completions \
145	-H "Content-Type: application/json" \
146	-d '{
147	"model": "Qwen/Qwen2.5-1.5B-Instruct",
148	"messages": [{"role": "user", "content": "Say hello!"}],
149	"max_tokens": 50
150	}'
151	```
152
153	## Troubleshooting
154
155	Virtual environment not found:
156	- Ensure the path provided with `--venv` exists and is a valid virtual environment
157	- Check that the activation script exists (`bin/activate` on Linux/macOS or `Scripts/activate` on Windows)
158	- Check and install uv, and create a new virtual environment with uv: `uv venv /path/to/venv` (suggested); or with pip: `python3 -m venv /path/to/venv`
159
160	Server won't start:
161	- Check if the port is already in use: `l

People who liked this also viewed

6 picks

Type

microsoft-foundryDeploy, evaluate, fine-tune, and manage Foundry agents end-to-end with azd: hosted agent scaffold/run/deploy, prompt agent create, batch eval, continuous eval, prompt optimizer, Agent Optimizer scaffoSkillsJul 2026476k1.3k
azure-deployExecute Azure deployments for ALREADY-PREPARED applications that have existing .azure/deployment-plan.md and infrastructure files. DO NOT use this skill when the user asks to CREATE a new application SkillsJul 2026472k1.3k
azure-preparePrepare azd-based Azure projects for deployment: generates azure.yaml, infrastructure (Bicep/Terraform), and Dockerfiles for the Azure Developer CLI (azd) workflow. USE ONLY when the user explicitly wSkillsJul 2026471k1.3k
azure-validate--- name: azure-validate description: "Pre-deployment validation for Azure readiness. Run deep checks on configuration, infrastructure (Bicep or Terraform), RBAC role assignments, managed identity pSkillsJul 2026471k1.3k
azure-aigatewayConfigure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbSkillsJul 2026470k1.3k
azure-hosted-copilot-sdkOfficial agent plugin providing skills and MCP server configurations for Azure scenarios.SkillsJul 2026424k1.3k

vllm-deploy-simple

89stars23forks

View on GitHub

How to install vllm-deploy-simple?

vllm-project/vllm-skills/vllm-deploy-simple

$npx -y skills add vllm-project/vllm-skills --skill vllm-deploy-simple

Installs into the current project.

Summary

Quick install and deploy vLLM, start serving with a simple LLM, and test OpenAI API.

Files · 1

View on GitHub

SKILL.md

1	# vLLM Simple Deployment
2
3	A simple skill to quickly install vLLM, start a server, and validate the OpenAI-compatible API.
4
5	## What this skill does
6
7	This skill provides a streamlined workflow to:
8	- Detect hardware backend (NVIDIA CUDA, AMD ROCm, Google TPU, or CPU)
9	- Install vLLM with appropriate backend support
10	- Start the vLLM server with configurable model and port
11	- Test the OpenAI-compatible API endpoint
12	- Validate the deployment is working correctly
13	- Support virtual environment isolation
14
15	## Prerequisites
16
17	- Python 3.10+
18	- GPU (NVIDIA CUDA, AMD ROCm) (recommended) or TPU or CPU
19	- pip or uv package manager
20	- curl (for API testing)
21	- Virtual environment (optional but recommended)
22
23	## Usage
24
25	### Create a venv
26
27	If user did not specify the venv path or asked to deploy in the current environment, create a venv using uv with python 3.12 in the current folder. If uv not found, make a folder in this path and use python to create a virtual environment.
28
29	### Run the complete workflow (suggested)
30
31	If user did not specify the venv path, model, or port, use default options:
32
33	```bash
34	# Default deployment options (--venv "." --model "Qwen/Qwen2.5-1.5B-Instruct" --port 8000 --gpu_memory_utilization 0.8)
35	scripts/quickstart.sh
36	```
37
38	Or with custom options:
39
40	```bash
41	# Use custom virtual environment
42	scripts/quickstart.sh --venv /path/to/venv
43
44	# Use custom model and port
45	scripts/quickstart.sh --model "Qwen/Qwen2.5-1.5B-Instruct" --port 8000
46
47	# Use custom GPU memory utilization
48	scripts/quickstart.sh --gpu_memory_utilization 0.6
49
50	# Combine all options
51	scripts/quickstart.sh --venv /path/to/venv --model "Qwen/Qwen2.5-1.5B-Instruct" --port 8000 --gpu_memory_utilization 0.8
52	```
53
54	This will:
55	1. Activate the virtual environment (if specified)
56	2. Detect hardware backend (CUDA/ROCm/TPU/CPU)
57	3. Install vLLM with appropriate backend support
58	4. Start the vLLM server in the background
59	5. Wait for the server to be ready
60	6. Test the API with a sample request
61	7. Display the server status
62
63	### Run individual commands (for step-by-step usage or troubleshooting)
64
65	Install vLLM:
66	```bash
67	scripts/quickstart.sh install
68	# Or with virtual environment
69	scripts/quickstart.sh install --venv /path/to/venv
70	```
71
72	Start the server:
73	```bash
74	scripts/quickstart.sh start
75	# Or with custom options
76	scripts/quickstart.sh start --venv /path/to/venv --model "Qwen/Qwen2.5-1.5B-Instruct" --port 8000 --gpu_memory_utilization 0.8
77	```
78
79	Test the API:
80	```bash
81	scripts/quickstart.sh test
82	# Or with custom port
83	scripts/quickstart.sh test --port 8000
84	```
85
86	Stop the server:
87	```bash
88	scripts/quickstart.sh stop
89	# Or with virtual environment
90	scripts/quickstart.sh stop --venv /path/to/venv
91	```
92
93	Check server status:
94	```bash
95	scripts/quickstart.sh status
96	```
97
98	Restart the server:
99	```bash
100	scripts/quickstart.sh restart
101	# Or with custom options
102	scripts/quickstart.sh restart --venv /path/to/venv --port 8000 --gpu_memory_utilization 0.8
103	```
104
105	## Configuration
106
107	The script supports the following command-line options:
108
109	```bash
110	scripts/quickstart.sh [command] [OPTIONS]
111
112	Commands:
113	install - Install vLLM and dependencies
114	start - Start the vLLM server
115	stop - Stop the vLLM server
116	test - Test the OpenAI-compatible API
117	status - Show server status
118	restart - Restart the server
119	all - Run complete workflow (default)
120
121	Options:
122	--model MODEL Model to use (default: Qwen/Qwen2.5-1.5B-Instruct)
123	--port PORT Port to run server on (default: 8000)
124	--venv VENV_PATH Virtual environment path (default: .)
125	--gpu_memory_utilization VRAM GPU memory utilization (default: 0.8)
126	```
127
128	### Hardware Backend Detection
129
130	The script automatically detects your hardware and installs the appropriate vLLM version:
131
132	- NVIDIA CUDA: Detected via `nvidia-smi` command
133	- AMD ROCm: Detected via `/dev/kfd` and `/dev/dri` devices
134	- Google TPU: Detected via `TPU_NAME` environment variable or `gcloud` command
135	- CPU: Fallback if no GPU/TPU detected
136
137	For Google TPU, the script installs `vllm-tpu` instead of the standard `vllm` package.
138
139	## API Testing
140
141	The test script sends a simple chat completion request:
142
143	```bash
144	curl http://localhost:8000/v1/chat/completions \
145	-H "Content-Type: application/json" \
146	-d '{
147	"model": "Qwen/Qwen2.5-1.5B-Instruct",
148	"messages": [{"role": "user", "content": "Say hello!"}],
149	"max_tokens": 50
150	}'
151	```
152
153	## Troubleshooting
154
155	Virtual environment not found:
156	- Ensure the path provided with `--venv` exists and is a valid virtual environment
157	- Check that the activation script exists (`bin/activate` on Linux/macOS or `Scripts/activate` on Windows)
158	- Check and install uv, and create a new virtual environment with uv: `uv venv /path/to/venv` (suggested); or with pip: `python3 -m venv /path/to/venv`
159
160	Server won't start:
161	- Check if the port is already in use: `l

People who liked this also viewed

6 picks

Type