deepstream-import-vision-model

Name: deepstream-import-vision-model
Author: NVIDIA-AI-IOT

76stars18forks

How to install deepstream-import-vision-model?

nvidia-ai-iot/deepstream_coding_agent/deepstream-import-vision-model

$npx -y skills add NVIDIA-AI-IOT/DeepStream_Coding_Agent --skill deepstream-import-vision-model

Installs into the current project.

Summary

Use this skill to bring any vision model from HuggingFace or NVIDIA NGC into an NVIDIA DeepStream pipeline with end-to-end automation: ONNX download, SafeTensors export, TRT engine build, custom nvinfer bbox parser, multi-stream benchmark, and PDF report. Object detection models

Files · 1

View on GitHub

SKILL.md

1	# DeepStream Import Vision Model
2
3	When this skill is active, read the relevant reference document before starting each phase. Do not rely on memory — reference documents contain exact script paths, bash variable conventions, log filename contracts, and critical parsing rules.
4
5	Current scope: Object detection models only. Fail fast on classification, segmentation, or other architectures detected in `config.json`.
6
7	## Pipeline Overview
8
9	\| Step \| Phase \| Reference \| What it does \|
10	\|------\|-------\|-----------\|--------------\|
11	\| 1–3 \| Model Acquire \| [references/model-acquire.md](references/model-acquire.md) \| Browse HF/NGC, detect format, download ONNX or export SafeTensors \|
12	\| 4–5 \| Engine Build \| [references/engine-build.md](references/engine-build.md) \| Build dynamic TRT engine, run trtexec BS=1 and BS=MAX_BS \|
13	\| 6–7 \| DS Pipeline \| [references/pipeline-run.md](references/pipeline-run.md) \| Custom bbox parser, nvinfer config, single-stream + multi-stream benchmarks \|
14	\| 8 \| Report \| [references/report-generation.md](references/report-generation.md) \| 5 charts, HTML, PDF benchmark report \|
15
16	Run the full pipeline autonomously without pausing for confirmation at each step.
17
18	## Pre-flight Checks
19
20	Run before starting:
21
22	```bash
23	# 1. GPU and drivers
24	nvidia-smi
25
26	# 2. TensorRT version match (must match between builder and DS runtime)
27	trtexec 2>&1 \| head -3
28	dpkg -l \| grep libnvinfer-bin
29
30	# 3. Shared Python venv — create once, reuse across all models
31	mkdir -p build
32	VENV=build/.venv_optimum
33	if [ ! -x "$VENV/bin/python3" ]; then
34	python3 -m venv "$VENV"
35	"$VENV/bin/pip" install --upgrade pip -q
36	"$VENV/bin/pip" install "optimum[exporters]>=1.20,<2.0" "torch<2.12" \
37	transformers onnxruntime matplotlib numpy markdown -q
38	fi
39
40	# 4. System tools
41	which wkhtmltopdf \|\| apt-get install -y wkhtmltopdf
42	which mediainfo \|\| apt-get install -y mediainfo
43	which deepstream-app # required for KITTI dump (Step 6g) and benchmark perf-measurement (Step 7c); shipped with DeepStream SDK
44
45	# 5. Sample video — only check default path when user has not provided a custom DS_VIDEO
46	if [ -z "$DS_VIDEO" ]; then
47	[ -f /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4 ] \|\| \
48	echo "WARNING: sample_720p.mp4 not found. Install DeepStream samples or set DS_VIDEO=/path/to/your.mp4"
49	fi
50	```
51
52	## Mandatory Output Structure
53
54	Create once `MODEL_NAME` is known (Step 1). Never dump files flat.
55
56	```
57	models/{model_name}/
58	model/ <- ONNX file(s)
59	parser/ <- .cpp, Makefile, .so
60	config/ <- nvinfer config, ds-app config, labels.txt
61	scripts/ <- run helper scripts
62	benchmarks/
63	engines/ <- _dynamic_b{MAX_BS}.engine, timing.cache, build logs
64	b1/ <- trtexec BS=1 log
65	b{MAX_BS}/ <- trtexec BS=MAX_BS log
66	ds/ <- DS benchmark logs
67	reports/ <- benchmark_report.md, .html, .pdf, benchmark_data.json
68	charts/ <- chart_*.png (5 charts)
69	samples/ <- output .mp4 or .ogv (theoraenc fallback), test frames
70	kitti_output/ <- KITTI detection .txt files
71	```
72
73	```bash
74	mkdir -p models/$MODEL_NAME/{model,parser,config,scripts,benchmarks/engines,benchmarks/ds,reports/charts,samples/kitti_output}
75	```
76
77	## Critical Rules
78
79	1. Engine naming — always `{model}_dynamic_b{MAX_BS}.engine`. Never bare `model_dynamic.engine`.
80	2. batch_size == num_streams — in DS runs, `batch-size` and stream count are always equal.
81	3. Log filenames are fixed — `trtexec_b1.log`, `trtexec_b${MAX_BS}.log`, `ds_s${N}_run1.log`, `ds_s${N}_run2.log`. No timestamps. Report generation reads exact paths.
82	4. Parser zero-init — always `NvDsInferObjectDetectionInfo obj = {};`. Required for DS 9.0 OBB support; bare `obj;` leaves `rotation_angle` uninitialized, causing tilted bounding boxes.
83	5. KITTI validation gate — do NOT proceed to Step 7 if KITTI frame count is zero or detection rate < 90%.
84	6. Shared venv — `build/.venv_optimum` reused across all models. Never create per-model venvs.
85	7. trtexec `--noDataTransfers` — GPU-only compute matches DeepStream's GPU-to-GPU data flow.
86	8. Report HTML+PDF — always use `skills/deepstream-import-vision-model/scripts/report/md-to-html-pdf.py`. Never write a custom HTML generator or call `wkhtmltopdf` directly.
87	9. Object detection only — reject non-detection architectures from `config.json` before building anything.
88	10. Encoder fallback (MANDATORY) — `x264enc` and `openh264enc` are prohibited. On NVENC-unavailable systems, use `theoraenc + oggmux` (LGPL; s

People who liked this also viewed

6 picks

Type

deepstream-import-vision-model

76stars18forks

View on GitHub

How to install deepstream-import-vision-model?

nvidia-ai-iot/deepstream_coding_agent/deepstream-import-vision-model

$npx -y skills add NVIDIA-AI-IOT/DeepStream_Coding_Agent --skill deepstream-import-vision-model

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# DeepStream Import Vision Model
2
3	When this skill is active, read the relevant reference document before starting each phase. Do not rely on memory — reference documents contain exact script paths, bash variable conventions, log filename contracts, and critical parsing rules.
4
5	Current scope: Object detection models only. Fail fast on classification, segmentation, or other architectures detected in `config.json`.
6
7	## Pipeline Overview
8
9	\| Step \| Phase \| Reference \| What it does \|
10	\|------\|-------\|-----------\|--------------\|
11	\| 1–3 \| Model Acquire \| [references/model-acquire.md](references/model-acquire.md) \| Browse HF/NGC, detect format, download ONNX or export SafeTensors \|
12	\| 4–5 \| Engine Build \| [references/engine-build.md](references/engine-build.md) \| Build dynamic TRT engine, run trtexec BS=1 and BS=MAX_BS \|
13	\| 6–7 \| DS Pipeline \| [references/pipeline-run.md](references/pipeline-run.md) \| Custom bbox parser, nvinfer config, single-stream + multi-stream benchmarks \|
14	\| 8 \| Report \| [references/report-generation.md](references/report-generation.md) \| 5 charts, HTML, PDF benchmark report \|
15
16	Run the full pipeline autonomously without pausing for confirmation at each step.
17
18	## Pre-flight Checks
19
20	Run before starting:
21
22	```bash
23	# 1. GPU and drivers
24	nvidia-smi
25
26	# 2. TensorRT version match (must match between builder and DS runtime)
27	trtexec 2>&1 \| head -3
28	dpkg -l \| grep libnvinfer-bin
29
30	# 3. Shared Python venv — create once, reuse across all models
31	mkdir -p build
32	VENV=build/.venv_optimum
33	if [ ! -x "$VENV/bin/python3" ]; then
34	python3 -m venv "$VENV"
35	"$VENV/bin/pip" install --upgrade pip -q
36	"$VENV/bin/pip" install "optimum[exporters]>=1.20,<2.0" "torch<2.12" \
37	transformers onnxruntime matplotlib numpy markdown -q
38	fi
39
40	# 4. System tools
41	which wkhtmltopdf \|\| apt-get install -y wkhtmltopdf
42	which mediainfo \|\| apt-get install -y mediainfo
43	which deepstream-app # required for KITTI dump (Step 6g) and benchmark perf-measurement (Step 7c); shipped with DeepStream SDK
44
45	# 5. Sample video — only check default path when user has not provided a custom DS_VIDEO
46	if [ -z "$DS_VIDEO" ]; then
47	[ -f /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.mp4 ] \|\| \
48	echo "WARNING: sample_720p.mp4 not found. Install DeepStream samples or set DS_VIDEO=/path/to/your.mp4"
49	fi
50	```
51
52	## Mandatory Output Structure
53
54	Create once `MODEL_NAME` is known (Step 1). Never dump files flat.
55
56	```
57	models/{model_name}/
58	model/ <- ONNX file(s)
59	parser/ <- .cpp, Makefile, .so
60	config/ <- nvinfer config, ds-app config, labels.txt
61	scripts/ <- run helper scripts
62	benchmarks/
63	engines/ <- _dynamic_b{MAX_BS}.engine, timing.cache, build logs
64	b1/ <- trtexec BS=1 log
65	b{MAX_BS}/ <- trtexec BS=MAX_BS log
66	ds/ <- DS benchmark logs
67	reports/ <- benchmark_report.md, .html, .pdf, benchmark_data.json
68	charts/ <- chart_*.png (5 charts)
69	samples/ <- output .mp4 or .ogv (theoraenc fallback), test frames
70	kitti_output/ <- KITTI detection .txt files
71	```
72
73	```bash
74	mkdir -p models/$MODEL_NAME/{model,parser,config,scripts,benchmarks/engines,benchmarks/ds,reports/charts,samples/kitti_output}
75	```
76
77	## Critical Rules
78
79	1. Engine naming — always `{model}_dynamic_b{MAX_BS}.engine`. Never bare `model_dynamic.engine`.
80	2. batch_size == num_streams — in DS runs, `batch-size` and stream count are always equal.
81	3. Log filenames are fixed — `trtexec_b1.log`, `trtexec_b${MAX_BS}.log`, `ds_s${N}_run1.log`, `ds_s${N}_run2.log`. No timestamps. Report generation reads exact paths.
82	4. Parser zero-init — always `NvDsInferObjectDetectionInfo obj = {};`. Required for DS 9.0 OBB support; bare `obj;` leaves `rotation_angle` uninitialized, causing tilted bounding boxes.
83	5. KITTI validation gate — do NOT proceed to Step 7 if KITTI frame count is zero or detection rate < 90%.
84	6. Shared venv — `build/.venv_optimum` reused across all models. Never create per-model venvs.
85	7. trtexec `--noDataTransfers` — GPU-only compute matches DeepStream's GPU-to-GPU data flow.
86	8. Report HTML+PDF — always use `skills/deepstream-import-vision-model/scripts/report/md-to-html-pdf.py`. Never write a custom HTML generator or call `wkhtmltopdf` directly.
87	9. Object detection only — reject non-detection architectures from `config.json` before building anything.
88	10. Encoder fallback (MANDATORY) — `x264enc` and `openh264enc` are prohibited. On NVENC-unavailable systems, use `theoraenc + oggmux` (LGPL; s

People who liked this also viewed

6 picks

Type