deepstream-dev

Name: deepstream-dev
Author: NVIDIA-AI-IOT

76stars18forks

How to install deepstream-dev?

nvidia-ai-iot/deepstream_coding_agent/deepstream-dev

$npx -y skills add NVIDIA-AI-IOT/DeepStream_Coding_Agent --skill deepstream-dev

Installs into the current project.

Summary

NVIDIA DeepStream SDK 9.0 development with Python pyservicemaker API. Use when building video analytics pipelines, GStreamer-based video processing, TensorRT inference integration, object detection/tracking, or Kafka/message broker integration.

Files · 1

View on GitHub

SKILL.md

1	# DeepStream Development Skill
2
3	When this skill is active, ALWAYS read the relevant reference documents before generating code. Do NOT rely on memory - the reference documents contain critical details about exact property names, correct API usage, and common pitfalls.
4
5	## SDK and Architecture Quick Reference
6
7	### DeepStream SDK 9.0 Version Requirements
8
9	- GStreamer: 1.24.2
10	- NVIDIA Driver: 590+
11	- CUDA: 13.1
12	- TensorRT: 10.14.1.48
13	- Platforms: Ubuntu 24.04 (x86_64 and ARM64/Jetson)
14
15	### Typical Pipeline Flow
16
17	```
18	Source → Stream Muxer → Inference → [Tracker] → OSD → Renderer
19	```
20	Components in `[brackets]` are optional -- only add them when the user explicitly requests them.
21
22	\| Stage \| Role \| Key Element(s) \| Required? \|
23	\|-------\|------\|-----------------\|-----------\|
24	\| Source \| Input from files, RTSP, cameras \| `nvurisrcbin` (preferred), `nvmultiurisrcbin`, `filesrc` \| Yes \|
25	\| Stream Muxer \| Batches streams for inference \| `nvstreammux` \| Yes \|
26	\| Inference \| TensorRT model execution \| `nvinfer`, `nvinferserver` \| Yes \|
27	\| Tracker \| Multi-object tracking across frames \| `nvtracker` \| Only if requested \|
28	\| OSD \| Draws bounding boxes, labels, overlays \| `nvosdbin` \| Yes (for visualization) \|
29	\| Renderer \| Display or save output \| `nveglglessink`, `nv3dsink`, `filesink` \| Yes \|
30
31	### Memory Model
32
33	DeepStream uses NVIDIA Video Memory Manager (NVMM) for zero-copy GPU buffer transfers. Caps strings use `memory:NVMM` to indicate GPU memory (e.g., `video/x-raw(memory:NVMM), format=NV12`).
34
35	## Critical Rules
36
37	1. Only Add Requested Components: Do NOT add pipeline elements the user did not ask for.
38	- Tracker (`nvtracker`): Only add when the user explicitly requests tracking or object IDs across frames
39	- Secondary GIEs: Only add when the user requests classification or attribute extraction
40	- Analytics (`nvdsanalytics`): Only add when the user requests line crossing, ROI counting, etc.
41	- Message broker (`nvmsgbroker`/`nvmsgconv`): Only add when the user requests Kafka/cloud messaging
42	- When in doubt, build the minimal working pipeline and let the user ask for additions
43
44	2. Default to `nvurisrcbin` for Sources: When the user says "camera", "stream", "video", or provides a file path:
45	- Always use `nvurisrcbin` -- it handles RTSP, HTTP, and local files (`file://`) transparently
46	- Only use `filesrc` + `qtdemux` + parser when the user explicitly needs raw file source control
47	- For RTSP/live sources, also set `live-source=1` on `nvstreammux` and `sync=0` on the sink
48	- Convert local paths to URI: `"file://" + os.path.abspath(path)`
49
50	3. Metadata Iteration: Use `.frame_items` and `.object_items` (returns iterators, NOT lists)
51	- NEVER use `len()` on these - iterate to count
52	- Iterator can only be consumed once
53
54	4. Request Pad Syntax: Use `"sink_%u"` template, NEVER literal pad names
55	```python
56	pipeline.link(("decoder", "mux"), ("", "sink_%u")) # CORRECT
57	# pipeline.link(("decoder", "mux"), ("", "sink_0")) # WRONG - will fail
58	```
59
60	5. Platform Detection for Sinks:
61	```python
62	import platform
63	sink_type = "nv3dsink" if platform.processor() == "aarch64" else "nveglglessink"
64	```
65
66	6. Buffer Cloning: Always clone buffers for async processing
67	```python
68	tensor = buffer.extract(0).clone() # CRITICAL
69	```
70
71	7. Queue Types:
72	- `queue.Queue` → Use with `threading.Thread`
73	- `multiprocessing.Queue` → Use with `multiprocessing.Process`
74	- Using wrong type causes silent data loss!
75
76	8. nvinfer Config Format:
77	- YAML: Use `property:` section (NOT `model:`), `key: value` with space after colon
78	- INI: Use `[property]` section, `key=value` with equals sign
79	- Section MUST be named `property`
80
81	9. nvmsgbroker is a SINK: Cannot have downstream elements - use `tee` to split pipeline
82
83	10. ALL Sinks Need async=0 for Tee Splits or Dynamic Sources: CRITICAL for state transitions
84	```python
85	# When using tee splits OR dynamic sources, ALL sinks MUST have async=0
86	pipeline.add("nveglglessink", "sink", {
87	"sync": 0, "qos": 0,
88	"async": 0 # CRITICAL - prevents state transition deadlock
89	})
90	```
91	Symptom if missing: Pipeline stays in PAUSED state, no video displays.
92
93	11. Built-in Probe Attachment: `measure_fps_probe` can only be attached to processing elements (e.g., `nvinfer`, `nvosdbin`), NOT to sink elements. Attaching to a sink raises `RuntimeError: Probe failure`.
94
95	12. Dynamic ONNX Models Require `infer-dims`: When the ONNX model has dynamic input shapes (

People who liked this also viewed

6 picks

Type

deepstream-dev

76stars18forks

View on GitHub

How to install deepstream-dev?

nvidia-ai-iot/deepstream_coding_agent/deepstream-dev

$npx -y skills add NVIDIA-AI-IOT/DeepStream_Coding_Agent --skill deepstream-dev

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# DeepStream Development Skill
2
3	When this skill is active, ALWAYS read the relevant reference documents before generating code. Do NOT rely on memory - the reference documents contain critical details about exact property names, correct API usage, and common pitfalls.
4
5	## SDK and Architecture Quick Reference
6
7	### DeepStream SDK 9.0 Version Requirements
8
9	- GStreamer: 1.24.2
10	- NVIDIA Driver: 590+
11	- CUDA: 13.1
12	- TensorRT: 10.14.1.48
13	- Platforms: Ubuntu 24.04 (x86_64 and ARM64/Jetson)
14
15	### Typical Pipeline Flow
16
17	```
18	Source → Stream Muxer → Inference → [Tracker] → OSD → Renderer
19	```
20	Components in `[brackets]` are optional -- only add them when the user explicitly requests them.
21
22	\| Stage \| Role \| Key Element(s) \| Required? \|
23	\|-------\|------\|-----------------\|-----------\|
24	\| Source \| Input from files, RTSP, cameras \| `nvurisrcbin` (preferred), `nvmultiurisrcbin`, `filesrc` \| Yes \|
25	\| Stream Muxer \| Batches streams for inference \| `nvstreammux` \| Yes \|
26	\| Inference \| TensorRT model execution \| `nvinfer`, `nvinferserver` \| Yes \|
27	\| Tracker \| Multi-object tracking across frames \| `nvtracker` \| Only if requested \|
28	\| OSD \| Draws bounding boxes, labels, overlays \| `nvosdbin` \| Yes (for visualization) \|
29	\| Renderer \| Display or save output \| `nveglglessink`, `nv3dsink`, `filesink` \| Yes \|
30
31	### Memory Model
32
33	DeepStream uses NVIDIA Video Memory Manager (NVMM) for zero-copy GPU buffer transfers. Caps strings use `memory:NVMM` to indicate GPU memory (e.g., `video/x-raw(memory:NVMM), format=NV12`).
34
35	## Critical Rules
36
37	1. Only Add Requested Components: Do NOT add pipeline elements the user did not ask for.
38	- Tracker (`nvtracker`): Only add when the user explicitly requests tracking or object IDs across frames
39	- Secondary GIEs: Only add when the user requests classification or attribute extraction
40	- Analytics (`nvdsanalytics`): Only add when the user requests line crossing, ROI counting, etc.
41	- Message broker (`nvmsgbroker`/`nvmsgconv`): Only add when the user requests Kafka/cloud messaging
42	- When in doubt, build the minimal working pipeline and let the user ask for additions
43
44	2. Default to `nvurisrcbin` for Sources: When the user says "camera", "stream", "video", or provides a file path:
45	- Always use `nvurisrcbin` -- it handles RTSP, HTTP, and local files (`file://`) transparently
46	- Only use `filesrc` + `qtdemux` + parser when the user explicitly needs raw file source control
47	- For RTSP/live sources, also set `live-source=1` on `nvstreammux` and `sync=0` on the sink
48	- Convert local paths to URI: `"file://" + os.path.abspath(path)`
49
50	3. Metadata Iteration: Use `.frame_items` and `.object_items` (returns iterators, NOT lists)
51	- NEVER use `len()` on these - iterate to count
52	- Iterator can only be consumed once
53
54	4. Request Pad Syntax: Use `"sink_%u"` template, NEVER literal pad names
55	```python
56	pipeline.link(("decoder", "mux"), ("", "sink_%u")) # CORRECT
57	# pipeline.link(("decoder", "mux"), ("", "sink_0")) # WRONG - will fail
58	```
59
60	5. Platform Detection for Sinks:
61	```python
62	import platform
63	sink_type = "nv3dsink" if platform.processor() == "aarch64" else "nveglglessink"
64	```
65
66	6. Buffer Cloning: Always clone buffers for async processing
67	```python
68	tensor = buffer.extract(0).clone() # CRITICAL
69	```
70
71	7. Queue Types:
72	- `queue.Queue` → Use with `threading.Thread`
73	- `multiprocessing.Queue` → Use with `multiprocessing.Process`
74	- Using wrong type causes silent data loss!
75
76	8. nvinfer Config Format:
77	- YAML: Use `property:` section (NOT `model:`), `key: value` with space after colon
78	- INI: Use `[property]` section, `key=value` with equals sign
79	- Section MUST be named `property`
80
81	9. nvmsgbroker is a SINK: Cannot have downstream elements - use `tee` to split pipeline
82
83	10. ALL Sinks Need async=0 for Tee Splits or Dynamic Sources: CRITICAL for state transitions
84	```python
85	# When using tee splits OR dynamic sources, ALL sinks MUST have async=0
86	pipeline.add("nveglglessink", "sink", {
87	"sync": 0, "qos": 0,
88	"async": 0 # CRITICAL - prevents state transition deadlock
89	})
90	```
91	Symptom if missing: Pipeline stays in PAUSED state, no video displays.
92
93	11. Built-in Probe Attachment: `measure_fps_probe` can only be attached to processing elements (e.g., `nvinfer`, `nvosdbin`), NOT to sink elements. Attaching to a sink raises `RuntimeError: Probe failure`.
94
95	12. Dynamic ONNX Models Require `infer-dims`: When the ONNX model has dynamic input shapes (

People who liked this also viewed

6 picks

Type