video-search-and-summarization
starsforks
How to install video-search-and-summarization?
nvidia-ai-blueprints/video-search-and-summarization
$git clone https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarizationThe NVIDIA Blueprint for Video Search and Summarization (VSS) provides a suite of reference architectures for building vision agents and AI-powered video analytics applications. Those architectures bring together accelerated vision
README.md
| 1 | <h2>NVIDIA AI Blueprint: Video Search and Summarization (VSS)</h2> |
| 2 | |
| 3 | ### Table of Contents |
| 4 | - [Overview](#overview) |
| 5 | - [Use Case / Problem Description](#use-case--problem-description) |
| 6 | - [Agent Workflows](#agent-workflows) |
| 7 | - [Software Components](#software-components) |
| 8 | - [Target Audience](#target-audience) |
| 9 | - [Repository Structure Overview](#repository-structure-overview) |
| 10 | - [Documentation](#documentation) |
| 11 | - [Prerequisites](#prerequisites) |
| 12 | - [Hardware Requirements](#hardware-requirements) |
| 13 | - [Quickstart Guide](#quickstart-guide) |
| 14 | - [Contributing](#contributing) |
| 15 | - [License](#license) |
| 16 | |
| 17 | ## Overview |
| 18 | |
| 19 | The [NVIDIA Blueprint for Video Search and Summarization (VSS)](https://docs.nvidia.com/vss/latest/index.html) provides a suite of reference architectures for building vision agents and AI-powered video analytics applications. Those architectures bring together accelerated vision microservices, vision language models (VLMs), and large language models (LLMs) so you can use them in existing applications, as standalone microservices, or as part of a larger vision agent. |
| 20 | |
| 21 | VSS is organized into three areas of processing and analysis: **real-time video intelligence** (feature extraction, embeddings, and stream understanding with results published to a message broker), **downstream analytics** (enrichment of metadata into trajectories, incidents, and verified alerts), and **agentic and offline processing** (orchestrated tools for search, Q&A, summarization, and clip retrieval, including via the Model Context Protocol). |
| 22 | |
| 23 | This repository implements the blueprint and powers the [NVIDIA build experience](https://build.nvidia.com/nvidia/video-search-and-summarization) for natural-language video agents—search, summarization, visual Q&A, and related workflows—backed by generative AI, VLMs and LLMs, and [NVIDIA NIM](https://build.nvidia.com/) microservices as configured in the stacks below. |
| 24 | |
| 25 | ## Use Case / Problem Description |
| 26 | |
| 27 | The NVIDIA AI Blueprint for Video Search and Summarization addresses the challenge of deploying visual agents capable of interacting with large volumes of video data, both stored and streamed. This can be used to create vision AI agents, that can be applied to a multitude of use cases such as monitoring smart spaces, warehouse automation, and SOP validation. This is important where quick and accurate video analysis can lead to better decision-making and enhanced operational efficiency. |
| 28 | |
| 29 | ## Agent Workflows |
| 30 | We provide multiple reference [Agent Workflows](https://docs.nvidia.com/vss/latest/agent-workflows.html) which demonstrate how the individual components can be leveraged by an agent: |
| 31 | |
| 32 | | Workflow | Description | |
| 33 | |----------|-------------| |
| 34 | | [Q&A and Report Generation (Quickstart)](https://docs.nvidia.com/vss/latest/quickstart.html) | Video retrieval, VLM-based Q&A, and report generation on short video clips | |
| 35 | | [Alert Verification](https://docs.nvidia.com/vss/latest/agent-workflow-alert-verification.html) | Realtime processing of videos using perception (object detection, tracking) and behavior analytics to generate alerts, which are subsequently verified with VLM to reduce false positives | |
| 36 | | [Real-Time Alerts](https://docs.nvidia.com/vss/latest/agent-workflow-rt-alert.html) | Continuous processing of video streams through VLM for anomaly detection | |
| 37 | | [Video Search](https://docs.nvidia.com/vss/latest/agent-workflow-search.html) | Natural language search across video archives using video embeddings (alpha) | |
| 38 | | [Long Video Summarization](https://docs.nvidia.com/vss/latest/agent-workflow-lvs.html) | Analysis and summarization of extended video recordings through chunking and aggregation of dense captions | |
| 39 | |
| 40 | ## Software Components |
| 41 | <div align="center"> |
| 42 | <img src="https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization/raw/main/assets/vss-architecture.png" width="800"> |
| 43 | </div> |
| 44 | |
| 45 | 1. **NIM microservices**: Here are models used in this blueprint: |
| 46 | |
| 47 | - [Cosmos3 Nano Reasoner](https://build.nvidia.com/nvidia/cosmos3-nano-reasoner |
People who liked this also viewed
6 picksType
- elevenlabs-mcpThe official ElevenLabs MCP serverMCP ServersJul 2026
- MonetEdit Videos and Design Images with Claude code or CodexMCP ServersJul 2026
- scribe.jsJavaScript OCR and text extraction for images and PDFs.MCP ServersJul 2026
- gliana-aiPay-per-call generative AI — 60+ models (image, video, music, speech). No signup, no API key; pay per call from your wallet over MPP/x402.MCP Serversrecently
- nanobanana-mcp-serverAI image generation MCP server powered by Google Gemini, with smart model selection and 4K outputMCP ServersMay 2026
- FramesailCreate long-form (faceless YouTube) videos end to end from any MCP client: script, locked character references, storyboard, voiceover, and final video editing — with characters and style held consistent across every shot. Making long-form MCP Serversrecently
