NVIDIA-AI-Blueprints/video-search-and-summarization

---

name: benchmark-video-search

description: Measure retrieval quality and latency of a deployed VSS search profile by ingesting a labelled dataset and running the vss CLI across retrieval paths. Use when the user asks to "benchmark video search", "compare search recall between builds", or "profile search latency" on a deployed VSS profile.

---

name: profile-vllm-performance

description: Profile vLLM and RT-VLM inference to identify and remove GPU-idle gaps, underfilled batches, transfer stalls, serialized multimodal work, scheduler gaps, or KV pressure. Use this skill when GPU utilization is unexpectedly low or bursty, capacity trails another runtime, TTFT or ITL regresses, or multimodal prefill and decode appear serialized. Not for running a first benchmark without a reproducible fixed-load signal.

---

name: rtvi-vlm-perf-testing

description: Plan, run, and diagnose reproducible RT-VLM GPU performance canaries and benchmarks. Use this skill when running fresh-container stream-capacity, semantic-isolation, latency, throughput, or regression experiments against an RTVI microservices checkout.

---

name: vss-benchmark-video-summarization

description: Benchmark a deployed LVS instance — set up test media, run single-file latency and burst-throughput tests, analyze GPU and latency metrics, and get configuration recommendations to improve performance.

---

name: vss-benchmark-vlm-qa

description: Benchmark video Q&A accuracy and latency of a deployed RT-VLM (Cosmos Reason 3) via vss vlm run, using questions and videos from the DSS vss-devx-base dataset. Replaces the deprecated nat eval / vss-agent QA path. Not for tool-calling or trajectory evaluation, and not for LVS summarization throughput.

---

name: vss-evaluate-caption-accuracy

description: Measure whether an RT-VLM configuration change altered caption quality — capture paired baseline and candidate captions for a set of videos, score both against a ground truth with an LLM judge, and emit an accuracy and processing-time table. Use when changing frame selection, decode, or model settings and you need evidence there is no accuracy regression.

---

name: rtvi-byom-porting

description: Use when adding, debugging, or validating a bring-your-own VLM in VSS RT-VLM, including custom Hugging Face or NGC checkpoints, vLLM adapters or plugins, model shims, and model-specific runtime dependencies. Not for selecting an already-supported model or ordinary RT-VLM deployment.

---

name: vss-deploy-dense-captioning

description: Use this skill when deploying standalone RT-VLM dense captioning or calling its REST API (uploads, captions, streams, chat-completions, Kafka). Not for VSS profile deploy or video-search ingestion.

---

name: vss-deploy-detection-tracking-2d

description: Use this skill when the user wants to deploy, run, debug, tear down, or call the REST API of the RTVI-CV 2D detection / tracking microservice. Trigger when the user says things like 'deploy rtvi-cv', 'start warehouse 2d', 'add a stream', 'check rtvi-cv health', or 'stop the perception container'. Not for VLM, embedding, or analytics — use the matching vss-* skill.

---

name: vss-deploy-detection-tracking-3d

description: Use when deploying or operating standalone RTVI-CV-3D / MV3DT multi-camera 3D tracking for calibrated MP4/file inputs and live RTSP streams: missing-calibration handoff to AMC skills, the 4-camera sample dataset, camera config, BEV Fusion, live OSD or saved grid/BEV outputs, bundled brokers, basic external MQTT/Kafka brokers, verification, and teardown. Trigger for generic MV3DT, RTVI-CV-3D, multi-view 3D tracking, multi-cam tracking, or sample MV3DT dataset requests. Explicit warehouse blueprint/profile MV3DT requests route to vss-build-vision-ai; single-camera 2D tracking routes to the 2D tracking or DeepStream skills. Not for full warehouse blueprint deployment, single-camera 2D tracking, camera calibration itself, or VSS summarization, Q&A, and RAG workflows.

---

name: vss-deploy-video-embedding

description: Use this skill when deploying, operating, integrating, or customizing the VSS RT-Embed Video Embedding microservice. Covers standalone Docker Compose deployment, the `/v1` REST API for text/video embeddings and live streams, Redis/Kafka/OTel integration, troubleshooting, and bring-your-own-model (BYOM) custom embedding backends, with VideoPrism as an example. Do not use for RT-CV, RT-VLM, VSS Agent, or general VSS deployment work that does not include RT-Embed.

---

name: vss-deploy-warehouse-helm

description: Use when the user asks to deploy, upgrade, or size the VSS warehouse blueprint (2D / 3D / MV3DT) on Kubernetes via Helm — as opposed to Docker Compose, which is covered by vss-build-vision-ai's warehouse reference. Handles GPU-aware NUM_STREAMS capping so the deployment matches what the perception pipeline can actually sustain.

---

name: vss-setup-behavior-analytics

description: Use this skill to deploy the vss-behavior-analytics service standalone (entrypoint, config-source, optional calibration). Not for the full warehouse deploy.

---

name: vss-setup-video-analytics-api

description: Use to deploy the vss-video-analytics-api REST service standalone with its Elasticsearch ingest-pipeline, selectable Kafka/Redis stream type, and Kafka-topic readiness gates when Kafka is selected. Not for full warehouse deploy.

---

name: vss-ask-video

description: Use this skill when answering a question about previously analyzed or freshly scoped VSS video, or when reading a stored VSS memory job or record by id, or whenever a question should be answered by running the `vss memory introspect` command. Route through hot context, agent Markdown notes, `vss memory get` or `vss memory query`, `vss memory introspect`, or an exact-window `vss vlm run`. Not for video retrieval or metadata-answerable questions.

---

name: vss-generate-video-report

description: Use this skill when producing a VSS analysis report — Mode A per-clip VLM, Mode B incident-range via video-analytics, Mode C SOP compliance via the SOP tools. Not for standalone video summarization, real-time alerts or ad-hoc Q&A.

---

name: vss-generate-video-report-rag

description: Generates VSS video summary reports with LVS HITL and optional Enterprise RAG document grounding. Trigger when the user asks for a frag/RAG-assisted video report, knowledge-enhanced analysis, or Enterprise RAG context in a video summary.

---

name: vss-manage-alerts

description: Use this skill when operating VSS alert workflows — real-time monitoring, Alert-Bridge subscriptions, verification verdicts, on-demand verification, always-on operation, Slack notifications, incident queries, or camera onboarding. Not for non-alert analytics.

---

name: vss-manage-video-io-storage

description: Use to drive `vss vios` for sensor list, timelines, clips, snapshots, and add/delete of video or stream sources, and the VIOS REST API only for what that CLI does not cover (sensor info/status/settings, storage and recorder status, WebRTC, the RTSP proxy, network scan, device settings, bytes to disk, the NvStreamer API), when the caller names a REST endpoint, or to debug VIOS. Also provisions a source into an Agent-backed or headless deployment, which the deployment fans out to the perception consumers (RT-CV/RT-Embed/RT-VLM). Not for VLM inference, semantic search.

---

name: vss-query-analytics

description: Use this skill for read-only incident, occupancy, speed, place, and analytics-sensor questions through the project-local VSS CLI. Not for live VLM, incident-range narrative reports, deployment, or alert-rule management.

---

name: vss-search-archive

description: Use this skill when a user wants to search archived VSS video that is already registered in a configured deployment — by natural-language, similarity, attribute, object-ID, or lexical tag query. Not for fresh clip Q&A, live captioning, video summarization, deployment, or source ingestion/deletion.

---

name: vss-summarize-video

description: Use when summarizing a recorded video through HITL-gated LVS, falling back to `vss vlm run` when LVS is not ready. Not for reports, archive search, or live RTSP captioning.

---

name: vss-generate-video-calibration

description: Use this skill when running AutoMagicCalib on local MP4s, RTSP, or the bundled sample dataset, or when deploying vss-auto-calibration. Do not use for non-AMC calibration or runtime analytics.

---

name: vss-build-vision-ai

description: Add agent-ready vision capabilities — dense captioning, detection, search, alerting, summarization — to an agent or application through a customizable, self-contained vision stack built on the NVIDIA VSS Blueprint. Use this skill when a developer or agent wants to give their app vision: pick capabilities via guided intake ("build a vision agent", "add vision capabilities") or describe them in natural language ("create a profile for streaming dense captioning", "add agentic search to my base deployment", "deploy warehouse 3d"). Route, compose, configure, and deploy stock base, alerts, LVS, search developer profiles, or the warehouse industry profile and lean custom combinations expressed as delta overlays using one current developer profile as the Foundation. Not for operating a stack that is already deployed — searching, asking about a video, summarizing, managing alerts, or generating a report — and not for deploying a single microservice on its own; use the matching vss-* skill for those.