vss-deploy-video-embedding

VSS Video Embedding (RT-Embed)

Use this skill for the RT-Embed video embedding microservice, including the standard Cosmos-Embed1 deployment path and custom/BYOM embedding model work.

Trigger phrases: vss-deploy-video-embedding, RT-Embed, rtvi-embed, video embedding service, Cosmos-Embed1, embed live stream, embed video file, generate video embeddings, text embedding for video search, RT-Embed BYOM, VideoPrism embed, custom embed model, MODEL_IMPLEMENTATION_PATH, MODEL_REPOSITORY_SCRIPT_PATH, bring your own embedding model.

Do not use this skill for RT-CV, RT-VLM, VSS Agent, or general VSS deployment work unless the request deploys, operates, integrates, or customizes RT-Embed.

When to Use

• Deploy, size, upgrade, roll back, or tear down standalone RT-Embed (Cosmos-Embed1 or a custom/BYOM model)
• Call RT-Embed's /v1 API for text/video embeddings, live streams, model listing, health, metrics
• Wire RT-Embed into another service with Redis, Kafka, OpenTelemetry, auth, or storage
• Add or validate a custom/BYOM embedding backend
• Debug RT-Embed readiness, model/cache startup, or Redis/Kafka reachability

Service Snapshot

• Skill: vss-deploy-video-embedding.
• Legacy 3.1 name: RT-Embed.
• Compose service: rtvi-embed.
• Container name: vss-rtvi-embed.
• Image: ghcr.io/nvidia-ai-blueprints/vss/vss-rt-embed (override with VSS_RT_EMBED_IMAGE).
• Default tag: develop-latest (override with VSS_RT_EMBED_TAG; use develop-latest-sbsa for an SBSA/DGX Spark host).
• Profile: rtvi-embed.
• Container port: 8000 (host-side ${RTVI_EMBED_PORT}).
• Default model: cosmos-embed1-448p from nvidia/Cosmos-Embed1-448p.
• BYOM loader variables: MODEL_PATH, MODEL_IMPLEMENTATION_PATH, MODEL_REPOSITORY_SCRIPT_PATH.
• Health endpoint: GET /v1/ready.
• Healthcheck startup grace: 1200s (20 minutes) on first boot.

Route First

Choose one primary path before acting. Load the linked reference and follow it; do not duplicate full workflows from this top-level file.

User intent	Use this path
Deploy, size, upgrade, roll back, or tear down standalone RT-Embed with the default Cosmos-Embed1 model	references/deploy-vss-deploy-video-embedding.md
Call RT-Embed APIs for files, text/video embeddings, live streams, model listing, health, metrics, metadata, or manifests	references/rest-api.md
Wire RT-Embed into another service or deployment with Redis, Kafka, OpenTelemetry, auth, storage, or env var mapping	references/integrate-vss-deploy-video-embedding.md and references/environment.md
Use decoded-frame IPC from a compatible RTVI CV producer	references/environment.md#decoded-frame-ipc and references/integrate-vss-deploy-video-embedding.md
Add, wire, or validate a custom/BYOM embedding backend, with VideoPrism as an example	references/byom-custom-model.md
Debug readiness, model/cache startup, permissions, Redis/Kafka reachability, API failures, or observability	references/troubleshooting.md

Selection rules:

• If asked which skill handles default Cosmos-Embed1 RT-Embed deployment, answer: use vss-deploy-video-embedding; this is the default deployment path, not the BYOM/custom-model path.
• For normal RT-Embed or Cosmos-Embed1 deployment, use the deployment reference. In the answer, explicitly say that this is the default RT-Embed deployment path. Also explicitly distinguish it from BYOM/custom model integration: BYOM is only for adding or validating non-default custom embedding backends such as VideoPrism, and is not needed for the default Cosmos-Embed1 model.
• For BYOM, custom embedding models, VideoPrism examples, or model implementation path questions, use the BYOM reference first, then deployment/API references only as needed.
• For direct endpoint calls, use the API reference and reuse deployment context only when the service is not already running.
• Decoded-frame IPC requires a compatible RTVI CV producer on the same host, a shared socket directory accessible to UID/GID 1001, and a matching camera ID. The consumer uses the fixed /run/rtvi-ipc/nvds_ipc_{camera_id}.sock socket contract. IPC camera, sensor, and stream IDs must be non-empty and contain only ASCII letters, digits, ., _, and -; standard UUIDs are valid. It applies only to live RTSP processing.
• If the request mixes deployment and BYOM, load BYOM first to establish model path requirements, then use the deployment reference to run the service.

Operating Rules

• Do not deploy a full VSS profile for standalone RT-Embed. Work from deploy/docker/services/rtvi/rtvi-embed unless the user explicitly asks for a profile deployment.
• Never let sudo prompt interactively. Prefer plain docker; otherwise use sudo -n docker and stop with the exact manual command if passwordless sudo is unavailable.
• Do not expose full values of NGC_API_KEY, HF_TOKEN, bearer tokens, or model repository credentials in prompts, logs, or final answers.
• Do not shorten the start_period: 1200s healthcheck during first boot. Cosmos model download and Triton model repository generation can take up to 20 minutes.
• In standalone mode, disable missing peers with MESSAGE_BUS=, ERROR_BUS=, and ENABLE_REDIS_ERROR_MESSAGES=false unless the corresponding Kafka or Redis service is started and reachable.
• For BYOM models that are video-only, require an explicit text endpoint decision: either a compatible text encoder in the same embedding space or a clear 4xx response for /v1/generate_text_embeddings.

Quick Reference

• Deployment details: image, GPU, storage, startup, readiness, upgrade, and teardown live in references/deploy-vss-deploy-video-embedding.md.
• API details: file upload, text/video embeddings, live-stream control, models, health, metadata, and metrics live in references/rest-api.md.
• Integration details: inputs/outputs, Redis/Kafka/OTel, auth, networking, and Compose snippets live in references/integrate-vss-deploy-video-embedding.md.
• Environment matrix: host-to-container renames, optional volumes, and secret-sensitive variables live in references/environment.md.
• BYOM details: custom model contract, Docker/Helm overrides, model path variables, and VideoPrism example validation live in references/byom-custom-model.md.
• Troubleshooting details: common startup, cache, permission, bus, and API failures live in references/troubleshooting.md.

References

File	When to read
references/README.md	Table of contents for all reference files.
references/deploy-vss-deploy-video-embedding.md	Deployment reference: image, GPU, storage, startup, prerequisites, known issues.
references/rest-api.md	Full REST endpoint catalog with worked curl examples for file uploads, video/text embeddings, live streams, and health/metrics.
references/integrate-vss-deploy-video-embedding.md	Integration reference: peers, inputs/outputs, env vars, network, example Compose snippet.
references/environment.md	Complete environment-variable matrix, including host-to-container renames and secret-sensitive variables.
references/byom-custom-model.md	BYOM reference: custom model contract, path overrides, Docker/Helm wiring, and VideoPrism example validation checklist.
references/troubleshooting.md	Operational diagnostics for startup, model/cache, runtime, and observability issues.