Media playback

OpenClaw chat clients play assistant audio and video attachments inline. The Gateway keeps those attachments behind session-scoped access, serves seekable byte ranges, and can prepare a portable playback rendition for recognized formats that are not safe across every client.

This page covers playback in OpenClaw clients. Channel delivery, inbound media understanding, and live voice conversations use separate paths; see Image and media support, Media understanding, and Talk mode.

Client support

Portable formats

The Gateway classifies these formats as the portable native set shared by the browser, Apple players, and Android Media3:

The Linux companion can also play formats supplied by its installed GStreamer plugins. Browser and operating-system updates may add native formats, but the table above is the cross-client contract OpenClaw targets.

Lazy playback renditions

Both Gateway byte routes accept ?playback=1: the managed attachment route under /api/chat/media/outgoing/.../full and the Control UI assistant-media route. Attachment metadata can report playback: "native" or playback: "transcode" so a client can choose the rendition deliberately.

Playback conversion is lazy:

1. A native source passes through unchanged.
2. A recognized non-portable source starts a bounded ffmpeg job. The route returns HTTP 202 with { "status": "preparing" } while the rendition is being prepared.
3. A later request receives the cached M4A or MP4 rendition.
4. If inspection or conversion is unavailable, fails, or exceeds a limit, the route falls back to the original bytes. The client can then show its unplayable-media fallback and keep the download action available.

The Control UI checks rendition readiness with HEAD before loading the inline player. It shows Preparing playback while conversion is pending and keeps the download action available when playback is unavailable.

Transcoding accepts sources up to 20 minutes and never raises the normal audio or video byte cap. Cached playback renditions use a fixed seven-day retention that Gateway maintenance enforces at startup and hourly, independently of attachments.ttlHours.

Managed attachments and access

To attach a local file in an assistant reply, put MEDIA:/absolute/path/movie.mp4 on its own line. The session's existing media access policy must allow the path. The Control UI stages allowed local audio and video with bounded streaming and plays them through the Gateway's authenticated, seekable media route. Use a portal for an app or development server; use MEDIA:<path> for files.

Remote MEDIA: references must be public HTTPS URLs without credentials. Rejected references produce a visible attachment failure with instructions to use an allowed URL or local path; they do not appear as raw directive links. Inline prose and fenced code examples mentioning MEDIA: remain text.

Agent-produced audio and video are stored as managed media artifacts. Images keep their separate managed-image artifact family. Native clients resolve the artifact through artifacts.download, which returns inline base64 bytes when the artifact is byte-backed or a short-lived, ticketed URL when it is Gateway-managed.

Managed download tickets check current session access and the selected message's attachment reference in visible history. For indexed messages, issuing a ticket does not read the original file or validate unrelated transcript payloads. The HTTP request transfers the file separately.

Download filenames preserve Unicode characters and literal percent sequences such as %20.

Native clients resolve ticketed media against the connected Gateway URL, preserving its reverse-proxy path prefix. A Gateway reached at wss://gateway.example/openclaw loads managed media beneath https://gateway.example/openclaw/api/chat/media/outgoing/, not the server root.

The ticketed byte routes support:

• Range requests with HTTP 206 Partial Content for seeking
• ETag and If-Range for safe resume of immutable managed originals
• HEAD requests with the same content metadata and no response body

For immutable originals, If-None-Match compares complete quoted tags using weak comparison. Commas and asterisks inside a quoted tag are literal; only a standalone * is a wildcard. A nonmatching tag leaves the normal full or ranged response intact.

Local assistant files can change, and playback renditions can become available after a conversion retry. These responses revalidate without reusable validators: cached ETags or modification dates cannot suppress fresh bytes, and If-Range requests receive the full representation. Ordinary Range requests still support seeking. Managed playback responses remain private to the client cache.

Do not copy a ticketed URL into durable configuration. Clients reacquire a ticket from the authenticated Gateway when needed.

Metadata and limits

Chat attachments may include sizeBytes, durationMs, width, and height. OpenClaw also uses ffprobe, when available, to fill audio duration and video duration/dimensions for media facts and the Control UI ?meta=1 availability check. Video dimensions account for non-square pixels and quarter-turn display rotation; image dimensions account for EXIF orientation. Checking is best-effort: a missing or failed check leaves fields absent instead of rejecting the attachment. The Gateway shares concurrent metadata inspections for the same local file and reuses successful results while that file is unchanged. Replacing or editing the file triggers a fresh inspection; failed checks remain retryable. Distinct files wait in a bounded inspection queue. If the queue is full, metadata reports temporary unavailability that you can retry, and playback remains preparing. Disconnected requests stop waiting, and queued checks with no remaining viewers release their queue slots immediately. Queued reads recheck current access before opening and checking the file. A busy inspector does not discard outgoing attachments; their optional playback metadata can remain absent. Outgoing reply creation uses immediate inspection admission and does not wait behind queued viewer requests.

Control UI managed assistant attachments use these per-file caps:

These are playback/storage caps, not the separate media-understanding limits. The larger local-file limit does not change channel outbound limits, remote or data URL ingestion limits, or the limits of buffer-based remote workspace readers. It also does not raise the transcoding budget; large native MP4 files can play directly without conversion. For transcription and description limits, see Image and media support.

Troubleshooting

Duration or dimensions are missing

Check that ffprobe is installed on the Gateway host and visible on its PATH:

ffprobe -version

Playback of an already portable file can still work without metadata.

A recognized format downloads instead of playing

Check both media tools on the Gateway host:

ffmpeg -version
ffprobe -version

ffprobe classifies codecs and duration; ffmpeg creates the portable rendition. If either step cannot safely handle the source, OpenClaw serves the original file and the client keeps its fallback/download path.

Playback stays in preparing state

The first rendition request is asynchronous. Wait briefly and retry. Very large, longer than 20-minute, unprobeable, or unsupported sources remain on the original-byte fallback instead of blocking the Gateway.

Linux reports a codec error

Use the package and source-build instructions in Linux media codecs. The .deb depends on the required GStreamer plugin packages; the AppImage carries the media framework and codecs installed by the release build.

Android shows a media row while offline

That is expected. Android caches the transcript metadata, not the attachment bytes or its short-lived download capability. Reconnect, then play again so the app can request a new ticket.

Related

• Image and media support
• Audio and voice notes
• Media overview
• Text to speech
• Linux app