speech-generation

ElevenLabs Text-to-Speech (via Letta)

Generate natural speech from text - supports 90+ languages, multiple models for quality vs latency tradeoffs. Examples default to eleven_v4, the latest and highest-quality model.

This skill is adapted from ElevenLabs' text-to-speech skill (elevenlabs/skills, MIT; see LICENSE). Instead of calling ElevenLabs with your own key, you call Letta's hosted endpoint POST /v1/audio/speech. It forwards the request to ElevenLabs with Letta's key and charges the organization's Letta credits, so you do not need an ElevenLabs API key. The request body uses the same field names as ElevenLabs "Create speech"; the voice ID goes in the body.

Quick Start

base_url="${LETTA_BASE_URL%/}"

curl -sS -X POST "$base_url/v1/audio/speech" \
  -H "Authorization: Bearer $LETTA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text":"Hello, welcome to ElevenLabs!","voice_id":"JBFqnCBsd6RMkjVDRZzb","model_id":"eleven_v4"}' \
  > speech-response.json

python3 - <<'PY'
import base64, json

with open("speech-response.json") as f:
    response = json.load(f)
if "audio_b64" not in response:
    raise SystemExit(response.get("message", response))

with open("output.mp3", "wb") as f:
    f.write(base64.b64decode(response["audio_b64"]))
print("saved output.mp3")
PY

Use the runtime-provided LETTA_BASE_URL and LETTA_API_KEY together. Build URLs relative to ${LETTA_BASE_URL%/} and do not hardcode https://api.letta.com. If either variable is missing, the user needs to authenticate with Letta Cloud; do not ask for an ElevenLabs key.

Share the saved file the way your current surface supports: attach or upload it to the conversation or channel you are replying in. Never paste the base64 into a reply. Keep credit amounts out of replies unless the user asks about cost.

Models

Model ID	Languages	Latency	Best For
eleven_v4	90+	Standard	Default. Highest quality, emotional range, audio tags, voice cloning accuracy
eleven_v4_turbo	90+	~100ms	Faster v4 for shorter, time-sensitive replies
eleven_flash_v2_5	32	~75ms	Lowest latency, lowest cost
eleven_flash_v2	English	~75ms	English-only, lowest latency
eleven_multilingual_v2	29	Standard	Previous generation; most stable on very long-form generations
eleven_v3	70+	Standard	Previous generation expressive model

eleven_turbo_v2_5 and eleven_turbo_v2 are superseded by the Flash models (same output, lower latency) — use eleven_flash_v2_5 / eleven_flash_v2 instead.

Eleven v4 notes: supports only stability and similarity_boost voice settings (no style or speed) and does not support SSML. Direct delivery with audio tags such as [whispers], [laughs], [sarcastic] (see "Audio Tags").

Voice IDs

The Letta endpoint accepts ElevenLabs premade voices and voices Letta provides. If your memory names your own voice ID, use it whenever you speak as yourself. Other voices (cloned or designed voices from someone's own ElevenLabs account) are rejected with 400; to use one, call ElevenLabs directly with the user's own ELEVENLABS_API_KEY secret and ElevenLabs' text-to-speech skill.

Premade voices:

• JBFqnCBsd6RMkjVDRZzb - George (male, warm storyteller)
• EXAVITQu4vr4xnSDxMaL - Sarah (female, mature, reassuring)
• cgSgspJ2msm6clMCkdW9 - Jessica (female, playful, conversational)
• onwK4e9ZLuTAKqWW03F9 - Daniel (male, steady broadcaster)
• SAz9YHcvj6GT2YYXdXww - River (neutral, relaxed)

Voice Settings

Fine-tune how the voice sounds:

• Stability: How consistent the voice stays. Lower values = more emotional range and variation, but can sound unstable. Higher = steady, predictable delivery.
• Similarity boost: How closely to match the original voice sample. Higher values sound more like the original but may amplify audio artifacts.
• Style: Exaggerates the voice's unique style characteristics (v2/v3 models; not supported on Eleven v4).
• Speed: Speech speed multiplier (v2/v3 models; not supported on Eleven v4).
• Speaker boost: Post-processing that enhances clarity and voice similarity.

{
  "text": "Customize my voice settings.",
  "voice_id": "JBFqnCBsd6RMkjVDRZzb",
  "model_id": "eleven_v4",
  "voice_settings": {
    "stability": 0.5,
    "similarity_boost": 0.75
  }
}

stability: lower = more expressive, higher = more consistent. similarity_boost: higher = closer to the reference voice. See references/voice-settings.md for ranges and use-case presets.

Audio Tags

Eleven v4 reads bracketed, natural-language cues in text as stage directions:

• Place a tag before the words it affects: [shouts] I can't believe you said that to me.
• A tag's emotion carries forward, so add another tag only where the delivery should shift.
• Combine tags inside one set of brackets: [whispering, playful]. Plain-language tags work, such as [nervous, trying to sound confident] or [phone buzzing].
• Use one emotion per clause; contrasting tags on the same words blur the performance.
• Pair tags with punctuation: ellipses slow a line, dashes cut a speaker off, and [pause] / [long pause] add beats.
• If a delivery misses, swap the tag (try [let down] instead of [despair]) and regenerate rather than rewriting the line.

Language Selection

Use language_code with models that support language enforcement to guide pronunciation and text normalization. Unsupported language codes are ignored, and language_code is not supported on eleven_multilingual_v2.

{
  "text": "Bonjour, comment allez-vous?",
  "voice_id": "JBFqnCBsd6RMkjVDRZzb",
  "model_id": "eleven_v4",
  "language_code": "fr"
}

Text Normalization

Controls how numbers, dates, and abbreviations are converted to spoken words. For example, "01/15/2026" becomes "January fifteenth, twenty twenty-six":

• "auto" (default): Model decides based on context
• "on": Always normalize (use when you want natural speech)
• "off": Speak literally (use when you want "zero one slash one five...")

{
  "text": "Call 1-800-555-0123 on 01/15/2026",
  "voice_id": "JBFqnCBsd6RMkjVDRZzb",
  "model_id": "eleven_v4",
  "apply_text_normalization": "on"
}

Request Stitching

When generating long audio in multiple requests, the audio can have pops, unnatural pauses, or tone shifts at the boundaries. Request stitching solves this by letting each request know what comes before/after it:

{"text": "This is the first part.", "voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_v4", "next_text": "And this continues the story."}

{"text": "And this continues the story.", "voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_v4", "previous_text": "This is the first part."}

Each response includes request_id. Instead of previous_text, you can pass up to three earlier request_id values in previous_request_ids (and later ones in next_request_ids when regenerating a middle part). Keep the same voice, model, and voice settings for every part, split long text at sentence or paragraph boundaries (each request takes up to 10,000 characters), and concatenate the files in order, for example with ffmpeg -f concat.

Output Formats

Pass output_format in the body.

Format	Description
mp3_44100_128	MP3 44.1kHz 128kbps (default) - compressed, good for web/apps
mp3_44100_192	MP3 44.1kHz 192kbps - higher quality compressed
mp3_44100_64	MP3 44.1kHz 64kbps - lower quality, smaller files
mp3_22050_32	MP3 22.05kHz 32kbps - smallest MP3 files
pcm_16000	Raw PCM 16kHz - use for real-time processing
pcm_22050	Raw PCM 22.05kHz
pcm_24000	Raw PCM 24kHz
pcm_44100	Raw PCM 44.1kHz - CD quality
pcm_48000	Raw PCM 48kHz - highest quality
ulaw_8000	μ-law 8kHz - standard for phone systems (Twilio, telephony)
alaw_8000	A-law 8kHz - telephony (alternative to μ-law)
opus_48000_64	Opus 48kHz 64kbps - efficient codec, small files
wav_44100	WAV 44.1kHz - uncompressed with headers

The Letta endpoint returns the whole file at once (as audio_b64); it does not stream.

Response

{
  "model_id": "eleven_v4",
  "voice_id": "JBFqnCBsd6RMkjVDRZzb",
  "output_format": "mp3_44100_128",
  "audio_b64": "<base64>",
  "request_id": "Frhov0LQNC3M9HcHKY3t",
  "characters": 46,
  "billing": { "credits_charged": 4, "transaction_id": "...", "new_credit_balance": "..." }
}

Error Handling

Errors return JSON with a message:

• 400 "Voice … is not available": use your own voice or a premade voice (see "Voice IDs").
• 400 with an ElevenLabs message: invalid parameters (check model_id, voice settings the model does not support, or text length).
• 402: the organization is out of Letta credits. Tell the user; do not retry.
• 500: retry once, then report the failure.

References

• Voice Settings