chatterbox-tts/docs/superpowers/specs/2026-08-08-chatterbox-tts-design.md

Chatterbox TTS service design

Goal

Run ResembleAI Chatterbox as an independent GPU-backed text-to-speech service, available at https://chatterbox.dimensionlab.net, while preserving the existing Kokoro service unchanged.

Architecture

chatterbox-tts is a small Python FastAPI application in /home/vince/ai/apps/chatterbox-tts. A user-level systemd unit runs it on loopback port 8881. Caddy terminates TLS for chatterbox.dimensionlab.net and reverse-proxies that hostname to 127.0.0.1:8881.

The application loads the English ResembleAI/chatterbox model on CUDA lazily. It keeps the model resident while requests are active, then frees its GPU allocation after five minutes without a synthesis request. A single-request GPU lock serializes synthesis and model unload operations. This lets it coexist with the existing Kokoro service and other GPU workloads without competing requests exhausting VRAM.

API contract

The service provides:

• POST /v1/audio/speech, accepting the OpenAI speech request fields model, input, voice, response_format, and speed.
• GET /v1/models, returning the available Chatterbox model identifier.
• GET /health, returning a small health response without forcing model load.

POST /v1/audio/speech returns generated WAV audio. model defaults to chatterbox; response_format accepts wav; and input is capped at 2,000 characters. The request's voice field is a path relative to the dedicated /home/vince/ai/apps/chatterbox-tts/voices directory. The resolved path must remain within that directory, exist, and be readable by the service user before it is supplied as Chatterbox's reference clip. Invalid, missing, or escaping paths fail with a clear 400 response. No upload endpoint or voice catalogue is part of this first version.

Failure handling

Malformed requests, unsupported formats, invalid paths, model-download/load failures, and synthesis errors produce JSON errors. A process crash is restarted by systemd. Caddy continues to expose only the public hostname; the Python service is not publicly bound.

Verification

Verify the unit reaches active status, loopback port 8881 serves /health, and the public hostname reaches it through Caddy. Exercise /v1/models and a short /v1/audio/speech request using a known local reference WAV, checking that the response is valid WAV audio.