53 lines
2.3 KiB
Markdown
53 lines
2.3 KiB
Markdown
# Chatterbox TTS service design
|
|
|
|
## Goal
|
|
|
|
Run ResembleAI Chatterbox as an independent GPU-backed text-to-speech service,
|
|
available at `https://chatterbox.dimensionlab.net`, while preserving the
|
|
existing Kokoro service unchanged.
|
|
|
|
## Architecture
|
|
|
|
`chatterbox-tts` is a small Python FastAPI application in
|
|
`/home/vince/ai/apps/chatterbox-tts`. A user-level systemd unit runs it on
|
|
loopback port 8881. Caddy terminates TLS for `chatterbox.dimensionlab.net` and
|
|
reverse-proxies that hostname to `127.0.0.1:8881`.
|
|
|
|
The application loads the English `ResembleAI/chatterbox` model on CUDA lazily.
|
|
It keeps the model resident while requests are active, then frees its GPU
|
|
allocation after five minutes without a synthesis request. A single-request GPU
|
|
lock serializes synthesis and model unload operations. This lets it coexist
|
|
with the existing Kokoro service and other GPU workloads without competing
|
|
requests exhausting VRAM.
|
|
|
|
## API contract
|
|
|
|
The service provides:
|
|
|
|
- `POST /v1/audio/speech`, accepting the OpenAI speech request fields
|
|
`model`, `input`, `voice`, `response_format`, and `speed`.
|
|
- `GET /v1/models`, returning the available Chatterbox model identifier.
|
|
- `GET /health`, returning a small health response without forcing model load.
|
|
|
|
`POST /v1/audio/speech` returns generated WAV audio. `model` defaults to
|
|
`chatterbox`; `response_format` accepts `wav`; and `input` is capped at 2,000
|
|
characters. The request's `voice` field is a path relative to the dedicated
|
|
`/home/vince/ai/apps/chatterbox-tts/voices` directory. The resolved path must
|
|
remain within that directory, exist, and be readable by the service user before
|
|
it is supplied as Chatterbox's reference clip. Invalid, missing, or escaping
|
|
paths fail with a clear 400 response. No upload endpoint or voice catalogue is
|
|
part of this first version.
|
|
|
|
## Failure handling
|
|
|
|
Malformed requests, unsupported formats, invalid paths, model-download/load
|
|
failures, and synthesis errors produce JSON errors. A process crash is restarted
|
|
by systemd. Caddy continues to expose only the public hostname; the Python
|
|
service is not publicly bound.
|
|
|
|
## Verification
|
|
|
|
Verify the unit reaches active status, loopback port 8881 serves `/health`, and
|
|
the public hostname reaches it through Caddy. Exercise `/v1/models` and a
|
|
short `/v1/audio/speech` request using a known local reference WAV, checking
|
|
that the response is valid WAV audio.
|