166 lines
6.8 KiB
Markdown
166 lines
6.8 KiB
Markdown
# Chatterbox TTS Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Deploy an OpenAI-compatible Chatterbox text-to-speech service at `chatterbox.dimensionlab.net`.
|
|
|
|
**Architecture:** A FastAPI application on loopback port 8881 wraps the CUDA Chatterbox model and exposes the standard speech endpoint. The application confines reference audio to its `voices/` directory, serializes GPU synthesis, and unloads the cached model after exactly five idle minutes. A user systemd service supervises it, and Caddy exposes the public hostname.
|
|
|
|
**Tech Stack:** Python 3, FastAPI, Uvicorn, PyTorch, torchaudio, chatterbox-tts, systemd user units, Caddy.
|
|
|
|
## Global Constraints
|
|
|
|
- Do not change the Kokoro service or its Caddy route.
|
|
- Bind Uvicorn to `127.0.0.1:8881` only.
|
|
- Use the English `ResembleAI/chatterbox` model on CUDA.
|
|
- Accept reference audio only below `/home/vince/ai/apps/chatterbox-tts/voices`.
|
|
- Set model idle unload to exactly 300 seconds.
|
|
- Permit at most one in-flight model synthesis request.
|
|
- Limit `input` to 2,000 characters.
|
|
- Support WAV responses in the first release.
|
|
|
|
---
|
|
|
|
### Task 1: Create and test the API module
|
|
|
|
**Files:**
|
|
- Create: `/home/vince/ai/apps/chatterbox-tts/app.py`
|
|
- Create: `/home/vince/ai/apps/chatterbox-tts/tests/test_app.py`
|
|
- Create: `/home/vince/ai/apps/chatterbox-tts/requirements.txt`
|
|
- Create: `/home/vince/ai/apps/chatterbox-tts/voices/.gitkeep`
|
|
|
|
**Interfaces:**
|
|
- Produces: `app: FastAPI` with `GET /health`, `GET /v1/models`, and `POST /v1/audio/speech`.
|
|
- Produces: `resolve_voice_path(voice: str) -> Path` and `SpeechRequest` request validation.
|
|
|
|
- [ ] **Step 1: Write failing API tests**
|
|
|
|
```python
|
|
def test_models_and_health(client):
|
|
assert client.get('/health').json() == {'status': 'ok'}
|
|
assert client.get('/v1/models').json()['data'][0]['id'] == 'chatterbox'
|
|
|
|
def test_voice_must_stay_under_voice_directory(client):
|
|
response = client.post('/v1/audio/speech', json={
|
|
'input': 'Hello', 'voice': '../secret.wav', 'response_format': 'wav'
|
|
})
|
|
assert response.status_code == 400
|
|
|
|
def test_input_is_limited(client):
|
|
response = client.post('/v1/audio/speech', json={
|
|
'input': 'x' * 2001, 'voice': 'reference.wav', 'response_format': 'wav'
|
|
})
|
|
assert response.status_code == 422
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to confirm they fail before implementation**
|
|
|
|
Run: `cd /home/vince/ai/apps/chatterbox-tts && .venv/bin/python -m pytest tests/test_app.py -v`
|
|
|
|
Expected: FAIL because `app.py` and the `app` module do not exist.
|
|
|
|
- [ ] **Step 3: Implement the API and GPU model manager**
|
|
|
|
Implement a `ModelManager` protected by `threading.Lock`. It must lazily call `ChatterboxTTS.from_pretrained(device='cuda')`, update `last_request_time` at synthesis start, and have a daemon that clears the model and calls `torch.cuda.empty_cache()` when idle for 300 seconds. Guard the full inference call with the same lock. Resolve `voice` with `Path(VOICES_DIR, voice).resolve()` and reject it unless `relative_to(VOICES_DIR.resolve())` succeeds. Use `torchaudio.save` to write the generated tensor to an in-memory WAV buffer.
|
|
|
|
```python
|
|
@app.post('/v1/audio/speech')
|
|
def speech(request: SpeechRequest) -> Response:
|
|
reference = resolve_voice_path(request.voice)
|
|
wav, sample_rate = model_manager.generate(request.input, reference)
|
|
buffer = io.BytesIO()
|
|
torchaudio.save(buffer, wav.cpu(), sample_rate, format='wav')
|
|
return Response(buffer.getvalue(), media_type='audio/wav')
|
|
```
|
|
|
|
- [ ] **Step 4: Run the focused tests**
|
|
|
|
Run: `cd /home/vince/ai/apps/chatterbox-tts && .venv/bin/python -m pytest tests/test_app.py -v`
|
|
|
|
Expected: PASS without loading the CUDA model for health, model-list, length, or invalid-path cases.
|
|
|
|
### Task 2: Install isolated dependencies and verify real synthesis
|
|
|
|
**Files:**
|
|
- Modify: `/home/vince/ai/apps/chatterbox-tts/requirements.txt`
|
|
- Create: `/home/vince/ai/apps/chatterbox-tts/tests/test_synthesis.py`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `app.model_manager.generate(text, voice_path)`.
|
|
- Produces: a WAV-producing Chatterbox runtime in `.venv`.
|
|
|
|
- [ ] **Step 1: Write the failing real-synthesis test**
|
|
|
|
```python
|
|
def test_generate_returns_audio_tensor_and_sample_rate(reference_wav):
|
|
wav, sample_rate = model_manager.generate('A short Chatterbox smoke test.', reference_wav)
|
|
assert wav.numel() > 0
|
|
assert sample_rate > 0
|
|
```
|
|
|
|
- [ ] **Step 2: Install the project environment**
|
|
|
|
Run: `cd /home/vince/ai/apps/chatterbox-tts && python3 -m venv .venv && .venv/bin/pip install --upgrade pip && .venv/bin/pip install -r requirements.txt`
|
|
|
|
Expected: `chatterbox-tts`, CUDA-compatible PyTorch/torchaudio, FastAPI, Uvicorn, Pydantic, and pytest install into `.venv` only.
|
|
|
|
- [ ] **Step 3: Add a short, consented reference WAV to `voices/`**
|
|
|
|
Use one existing, locally owned test WAV as `voices/reference.wav`; do not copy any credentials or unrelated files.
|
|
|
|
- [ ] **Step 4: Run the real-synthesis test**
|
|
|
|
Run: `cd /home/vince/ai/apps/chatterbox-tts && .venv/bin/python -m pytest tests/test_synthesis.py -v -s`
|
|
|
|
Expected: PASS and the CUDA model is released after the test process exits.
|
|
|
|
### Task 3: Supervise and publish the service
|
|
|
|
**Files:**
|
|
- Create: `/home/vince/.config/systemd/user/chatterbox-tts.service`
|
|
- Create: `/etc/caddy/conf.d/chatterbox.caddy`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `app:app` from `app.py`.
|
|
- Produces: loopback service on port 8881 and HTTPS public hostname.
|
|
|
|
- [ ] **Step 1: Create the systemd user unit**
|
|
|
|
Use `WorkingDirectory=/home/vince/ai/apps/chatterbox-tts` and:
|
|
|
|
```ini
|
|
ExecStart=/home/vince/ai/apps/chatterbox-tts/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 8881
|
|
Restart=on-failure
|
|
RestartSec=5
|
|
```
|
|
|
|
- [ ] **Step 2: Create the Caddy route**
|
|
|
|
```caddyfile
|
|
chatterbox.dimensionlab.net {
|
|
import cloudflare_tls
|
|
reverse_proxy 127.0.0.1:8881
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Validate configuration and activate services**
|
|
|
|
Run: `systemctl --user daemon-reload && systemctl --user enable --now chatterbox-tts.service && /usr/local/bin/caddy-cloudflare validate --config /etc/caddy/Caddyfile && sudo systemctl reload caddy`
|
|
|
|
Expected: the user unit is active, Caddy validates, and the public route is loaded.
|
|
|
|
- [ ] **Step 4: Verify the full public API path**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
curl -fsS https://chatterbox.dimensionlab.net/health
|
|
curl -fsS https://chatterbox.dimensionlab.net/v1/models
|
|
curl -fsS -o /tmp/chatterbox-smoke.wav \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"model":"chatterbox","input":"Service smoke test.","voice":"reference.wav","response_format":"wav"}' \
|
|
https://chatterbox.dimensionlab.net/v1/audio/speech
|
|
file /tmp/chatterbox-smoke.wav
|
|
```
|
|
|
|
Expected: health and models return JSON, speech returns an RIFF/WAVE file, and no listener is exposed on a non-loopback interface for port 8881.
|