Compare commits

..
139 changed files with 1068 additions and 9418 deletions
-15
View File
@@ -1,15 +0,0 @@
*.py text eol=lf
*.md text eol=lf
*.yml text eol=lf
*.yaml text eol=lf
*.toml text eol=lf
*.json text eol=lf
*.txt text eol=lf
*.html text eol=lf
*.css text eol=lf
*.js text eol=lf
*.sh text eol=lf
*.cfg text eol=lf
*.ini text eol=lf
*.svg text eol=lf
*.j2 text eol=lf
+12 -32
View File
@@ -1,9 +1,7 @@
name: CI
run-name: CI
on:
name: pip install
run-name: pip install
on:
push:
branches: [main]
paths:
- '**.py'
- 'pyproject.toml'
@@ -13,41 +11,23 @@ on:
- 'pyproject.toml'
- '.github/workflows/**'
workflow_dispatch:
jobs:
test:
install-and-run:
strategy:
matrix:
os: [ubuntu-latest, macos-14, windows-latest]
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ['3.12']
fail-fast: false
continue-on-error: true
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v6
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install uv
uses: astral-sh/setup-uv@v8.3.1
with:
enable-cache: true
prune-cache: false
cache-dependency-glob: pyproject.toml
- name: Install system dependencies (Ubuntu)
if: runner.os == 'Linux'
run: sudo apt-get update && sudo apt-get install -y libegl1
- name: Install dependencies
run: uv pip install --system .[dev]
env:
UV_LINK_MODE: copy
- name: Run tests
env:
QT_QPA_PLATFORM: offscreen
run: pytest tests/ -v --tb=short
- name: Install from repository
run: python -m pip install .
#- name: Run abogen
# run: abogen
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v4
- name: Login to Github Container Registry
# Only if we need to push an image
+18 -13
View File
@@ -38,6 +38,8 @@ This method handles everything automatically - installing all dependencies inclu
#### <b>OPTION 2: Install using uv</b>
First, [install uv](https://docs.astral.sh/uv/getting-started/installation/) if you haven't already.
The CUDA extras install both GPU-accelerated Kokoro (via PyTorch) and Supertonic (via onnxruntime-gpu).
```bash
# For NVIDIA GPUs (CUDA 12.8) - Recommended
uv tool install --python 3.12 abogen[cuda] --extra-index-url https://download.pytorch.org/whl/cu128 --index-strategy unsafe-best-match
@@ -65,6 +67,9 @@ venv\Scripts\activate
# We need to use an older version of PyTorch (2.8.0) until this issue is fixed: https://github.com/pytorch/pytorch/issues/166628
pip install torch==2.8.0+cu128 torchvision==0.23.0+cu128 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128
# Also install onnxruntime-gpu for Supertonic GPU acceleration:
pip install onnxruntime-gpu
# For AMD GPUs:
# Not supported yet, because ROCm is not available on Windows. Use Linux if you have AMD GPU.
@@ -173,7 +178,7 @@ Abogen offers **two interfaces**, but currently they have different feature sets
| Command | Interface | Features |
|---------|-----------|----------|
| `abogen` | PyQt6 Desktop GUI | Stable core features |
| `abogen` | PyQt6 Desktop GUI | Stable core features + **Supertonic TTS**|
| `abogen-web` | Flask Web UI | Core features + **Supertonic TTS**, **LLM Normalization**, **Audiobookshelf Integration** and more! |
> **Note:** The Web UI is under active development. We are working to integrate these new features into the PyQt desktop app. until then, the Web UI provides the most feature-rich experience.
@@ -407,18 +412,18 @@ When Audiobookshelf sits behind Nginx Proxy Manager (NPM), make sure the API pat
1. Create a **Proxy Host** that points to your ABS container or host (default forward port `13378`).
2. Under the **SSL** tab, enable your certificate and tick **Force SSL** if you want HTTPS only.
3. In the **Advanced** tab, append the snippet below so bearer tokens, client IPs, and large uploads survive the proxy hop:
```nginx
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header Authorization $http_authorization;
client_max_body_size 5g;
proxy_read_timeout 300s;
proxy_connect_timeout 300s;
```
```nginx
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header Authorization $http_authorization;
client_max_body_size 5g;
proxy_read_timeout 300s;
proxy_connect_timeout 300s;
```
4. Disable **Block Common Exploits** (it strips Authorization headers in some NPM builds).
5. Enable **Websockets Support** on the main proxy screen (Audiobookshelf uses it for the web UI, and it keeps the reverse proxy configuration consistent).
6. If you publish Audiobookshelf under a path prefix (for example `/abs`), add a **Custom Location** with `Location: /abs/` and set the **Forward Path** to `/`. That rewrite strips the `/abs` prefix before traffic reaches Audiobookshelf so `/abs/api/...` on the internet becomes `/api/...` on the backend. Use the same prefixed URL in Abogens “Base URL” field.
+14
View File
@@ -323,6 +323,13 @@ if /I "%IS_NVIDIA%"=="true" (
pause
exit /b
)
echo Installing onnxruntime-gpu for Supertonic GPU acceleration...
%PYTHON_CONSOLE_PATH% -m uv pip install --system onnxruntime-gpu
if errorlevel 1 (
echo Failed to install onnxruntime-gpu.
pause
exit /b
)
) else (
echo CUDA is available on NVIDIA GPU.
)
@@ -348,6 +355,13 @@ if /I "%IS_NVIDIA%"=="true" (
pause
exit /b
)
echo Installing onnxruntime-gpu for Supertonic GPU acceleration...
%PYTHON_CONSOLE_PATH% -m uv pip install --system onnxruntime-gpu
if errorlevel 1 (
echo Failed to install onnxruntime-gpu.
pause
exit /b
)
)
)
Binary file not shown.

Before

Width:  |  Height:  |  Size: 571 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 992 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 786 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 877 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 832 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 521 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 868 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 808 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 372 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 883 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 465 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 885 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 381 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 855 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 917 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 441 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 856 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 837 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 875 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 617 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 843 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 875 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 891 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 851 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.1 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 431 B

+30 -30
View File
@@ -1,31 +1,31 @@
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
<svg height="800px" width="800px" version="1.1" id="_x32_" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"
viewBox="0 0 512 512" xml:space="preserve">
<style type="text/css">
.st0{fill:#808080;}
</style>
<g>
<path class="st0" d="M502.325,307.303l-39.006-30.805c-6.215-4.908-9.665-12.429-9.668-20.348c0-0.084,0-0.168,0-0.252
c-0.014-7.936,3.44-15.478,9.667-20.396l39.007-30.806c8.933-7.055,12.093-19.185,7.737-29.701l-17.134-41.366
c-4.356-10.516-15.167-16.86-26.472-15.532l-49.366,5.8c-7.881,0.926-15.656-1.966-21.258-7.586
c-0.059-0.06-0.118-0.119-0.177-0.178c-5.597-5.602-8.476-13.36-7.552-21.225l5.799-49.363
c1.328-11.305-5.015-22.116-15.531-26.472L337.004,1.939c-10.516-4.356-22.646-1.196-29.701,7.736l-30.805,39.005
c-4.908,6.215-12.43,9.665-20.349,9.668c-0.084,0-0.168,0-0.252,0c-7.935,0.014-15.477-3.44-20.395-9.667L204.697,9.675
c-7.055-8.933-19.185-12.092-29.702-7.736L133.63,19.072c-10.516,4.356-16.86,15.167-15.532,26.473l5.799,49.366
c0.926,7.881-1.964,15.656-7.585,21.257c-0.059,0.059-0.118,0.118-0.178,0.178c-5.602,5.598-13.36,8.477-21.226,7.552
l-49.363-5.799c-11.305-1.328-22.116,5.015-26.472,15.531L1.939,174.996c-4.356,10.516-1.196,22.646,7.736,29.701l39.006,30.805
c6.215,4.908,9.665,12.429,9.668,20.348c0,0.084,0,0.167,0,0.251c0.014,7.935-3.44,15.477-9.667,20.395L9.675,307.303
c-8.933,7.055-12.092,19.185-7.736,29.701l17.134,41.365c4.356,10.516,15.168,16.86,26.472,15.532l49.366-5.799
c7.882-0.926,15.656,1.965,21.258,7.586c0.059,0.059,0.118,0.119,0.178,0.178c5.597,5.603,8.476,13.36,7.552,21.226l-5.799,49.364
c-1.328,11.305,5.015,22.116,15.532,26.472l41.366,17.134c10.516,4.356,22.646,1.196,29.701-7.736l30.804-39.005
c4.908-6.215,12.43-9.665,20.348-9.669c0.084,0,0.168,0,0.251,0c7.936-0.014,15.478,3.44,20.396,9.667l30.806,39.007
c7.055,8.933,19.185,12.093,29.701,7.736l41.366-17.134c10.516-4.356,16.86-15.168,15.532-26.472l-5.8-49.366
c-0.926-7.881,1.965-15.656,7.586-21.257c0.059-0.059,0.119-0.119,0.178-0.178c5.602-5.597,13.36-8.476,21.225-7.552l49.364,5.799
c11.305,1.328,22.117-5.015,26.472-15.531l17.134-41.365C514.418,326.488,511.258,314.358,502.325,307.303z M281.292,329.698
c-39.68,16.436-85.172-2.407-101.607-42.087c-16.436-39.68,2.407-85.171,42.087-101.608c39.68-16.436,85.172,2.407,101.608,42.088
C339.815,267.771,320.972,313.262,281.292,329.698z"/>
</g>
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
<svg height="800px" width="800px" version="1.1" id="_x32_" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"
viewBox="0 0 512 512" xml:space="preserve">
<style type="text/css">
.st0{fill:#808080;}
</style>
<g>
<path class="st0" d="M502.325,307.303l-39.006-30.805c-6.215-4.908-9.665-12.429-9.668-20.348c0-0.084,0-0.168,0-0.252
c-0.014-7.936,3.44-15.478,9.667-20.396l39.007-30.806c8.933-7.055,12.093-19.185,7.737-29.701l-17.134-41.366
c-4.356-10.516-15.167-16.86-26.472-15.532l-49.366,5.8c-7.881,0.926-15.656-1.966-21.258-7.586
c-0.059-0.06-0.118-0.119-0.177-0.178c-5.597-5.602-8.476-13.36-7.552-21.225l5.799-49.363
c1.328-11.305-5.015-22.116-15.531-26.472L337.004,1.939c-10.516-4.356-22.646-1.196-29.701,7.736l-30.805,39.005
c-4.908,6.215-12.43,9.665-20.349,9.668c-0.084,0-0.168,0-0.252,0c-7.935,0.014-15.477-3.44-20.395-9.667L204.697,9.675
c-7.055-8.933-19.185-12.092-29.702-7.736L133.63,19.072c-10.516,4.356-16.86,15.167-15.532,26.473l5.799,49.366
c0.926,7.881-1.964,15.656-7.585,21.257c-0.059,0.059-0.118,0.118-0.178,0.178c-5.602,5.598-13.36,8.477-21.226,7.552
l-49.363-5.799c-11.305-1.328-22.116,5.015-26.472,15.531L1.939,174.996c-4.356,10.516-1.196,22.646,7.736,29.701l39.006,30.805
c6.215,4.908,9.665,12.429,9.668,20.348c0,0.084,0,0.167,0,0.251c0.014,7.935-3.44,15.477-9.667,20.395L9.675,307.303
c-8.933,7.055-12.092,19.185-7.736,29.701l17.134,41.365c4.356,10.516,15.168,16.86,26.472,15.532l49.366-5.799
c7.882-0.926,15.656,1.965,21.258,7.586c0.059,0.059,0.118,0.119,0.178,0.178c5.597,5.603,8.476,13.36,7.552,21.226l-5.799,49.364
c-1.328,11.305,5.015,22.116,15.532,26.472l41.366,17.134c10.516,4.356,22.646,1.196,29.701-7.736l30.804-39.005
c4.908-6.215,12.43-9.665,20.348-9.669c0.084,0,0.168,0,0.251,0c7.936-0.014,15.478,3.44,20.396,9.667l30.806,39.007
c7.055,8.933,19.185,12.093,29.701,7.736l41.366-17.134c10.516-4.356,16.86-15.168,15.532-26.472l-5.8-49.366
c-0.926-7.881,1.965-15.656,7.586-21.257c0.059-0.059,0.119-0.119,0.178-0.178c5.602-5.597,13.36-8.476,21.225-7.552l49.364,5.799
c11.305,1.328,22.117-5.015,26.472-15.531l17.134-41.365C514.418,326.488,511.258,314.358,502.325,307.303z M281.292,329.698
c-39.68,16.436-85.172-2.407-101.607-42.087c-16.436-39.68,2.407-85.171,42.087-101.608c39.68-16.436,85.172,2.407,101.608,42.088
C339.815,267.771,320.972,313.262,281.292,329.698z"/>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 2.5 KiB

After

Width:  |  Height:  |  Size: 2.6 KiB

+109
View File
@@ -29,6 +29,57 @@ LANGUAGE_DESCRIPTIONS = {
"z": "Mandarin Chinese",
}
# Mapping from Kokoro single-letter language codes to ISO 3166-1 alpha-2 country codes
# Used for loading flag icons
KOKORO_LANG_TO_COUNTRY = {
"a": "us", # American English -> United States
"b": "gb", # British English -> United Kingdom
"e": "es", # Spanish -> Spain
"f": "fr", # French -> France
"h": "in", # Hindi -> India
"i": "it", # Italian -> Italy
"j": "jp", # Japanese -> Japan
"p": "br", # Brazilian Portuguese -> Brazil
"z": "cn", # Mandarin Chinese -> China
}
# Mapping from Supertonic ISO 639-1 language codes to ISO 3166-1 alpha-2 country codes
# Used for loading flag icons in the Supertonic language picker
SUPERTONIC_LANG_TO_COUNTRY = {
"en": "gb",
"ko": "kr",
"ja": "jp",
"ar": "ae",
"bg": "bg",
"cs": "cz",
"da": "dk",
"de": "de",
"el": "gr",
"es": "es",
"et": "ee",
"fi": "fi",
"fr": "fr",
"hi": "in",
"hr": "hr",
"hu": "hu",
"id": "id",
"it": "it",
"lt": "lt",
"lv": "lv",
"nl": "nl",
"pl": "pl",
"pt": "pt",
"ro": "ro",
"ru": "ru",
"sk": "sk",
"sl": "si",
"sv": "se",
"tr": "tr",
"uk": "ua",
"vi": "vn",
"na": "na",
}
# Supported sound formats
SUPPORTED_SOUND_FORMATS = [
"wav",
@@ -63,6 +114,64 @@ SUPPORTED_INPUT_FORMATS = [
# 384 if self.lang_code in 'ab':
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION = list(LANGUAGE_DESCRIPTIONS.keys())
# Voice and sample text constants
VOICES_INTERNAL = [
"af_alloy",
"af_aoede",
"af_bella",
"af_heart",
"af_jessica",
"af_kore",
"af_nicole",
"af_nova",
"af_river",
"af_sarah",
"af_sky",
"am_adam",
"am_echo",
"am_eric",
"am_fenrir",
"am_liam",
"am_michael",
"am_onyx",
"am_puck",
"am_santa",
"bf_alice",
"bf_emma",
"bf_isabella",
"bf_lily",
"bm_daniel",
"bm_fable",
"bm_george",
"bm_lewis",
"ef_dora",
"em_alex",
"em_santa",
"ff_siwis",
"hf_alpha",
"hf_beta",
"hm_omega",
"hm_psi",
"if_sara",
"im_nicola",
"jf_alpha",
"jf_gongitsune",
"jf_nezumi",
"jf_tebukuro",
"jm_kumo",
"pf_dora",
"pm_alex",
"pm_santa",
"zf_xiaobei",
"zf_xiaoni",
"zf_xiaoxiao",
"zf_xiaoyi",
"zm_yunjian",
"zm_yunxi",
"zm_yunxia",
"zm_yunyang",
]
# Voice and sample text mapping
SAMPLE_VOICE_TEXTS = {
"a": "This is a sample of the selected voice.",
+15 -5
View File
@@ -2,14 +2,13 @@
from __future__ import annotations
import atexit
import os
import platform
import signal
import sys
# Initialise global shutdown handling (atexit, signals, Qt) as early as possible.
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
from abogen.utils import load_config
from abogen.utils import load_config, prevent_sleep_end
from abogen.webui.app import main as _run_web_ui
# Configure Hugging Face Hub behaviour (mirrors legacy GUI defaults).
@@ -28,6 +27,17 @@ os.environ.setdefault("MIOPEN_CONV_PRECISE_ROCM_TUNING", "0")
if platform.system() == "Darwin" and platform.processor() == "arm":
os.environ.setdefault("PYTORCH_ENABLE_MPS_FALLBACK", "1")
atexit.register(prevent_sleep_end)
def _cleanup_sleep(signum, _frame):
prevent_sleep_end()
sys.exit(0)
signal.signal(signal.SIGINT, _cleanup_sleep)
signal.signal(signal.SIGTERM, _cleanup_sleep)
def main() -> None:
"""Launch the Flask-based web UI."""
+4 -5
View File
@@ -21,8 +21,7 @@ from PyQt6.QtWidgets import (
)
from PyQt6.QtCore import QThread, pyqtSignal
from abogen.constants import COLORS
from abogen.tts_plugin.utils import get_voices
from abogen.constants import COLORS, VOICES_INTERNAL
from abogen.spacy_utils import SPACY_MODELS
import abogen.hf_tracker
@@ -115,7 +114,7 @@ class PreDownloadWorker(QThread):
self._voices_success = False
return
voice_list = get_voices("kokoro")
voice_list = VOICES_INTERNAL
for idx, voice in enumerate(voice_list, start=1):
if self._cancelled:
self._voices_success = False
@@ -463,14 +462,14 @@ class PreDownloadDialog(QDialog):
try:
from huggingface_hub import try_to_load_from_cache
for voice in get_voices("kokoro"):
for voice in VOICES_INTERNAL:
if not try_to_load_from_cache(
repo_id="hexgrad/Kokoro-82M", filename=f"voices/{voice}.pt"
):
missing.append(voice)
except Exception:
# If HF missing, report all as missing
return False, list(get_voices("kokoro"))
return False, list(VOICES_INTERNAL)
return (len(missing) == 0), missing
def _check_kokoro_model(self) -> bool:
+134 -106
View File
@@ -5,7 +5,6 @@ import hashlib # For generating unique cache filenames
from platformdirs import user_desktop_dir
from PyQt6.QtCore import QThread, pyqtSignal, Qt, QTimer
from PyQt6.QtWidgets import QCheckBox, QVBoxLayout, QDialog, QLabel, QDialogButtonBox
import numpy as np
import soundfile as sf
from abogen.utils import (
create_process,
@@ -21,6 +20,7 @@ from abogen.constants import (
SUPPORTED_SOUND_FORMATS,
SUPPORTED_SUBTITLE_FORMATS,
)
from abogen.tts_supertonic import SupertonicPipeline, SUPERTONIC_AVAILABLE_LANGS, DEFAULT_SUPERTONIC_VOICES
from abogen.voice_formulas import get_new_voice
import abogen.hf_tracker as hf_tracker
import static_ffmpeg
@@ -260,21 +260,30 @@ class ConversionThread(QThread):
output_folder,
subtitle_mode,
output_format,
backend,
np_module,
kpipeline_class,
start_time,
total_char_count,
use_gpu=True,
from_queue=False,
save_base_path=None,
): # Add use_gpu parameter
tts_provider="kokoro",
supertonic_language="en",
supertonic_total_steps=8,
):
super().__init__()
self._chapter_options_event = threading.Event()
self._timestamp_response_event = threading.Event()
self.backend = backend
self.np = np_module
self.KPipeline = kpipeline_class
self.file_name = file_name
self.lang_code = lang_code
self.speed = speed
self.voice = voice
self.tts_provider = tts_provider
self.supertonic_language = supertonic_language
self.supertonic_total_steps = supertonic_total_steps
self.sample_rate = 44100 if tts_provider == "supertonic" else 24000
self.save_option = save_option
self.output_folder = output_folder
self.subtitle_mode = subtitle_mode
@@ -426,6 +435,10 @@ class ConversionThread(QThread):
)
self.log_updated.emit(f"- Voice: {self.voice}")
self.log_updated.emit(f"- Speed: {self.speed}")
tts_provider_label = self.tts_provider.capitalize()
if self.tts_provider == "supertonic":
tts_provider_label += f" (lang={self.supertonic_language}, steps={self.supertonic_total_steps})"
self.log_updated.emit(f"- TTS Engine: {tts_provider_label}")
self.log_updated.emit(f"- Subtitle mode: {self.subtitle_mode}")
self.log_updated.emit(f"- Output format: {self.output_format}")
self.log_updated.emit(
@@ -489,6 +502,26 @@ class ConversionThread(QThread):
self.log_updated.emit(("\nInitializing TTS pipeline...", "grey"))
# Set device based on use_gpu setting and platform
if self.use_gpu:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps" # Use MPS for Apple Silicon
else:
device = "cuda" # Use CUDA for other platforms
else:
device = "cpu"
if self.tts_provider == "supertonic":
tts = SupertonicPipeline(
sample_rate=self.sample_rate,
lang=self.supertonic_language,
total_steps=self.supertonic_total_steps,
)
else:
tts = self.KPipeline(
lang_code=self.lang_code, repo_id="hexgrad/Kokoro-82M", device=device
)
# Check if the input is a subtitle file or timestamp text file
is_subtitle_file = False
is_timestamp_text = False
@@ -524,7 +557,7 @@ class ConversionThread(QThread):
# Process subtitle files separately
if is_subtitle_file or is_timestamp_text:
self._process_subtitle_file(self.backend, base_path, is_timestamp_text)
self._process_subtitle_file(tts, base_path, is_timestamp_text)
return
if self.is_direct_text:
@@ -733,7 +766,7 @@ class ConversionThread(QThread):
merged_out_path = f"{base_filepath_no_ext}.{self.output_format}"
subtitle_entries = []
current_time = 0.0
rate = 24000
rate = self.sample_rate
subtitle_mode = self.subtitle_mode
self.etr_start_time = time.time()
self.processed_char_count = 0
@@ -749,7 +782,7 @@ class ConversionThread(QThread):
merged_out_file = sf.SoundFile(
merged_out_path,
"w",
samplerate=24000,
samplerate=self.sample_rate,
channels=1,
format=self.output_format,
)
@@ -765,62 +798,18 @@ class ConversionThread(QThread):
# Prepare ffmpeg command for m4b output
cmd = [
"ffmpeg",
"-y",
"-thread_queue_size",
"32768",
"-f",
"f32le",
"-ar",
"24000",
"-ac",
"1",
"-i",
"pipe:0",
]
if cover_path and os.path.exists(cover_path):
cmd.extend(
[
"-i",
cover_path,
"-map",
"0:a",
"-map",
"1",
"-c:v",
"copy",
"-disposition:v",
"attached_pic",
]
)
cmd.extend(
[
"-c:a",
"aac",
"-q:a",
"2",
"-movflags",
"+faststart+use_metadata_tags",
"-y",
"-thread_queue_size",
"32768",
"-f",
"f32le",
"-ar",
str(self.sample_rate),
"-ac",
"1",
"-i",
"pipe:0",
]
)
cmd += metadata_options
cmd.append(merged_out_path)
ffmpeg_proc = create_process(cmd, stdin=subprocess.PIPE, text=False)
elif self.output_format == "opus":
static_ffmpeg.add_paths()
cmd = [
"ffmpeg",
"-y",
"-thread_queue_size",
"32768",
"-f",
"f32le",
"-ar",
"24000",
"-ac",
"1",
"-i",
"pipe:0",
]
cmd.extend(["-c:a", "libopus", "-b:a", "24000"])
cmd.append(merged_out_path)
ffmpeg_proc = create_process(cmd, stdin=subprocess.PIPE, text=False)
@@ -902,7 +891,7 @@ class ConversionThread(QThread):
merged_out_path = None
subtitle_entries = []
current_time = 0.0
rate = 24000
rate = self.sample_rate
subtitle_mode = self.subtitle_mode
self.etr_start_time = time.time()
self.processed_char_count = 0
@@ -955,7 +944,7 @@ class ConversionThread(QThread):
chapter_out_file = sf.SoundFile(
chapter_out_path,
"w",
samplerate=24000,
samplerate=self.sample_rate,
channels=1,
format=separate_chapters_format,
)
@@ -970,7 +959,7 @@ class ConversionThread(QThread):
"-f",
"f32le",
"-ar",
"24000",
str(self.sample_rate),
"-ac",
"1",
"-i",
@@ -1057,7 +1046,7 @@ class ConversionThread(QThread):
for segment_idx, (voice_name, segment_text) in enumerate(voice_segments):
# Load voice for this segment (with caching)
try:
loaded_voice = self.load_voice_cached(voice_name, self.backend)
loaded_voice = self.load_voice_cached(voice_name, tts)
if segment_idx > 0:
voice_display = voice_name if len(voice_name) < 50 else voice_name[:47] + "..."
self.log_updated.emit((f" → Voice: {voice_display}", "grey"))
@@ -1066,7 +1055,7 @@ class ConversionThread(QThread):
(f"⚠ Voice loading error for '{voice_name}', continuing with previous", "orange")
)
if segment_idx == 0:
loaded_voice = self.load_voice_cached(self.voice, self.backend)
loaded_voice = self.load_voice_cached(self.voice, tts)
# Determine if spaCy segmentation should be used for PRE-TTS segmentation
# Only non-English languages use spaCy for pre-segmentation
@@ -1152,7 +1141,7 @@ class ConversionThread(QThread):
print("Using split pattern: (unprintable)")
for text_segment in text_segments:
for result in self.backend(
for result in tts(
text_segment,
voice=loaded_voice,
speed=self.speed,
@@ -1352,9 +1341,9 @@ class ConversionThread(QThread):
# Add silence between chapters for merged output (except after the last chapter)
if merge_chapters_at_end and chapter_idx < total_chapters:
silence_samples = int(
self.silence_duration * 24000
self.silence_duration * self.sample_rate
) # Silence duration at 24,000 Hz
silence_audio = np.zeros(silence_samples, dtype="float32")
silence_audio = self.np.zeros(silence_samples, dtype="float32")
silence_bytes = silence_audio.tobytes()
if merged_out_file:
@@ -1583,7 +1572,7 @@ class ConversionThread(QThread):
parent_dir, f"{sanitized_base_name}{suffix}"
)
merged_out_path = f"{base_filepath_no_ext}.{self.output_format}"
rate = 24000
rate = self.sample_rate
# Setup audio output
merged_out_file, ffmpeg_proc = None, None
@@ -1693,7 +1682,7 @@ class ConversionThread(QThread):
max_end_time = max(
(end for _, end, _ in subtitles if end is not None), default=0
)
audio_buffer = np.zeros(
audio_buffer = self.np.zeros(
int(max_end_time * rate) + rate, dtype="float32"
)
@@ -1757,7 +1746,7 @@ class ConversionThread(QThread):
# Generate TTS audio
tts_results = [
r
for r in self.backend(
for r in tts(
processed_text,
voice=loaded_voice,
speed=self.speed,
@@ -1775,11 +1764,11 @@ class ConversionThread(QThread):
# Concatenate audio and determine duration
full_audio = (
np.concatenate(
self.np.concatenate(
[a.numpy() if hasattr(a, "numpy") else a for a in audio_chunks]
)
if audio_chunks
else np.zeros(
else self.np.zeros(
int((subtitle_duration or 0) * rate), dtype="float32"
)
)
@@ -1813,8 +1802,8 @@ class ConversionThread(QThread):
num_stages = max(
1,
int(
np.ceil(
np.log(speed_factor) / np.log(2.0)
self.np.ceil(
self.np.log(speed_factor) / self.np.log(2.0)
)
),
)
@@ -1847,7 +1836,7 @@ class ConversionThread(QThread):
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
full_audio = np.frombuffer(
full_audio = self.np.frombuffer(
speed_proc.communicate(input=full_audio.tobytes())[0],
dtype="float32",
)
@@ -1861,7 +1850,7 @@ class ConversionThread(QThread):
tts_results = [
r
for r in self.backend(
for r in tts(
processed_text,
voice=loaded_voice,
speed=new_speed,
@@ -1872,14 +1861,14 @@ class ConversionThread(QThread):
audio_chunks = [r.audio for r in tts_results]
full_audio = (
np.concatenate(
self.np.concatenate(
[
a.numpy() if hasattr(a, "numpy") else a
for a in audio_chunks
]
)
if audio_chunks
else np.zeros(
else self.np.zeros(
int(subtitle_duration * rate), dtype="float32"
)
)
@@ -1896,10 +1885,10 @@ class ConversionThread(QThread):
# Pad or trim to subtitle duration
target_samples = int(subtitle_duration * rate)
if len(full_audio) < target_samples:
full_audio = np.concatenate(
full_audio = self.np.concatenate(
[
full_audio,
np.zeros(
self.np.zeros(
target_samples - len(full_audio), dtype="float32"
),
]
@@ -1912,10 +1901,10 @@ class ConversionThread(QThread):
end_sample = start_sample + len(full_audio)
if end_sample > len(audio_buffer):
# Extend buffer if needed
audio_buffer = np.concatenate(
audio_buffer = self.np.concatenate(
[
audio_buffer,
np.zeros(
self.np.zeros(
end_sample - len(audio_buffer), dtype="float32"
),
]
@@ -1957,7 +1946,7 @@ class ConversionThread(QThread):
self.progress_updated.emit(percent, etr_str)
# Normalize audio buffer to prevent clipping from mixed overlaps
max_amplitude = np.abs(audio_buffer).max()
max_amplitude = self.np.abs(audio_buffer).max()
if max_amplitude > 1.0:
self.log_updated.emit(
f"\n -> Normalizing audio (peak: {max_amplitude:.2f})"
@@ -2397,10 +2386,6 @@ class ConversionThread(QThread):
self.cancel_requested = True
self.should_cancel = True
self.waiting_for_user_input = False
# Clear voice cache (instance and module-level)
self.voice_cache.clear()
from abogen.voice_cache import clear_voice_cache
clear_voice_cache()
# Terminate subprocess if running
if self.process:
try:
@@ -2430,19 +2415,28 @@ class VoicePreviewThread(QThread):
def __init__(
self,
backend,
np_module,
kpipeline_class,
lang_code,
voice,
speed,
use_gpu=False,
parent=None,
tts_provider="kokoro",
supertonic_language="en",
supertonic_total_steps=8,
):
super().__init__(parent)
self.backend = backend
self.np_module = np_module
self.kpipeline_class = kpipeline_class
self.lang_code = lang_code
self.voice = voice
self.speed = speed
self.use_gpu = use_gpu
self.tts_provider = tts_provider
self.supertonic_language = supertonic_language
self.supertonic_total_steps = supertonic_total_steps
self.sample_rate = 44100 if tts_provider == "supertonic" else 24000
# Cache location for preview audio
self.cache_dir = get_user_cache_path("preview_cache")
@@ -2452,6 +2446,11 @@ class VoicePreviewThread(QThread):
def _get_cache_path(self):
"""Generate a unique filename for the voice with its parameters"""
if self.tts_provider == "supertonic":
voice_id = self.voice or "M1"
filename = f"st_{voice_id}_{self.supertonic_language}_steps{self.supertonic_total_steps}_{self.speed:.2f}.wav"
return os.path.join(self.cache_dir, filename)
# For a voice formula, use a hash of the formula
if "*" in self.voice:
voice_id = (
@@ -2466,27 +2465,56 @@ class VoicePreviewThread(QThread):
def run(self):
print(
f"\nVoice: {self.voice}\nLanguage: {self.lang_code}\nSpeed: {self.speed}\nGPU: {self.use_gpu}\n"
f"\nVoice: {self.voice}\nLanguage: {self.lang_code}\nSpeed: {self.speed}\nGPU: {self.use_gpu}\nTTS Provider: {self.tts_provider}\n"
)
# Generate the preview and save to cache
try:
if self.tts_provider == "supertonic":
from abogen.tts_supertonic import SupertonicPipeline
# Enable voice formula support for preview
if "*" in self.voice:
loaded_voice = get_new_voice(self.backend, self.voice, self.use_gpu)
tts = SupertonicPipeline(
sample_rate=self.sample_rate,
lang=self.supertonic_language,
total_steps=self.supertonic_total_steps,
)
loaded_voice = self.voice or "M1"
sample_text = "Hello, this is a sample of the selected voice."
audio_segments = []
for result in tts(
sample_text,
voice=loaded_voice,
speed=self.speed,
split_pattern=None,
):
audio_segments.append(result.audio)
else:
loaded_voice = self.voice
sample_text = get_sample_voice_text(self.lang_code)
audio_segments = []
for result in self.backend(
sample_text, voice=loaded_voice, speed=self.speed, split_pattern=None
):
audio_segments.append(result.audio)
# Set device based on use_gpu setting and platform
if self.use_gpu:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps"
else:
device = "cuda"
else:
device = "cpu"
tts = self.kpipeline_class(
lang_code=self.lang_code, repo_id="hexgrad/Kokoro-82M", device=device
)
if "*" in self.voice:
loaded_voice = get_new_voice(tts, self.voice, self.use_gpu)
else:
loaded_voice = self.voice
sample_text = get_sample_voice_text(self.lang_code)
audio_segments = []
for result in tts(
sample_text, voice=loaded_voice, speed=self.speed, split_pattern=None
):
audio_segments.append(result.audio)
if audio_segments:
audio = np.concatenate(audio_segments)
# Save directly to the cache path
sf.write(self.cache_path, audio, 24000)
audio = self.np_module.concatenate(audio_segments)
sf.write(self.cache_path, audio, self.sample_rate)
self.temp_wav = self.cache_path
self.finished.emit()
except Exception as e:
+256 -55
View File
@@ -9,6 +9,7 @@ from abogen.pyqt.queue_manager_gui import QueueManager
from abogen.pyqt.queued_item import QueuedItem
import abogen.hf_tracker as hf_tracker
import hashlib # Added for cache path generation
from abogen.tts_supertonic import SUPERTONIC_AVAILABLE_LANGS, DEFAULT_SUPERTONIC_VOICES
from PyQt6.QtWidgets import (
QApplication,
QWidget,
@@ -82,11 +83,13 @@ from abogen.constants import (
GITHUB_URL,
PROGRAM_DESCRIPTION,
LANGUAGE_DESCRIPTIONS,
VOICES_INTERNAL,
KOKORO_LANG_TO_COUNTRY,
SUPERTONIC_LANG_TO_COUNTRY,
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
COLORS,
SUBTITLE_FORMATS,
)
from abogen.tts_plugin.utils import get_voices
import threading
from abogen.pyqt.voice_formula_gui import VoiceFormulaDialog
from abogen.voice_profiles import load_profiles
@@ -970,6 +973,9 @@ class abogen(QWidget):
self.fix_nonstandard_punctuation = self.config.get(
"fix_nonstandard_punctuation", False
)
self.tts_provider_config = self.config.get("tts_provider", "kokoro")
self.supertonic_language_config = self.config.get("supertonic_language", "en")
self.supertonic_total_steps_config = self.config.get("supertonic_total_steps", 8)
self._pending_close_event = None
self.gpu_ok = False # Initialize GPU availability status
@@ -1018,6 +1024,16 @@ class abogen(QWidget):
else:
self.mixed_voice_state = entry
self.selected_lang = entry[0][0] if entry and entry[0] else None
# Restore TTS provider and supertonic settings from config
provider_text = "Supertonic" if self.tts_provider_config == "supertonic" else "Kokoro"
idx_st = self.provider_combo.findText(provider_text)
if idx_st >= 0:
self.provider_combo.setCurrentIndex(idx_st)
self.st_lang_combo.setCurrentText(self.supertonic_language_config)
idx_steps = self.st_steps_combo.findData(self.supertonic_total_steps_config)
if idx_steps >= 0:
self.st_steps_combo.setCurrentIndex(idx_steps)
if self.save_option == "Choose output folder" and self.selected_output_folder:
self.save_path_label.setText(self.selected_output_folder)
self.save_path_row_widget.show()
@@ -1107,6 +1123,53 @@ class abogen(QWidget):
speed_layout.addWidget(self.speed_label)
controls_layout.addLayout(speed_layout)
self.speed_slider.valueChanged.connect(self.update_speed_label)
# TTS Provider selection
provider_layout = QHBoxLayout()
provider_layout.setSpacing(7)
provider_label = QLabel("TTS Engine:", self)
provider_layout.addWidget(provider_label)
self.provider_combo = QComboBox(self)
self.provider_combo.addItem("Kokoro", "kokoro")
self.provider_combo.addItem("Supertonic", "supertonic")
self.provider_combo.setStyleSheet("QComboBox { min-height: 20px; padding: 6px 12px; }")
self.provider_combo.currentIndexChanged.connect(self.on_provider_changed)
provider_layout.addWidget(self.provider_combo)
controls_layout.addLayout(provider_layout)
# Supertonic-specific controls (language + steps), hidden by default
self.supertonic_row = QWidget()
supertonic_row_layout = QHBoxLayout(self.supertonic_row)
supertonic_row_layout.setContentsMargins(0, 0, 0, 0)
supertonic_row_layout.setSpacing(7)
st_lang_label = QLabel("Language:", self)
supertonic_row_layout.addWidget(st_lang_label)
self.st_lang_combo = QComboBox(self)
for code in SUPERTONIC_AVAILABLE_LANGS:
country_code = SUPERTONIC_LANG_TO_COUNTRY.get(code, code)
flag = get_resource_path("abogen.assets.flags", f"{country_code}.png")
icon_st = QIcon(flag) if flag and os.path.exists(flag) else QIcon()
self.st_lang_combo.addItem(icon_st, code, code)
self.st_lang_combo.setCurrentText("en")
self.st_lang_combo.setStyleSheet("QComboBox { min-height: 20px; padding: 6px 12px; }")
self.st_lang_combo.currentTextChanged.connect(self._on_st_lang_changed)
supertonic_row_layout.addWidget(self.st_lang_combo)
st_steps_label = QLabel("Steps:", self)
supertonic_row_layout.addWidget(st_steps_label)
self.st_steps_combo = QComboBox(self)
for val in range(2, 16):
self.st_steps_combo.addItem(str(val), val)
self.st_steps_combo.setCurrentIndex(self.st_steps_combo.findData(8))
self.st_steps_combo.setStyleSheet("QComboBox { min-height: 20px; padding: 6px 12px; }")
self.st_steps_combo.currentIndexChanged.connect(self._on_st_steps_changed)
supertonic_row_layout.addWidget(self.st_steps_combo)
supertonic_row_layout.addStretch()
self.supertonic_row.hide()
controls_layout.addWidget(self.supertonic_row)
# Voice selection
voice_layout = QHBoxLayout()
voice_layout.setSpacing(7)
@@ -1764,6 +1827,12 @@ class abogen(QWidget):
Update the enabled state of subtitle options based on the selected language.
For non-English languages, only sentence-based and line-based modes are supported.
"""
provider = self.provider_combo.currentData()
if provider == "supertonic":
self.subtitle_combo.setEnabled(False)
self.subtitle_format_combo.setEnabled(False)
return
# Check if current file is a subtitle file
is_subtitle_input = False
if self.selected_file and self.selected_file.lower().endswith(
@@ -1823,6 +1892,48 @@ class abogen(QWidget):
# Enable/disable subtitle options based on language
self.update_subtitle_options_availability()
def on_provider_changed(self, index):
provider = self.provider_combo.itemData(index)
self.config["tts_provider"] = provider
save_config(self.config)
is_supertonic = provider == "supertonic"
# Show/hide Supertonic controls
self.supertonic_row.setVisible(is_supertonic)
# Update subtitles availability
self.update_subtitle_options_availability()
# Repopulate voice list
self.populate_profiles_in_voice_combo()
# Clear/reset mixed voice state when switching provider
if is_supertonic:
self.mixed_voice_state = None
self.btn_voice_formula_mixer.setEnabled(False)
self.voice_combo.setToolTip(
"Supertonic voices:\n"
"M1-M5 = Male voices\n"
"F1-F5 = Female voices"
)
else:
self.btn_voice_formula_mixer.setEnabled(True)
self.voice_combo.setToolTip(
"The first character represents the language:\n"
'"a" => American English\n"b" => British English\n"e" => Spanish\n"f" => French\n"h" => Hindi\n"i" => Italian\n"j" => Japanese\n"p" => Brazilian Portuguese\n"z" => Mandarin Chinese\nThe second character represents the gender:\n"m" => Male\n"f" => Female'
)
def _on_st_lang_changed(self, lang):
self.config["supertonic_language"] = lang
save_config(self.config)
if self.provider_combo.currentData() == "supertonic":
self.selected_lang = lang
self.update_subtitle_options_availability()
def _on_st_steps_changed(self):
self.config["supertonic_total_steps"] = self.st_steps_combo.currentData()
save_config(self.config)
def on_voice_combo_changed(self, index):
data = self.voice_combo.itemData(index)
if isinstance(data, str) and data.startswith("profile:"):
@@ -1831,10 +1942,26 @@ class abogen(QWidget):
from abogen.voice_profiles import load_profiles
entry = load_profiles().get(pname, {})
# set mixed voices and language
if isinstance(entry, dict):
self.mixed_voice_state = entry.get("voices", [])
self.selected_lang = entry.get("language")
entry_provider = str(entry.get("provider", "")).strip().lower()
if entry_provider == "supertonic":
# Switch provider to Supertonic if not already
if self.provider_combo.currentData() != "supertonic":
self.provider_combo.setCurrentIndex(1)
self.mixed_voice_state = None
self.selected_lang = entry.get("language", self.st_lang_combo.currentText())
# Sync supertonic controls from profile
profile_steps = entry.get("total_steps")
if profile_steps is not None:
idx_steps = self.st_steps_combo.findData(int(profile_steps))
if idx_steps >= 0:
self.st_steps_combo.setCurrentIndex(idx_steps)
profile_lang = entry.get("language")
if profile_lang and profile_lang in SUPERTONIC_AVAILABLE_LANGS:
self.st_lang_combo.setCurrentText(profile_lang)
else:
self.mixed_voice_state = entry.get("voices", [])
self.selected_lang = entry.get("language")
else:
self.mixed_voice_state = entry
self.selected_lang = entry[0][0] if entry and entry[0] else None
@@ -1847,7 +1974,12 @@ class abogen(QWidget):
else:
self.mixed_voice_state = None
self.selected_profile_name = None
self.selected_voice, self.selected_lang = data, data[0]
self.selected_voice = data
provider = self.provider_combo.currentData()
if provider == "supertonic":
self.selected_lang = self.st_lang_combo.currentText()
else:
self.selected_lang = data[0] if data else ""
self.config["selected_voice"] = data
if "selected_profile_name" in self.config:
del self.config["selected_profile_name"]
@@ -1866,19 +1998,40 @@ class abogen(QWidget):
def populate_profiles_in_voice_combo(self):
# preserve current voice or profile
current = self.voice_combo.currentData()
provider = self.provider_combo.currentData()
self.voice_combo.blockSignals(True)
self.voice_combo.clear()
# re-add profiles
# re-add profiles matching current provider
profile_icon = QIcon(get_resource_path("abogen.assets", "profile.png"))
for pname in load_profiles().keys():
self.voice_combo.addItem(profile_icon, pname, f"profile:{pname}")
for pname, entry in load_profiles().items():
entry_provider = ""
if isinstance(entry, dict):
entry_provider = str(entry.get("provider", "")).strip().lower()
if provider == "supertonic":
if entry_provider == "supertonic":
self.voice_combo.addItem(profile_icon, pname, f"profile:{pname}")
else:
if entry_provider != "supertonic":
self.voice_combo.addItem(profile_icon, pname, f"profile:{pname}")
# re-add voices
for v in get_voices("kokoro"):
icon = QIcon()
flag_path = get_resource_path("abogen.assets.flags", f"{v[0]}.png")
if flag_path and os.path.exists(flag_path):
icon = QIcon(flag_path)
self.voice_combo.addItem(icon, f"{v}", v)
if provider == "supertonic":
for v in DEFAULT_SUPERTONIC_VOICES:
icon = QIcon()
if v.startswith("F"):
icon_path = get_resource_path("abogen.assets", "female.png")
else:
icon_path = get_resource_path("abogen.assets", "male.png")
if icon_path and os.path.exists(icon_path):
icon = QIcon(icon_path)
self.voice_combo.addItem(icon, f"{v}", v)
else:
for v in VOICES_INTERNAL:
icon = QIcon()
country_code = KOKORO_LANG_TO_COUNTRY.get(v[0], v[0])
flag_path = get_resource_path("abogen.assets.flags", f"{country_code}.png")
if flag_path and os.path.exists(flag_path):
icon = QIcon(flag_path)
self.voice_combo.addItem(icon, f"{v}", v)
# restore selection
idx = -1
if self.selected_profile_name:
@@ -2069,6 +2222,9 @@ class abogen(QWidget):
save_base_path=save_base_path,
save_chapters_separately=getattr(self, "save_chapters_separately", None),
merge_chapters_at_end=getattr(self, "merge_chapters_at_end", None),
tts_provider=self.provider_combo.currentData(),
supertonic_language=self.st_lang_combo.currentText(),
supertonic_total_steps=self.st_steps_combo.currentData(),
)
# Prevent adding duplicate items to the queue
@@ -2212,6 +2368,15 @@ class abogen(QWidget):
self.config["replace_numerals"] = self.replace_numerals
self.config["fix_nonstandard_punctuation"] = self.fix_nonstandard_punctuation
# TTS provider settings
tts_provider = getattr(queued_item, "tts_provider", "kokoro")
self.provider_combo.setCurrentText("Supertonic" if tts_provider == "supertonic" else "Kokoro")
self.st_lang_combo.setCurrentText(getattr(queued_item, "supertonic_language", "en"))
steps_val = getattr(queued_item, "supertonic_total_steps", 8)
idx_steps = self.st_steps_combo.findData(steps_val)
if idx_steps >= 0:
self.st_steps_combo.setCurrentIndex(idx_steps)
# Sync Voice/Profile in config
self.config["selected_voice"] = self.selected_voice
if "selected_profile_name" in self.config:
@@ -2234,6 +2399,8 @@ class abogen(QWidget):
self.current_queue_index = 0 # Reset for next time
def get_voice_formula(self) -> str:
if self.provider_combo.currentData() == "supertonic":
return self._get_supertonic_voice()
if self.mixed_voice_state:
formula_components = [
f"{name}*{weight}" for name, weight in self.mixed_voice_state
@@ -2243,6 +2410,8 @@ class abogen(QWidget):
return self.selected_voice
def get_selected_lang(self, voice_formula) -> str:
if self.provider_combo.currentData() == "supertonic":
return self.st_lang_combo.currentText()
if self.selected_profile_name:
from abogen.voice_profiles import load_profiles
@@ -2316,9 +2485,9 @@ class abogen(QWidget):
file_size_str = "Unknown"
# pipeline_loaded_callback remains unchanged
def pipeline_loaded_callback(backend, error):
def pipeline_loaded_callback(np_module, kpipeline_class, error):
if error:
self.update_log((f"Error loading TTS backend: {error}", "red"))
self.update_log((f"Error loading numpy or KPipeline: {error}", "red"))
prevent_sleep_end()
return
@@ -2332,6 +2501,10 @@ class abogen(QWidget):
# determine selected language: use profile setting if profile selected, else voice code
selected_lang = self.get_selected_lang(voice_formula)
tts_provider = self.provider_combo.currentData()
supertonic_language = self.st_lang_combo.currentText()
supertonic_total_steps = self.st_steps_combo.currentData()
self.conversion_thread = ConversionThread(
self.selected_file,
selected_lang,
@@ -2341,13 +2514,17 @@ class abogen(QWidget):
self.selected_output_folder,
subtitle_mode=actual_subtitle_mode,
output_format=self.selected_format,
backend=backend,
np_module=np_module,
kpipeline_class=kpipeline_class,
start_time=self.start_time,
total_char_count=self.char_count,
use_gpu=self.gpu_ok,
from_queue=from_queue,
save_base_path=self.displayed_file_path, # Pass the save base path (original file for EPUB)
) # Use gpu_ok status
save_base_path=self.displayed_file_path,
tts_provider=tts_provider,
supertonic_language=supertonic_language,
supertonic_total_steps=supertonic_total_steps,
)
# Pass the displayed file path to the log_updated signal handler in ConversionThread
self.conversion_thread.display_path = display_path
# Pass the file size string
@@ -2424,22 +2601,15 @@ class abogen(QWidget):
# Store gpu_ok status to use when creating the conversion thread
self.gpu_ok = gpu_ok
self.update_log((gpu_msg, gpu_ok))
self.update_log("Loading modules...")
# Determine device based on GPU availability
if gpu_ok:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps"
else:
device = "cuda"
tts_provider = self.provider_combo.currentData()
if tts_provider == "supertonic":
# Supertonic doesn't need KPipeline, call callback directly
import numpy as np
pipeline_loaded_callback(np, None, None)
else:
device = "cpu"
lang_code = self.selected_lang or "a"
load_thread = LoadPipelineThread(
pipeline_loaded_callback, lang_code=lang_code, device=device
)
load_thread.start()
self.update_log("Loading modules...")
load_thread = LoadPipelineThread(pipeline_loaded_callback)
load_thread.start()
threading.Thread(target=gpu_and_load, daemon=True).start()
@@ -2752,9 +2922,32 @@ class abogen(QWidget):
"Open File Error", f"Could not open file:\n{e}"
)
def _get_supertonic_voice(self) -> str:
"""Resolve the effective Supertonic voice from the current combo selection."""
if self.selected_profile_name:
from abogen.voice_profiles import load_profiles
entry = load_profiles().get(self.selected_profile_name, {})
if isinstance(entry, dict):
return str(entry.get("voice", "M1"))
return "M1"
current_data = self.voice_combo.currentData()
if current_data and isinstance(current_data, str) and not current_data.startswith("profile:"):
return current_data
return "M1"
def _get_preview_cache_path(self):
"""Generate the expected cache path for the current voice settings."""
speed = self.speed_slider.value() / 100.0
provider = self.provider_combo.currentData()
if provider == "supertonic":
voice_to_cache = self._get_supertonic_voice()
lang_to_cache = self.st_lang_combo.currentText()
steps = self.st_steps_combo.currentData()
cache_dir = get_user_cache_path("preview_cache")
filename = f"st_{voice_to_cache}_{lang_to_cache}_steps{steps}_{speed:.2f}.wav"
return os.path.join(cache_dir, filename)
voice_to_cache = ""
lang_to_cache = ""
@@ -2859,6 +3052,13 @@ class abogen(QWidget):
self.btn_voice_formula_mixer.setEnabled(False) # Disable mixer button
self.btn_start.setEnabled(False) # Disable start button during preview
# For Supertonic, skip KPipeline loading and use SupertonicPipeline directly
if self.provider_combo.currentData() == "supertonic":
import numpy as np
self.loading_movie.start()
self._on_pipeline_loaded_for_preview(np, None, None)
return
# Start loading animation - ensure signal connection is always active
if hasattr(self, "loading_movie"):
# Disconnect previous connections to avoid multiple connections
@@ -2875,27 +3075,18 @@ class abogen(QWidget):
)
self.loading_movie.start()
# Determine device based on GPU availability
if self.gpu_ok:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps"
else:
device = "cuda"
else:
device = "cpu"
def pipeline_loaded_callback(np_module, kpipeline_class, error):
self._on_pipeline_loaded_for_preview(np_module, kpipeline_class, error)
lang = self.selected_lang or "a"
load_thread = LoadPipelineThread(
self._on_pipeline_loaded_for_preview, lang_code=lang, device=device
)
load_thread = LoadPipelineThread(pipeline_loaded_callback)
load_thread.start()
def _on_pipeline_loaded_for_preview(self, backend, error):
def _on_pipeline_loaded_for_preview(self, np_module, kpipeline_class, error):
# stop loading animation and restore icon on error
if error:
self.loading_movie.stop()
self._show_error_message_box(
"Loading Error", f"Error loading TTS backend: {error}"
"Loading Error", f"Error loading numpy or KPipeline: {error}"
)
self.btn_preview.setIcon(self.play_icon)
self.btn_preview.setEnabled(True)
@@ -2926,14 +3117,28 @@ class abogen(QWidget):
else None
)
else:
lang = self.selected_voice[0]
voice = self.selected_voice
if self.provider_combo.currentData() == "supertonic":
voice = self._get_supertonic_voice()
else:
voice = self.selected_voice or ""
tts_provider = self.provider_combo.currentData()
supertonic_language = self.st_lang_combo.currentText()
supertonic_total_steps = self.st_steps_combo.currentData()
if tts_provider == "supertonic":
lang = supertonic_language
else:
lang = self.selected_voice[0] if self.selected_voice else ""
# use same gpu/cpu logic as in conversion
gpu_msg, gpu_ok = get_gpu_acceleration(self.use_gpu)
self.preview_thread = VoicePreviewThread(
backend, lang, voice, speed, gpu_ok
np_module, kpipeline_class, lang, voice, speed, gpu_ok,
tts_provider=tts_provider,
supertonic_language=supertonic_language,
supertonic_total_steps=supertonic_total_steps,
)
self.preview_thread.finished.connect(self._play_preview_audio)
self.preview_thread.error.connect(self._preview_error)
@@ -3236,16 +3441,12 @@ class abogen(QWidget):
)
box.setDefaultButton(QMessageBox.StandardButton.No)
if box.exec() == QMessageBox.StandardButton.Yes:
from abogen import shutdown
shutdown.request_shutdown()
self.cleanup_conversion_thread()
self.cleanup_preview_threads()
event.accept()
else:
event.ignore()
else:
from abogen import shutdown
shutdown.request_shutdown()
self.cleanup_conversion_thread()
self.cleanup_preview_threads()
event.accept()
+23 -5
View File
@@ -1,10 +1,10 @@
import os
import sys
import platform
import atexit
import signal
from abogen.utils import get_resource_path, load_config, prevent_sleep_end
# Initialise global shutdown handling (atexit, signals, Qt) as early as possible.
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
# Fix PyTorch DLL loading issue ([WinError 1114]) on Windows before importing PyQt6
if platform.system() == "Windows":
@@ -94,7 +94,6 @@ os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1" # Disable Hugging Face telemetry
os.environ["HF_HUB_ETAG_TIMEOUT"] = "10" # Metadata request timeout (seconds)
os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = "10" # File download timeout (seconds)
os.environ["HF_HUB_DISABLE_SYMLINKS_WARNING"] = "1" # Disable symlinks warning
from abogen.utils import load_config
if load_config().get("disable_kokoro_internet", False):
print("INFO: Kokoro's internet access is disabled.")
os.environ["HF_HUB_OFFLINE"] = "1" # Disable Hugging Face Hub internet access
@@ -106,6 +105,25 @@ from abogen.constants import PROGRAM_NAME, VERSION
os.environ["MIOPEN_FIND_MODE"] = "FAST"
os.environ["MIOPEN_CONV_PRECISE_ROCM_TUNING"] = "0"
# Reset sleep states
atexit.register(prevent_sleep_end)
# Also handle signals (Ctrl+C, kill, etc.)
def _cleanup_sleep(signum, frame):
prevent_sleep_end()
sys.exit(0)
signal.signal(signal.SIGINT, _cleanup_sleep)
signal.signal(signal.SIGTERM, _cleanup_sleep)
# Ensure sys.stdout and sys.stderr are valid in GUI mode
if sys.stdout is None:
sys.stdout = open(os.devnull, "w")
if sys.stderr is None:
sys.stderr = open(os.devnull, "w")
# Enable MPS GPU acceleration on Mac Apple Silicon
if platform.system() == "Darwin" and platform.processor() == "arm":
os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"
@@ -166,4 +184,4 @@ def main():
if __name__ == "__main__":
main()
main()
+4 -5
View File
@@ -21,8 +21,7 @@ from PyQt6.QtWidgets import (
)
from PyQt6.QtCore import QThread, pyqtSignal
from abogen.constants import COLORS
from abogen.tts_plugin.utils import get_voices
from abogen.constants import COLORS, VOICES_INTERNAL
from abogen.spacy_utils import SPACY_MODELS
import abogen.hf_tracker
@@ -115,7 +114,7 @@ class PreDownloadWorker(QThread):
self._voices_success = False
return
voice_list = get_voices("kokoro")
voice_list = VOICES_INTERNAL
for idx, voice in enumerate(voice_list, start=1):
if self._cancelled:
self._voices_success = False
@@ -463,14 +462,14 @@ class PreDownloadDialog(QDialog):
try:
from huggingface_hub import try_to_load_from_cache
for voice in get_voices("kokoro"):
for voice in VOICES_INTERNAL:
if not try_to_load_from_cache(
repo_id="hexgrad/Kokoro-82M", filename=f"voices/{voice}.pt"
):
missing.append(voice)
except Exception:
# If HF missing, report all as missing
return False, list(get_voices("kokoro"))
return False, list(VOICES_INTERNAL)
return (len(missing) == 0), missing
def _check_kokoro_model(self) -> bool:
+4
View File
@@ -26,3 +26,7 @@ class QueuedItem:
replace_all_caps: bool = False
replace_numerals: bool = False
fix_nonstandard_punctuation: bool = False
# TTS Provider fields
tts_provider: str = "kokoro"
supertonic_language: str = "en"
supertonic_total_steps: int = 8
+8 -5
View File
@@ -28,11 +28,12 @@ from PyQt6.QtWidgets import (
from PyQt6.QtCore import Qt, QTimer, QPoint, QRect, QSize
from PyQt6.QtGui import QPixmap, QIcon, QAction
from abogen.constants import (
VOICES_INTERNAL,
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
LANGUAGE_DESCRIPTIONS,
KOKORO_LANG_TO_COUNTRY,
COLORS,
)
from abogen.tts_plugin.utils import get_voices
import re
import platform
from abogen.utils import get_resource_path
@@ -179,7 +180,7 @@ class VoiceMixer(QWidget):
layout.addWidget(QLabel(name), alignment=Qt.AlignmentFlag.AlignCenter)
# Voice name label with gender icon
is_female = self.voice_name in get_voices("kokoro") and self.voice_name[1] == "f"
is_female = self.voice_name in VOICES_INTERNAL and self.voice_name[1] == "f"
# Icons layout (flag and gender)
icons_layout = QHBoxLayout()
@@ -189,8 +190,9 @@ class VoiceMixer(QWidget):
) # Center the icons horizontally
# Flag icon
country_code = KOKORO_LANG_TO_COUNTRY.get(language_code, language_code)
flag_icon_path = get_resource_path(
"abogen.assets.flags", f"{language_code}.png"
"abogen.assets.flags", f"{country_code}.png"
)
gender_icon_path = get_resource_path(
"abogen.assets", "female.png" if is_female else "male.png"
@@ -512,7 +514,8 @@ class VoiceFormulaDialog(QDialog):
header_row.addWidget(QLabel("Language:"))
self.language_combo = QComboBox()
for code, desc in LANGUAGE_OPTIONS:
flag = get_resource_path("abogen.assets.flags", f"{code}.png")
country_code = KOKORO_LANG_TO_COUNTRY.get(code, code)
flag = get_resource_path("abogen.assets.flags", f"{country_code}.png")
if flag and os.path.exists(flag):
self.language_combo.addItem(QIcon(flag), desc, code)
else:
@@ -772,7 +775,7 @@ class VoiceFormulaDialog(QDialog):
def add_voices(self, initial_state):
first_enabled_voice = None
for voice in get_voices("kokoro"):
for voice in VOICES_INTERNAL:
language_code = voice[0] # First character is the language code
matching_voice = next(
(item for item in initial_state if item[0] == voice), None
-160
View File
@@ -1,160 +0,0 @@
"""Graceful shutdown - single module, no over-engineering."""
from __future__ import annotations
import atexit
import gc
import signal
import sys
from typing import Callable
_CLEANUP_FUNCS: list[Callable[[], None]] = []
_EXECUTED = False
def register_cleanup(fn: Callable[[], None]) -> None:
"""Register a cleanup function to run on shutdown."""
_CLEANUP_FUNCS.append(fn)
def _run_cleanups() -> None:
global _EXECUTED
if _EXECUTED:
return
_EXECUTED = True
for fn in _CLEANUP_FUNCS:
try:
fn()
except Exception:
pass
# ---- Register built-in cleanup functions ----
# 1. Restore sleep prevention
def _restore_sleep() -> None:
try:
from abogen.utils import prevent_sleep_end
prevent_sleep_end()
except Exception:
pass
register_cleanup(_restore_sleep)
# 2. Shutdown web UI ConversionService
def _shutdown_conversion_service() -> None:
try:
from abogen.webui.service import get_service
svc = get_service()
if svc is not None:
svc.shutdown()
except Exception:
pass
register_cleanup(_shutdown_conversion_service)
# 3. Clear TTS pipelines and GPU memory
def _cleanup_tts_pipelines() -> None:
# Clear web UI pipeline cache
try:
from abogen.webui.conversion_runner import _PIPELINES
_PIPELINES.clear()
except Exception:
pass
# Clear PyQt conversion thread voice cache
try:
from abogen.pyqt.conversion import ConversionThread
if hasattr(ConversionThread, "voice_cache"):
ConversionThread.voice_cache.clear()
except Exception:
pass
gc.collect()
# Release CUDA cache
try:
import torch
if torch.cuda.is_available():
torch.cuda.empty_cache()
torch.cuda.ipc_collect()
except Exception:
pass
register_cleanup(_cleanup_tts_pipelines)
# 4. Clear global voice cache
def _clear_voice_cache() -> None:
try:
from abogen.voice_cache import clear_voice_cache
clear_voice_cache()
except Exception:
pass
register_cleanup(_clear_voice_cache)
# 5. Terminate child processes (ffmpeg, etc.)
def _terminate_subprocesses() -> None:
try:
import psutil
except Exception:
return
try:
current = psutil.Process()
for child in current.children(recursive=True):
try:
child.terminate()
except Exception:
pass
gone, alive = psutil.wait_procs(current.children(recursive=True), timeout=3)
for proc in alive:
try:
proc.kill()
except Exception:
pass
except Exception:
pass
register_cleanup(_terminate_subprocesses)
def register_shutdown() -> None:
"""Install process-wide shutdown hooks (atexit, signals, Qt)."""
if register_shutdown._registered:
return
register_shutdown._registered = True
atexit.register(_run_cleanups)
# POSIX signals
for sig in (signal.SIGINT, signal.SIGTERM):
try:
signal.signal(sig, _on_signal)
except Exception:
pass
# Qt hook
try:
from PyQt6.QtWidgets import QApplication
app = QApplication.instance()
if app is not None:
app.aboutToQuit.connect(_run_cleanups)
except Exception:
pass
register_shutdown._registered = False
def _on_signal(signum: int, _frame) -> None:
_run_cleanups()
sys.exit(0)
def request_shutdown() -> None:
"""Programmatically trigger cleanup (e.g., from GUI closeEvent)."""
_run_cleanups()
__all__ = ["register_shutdown", "request_shutdown", "register_cleanup"]
+7 -7
View File
@@ -466,7 +466,7 @@ def sanitize_name_for_os(name, is_folder=True):
def validate_voice_name(voice_name):
"""Validate voice name against available voices (case-insensitive).
"""Validate voice name against VOICES_INTERNAL list (case-insensitive).
Handles both single voices and formulas like 'af_heart*0.5 + am_echo*0.5'.
Args:
@@ -477,10 +477,10 @@ def validate_voice_name(voice_name):
- is_valid: True if all voices in the name/formula are valid
- invalid_voice_name: The first invalid voice found, or None if all valid
"""
from abogen.tts_plugin.utils import get_voices
from abogen.constants import VOICES_INTERNAL
# Create case-insensitive lookup set (done once per call)
voice_lookup_lower = {v.lower() for v in get_voices("kokoro")}
voice_lookup_lower = {v.lower() for v in VOICES_INTERNAL}
voice_name = voice_name.strip()
# Check if it's a formula (contains *)
@@ -505,7 +505,7 @@ def split_text_by_voice_markers(text, default_voice):
"""Split text by voice markers, returning list of (voice, text) tuples.
IMPORTANT: Returns the last voice used so it can persist across chapters.
Voice names are normalized to lowercase to match canonical voice names.
Voice names are normalized to lowercase to match VOICES_INTERNAL.
Args:
text: Text potentially containing <<VOICE:name>> markers
@@ -518,7 +518,7 @@ def split_text_by_voice_markers(text, default_voice):
- valid_count: Number of valid voice markers processed
- invalid_count: Number of invalid voice markers skipped
"""
from abogen.tts_plugin.utils import get_voices
from abogen.constants import VOICES_INTERNAL
voice_splits = list(_VOICE_MARKER_SEARCH_PATTERN.finditer(text))
@@ -560,7 +560,7 @@ def split_text_by_voice_markers(text, default_voice):
# Find the canonical (lowercase) voice name
voice_part_lower = voice_part.strip().lower()
canonical_voice = next(
(v for v in get_voices("kokoro") if v.lower() == voice_part_lower),
(v for v in VOICES_INTERNAL if v.lower() == voice_part_lower),
voice_part.strip()
)
normalized_parts.append(f"{canonical_voice}*{weight.strip()}")
@@ -569,7 +569,7 @@ def split_text_by_voice_markers(text, default_voice):
# Find the canonical (lowercase) voice name
voice_name_lower = voice_name.lower()
current_voice = next(
(v for v in get_voices("kokoro") if v.lower() == voice_name_lower),
(v for v in VOICES_INTERNAL if v.lower() == voice_name_lower),
voice_name
)
valid_markers += 1
-170
View File
@@ -1,170 +0,0 @@
"""TTS Plugin Architecture - Public API.
This package defines the frozen Plugin API for the TTS Plugin Architecture.
All public interfaces are fully defined but contain no business logic.
Public modules:
- types: Core domain value objects (AudioFormat, Duration, VoiceSelection, etc.)
- errors: Error hierarchy (EngineError and subtypes)
- manifest: Plugin manifest types (PluginManifest, EngineManifest, etc.)
- engine: Engine and EngineSession protocols
- capabilities: Optional capability interfaces (VoiceLister, PreviewGenerator, etc.)
- host_context: HostContext dataclass
- plugin: Plugin contract (create_engine function signature)
- loader: Plugin discovery and loading
- plugin_manager: Plugin management and engine creation
- utils: Direct utility functions (get_voices, create_pipeline, etc.)
Usage:
from abogen.tts_plugin import (
# Types
AudioFormat,
Duration,
VoiceSelection,
ParameterValues,
SynthesisRequest,
SynthesizedAudio,
EngineConfig,
# Errors
EngineError,
ModelNotFoundError,
ModelLoadError,
NetworkError,
InvalidInputError,
ConfigurationError,
CancelledError,
InternalError,
# Manifest
PluginManifest,
EngineManifest,
VoiceSourceManifest,
VoiceManifest,
ParameterManifest,
AudioFormatManifest,
EnumOption,
RequirementManifest,
GpuRequirement,
ModelManifest,
# Engine
Engine,
EngineSession,
# Capabilities
VoiceLister,
PreviewGenerator,
StreamingSynthesizer,
CancelableSession,
# Host Context
HostContext,
HttpClient,
# Plugin Manager
get_plugin_manager,
reset_plugin_manager,
# Utils
get_voices,
get_default_voice,
is_plugin_registered,
resolve_voice_to_plugin,
create_pipeline,
)
"""
from abogen.tts_plugin.capabilities import (
CancelableSession,
PreviewGenerator,
StreamingSynthesizer,
VoiceLister,
)
from abogen.tts_plugin.engine import Engine, EngineSession
from abogen.tts_plugin.errors import (
CancelledError,
ConfigurationError,
EngineError,
InternalError,
InvalidInputError,
ModelLoadError,
ModelNotFoundError,
NetworkError,
)
from abogen.tts_plugin.host_context import HttpClient, HostContext
from abogen.tts_plugin.manifest import (
AudioFormatManifest,
EngineManifest,
EnumOption,
GpuRequirement,
ModelManifest,
ParameterManifest,
PluginManifest,
RequirementManifest,
VoiceManifest,
VoiceSourceManifest,
)
from abogen.tts_plugin.types import (
AudioFormat,
Duration,
EngineConfig,
ParameterValues,
SynthesisRequest,
SynthesizedAudio,
VoiceSelection,
)
# Plugin Manager and Utils
from abogen.tts_plugin.plugin_manager import get_plugin_manager, reset_plugin_manager
from abogen.tts_plugin.utils import (
create_pipeline,
get_default_voice,
get_voices,
is_plugin_registered,
resolve_voice_to_plugin,
)
__all__ = [
# Types
"AudioFormat",
"Duration",
"VoiceSelection",
"ParameterValues",
"SynthesisRequest",
"SynthesizedAudio",
"EngineConfig",
# Errors
"EngineError",
"ModelNotFoundError",
"ModelLoadError",
"NetworkError",
"InvalidInputError",
"ConfigurationError",
"CancelledError",
"InternalError",
# Manifest
"PluginManifest",
"EngineManifest",
"VoiceSourceManifest",
"VoiceManifest",
"ParameterManifest",
"AudioFormatManifest",
"EnumOption",
"RequirementManifest",
"GpuRequirement",
"ModelManifest",
# Engine
"Engine",
"EngineSession",
# Capabilities
"VoiceLister",
"PreviewGenerator",
"StreamingSynthesizer",
"CancelableSession",
# Host Context
"HostContext",
"HttpClient",
# Plugin Manager
"get_plugin_manager",
"reset_plugin_manager",
# Utils
"get_voices",
"get_default_voice",
"is_plugin_registered",
"resolve_voice_to_plugin",
"create_pipeline",
]
-103
View File
@@ -1,103 +0,0 @@
"""Capability interfaces for the TTS Plugin Architecture.
This module defines optional capability interfaces that engines can implement.
Capabilities are additive; implementing new capabilities doesn't break old plugins.
"""
from __future__ import annotations
from typing import Iterator, Protocol, runtime_checkable
from abogen.tts_plugin.manifest import VoiceManifest
from abogen.tts_plugin.types import SynthesisRequest, SynthesizedAudio, VoiceSelection
@runtime_checkable
class VoiceLister(Protocol):
"""Protocol for listing available voices.
Engines that support voice listing should implement this interface.
"""
def listVoices(self, sourceId: str) -> list[VoiceManifest]:
"""List available voices for a given source.
Args:
sourceId: The voice source identifier.
Returns:
List of VoiceManifest describing available voices.
Raises:
EngineError: On failure.
"""
...
@runtime_checkable
class PreviewGenerator(Protocol):
"""Protocol for generating voice previews.
Engines that support voice preview should implement this interface.
"""
def generatePreview(self, voice: VoiceSelection, text: str) -> SynthesizedAudio:
"""Generate a preview audio for a voice.
Args:
voice: Voice selection for the preview.
text: Text to use for the preview.
Returns:
SynthesizedAudio with the preview audio data.
Raises:
EngineError: On failure.
"""
...
@runtime_checkable
class StreamingSynthesizer(Protocol):
"""Protocol for streaming synthesis.
Optional capability of EngineSession, not Engine.
Engines that support streaming synthesis should implement this interface.
"""
def synthesizeStream(self, request: SynthesisRequest) -> Iterator[bytes]:
"""Synthesize audio in streaming mode.
Args:
request: The synthesis request.
Yields:
Audio chunks as they become available.
Raises:
CancelledError: If cancel() is called during iteration.
EngineError: On synthesis failure.
"""
...
# This is a generator function; implementation will use yield
yield b"" # pragma: no cover
@runtime_checkable
class CancelableSession(Protocol):
"""Protocol for cancellation support.
Optional capability for engines that support cancellation.
cancel() causes synthesize() to raise CancelledError.
"""
def cancel(self) -> None:
"""Cancel in-progress synthesis.
After cancellation, synthesize() raises CancelledError.
The session remains usable after cancellation.
Raises:
EngineError: If called after dispose().
"""
...
-95
View File
@@ -1,95 +0,0 @@
"""Engine interfaces for the TTS Plugin Architecture.
This module defines the core Engine and EngineSession protocols.
These are the primary interfaces that plugin implementations must satisfy.
"""
from __future__ import annotations
from typing import Protocol, runtime_checkable
from abogen.tts_plugin.types import SynthesisRequest, SynthesizedAudio
@runtime_checkable
class EngineSession(Protocol):
"""Protocol for a session that owns mutable execution state.
An EngineSession is created by Engine.createSession() and owns
mutable execution state isolated from other concurrent work.
It is NOT thread-safe.
Lifecycle:
1. Created by Engine.createSession()
2. Used for synthesis via synthesize()
3. Disposed via dispose()
After dispose(), all methods except dispose() raise EngineError.
"""
def synthesize(self, request: SynthesisRequest) -> SynthesizedAudio:
"""Synthesize audio from text.
Args:
request: The synthesis request containing text, voice, parameters, and format.
Returns:
SynthesizedAudio with the synthesized audio data.
Raises:
EngineError: On synthesis failure. Session remains usable after error.
EngineError: If called after dispose().
"""
...
def dispose(self) -> None:
"""Release session resources.
This method is idempotent and safe to call multiple times.
It never raises exceptions (catches and logs internally).
After dispose(), all methods except dispose() raise EngineError.
"""
...
@runtime_checkable
class Engine(Protocol):
"""Protocol for a TTS engine that creates sessions.
An Engine is a factory for EngineSession instances. It is stateless
and thread-safe for createSession().
Lifecycle:
1. Created via create_engine() (plugin contract)
2. Sessions created via createSession()
3. Disposed via dispose()
Thread Safety:
- createSession() is thread-safe and can be called from any thread.
- dispose() must be called after all sessions are disposed.
- Disposing engine while sessions are alive violates API contract.
"""
def createSession(self) -> EngineSession:
"""Create a new session for synthesis.
Returns:
A new EngineSession instance. Ownership transfers to caller.
Raises:
EngineError: On failure. No partially initialized session is returned.
"""
...
def dispose(self) -> None:
"""Release engine resources.
Caller must ensure all sessions created by this engine are disposed
before calling dispose(). Disposing an engine while any session is
still alive violates the API contract; behavior is undefined.
This method is idempotent and safe to call multiple times.
It never raises exceptions (catches and logs internally).
After dispose(), all methods except dispose() raise EngineError.
"""
...
-62
View File
@@ -1,62 +0,0 @@
"""Error hierarchy for the TTS Plugin Architecture.
This module defines typed exceptions that engines raise.
Engines should never raise raw exceptions; they must use EngineError or its subtypes.
"""
from __future__ import annotations
class EngineError(Exception):
"""Base exception for all engine errors.
All engine operations that can fail should raise EngineError or one of its subtypes.
After dispose(), all methods except dispose() raise EngineError.
"""
pass
class ModelNotFoundError(EngineError):
"""Raised when a required model is not found."""
pass
class ModelLoadError(EngineError):
"""Raised when a model fails to load."""
pass
class NetworkError(EngineError):
"""Raised when a network operation fails."""
pass
class InvalidInputError(EngineError):
"""Raised when invalid input is provided to the engine."""
pass
class ConfigurationError(EngineError):
"""Raised when there is a configuration error."""
pass
class CancelledError(EngineError):
"""Raised when an operation is cancelled.
This is raised by synthesize() when cancel() is called during synthesis.
"""
pass
class InternalError(EngineError):
"""Raised when an internal engine error occurs."""
pass
-46
View File
@@ -1,46 +0,0 @@
"""Host context for the TTS Plugin Architecture.
This module defines the HostContext dataclass that provides minimal
host services to plugins. It is the only interface through which
plugins can access host functionality.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from pathlib import Path
from typing import Protocol, runtime_checkable
@runtime_checkable
class HttpClient(Protocol):
"""Protocol for HTTP client provided by host.
Plugins can use this for network requests (e.g., API-based engines).
"""
def get(self, url: str, **kwargs: object) -> object:
"""Perform an HTTP GET request."""
...
def post(self, url: str, **kwargs: object) -> object:
"""Perform an HTTP POST request."""
...
@dataclass(frozen=True)
class HostContext:
"""Minimal host context provided to plugins.
Contains only essential host services. No business logic.
Attributes:
config_dir: Directory for API keys, preferences, and configuration.
logger: Logger for plugin logging.
http_client: HTTP client for network requests.
"""
config_dir: Path
logger: logging.Logger
http_client: HttpClient
-365
View File
@@ -1,365 +0,0 @@
"""Plugin loader infrastructure for the TTS Plugin Architecture.
This module provides functionality to discover, import, validate, and load
TTS plugins. It handles both valid and invalid plugins, providing diagnostic
messages for errors.
The loader does NOT:
- Create Engine instances (that's the plugin's create_engine() responsibility)
- Manage plugin lifecycle (that's the Plugin Manager's responsibility)
- Implement any TTS engine functionality
"""
from __future__ import annotations
import importlib
import re
import sys
import types
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Callable
from abogen.tts_plugin.manifest import ModelManifest, PluginManifest
# Host API version for compatibility checking
HOST_API_VERSION = "1.0"
@dataclass(frozen=True)
class PluginLoadError:
"""Diagnostic information for a failed plugin load.
Attributes:
plugin_id: Plugin identifier if available, otherwise directory name.
path: Path to the plugin directory.
errors: List of error messages describing what went wrong.
"""
plugin_id: str
path: Path
errors: tuple[str, ...] = field(default_factory=tuple)
@dataclass(frozen=True)
class PluginLoadResult:
"""Result of loading a plugin.
Attributes:
success: Whether the plugin loaded successfully.
manifest: The plugin manifest if successful.
model_requirements: Model requirements if successful.
create_engine: The create_engine function if successful.
module: The plugin module if successful.
error: Error information if failed.
"""
success: bool
manifest: PluginManifest | None = None
model_requirements: tuple[ModelManifest, ...] | None = None
create_engine: Callable[..., Any] | None = None
module: types.ModuleType | None = None
error: PluginLoadError | None = None
def _parse_api_version(version: str) -> tuple[int, int] | None:
"""Parse an api_version string into (major, minor) tuple.
Args:
version: Version string in format "MAJOR.MINOR".
Returns:
Tuple of (major, minor) or None if invalid format.
"""
match = re.match(r"^(\d+)\.(\d+)$", version)
if match:
return int(match.group(1)), int(match.group(2))
return None
def _check_api_version_compatibility(plugin_version: str) -> str | None:
"""Check if plugin api_version is compatible with host.
Architecture spec:
- Format: semver (MAJOR.MINOR)
- Compatibility: Host rejects plugin if major version differs
- Minor version: backward compatible, Host accepts higher minor
Args:
plugin_version: Plugin's api_version string.
Returns:
Error message if incompatible, None if compatible.
"""
plugin_ver = _parse_api_version(plugin_version)
if plugin_ver is None:
return f"Invalid api_version format: '{plugin_version}'. Expected format: MAJOR.MINOR"
host_ver = _parse_api_version(HOST_API_VERSION)
if host_ver is None:
return f"Invalid host api_version format: '{HOST_API_VERSION}'"
if plugin_ver[0] != host_ver[0]:
return (
f"api_version major mismatch: plugin={plugin_ver[0]}, host={host_ver[0]}. "
f"Major version must match for compatibility."
)
return None
def _validate_manifest(module: types.ModuleType, plugin_dir: Path) -> list[str]:
"""Validate that a plugin module has required exports.
Args:
module: The imported plugin module.
plugin_dir: Path to the plugin directory.
Returns:
List of error messages (empty if valid).
"""
errors: list[str] = []
# Check PLUGIN_MANIFEST
manifest = getattr(module, "PLUGIN_MANIFEST", None)
if manifest is None:
errors.append("Missing PLUGIN_MANIFEST export")
elif not isinstance(manifest, PluginManifest):
errors.append(
f"PLUGIN_MANIFEST must be a PluginManifest instance, "
f"got {type(manifest).__name__}"
)
# Check MODEL_REQUIREMENTS
model_reqs = getattr(module, "MODEL_REQUIREMENTS", None)
if model_reqs is None:
errors.append("Missing MODEL_REQUIREMENTS export")
elif not isinstance(model_reqs, list):
errors.append(
f"MODEL_REQUIREMENTS must be a list, got {type(model_reqs).__name__}"
)
else:
for i, req in enumerate(model_reqs):
if not isinstance(req, ModelManifest):
errors.append(
f"MODEL_REQUIREMENTS[{i}] must be a ModelManifest instance, "
f"got {type(req).__name__}"
)
# Check create_engine
create_engine = getattr(module, "create_engine", None)
if create_engine is None:
errors.append("Missing create_engine export")
elif not callable(create_engine):
errors.append(
f"create_engine must be callable, got {type(create_engine).__name__}"
)
return errors
def _validate_capabilities(manifest: PluginManifest) -> list[str]:
"""Validate plugin capabilities.
Args:
manifest: The plugin manifest to validate.
Returns:
List of error messages (empty if valid).
"""
errors: list[str] = []
# Known capabilities (can be extended)
known_capabilities = frozenset({
"voice_list",
"preview",
"voice_clone",
"voice_blend",
"streaming",
"cancel",
})
for cap in manifest.capabilities:
if cap not in known_capabilities:
errors.append(f"Unknown capability: '{cap}'")
return errors
def _validate_api_version(manifest: PluginManifest) -> list[str]:
"""Validate api_version compatibility.
Args:
manifest: The plugin manifest to validate.
Returns:
List of error messages (empty if valid).
"""
errors: list[str] = []
error = _check_api_version_compatibility(manifest.api_version)
if error:
errors.append(error)
return errors
def load_plugin_from_dir(plugin_dir: Path) -> PluginLoadResult:
"""Load and validate a plugin from a directory.
The plugin directory must contain an __init__.py that exports:
- PLUGIN_MANIFEST: PluginManifest
- MODEL_REQUIREMENTS: list[ModelManifest]
- create_engine: Callable
Args:
plugin_dir: Path to the plugin directory.
Returns:
PluginLoadResult with success status and either plugin data or error info.
"""
plugin_id = plugin_dir.name
errors: list[str] = []
# Check if directory exists
if not plugin_dir.exists():
return PluginLoadResult(
success=False,
error=PluginLoadError(
plugin_id=plugin_id,
path=plugin_dir,
errors=(f"Plugin directory does not exist: {plugin_dir}",),
),
)
# Check for __init__.py
init_file = plugin_dir / "__init__.py"
if not init_file.exists():
return PluginLoadResult(
success=False,
error=PluginLoadError(
plugin_id=plugin_id,
path=plugin_dir,
errors=("Missing __init__.py in plugin directory",),
),
)
# Import the module
module_name = f"abogen.tts_plugin._loaded.{plugin_id}"
try:
# Remove from cache if already imported (for testing)
if module_name in sys.modules:
del sys.modules[module_name]
spec = importlib.util.spec_from_file_location(
module_name, init_file, submodule_search_locations=[]
)
if spec is None or spec.loader is None:
return PluginLoadResult(
success=False,
error=PluginLoadError(
plugin_id=plugin_id,
path=plugin_dir,
errors=(f"Failed to create module spec for {init_file}",),
),
)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)
except Exception as e:
# Clean up module from sys.modules on import failure
if module_name in sys.modules:
del sys.modules[module_name]
return PluginLoadResult(
success=False,
error=PluginLoadError(
plugin_id=plugin_id,
path=plugin_dir,
errors=(f"Failed to import plugin module: {e}",),
),
)
# Validate manifest
manifest_errors = _validate_manifest(module, plugin_dir)
errors.extend(manifest_errors)
# If manifest is valid, perform additional validation
manifest = getattr(module, "PLUGIN_MANIFEST", None)
if isinstance(manifest, PluginManifest):
# Validate api_version
api_errors = _validate_api_version(manifest)
errors.extend(api_errors)
# Validate capabilities
cap_errors = _validate_capabilities(manifest)
errors.extend(cap_errors)
# Use manifest id if available
plugin_id = manifest.id
# Check if any errors occurred
if errors:
# Clean up module from sys.modules
if module_name in sys.modules:
del sys.modules[module_name]
return PluginLoadResult(
success=False,
error=PluginLoadError(
plugin_id=plugin_id,
path=plugin_dir,
errors=tuple(errors),
),
)
# Get MODEL_REQUIREMENTS
model_requirements = tuple(getattr(module, "MODEL_REQUIREMENTS", []))
create_engine = getattr(module, "create_engine", None)
return PluginLoadResult(
success=True,
manifest=manifest,
model_requirements=model_requirements,
create_engine=create_engine,
module=module,
)
def discover_plugins(plugin_dirs: list[Path]) -> list[PluginLoadResult]:
"""Discover and load plugins from multiple directories.
Args:
plugin_dirs: List of directories to scan for plugins.
Returns:
List of PluginLoadResult, one per plugin directory found.
"""
results: list[PluginLoadResult] = []
for plugin_dir in plugin_dirs:
if not plugin_dir.exists():
continue
# Scan for subdirectories (each is a potential plugin)
for item in sorted(plugin_dir.iterdir()):
if item.is_dir() and not item.name.startswith("."):
result = load_plugin_from_dir(item)
results.append(result)
return results
def load_plugin(
plugin_dir: Path,
) -> PluginLoadResult:
"""Load a single plugin from a directory.
This is the main entry point for loading a plugin.
Args:
plugin_dir: Path to the plugin directory.
Returns:
PluginLoadResult with success status and either plugin data or error info.
"""
return load_plugin_from_dir(plugin_dir)
-189
View File
@@ -1,189 +0,0 @@
"""Plugin manifest types for the TTS Plugin Architecture.
This module contains static metadata types that describe plugins.
These types have no dependencies and are immutable.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
@dataclass(frozen=True)
class AudioFormatManifest:
"""Manifest describing an audio format.
Attributes:
mime: MIME type of the audio.
extension: File extension.
"""
mime: str
extension: str
@dataclass(frozen=True)
class EnumOption:
"""Manifest describing an enum option for a parameter.
Attributes:
value: The enum value.
label: Human-readable label.
"""
value: str
label: str
@dataclass(frozen=True)
class ParameterManifest:
"""Manifest describing a synthesis parameter.
Attributes:
id: Parameter identifier.
name: Human-readable name.
description: Parameter description.
type: Parameter type ("float", "int", "string", "boolean", "enum").
default: Default value.
min: Minimum value (optional, for numeric types).
max: Maximum value (optional, for numeric types).
step: Step size (optional, for numeric types).
options: Available options (optional, for enum type).
unit: Unit of measurement (optional).
group: Parameter group (optional).
"""
id: str
name: str
description: str
type: str
default: Any
min: float | None = None
max: float | None = None
step: float | None = None
options: tuple[EnumOption, ...] = field(default_factory=tuple)
unit: str | None = None
group: str | None = None
@dataclass(frozen=True)
class VoiceManifest:
"""Manifest describing a voice.
Attributes:
id: Voice identifier.
name: Human-readable name.
tags: Voice tags (e.g., language, style).
"""
id: str
name: str
tags: tuple[str, ...] = field(default_factory=tuple)
@dataclass(frozen=True)
class VoiceSourceManifest:
"""Manifest describing a voice source.
Attributes:
id: Voice source identifier.
name: Human-readable name.
type: Source type ("list", "speaker_id", "clone", "blend", "generate", "none").
config: Source-specific configuration.
"""
id: str
name: str
type: str
config: Any = None
@dataclass(frozen=True)
class EngineManifest:
"""Manifest describing engine capabilities.
Attributes:
voiceSources: Available voice sources.
parameters: Available synthesis parameters.
audioFormats: Supported audio formats.
"""
voiceSources: tuple[VoiceSourceManifest, ...] = field(default_factory=tuple)
parameters: tuple[ParameterManifest, ...] = field(default_factory=tuple)
audioFormats: tuple[AudioFormatManifest, ...] = field(default_factory=tuple)
@dataclass(frozen=True)
class GpuRequirement:
"""Manifest describing GPU requirements.
Attributes:
required: Whether GPU is required.
type: GPU type (e.g., "cuda", "rocm").
memory: Required GPU memory in GB.
"""
required: bool = False
type: str | None = None
memory: float | None = None
@dataclass(frozen=True)
class RequirementManifest:
"""Manifest describing plugin requirements.
Attributes:
gpu: GPU requirements (optional).
memory: Required RAM in GB (optional).
internet: Whether internet is required (optional).
"""
gpu: GpuRequirement | None = None
memory: float | None = None
internet: bool | None = None
@dataclass(frozen=True)
class ModelManifest:
"""Manifest describing a model requirement.
Attributes:
id: Model identifier.
name: Human-readable name.
size: Model size as string (e.g., "100MB", "2GB").
"""
id: str
name: str
size: str
@dataclass(frozen=True)
class PluginManifest:
"""Main manifest for a TTS plugin.
Attributes:
id: Plugin identifier (unique).
name: Human-readable name.
version: Plugin version.
api_version: API version (semver format: MAJOR.MINOR).
description: Plugin description.
author: Plugin author.
capabilities: List of capability identifiers.
requires: Plugin requirements.
engine: Engine manifest.
voices: Optional static voice catalog. None = not declared (use VoiceLister),
empty tuple = explicitly no static voices, non-empty = static catalog.
"""
id: str
name: str
version: str
api_version: str
description: str
author: str
capabilities: tuple[str, ...] = field(default_factory=tuple)
requires: RequirementManifest = field(default_factory=RequirementManifest)
engine: EngineManifest = field(default_factory=EngineManifest)
voices: tuple[VoiceManifest, ...] | None = None
-55
View File
@@ -1,55 +0,0 @@
"""Plugin contract for the TTS Plugin Architecture.
This module defines the plugin contract that all TTS plugins must implement.
Each plugin must export:
- PLUGIN_MANIFEST: PluginManifest instance
- MODEL_REQUIREMENTS: list of ModelManifest instances
- create_engine(): Factory function that creates an Engine
The create_engine() function is the entry point for plugin activation.
It must be atomic: succeed fully or raise and clean up.
"""
from __future__ import annotations
from pathlib import Path
from typing import Protocol, runtime_checkable
from abogen.tts_plugin.engine import Engine
from abogen.tts_plugin.host_context import HostContext
from abogen.tts_plugin.types import EngineConfig
@runtime_checkable
class Plugin(Protocol):
"""Protocol defining the plugin contract.
Every TTS plugin must implement this protocol by exporting:
- PLUGIN_MANIFEST: PluginManifest
- MODEL_REQUIREMENTS: list[ModelManifest]
- create_engine: Callable[[HostContext, Path | None, EngineConfig], Engine]
"""
def create_engine(
self,
context: HostContext,
model_path: Path | None,
config: EngineConfig,
) -> Engine:
"""Create an engine instance.
This is the factory function that creates an Engine from a plugin.
It must be atomic: succeed fully or raise EngineError and clean up.
Args:
context: Host services (config dir, logger, http client).
model_path: Resolved model path, or None for cloud/no-model engines.
config: Engine initialization settings.
Returns:
A fully initialized Engine instance.
Raises:
EngineError: On failure. Cleans up partially created resources.
"""
...
-153
View File
@@ -1,153 +0,0 @@
"""Plugin Manager
Provides a simple interface for consumers to access TTS engines via the
new Plugin Architecture. Discovers, loads, and manages plugins from the
plugins directory.
Usage:
from abogen.tts_plugin.plugin_manager import get_plugin_manager
manager = get_plugin_manager()
engine = manager.create_engine("kokoro", lang_code="a", device="cpu")
session = engine.create_session()
try:
result = session.synthesize("Hello world")
finally:
session.dispose()
"""
from typing import Any, Dict, List, Optional, Type
from abogen.tts_plugin.engine import Engine, EngineSession
from abogen.tts_plugin.manifest import PluginManifest
from abogen.tts_plugin.types import AudioFormat
class PluginManager:
"""Manages TTS plugins and provides a simple interface for consumers."""
def __init__(self) -> None:
self._plugins: Dict[str, dict] = {}
self._engines: Dict[str, Engine] = {}
self._loaded = False
def discover(self, plugins_dir: str = "plugins") -> None:
"""Discover and load all plugins from the given directory."""
import os
from pathlib import Path
from abogen.tts_plugin.loader import load_plugin_from_dir
self._plugins.clear()
self._engines.clear()
plugins_path = Path(plugins_dir)
if not plugins_path.exists():
self._loaded = True
return
for entry in plugins_path.iterdir():
if entry.is_dir() and (entry / "__init__.py").exists():
try:
result = load_plugin_from_dir(entry)
if result.success and result.manifest is not None:
self._plugins[result.manifest.id] = {
"manifest": result.manifest,
"create_engine": result.create_engine,
"module": result.module,
}
except Exception as e:
# Log error but continue with other plugins
print(f"Warning: Failed to load plugin from {entry}: {e}")
self._loaded = True
def _ensure_loaded(self) -> None:
"""Ensure plugins have been discovered."""
if not self._loaded:
self.discover()
def list_plugins(self) -> List[PluginManifest]:
"""Return manifests for all loaded plugins."""
self._ensure_loaded()
return [info["manifest"] for info in self._plugins.values()]
def get_plugin(self, plugin_id: str) -> Optional[dict]:
"""Get plugin info by ID."""
self._ensure_loaded()
return self._plugins.get(plugin_id)
def has_plugin(self, plugin_id: str) -> bool:
"""Check if a plugin is loaded."""
self._ensure_loaded()
return plugin_id in self._plugins
def create_engine(self, plugin_id: str, **kwargs: Any) -> Engine:
"""Create an engine instance for the given plugin.
Args:
plugin_id: The plugin identifier (e.g., "kokoro")
**kwargs: Arguments passed to the engine constructor
Returns:
An Engine instance
Raises:
KeyError: If plugin_id is not found
Exception: If engine creation fails
"""
self._ensure_loaded()
if plugin_id not in self._plugins:
raise KeyError(f"Plugin not found: {plugin_id}")
plugin_info = self._plugins[plugin_id]
create_engine_func = plugin_info["create_engine"]
# Create engine using the plugin's factory
engine = create_engine_func(**kwargs)
return engine
def get_or_create_engine(self, plugin_id: str, **kwargs: Any) -> Engine:
"""Get an existing engine or create a new one.
Engines are cached by plugin_id. If you need multiple instances
with different parameters, use create_engine() directly.
"""
self._ensure_loaded()
cache_key = plugin_id
if cache_key in self._engines:
return self._engines[cache_key]
engine = self.create_engine(plugin_id, **kwargs)
self._engines[cache_key] = engine
return engine
def dispose_all(self) -> None:
"""Dispose all cached engines."""
for engine in self._engines.values():
try:
engine.dispose()
except Exception:
pass # dispose() should never raise
self._engines.clear()
# Global singleton
_manager: Optional[PluginManager] = None
def get_plugin_manager() -> PluginManager:
"""Get the global PluginManager instance."""
global _manager
if _manager is None:
_manager = PluginManager()
return _manager
def reset_plugin_manager() -> None:
"""Reset the global PluginManager (for testing)."""
global _manager
if _manager is not None:
_manager.dispose_all()
_manager = None
-111
View File
@@ -1,111 +0,0 @@
"""Core domain types for the TTS Plugin Architecture.
This module contains immutable value objects that form the core domain.
These types have zero dependencies and are used across the plugin system.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Mapping
@dataclass(frozen=True)
class AudioFormat:
"""Immutable value object representing an audio format.
Attributes:
mime: MIME type of the audio (e.g., "audio/wav", "audio/mpeg").
extension: File extension (e.g., "wav", "mp3").
"""
mime: str
extension: str
@dataclass(frozen=True)
class Duration:
"""Immutable value object representing a time duration.
Attributes:
seconds: Duration in seconds.
"""
seconds: float
@dataclass(frozen=True)
class VoiceSelection:
"""Immutable value object for voice selection. Opaque to engine.
Attributes:
source: Voice source identifier (e.g., "builtin", "clone").
key: Voice key within the source.
payload: Optional payload for clone/blend sources.
"""
source: str
key: str
payload: Any = None
@dataclass(frozen=True)
class ParameterValues:
"""Immutable value object for synthesis parameters. Behaves like Mapping[str, Any].
Attributes:
values: Mapping of parameter names to their values.
"""
values: Mapping[str, Any] = field(default_factory=dict)
@dataclass(frozen=True)
class SynthesisRequest:
"""Immutable value object for a synthesis request.
Attributes:
text: Text to synthesize.
voice: Voice selection.
parameters: Synthesis parameters.
format: Desired audio output format.
"""
text: str
voice: VoiceSelection
parameters: ParameterValues
format: AudioFormat
@dataclass(frozen=True)
class SynthesizedAudio:
"""Immutable value object for synthesized audio result.
Attributes:
data: Raw audio bytes.
format: Audio format of the result.
duration: Duration of the audio.
"""
data: bytes
format: AudioFormat
duration: Duration
@dataclass(frozen=True)
class EngineConfig:
"""Immutable configuration of an Engine instance.
Contains parameters that define how a particular Engine instance is
created and that remain constant throughout the lifetime of that Engine.
Plugin implementations may ignore fields that are not applicable to them.
Attributes:
device: Device to use (e.g., "cpu", "cuda:0").
lang_code: Language code for the engine (e.g., "a" for Kokoro English).
Plugins that do not require a language code ignore this field.
"""
device: str = "cpu"
lang_code: str = "a"
-235
View File
@@ -1,235 +0,0 @@
"""TTS Plugin Architecture — direct utility functions.
Provides helpers that replace the former compatibility adapter by
calling the Plugin Manager directly.
"""
from __future__ import annotations
from typing import Any, Iterator
import numpy as np
from abogen.tts_plugin.plugin_manager import get_plugin_manager
def get_voices(plugin_id: str) -> tuple[str, ...]:
"""Return the voice-id tuple for *plugin_id*.
Uses the official Plugin Architecture: PluginManager Engine VoiceLister.
First checks plugin manifest for static voice catalog.
"""
import logging
import tempfile
from pathlib import Path
from abogen.tts_plugin.host_context import HostContext
from abogen.tts_plugin.types import EngineConfig
manager = get_plugin_manager()
if not manager.has_plugin(plugin_id):
return ()
# Check manifest for static voice catalog
plugin_info = manager.get_plugin(plugin_id)
if plugin_info is not None:
manifest = plugin_info.get("manifest")
if manifest is not None and manifest.voices is not None:
return tuple(v.id for v in manifest.voices)
ctx = HostContext(
config_dir=Path(tempfile.gettempdir()),
logger=logging.getLogger(f"abogen.utils.{plugin_id}"),
http_client=type("_StubHttpClient", (), {
"get": staticmethod(lambda url, **kw: None),
"post": staticmethod(lambda url, **kw: None),
})(),
)
try:
engine = manager.create_engine(
plugin_id,
context=ctx,
model_path=None,
config=EngineConfig(device="cpu"),
)
except Exception:
return ()
try:
from abogen.tts_plugin.capabilities import VoiceLister
if isinstance(engine, VoiceLister):
manifests = engine.listVoices("builtin")
return tuple(v.id for v in manifests)
return ()
except Exception:
return ()
finally:
engine.dispose()
def get_default_voice(plugin_id: str, fallback: str = "") -> str:
"""Return the first voice of *plugin_id*, or *fallback*."""
voices = get_voices(plugin_id)
return voices[0] if voices else fallback
def is_plugin_registered(plugin_id: str) -> bool:
"""Check whether *plugin_id* is loaded by the Plugin Manager."""
return get_plugin_manager().has_plugin(plugin_id)
def resolve_voice_to_plugin(spec: str, fallback: str = "kokoro") -> str:
"""Determine which plugin owns the given voice specification.
Resolution rules:
1. Empty spec -> fallback
2. Kokoro formula (contains '*' or '+') -> "kokoro"
3. Exact voice-id match against loaded plugins -> plugin id
4. Unknown voice -> fallback
"""
raw = str(spec or "").strip()
if not raw:
return fallback
if "*" in raw or "+" in raw:
return "kokoro"
upper = raw.upper()
manager = get_plugin_manager()
for manifest in manager.list_plugins():
for voice_source in manifest.engine.voiceSources:
if voice_source.type == "list" and isinstance(voice_source.config, dict):
try:
engine = manager.create_engine(manifest.id)
try:
if hasattr(engine, "listVoices"):
voice_manifests = engine.listVoices(voice_source.id)
voice_ids = [v.id.upper() for v in voice_manifests]
if upper in voice_ids:
return manifest.id
finally:
engine.dispose()
except Exception:
continue
return fallback
class Pipeline:
"""Callable wrapper around Engine / EngineSession.
Presents the same interface that old callers expect::
pipeline = create_pipeline("kokoro", lang_code="a", device="cpu")
for segment in pipeline(text, voice="af_nova", speed=1.0):
audio = segment.audio
"""
def __init__(self, engine: Any, **engine_kwargs: Any) -> None:
self._engine = engine
self._engine_kwargs = engine_kwargs
self._session: Any = None
def _ensure_session(self) -> Any:
if self._session is None:
self._session = self._engine.createSession()
return self._session
def __call__(
self,
text: str,
voice: str = "default",
speed: float = 1.0,
split_pattern: str | None = None,
**kwargs: Any,
) -> Iterator[Any]:
from abogen.tts_plugin.types import (
AudioFormat,
ParameterValues,
SynthesisRequest,
VoiceSelection,
)
session = self._ensure_session()
params: dict[str, Any] = {"speed": speed}
if split_pattern is not None:
params["split_pattern"] = split_pattern
params.update(kwargs)
request = SynthesisRequest(
text=text,
voice=VoiceSelection(source="builtin", key=voice),
parameters=ParameterValues(values=params),
format=AudioFormat(mime="audio/wav", extension="wav"),
)
result = session.synthesize(request)
audio_array = np.frombuffer(result.data, dtype=np.float32)
from dataclasses import dataclass
@dataclass
class Segment:
graphemes: str
audio: np.ndarray
yield Segment(graphemes=text, audio=audio_array)
def dispose(self) -> None:
if self._session is not None:
try:
self._session.dispose()
except Exception:
pass
self._session = None
def __del__(self) -> None:
self.dispose()
def create_pipeline(
plugin_id: str,
*,
lang_code: str = "a",
device: str = "cpu",
) -> Pipeline:
"""Create a callable TTS pipeline via the Plugin Architecture.
Builds a proper HostContext and EngineConfig, then delegates to the
PluginManager to create the engine. Returns a :class:`Pipeline` whose
``__call__`` interface matches the callable protocol used by consumers.
Args:
plugin_id: Plugin identifier (e.g., "kokoro", "supertonic").
lang_code: Language code for the engine.
device: Device to use (e.g., "cpu", "cuda:0").
Returns:
A callable Pipeline instance.
"""
import logging
import tempfile
from pathlib import Path
from abogen.tts_plugin.host_context import HostContext
from abogen.tts_plugin.types import EngineConfig
manager = get_plugin_manager()
ctx = HostContext(
config_dir=Path(tempfile.gettempdir()),
logger=logging.getLogger(f"abogen.pipeline.{plugin_id}"),
http_client=type("_StubHttpClient", (), {
"get": staticmethod(lambda url, **kw: None),
"post": staticmethod(lambda url, **kw: None),
})(),
)
config = EngineConfig(device=device, lang_code=lang_code)
engine = manager.create_engine(plugin_id, context=ctx, model_path=None, config=config)
return Pipeline(engine)
@@ -1,25 +1,39 @@
"""SuperTonic Pipeline — self-contained TTS pipeline for the plugin.
This module provides the SuperTonicPipeline class and supporting utilities
used by the SuperTonic plugin. It is independent of the legacy
abogen.tts_backends module.
"""
from __future__ import annotations
import ast
from dataclasses import dataclass
import logging
import math
import os
import re
from typing import Any, Iterable, Iterator, Optional
import numpy as np
logger = logging.getLogger(__name__)
DEFAULT_SUPERTONIC_VOICES = ("M1", "M2", "M3", "M4", "M5", "F1", "F2", "F3", "F4", "F5")
SUPERTONIC_AVAILABLE_LANGS = [
"en", "ko", "ja", "ar", "bg", "cs", "da", "de", "el",
"es", "et", "fi", "fr", "hi", "hr", "hu", "id", "it",
"lt", "lv", "nl", "pl", "pt", "ro", "ru", "sk", "sl",
"sv", "tr", "uk", "vi", "na",
]
@dataclass
class SupertonicSegment:
graphemes: str
audio: np.ndarray
def _ensure_float32_mono(wav: Any) -> np.ndarray:
arr = np.asarray(wav, dtype="float32")
if arr.ndim == 2:
# (n, 1) or (1, n) or (n, channels)
if arr.shape[0] == 1 and arr.shape[1] > 1:
arr = arr.reshape(-1)
else:
@@ -56,6 +70,7 @@ def _split_text(
else:
parts = [stripped]
# Enforce max length by hard-splitting long parts.
result: list[str] = []
for part in parts:
if len(part) <= max_chunk_length:
@@ -64,6 +79,7 @@ def _split_text(
start = 0
while start < len(part):
end = min(len(part), start + max_chunk_length)
# Try to split at whitespace.
if end < len(part):
ws = part.rfind(" ", start, end)
if ws > start + 40:
@@ -81,7 +97,8 @@ _UNSUPPORTED_CHARS_RE = re.compile(
def _parse_unsupported_characters(error: BaseException) -> list[str]:
"""Best-effort extraction of unsupported characters from SuperTonic errors."""
"""Best-effort extraction of unsupported characters from Supertonic errors."""
message = " ".join(
str(part) for part in getattr(error, "args", ()) if part is not None
) or str(error)
@@ -127,11 +144,16 @@ def _configure_supertonic_gpu() -> None:
available = ort.get_available_providers()
# Use CUDA if available, skip TensorRT (requires extra libs not always present)
# TensorrtExecutionProvider may be listed as available but fail at runtime
# if TensorRT libraries (libnvinfer.so) are not installed
providers = []
if "CUDAExecutionProvider" in available:
providers.append("CUDAExecutionProvider")
providers.append("CPUExecutionProvider")
# Patch supertonic's config and loader before TTS import
# We must patch both because loader imports the value at module load time
import supertonic.config as supertonic_config
import supertonic.loader as supertonic_loader
@@ -141,16 +163,7 @@ def _configure_supertonic_gpu() -> None:
except Exception as exc:
logger.warning("Could not configure supertonic GPU providers: %s", exc)
class SupertonicSegment:
"""A single synthesized audio segment."""
__slots__ = ("graphemes", "audio")
def __init__(self, graphemes: str, audio: np.ndarray) -> None:
self.graphemes = graphemes
self.audio = audio
SUPERTONIC_MAX_CHUNK_LENGTH = 500
class SupertonicPipeline:
"""Minimal adapter that mimics Kokoro's pipeline iteration interface."""
@@ -161,12 +174,16 @@ class SupertonicPipeline:
sample_rate: int,
auto_download: bool = True,
total_steps: int = 5,
max_chunk_length: int = 300,
max_chunk_length: int = SUPERTONIC_MAX_CHUNK_LENGTH,
lang: str = "en",
intra_op_num_threads: Optional[int] = None,
) -> None:
self.sample_rate = int(sample_rate)
self.total_steps = int(total_steps)
self.max_chunk_length = int(max_chunk_length)
self.lang = str(lang or "en")
# Configure GPU providers before importing TTS
_configure_supertonic_gpu()
try:
@@ -176,7 +193,8 @@ class SupertonicPipeline:
"Supertonic is not installed. Install it with `pip install supertonic`."
) from exc
self._tts = TTS(auto_download=auto_download)
threads = intra_op_num_threads if intra_op_num_threads is not None else os.cpu_count()
self._tts = TTS(auto_download=auto_download, intra_op_num_threads=threads)
def __call__(
self,
@@ -186,12 +204,14 @@ class SupertonicPipeline:
speed: float,
split_pattern: Optional[str] = None,
total_steps: Optional[int] = None,
lang: Optional[str] = None,
) -> Iterator[SupertonicSegment]:
voice_name = (voice or "").strip() or "M1"
steps = int(total_steps) if total_steps is not None else self.total_steps
steps = max(2, min(15, steps))
speed_value = float(speed) if speed is not None else 1.0
speed_value = max(0.7, min(2.0, speed_value))
language = str(lang or self.lang or "en")
style = self._tts.get_voice_style(voice_name=voice_name)
chunks = _split_text(
@@ -202,11 +222,13 @@ class SupertonicPipeline:
removed: set[str] = set()
last_exc: Exception | None = None
# Supertonic can raise ValueError for unsupported characters; strip and retry.
for attempt in range(3):
try:
wav, duration = self._tts.synthesize(
text=chunk_to_speak,
voice_style=style,
lang=language,
total_steps=steps,
speed=speed_value,
max_chunk_length=self.max_chunk_length,
@@ -225,23 +247,25 @@ class SupertonicPipeline:
chunk_to_speak, unsupported
).strip()
# If we didn't change anything, don't loop forever.
if sanitized == chunk_to_speak.strip():
raise
chunk_to_speak = sanitized
if not chunk_to_speak:
logger.warning(
"SuperTonic: dropped a chunk after removing unsupported characters: %s",
"Supertonic: dropped a chunk after removing unsupported characters: %s",
sorted(removed),
)
break
if attempt == 0:
logger.warning(
"SuperTonic: removed unsupported characters %s and retried.",
"Supertonic: removed unsupported characters %s and retried.",
sorted(removed),
)
else:
# Exhausted retries.
assert last_exc is not None
raise last_exc
@@ -250,6 +274,7 @@ class SupertonicPipeline:
audio = _ensure_float32_mono(wav)
# If duration is present, infer the source sample rate and resample if needed.
src_rate = self.sample_rate
try:
dur = float(duration)
+11 -10
View File
@@ -529,20 +529,21 @@ def prevent_sleep_end():
_sleep_procs[system] = None
def load_numpy_kpipeline():
import numpy as np
from kokoro import KPipeline # type: ignore[import-not-found]
return np, KPipeline
class LoadPipelineThread(Thread):
def __init__(self, callback, lang_code="a", device="cpu"):
def __init__(self, callback):
super().__init__()
self.callback = callback
self.lang_code = lang_code
self.device = device
def run(self):
try:
from abogen.tts_plugin.utils import create_pipeline
backend = create_pipeline(
"kokoro", lang_code=self.lang_code, device=self.device
)
self.callback(backend, None)
np_module, kpipeline_class = load_numpy_kpipeline()
self.callback(np_module, kpipeline_class, None)
except Exception as e:
self.callback(None, str(e))
self.callback(None, None, str(e))
+3 -12
View File
@@ -17,7 +17,7 @@ if LocalEntryNotFoundError is None: # pragma: no cover - fallback for tests
pass
from abogen.tts_plugin.utils import get_voices
from abogen.constants import VOICES_INTERNAL
_CACHE_LOCK = threading.Lock()
_CACHED_VOICES: Set[str] = set()
@@ -26,9 +26,8 @@ _BOOTSTRAPPED = False
def _normalize_targets(voices: Optional[Iterable[str]]) -> Set[str]:
kokoro_voices = get_voices("kokoro")
if not voices:
return set(kokoro_voices)
return set(VOICES_INTERNAL)
normalized: Set[str] = set()
for voice in voices:
if not voice:
@@ -36,7 +35,7 @@ def _normalize_targets(voices: Optional[Iterable[str]]) -> Set[str]:
voice_id = str(voice).strip()
if not voice_id:
continue
if voice_id in kokoro_voices:
if voice_id in VOICES_INTERNAL:
normalized.add(voice_id)
return normalized
@@ -144,11 +143,3 @@ def _ensure_single_voice_asset(
hf_hub_download(resume_download=True, **common_kwargs)
return True
def clear_voice_cache() -> None:
"""Clear the inprocess voice cache (used during shutdown)."""
with _CACHE_LOCK:
_CACHED_VOICES.clear()
global _BOOTSTRAPPED
_BOOTSTRAPPED = False
+2 -3
View File
@@ -1,7 +1,7 @@
import re
from typing import List, Tuple
from abogen.tts_plugin.utils import get_voices
from abogen.constants import VOICES_INTERNAL
# Calls parsing and loads the voice to gpu or cpu
@@ -22,7 +22,6 @@ def parse_formula_terms(formula: str) -> List[Tuple[str, float]]:
raise ValueError("Empty voice formula")
terms: List[Tuple[str, float]] = []
kokoro_voices = get_voices("kokoro")
for segment in formula.split("+"):
part = segment.strip()
if not part:
@@ -31,7 +30,7 @@ def parse_formula_terms(formula: str) -> List[Tuple[str, float]]:
raise ValueError("Each component must be in the form voice*weight")
voice_name, raw_weight = part.split("*", 1)
voice_name = voice_name.strip()
if voice_name not in kokoro_voices:
if voice_name not in VOICES_INTERNAL:
raise ValueError(f"Unknown voice: {voice_name}")
try:
weight = float(raw_weight.strip())
-33
View File
@@ -1,33 +0,0 @@
from dataclasses import dataclass
@dataclass(frozen=True)
class VoiceMetadata:
"""
Immutable metadata describing a voice from a TTS backend.
This model describes a voice independently of any backend implementation.
Backends populate these objects; the application consumes them.
The ``backend_id`` field is set by the backend itself (via
``self.metadata.id``) the application never hardcodes it.
This ensures renaming a backend does not require touching voice definitions.
"""
id: str
"""Unique voice identifier within the backend (e.g. ``"af_alloy"``, ``"M1"``)."""
display_name: str
"""Human-readable display name (e.g. ``"Alloy"``, ``"Male 1"``)."""
language: str
"""Language code — backend-specific format is acceptable (e.g. ``"a"``, ``"en"``)."""
gender: str
"""Gender category: ``"female"``, ``"male"``, or ``"unknown"``."""
backend_id: str
"""Identifier of the backend that owns this voice (e.g. ``"kokoro"``).
Set automatically by the backend never hardcoded in voice definitions.
"""
+5 -6
View File
@@ -2,7 +2,8 @@ import json
import os
from typing import Any, Dict, Iterable, List, Tuple
from abogen.tts_plugin.utils import get_voices, is_plugin_registered
from abogen.constants import VOICES_INTERNAL
from abogen.tts_supertonic import DEFAULT_SUPERTONIC_VOICES
from abogen.utils import get_user_config_path
@@ -69,8 +70,7 @@ def serialize_profiles() -> Dict[str, Dict[str, Iterable[Tuple[str, float]]]]:
def _normalize_supertonic_voice(value: Any) -> str:
raw = str(value or "").strip().upper()
supertonic_voices = get_voices("supertonic")
return raw if raw in supertonic_voices else "M1"
return raw if raw in DEFAULT_SUPERTONIC_VOICES else "M1"
def _coerce_supertonic_steps(value: Any) -> int:
@@ -101,7 +101,7 @@ def normalize_profile_entry(entry: Any) -> Dict[str, Any]:
return {}
provider = str(entry.get("provider") or "kokoro").strip().lower()
if not is_plugin_registered(provider):
if provider not in {"kokoro", "supertonic"}:
provider = "kokoro"
language = str(entry.get("language") or "a").strip().lower() or "a"
@@ -135,7 +135,6 @@ def normalize_profile_entry(entry: Any) -> Dict[str, Any]:
def _normalize_voice_entries(entries: Iterable) -> List[Tuple[str, float]]:
normalized: List[Tuple[str, float]] = []
kokoro_voices = get_voices("kokoro")
for item in entries or []:
if isinstance(item, dict):
voice = item.get("id") or item.get("voice")
@@ -144,7 +143,7 @@ def _normalize_voice_entries(entries: Iterable) -> List[Tuple[str, float]]:
voice, weight = item[0], item[1]
else:
continue
if voice not in kokoro_voices:
if voice not in VOICES_INTERNAL:
continue
if weight is None:
continue
+12 -11
View File
@@ -2,6 +2,7 @@ FROM nvidia/cuda:12.6.3-cudnn-runtime-ubuntu22.04
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \
VIRTUAL_ENV=/opt/venv \
PATH=/opt/venv/bin:$PATH
@@ -26,22 +27,22 @@ RUN python3 -m venv "$VIRTUAL_ENV"
WORKDIR /app
COPY pyproject.toml README.md ./
RUN pip install uv \
&& if [ -n "$TORCH_VERSION" ]; then \
uv pip install --system torch=="$TORCH_VERSION" torchvision=="$TORCH_VERSION" torchaudio=="$TORCH_VERSION" --index-url "$TORCH_INDEX_URL"; \
else \
uv pip install --system torch torchvision torchaudio --index-url "$TORCH_INDEX_URL"; \
fi \
&& uv pip install --system . \
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl \
&& uv pip install --system "mutagen>=1.47.0"
COPY abogen ./abogen
RUN pip install --upgrade pip \
&& if [ -n "$TORCH_VERSION" ]; then \
pip install torch=="$TORCH_VERSION" torchvision=="$TORCH_VERSION" torchaudio=="$TORCH_VERSION" --index-url "$TORCH_INDEX_URL"; \
else \
pip install torch torchvision torchaudio --index-url "$TORCH_INDEX_URL"; \
fi \
&& pip install --no-cache-dir . \
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl \
&& pip install --no-cache-dir "mutagen>=1.47.0"
# Install onnxruntime-gpu for CUDA acceleration (supertonic uses ONNX Runtime)
# Set USE_GPU=false to skip this for CPU-only deployments
RUN if [ "$USE_GPU" = "true" ]; then \
uv pip install --system onnxruntime-gpu; \
pip install --no-cache-dir onnxruntime-gpu; \
fi
ENV ABOGEN_HOST=0.0.0.0 \
+4 -3
View File
@@ -1,5 +1,6 @@
from __future__ import annotations
import atexit
import logging
import os
from pathlib import Path
@@ -7,8 +8,6 @@ from typing import Any, Optional
from flask import Flask
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
from abogen.utils import get_user_cache_path, get_user_output_path, get_user_settings_dir
from .conversion_runner import run_conversion_job
@@ -114,6 +113,8 @@ def create_app(config: Optional[dict[str, Any]] = None) -> Flask:
app.register_blueprint(books_bp, url_prefix="/find-books")
app.register_blueprint(api_bp, url_prefix="/api")
atexit.register(service.shutdown)
global _access_log_filter_attached
if not _access_log_filter_attached:
logging.getLogger("werkzeug").addFilter(_SuppressSuccessfulAccessFilter())
@@ -131,4 +132,4 @@ def main() -> None:
if __name__ == "__main__": # pragma: no cover
main()
main()
+98 -93
View File
@@ -20,7 +20,7 @@ import numpy as np
import soundfile as sf
import static_ffmpeg
from abogen.tts_plugin.utils import get_voices, is_plugin_registered, resolve_voice_to_plugin
from abogen.constants import VOICES_INTERNAL
from abogen.epub3.exporter import build_epub3_package
from abogen.kokoro_text_normalization import ApostropheConfig, normalize_for_pipeline, HAS_NUM2WORDS
from abogen.normalization_settings import (
@@ -39,14 +39,14 @@ from abogen.utils import (
get_user_cache_path,
get_user_output_path,
load_config,
load_numpy_kpipeline,
)
from abogen.tts_plugin.utils import create_pipeline
from abogen.voice_cache import ensure_voice_assets
from abogen.voice_formulas import extract_voice_ids, get_new_voice
from abogen.voice_profiles import load_profiles, normalize_profile_entry
from abogen.pronunciation_store import increment_usage
from abogen.llm_client import LLMClientError
from abogen.tts_supertonic import DEFAULT_SUPERTONIC_VOICES, SupertonicPipeline
from .service import Job, JobStatus
@@ -56,26 +56,25 @@ SAMPLE_RATE = 24000
def _supertonic_voice_from_spec(spec: Any, fallback: str) -> str:
"""Normalize a voice specification for Supertonic.
This function only performs Supertonic-specific normalization (uppercase conversion
and fallback handling). Backend resolution is handled by the registry.
"""
raw = str(spec or "").strip()
fallback_raw = str(fallback or "").strip()
# Normalize to uppercase for Supertonic voice IDs
upper = raw.upper() if raw else ""
# Supertonic voices are discrete IDs (M1/F3/...). If we see a Kokoro mix
# formula (contains '*' or '+'), ignore it and fall back to a safe voice.
if not raw or "*" in raw or "+" in raw:
raw = fallback_raw
if not raw or "*" in raw or "+" in raw:
raw = "M1"
# If empty or contains formula characters, use fallback
if not upper or "*" in upper or "+" in upper:
upper = fallback_raw.upper() if fallback_raw else ""
upper = raw.upper()
if upper in DEFAULT_SUPERTONIC_VOICES:
return upper
# If still empty, use default Supertonic voice
if not upper or "*" in upper or "+" in upper:
upper = "M1"
fallback_upper = fallback_raw.upper() if fallback_raw else ""
if fallback_upper in DEFAULT_SUPERTONIC_VOICES:
return fallback_upper
return upper
return "M1"
def _split_speaker_reference(value: Any) -> tuple[Optional[str], str]:
@@ -119,7 +118,15 @@ def _formula_from_kokoro_entry(entry: Mapping[str, Any]) -> str:
def _infer_provider_from_spec(value: Any, fallback: str = "kokoro") -> str:
return resolve_voice_to_plugin(str(value or ""), fallback=fallback)
raw = str(value or "").strip()
if not raw:
return fallback
upper = raw.upper()
if upper in DEFAULT_SUPERTONIC_VOICES:
return "supertonic"
if "*" in raw or "+" in raw:
return "kokoro"
return fallback
class _JobCancelled(Exception):
@@ -568,7 +575,7 @@ def _spec_to_voice_ids(spec: Any) -> Set[str]:
return set(extract_voice_ids(text))
except ValueError:
return set()
if text in get_voices("kokoro"):
if text in VOICES_INTERNAL:
return {text}
return set()
@@ -632,7 +639,7 @@ def _collect_required_voice_ids(job: Job) -> Set[str]:
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(_spec_to_voice_ids(payload.get(key)))
voices.update(get_voices("kokoro"))
voices.update(VOICES_INTERNAL)
return voices
@@ -1566,7 +1573,7 @@ def run_conversion_job(job: Job) -> None:
def get_pipeline(provider: str) -> Any:
nonlocal kokoro_cache_ready
provider_norm = str(provider or "kokoro").strip().lower() or "kokoro"
if not is_plugin_registered(provider_norm):
if provider_norm not in {"kokoro", "supertonic"}:
provider_norm = "kokoro"
existing = pipelines.get(provider_norm)
@@ -1574,8 +1581,10 @@ def run_conversion_job(job: Job) -> None:
return existing
if provider_norm == "supertonic":
pipelines[provider_norm] = create_pipeline(
"supertonic",
pipelines[provider_norm] = SupertonicPipeline(
sample_rate=SAMPLE_RATE,
auto_download=True,
total_steps=int(getattr(job, "supertonic_total_steps", 8) or 8),
)
return pipelines[provider_norm]
@@ -1585,12 +1594,16 @@ def run_conversion_job(job: Job) -> None:
device = "cpu"
if not disable_gpu:
device = _select_device()
# Create KPipeline instance directly (uses new Plugin Architecture)
pipelines[provider_norm] = create_pipeline(
"kokoro",
lang_code=job.language,
device=device
)
_np, KPipeline = load_numpy_kpipeline()
# Try to initialize with the selected device; fall back to CPU if CUDA fails
try:
pipelines[provider_norm] = KPipeline(lang_code=job.language, repo_id="hexgrad/Kokoro-82M", device=device)
except RuntimeError as e:
if "CUDA" in str(e) and device != "cpu":
job.add_log(f"CUDA initialization failed, falling back to CPU: {e}", level="warning")
pipelines[provider_norm] = KPipeline(lang_code=job.language, repo_id="hexgrad/Kokoro-82M", device="cpu")
else:
raise
if not kokoro_cache_ready:
_initialize_voice_cache(job)
kokoro_cache_ready = True
@@ -1605,7 +1618,7 @@ def run_conversion_job(job: Job) -> None:
provider = str(entry.get("provider") or "kokoro").strip().lower() or "kokoro"
if provider == "supertonic":
voice = str(entry.get("voice") or getattr(job, "voice", "M1") or "M1").strip() or "M1"
steps = int(entry.get("total_steps") or getattr(job, "supertonic_total_steps", 5) or 5)
steps = int(entry.get("total_steps") or getattr(job, "supertonic_total_steps", 8) or 8)
speed = float(entry.get("speed") or getattr(job, "speed", 1.0) or 1.0)
return "supertonic", _supertonic_voice_from_spec(voice, getattr(job, "voice", "M1")), speed, steps
formula = _formula_from_kokoro_entry(entry)
@@ -1621,7 +1634,7 @@ def run_conversion_job(job: Job) -> None:
"""Resolve a raw voice spec into (provider, resolved_spec, choice, speed, steps).
For Kokoro formulas, `choice` will be a resolved voice tensor (via `voice_formulas`).
For SuperTonic, `choice` will be a valid SuperTonic voice id.
For Supertonic, `choice` will be a valid Supertonic voice id.
"""
provider, resolved, speed, steps = resolve_voice_target(raw_spec)
@@ -1631,8 +1644,8 @@ def run_conversion_job(job: Job) -> None:
return provider, resolved, cached, speed, steps
if provider == "kokoro":
kokoro_backend = get_pipeline("kokoro")
choice = _resolve_voice(kokoro_backend, resolved, job.use_gpu)
kokoro_pipeline = get_pipeline("kokoro")
choice = _resolve_voice(kokoro_pipeline, resolved, job.use_gpu)
else:
choice = resolved
@@ -1761,8 +1774,8 @@ def run_conversion_job(job: Job) -> None:
voice_cache: Dict[str, Any] = {}
base_provider, base_voice_resolved, _, _ = resolve_voice_target(base_voice_spec)
if base_provider == "kokoro" and base_voice_resolved and "*" not in base_voice_resolved:
kokoro_backend = get_pipeline("kokoro")
voice_cache[f"kokoro:{base_voice_resolved}"] = _resolve_voice(kokoro_backend, base_voice_resolved, job.use_gpu)
kokoro_pipeline = get_pipeline("kokoro")
voice_cache[f"kokoro:{base_voice_resolved}"] = _resolve_voice(kokoro_pipeline, base_voice_resolved, job.use_gpu)
processed_chars = 0
subtitle_index = 1
current_time = 0.0
@@ -1792,8 +1805,8 @@ def run_conversion_job(job: Job) -> None:
fallback_key = next(iter(voice_cache.keys()), "")
if fallback_key and fallback_key != "__custom_mix":
intro_voice_spec = fallback_key.split(":", 1)[-1]
if not intro_voice_spec:
intro_voice_spec = get_default_voice("kokoro")
if not intro_voice_spec and VOICES_INTERNAL:
intro_voice_spec = VOICES_INTERNAL[0]
if intro_voice_spec:
intro_provider, _, intro_voice_choice, intro_speed, intro_steps = resolve_voice_choice(
@@ -1844,62 +1857,56 @@ def run_conversion_job(job: Job) -> None:
voice=voice_name,
speed=float(speed_override if speed_override is not None else job.speed),
split_pattern=split_pattern,
total_steps=int(supertonic_steps_override if supertonic_steps_override is not None else getattr(job, "supertonic_total_steps", 5)),
total_steps=int(supertonic_steps_override if supertonic_steps_override is not None else getattr(job, "supertonic_total_steps", 8)),
)
else:
kokoro_backend = get_pipeline("kokoro")
segment_iter = kokoro_backend(
kokoro_pipeline = get_pipeline("kokoro")
segment_iter = kokoro_pipeline(
normalized,
voice=voice_choice,
speed=float(speed_override if speed_override is not None else job.speed),
split_pattern=split_pattern,
)
try:
for segment in segment_iter:
canceller()
graphemes_raw = getattr(segment, "graphemes", "") or ""
graphemes = graphemes_raw.strip()
for segment in segment_iter:
canceller()
graphemes_raw = getattr(segment, "graphemes", "") or ""
graphemes = graphemes_raw.strip()
audio = _to_float32(getattr(segment, "audio", None))
if audio.size == 0:
continue
audio = _to_float32(getattr(segment, "audio", None))
if audio.size == 0:
continue
local_segments += 1
if chapter_sink:
chapter_sink.write(audio)
if audio_sink:
audio_sink.write(audio)
local_segments += 1
if chapter_sink:
chapter_sink.write(audio)
if audio_sink:
audio_sink.write(audio)
duration = len(audio) / SAMPLE_RATE
processed_chars += len(graphemes)
job.processed_characters = processed_chars
if job.total_characters:
job.progress = min(processed_chars / job.total_characters, 0.999)
else:
job.progress = 0.0 if processed_chars == 0 else 0.999
duration = len(audio) / SAMPLE_RATE
processed_chars += len(graphemes)
job.processed_characters = processed_chars
if job.total_characters:
job.progress = min(processed_chars / job.total_characters, 0.999)
else:
job.progress = 0.0 if processed_chars == 0 else 0.999
preview_text = graphemes or (graphemes_raw[:80] if graphemes_raw else "[silence]")
prefix = f"{preview_prefix} · " if preview_prefix else ""
job.add_log(f"{prefix}{processed_chars:,}/{job.total_characters or ''}: {preview_text[:80]}")
preview_text = graphemes or (graphemes_raw[:80] if graphemes_raw else "[silence]")
prefix = f"{preview_prefix} · " if preview_prefix else ""
job.add_log(f"{prefix}{processed_chars:,}/{job.total_characters or ''}: {preview_text[:80]}")
if subtitle_writer and audio_sink and graphemes:
subtitle_writer.write_segment(
index=subtitle_index,
text=graphemes,
start=current_time,
end=current_time + duration,
)
subtitle_index += 1
if subtitle_writer and audio_sink and graphemes:
subtitle_writer.write_segment(
index=subtitle_index,
text=graphemes,
start=current_time,
end=current_time + duration,
)
subtitle_index += 1
if audio_sink:
current_time += duration
if audio_sink:
current_time += duration
except OverflowError as exc:
job.add_log(
f"Skipped chunk — number too large for TTS conversion: {exc}",
level="warning",
)
return local_segments
def append_silence(
@@ -1943,8 +1950,8 @@ def run_conversion_job(job: Job) -> None:
if chapter_provider == "kokoro":
voice_choice = voice_cache.get(chapter_cache_key)
if voice_choice is None:
kokoro_backend = get_pipeline("kokoro")
voice_choice = _resolve_voice(kokoro_backend, chapter_voice_resolved, job.use_gpu)
kokoro_pipeline = get_pipeline("kokoro")
voice_choice = _resolve_voice(kokoro_pipeline, chapter_voice_resolved, job.use_gpu)
voice_cache[chapter_cache_key] = voice_choice
else:
voice_choice = chapter_voice_resolved
@@ -2088,9 +2095,9 @@ def run_conversion_job(job: Job) -> None:
if chunk_provider == "kokoro":
chunk_voice_choice = voice_cache.get(chunk_cache_key)
if chunk_voice_choice is None:
kokoro_backend = get_pipeline("kokoro")
kokoro_pipeline = get_pipeline("kokoro")
chunk_voice_choice = _resolve_voice(
kokoro_backend,
kokoro_pipeline,
chunk_voice_resolved,
job.use_gpu,
)
@@ -2232,8 +2239,8 @@ def run_conversion_job(job: Job) -> None:
if fallback_key and fallback_key != "__custom_mix":
# `voice_cache` keys are internal and include provider prefixes.
outro_voice_spec = fallback_key.split(":", 1)[-1]
if not outro_voice_spec:
outro_voice_spec = get_default_voice("kokoro")
if not outro_voice_spec and VOICES_INTERNAL:
outro_voice_spec = VOICES_INTERNAL[0]
if outro_text and outro_voice_spec:
outro_start_time = current_time
@@ -2405,11 +2412,6 @@ def run_conversion_job(job: Job) -> None:
# Explicitly release the pipeline and force garbage collection to prevent
# memory accumulation in the worker process, which can lead to host lockups.
for p in pipelines.values():
try:
p.dispose()
except Exception:
pass
pipelines.clear()
pipeline = None
gc.collect()
@@ -2443,14 +2445,17 @@ def _load_pipeline(job: Job):
disable_gpu = not job.use_gpu or not cfg.get("use_gpu", True)
provider = str(getattr(job, "tts_provider", "kokoro") or "kokoro").strip().lower()
if provider == "supertonic":
return create_pipeline(
"supertonic",
return SupertonicPipeline(
sample_rate=SAMPLE_RATE,
auto_download=True,
total_steps=int(getattr(job, "supertonic_total_steps", 8) or 8),
)
device = "cpu"
if not disable_gpu:
device = _select_device()
return create_pipeline("kokoro", lang_code=job.language, device=device)
_np, KPipeline = load_numpy_kpipeline()
return KPipeline(lang_code=job.language, repo_id="hexgrad/Kokoro-82M", device=device)
def _select_device() -> str:
@@ -2605,7 +2610,7 @@ def _build_ffmpeg_command(path: Path, fmt: str, metadata: Optional[Dict[str, str
def _resolve_voice(pipeline, voice_spec: str, use_gpu: bool):
if "*" in voice_spec:
# Voice formulas are a Kokoro-only feature (they require a pipeline that can
# load individual Kokoro voices). When running with SuperTonic (or when the
# load individual Kokoro voices). When running with Supertonic (or when the
# pipeline is otherwise unavailable), treat the spec as a plain string and
# allow downstream provider-specific resolution to choose a safe fallback.
if pipeline is None or not hasattr(pipeline, "load_single_voice"):
+3 -6
View File
@@ -15,7 +15,7 @@ from abogen.normalization_settings import build_apostrophe_config
from abogen.text_extractor import extract_from_path
from abogen.voice_cache import ensure_voice_assets
from abogen.webui.conversion_runner import SAMPLE_RATE, SPLIT_PATTERN, _select_device, _to_float32, _resolve_voice, _spec_to_voice_ids
from abogen.tts_plugin.utils import create_pipeline
from abogen.utils import load_numpy_kpipeline
_MARKER_RE = re.compile(re.escape(MARKER_PREFIX) + r"(?P<code>[A-Z0-9_]+)" + re.escape(MARKER_SUFFIX))
@@ -45,7 +45,8 @@ def _load_pipeline(language: str, use_gpu: bool) -> Any:
device = "cpu"
if use_gpu:
device = _select_device()
return create_pipeline("kokoro", lang_code=language, device=device)
_np, KPipeline = load_numpy_kpipeline()
return KPipeline(lang_code=language, repo_id="hexgrad/Kokoro-82M", device=device)
def _extract_cases_from_text(text: str) -> List[Tuple[str, str]]:
@@ -247,8 +248,4 @@ def run_debug_tts_wavs(
"sample_rate": SAMPLE_RATE,
}
(run_dir / "manifest.json").write_text(json.dumps(manifest, indent=2), encoding="utf-8")
try:
pipeline.dispose()
except Exception:
pass
return manifest
+5 -6
View File
@@ -34,7 +34,6 @@ from abogen.normalization_settings import (
)
from abogen.llm_client import list_models, LLMClientError
from abogen.kokoro_text_normalization import normalize_for_pipeline
from abogen.tts_plugin.utils import is_plugin_registered
from abogen.integrations.audiobookshelf import AudiobookshelfClient, AudiobookshelfConfig
from abogen.integrations.calibre_opds import (
CalibreOPDSClient,
@@ -64,7 +63,7 @@ def api_save_voice_profile() -> ResponseReturnValue:
if profile is None:
# Speaker Studio payload format
provider = str(payload.get("provider") or "kokoro").strip().lower()
if not is_plugin_registered(provider):
if provider not in {"kokoro", "supertonic"}:
provider = "kokoro"
if provider == "supertonic":
profile = {
@@ -163,7 +162,7 @@ def api_voice_profiles_preview() -> ResponseReturnValue:
formula = str(payload.get("formula") or "").strip()
profile_name = str(payload.get("profile") or "").strip()
provider = str(payload.get("tts_provider") or payload.get("provider") or "").strip().lower() or None
supertonic_total_steps = int(payload.get("supertonic_total_steps") or payload.get("total_steps") or settings.get("supertonic_total_steps") or 5)
supertonic_total_steps = int(payload.get("supertonic_total_steps") or payload.get("total_steps") or settings.get("supertonic_total_steps") or 8)
voice_spec = ""
resolved_provider = provider or "kokoro"
@@ -225,13 +224,13 @@ def api_speaker_preview() -> ResponseReturnValue:
speed_value = payload.get("speed")
speed = coerce_float(speed_value, 1.0)
tts_provider = str(payload.get("tts_provider") or "").strip().lower()
supertonic_total_steps = int(payload.get("supertonic_total_steps") or 5)
supertonic_total_steps = int(payload.get("supertonic_total_steps") or 8)
settings = load_settings()
use_gpu = settings.get("use_gpu", False)
base_spec, speaker_name = split_profile_spec(voice)
resolved_provider = tts_provider if is_plugin_registered(tts_provider) else ""
resolved_provider = tts_provider if tts_provider in {"kokoro", "supertonic"} else ""
if speaker_name:
entry = normalize_profile_entry(load_profiles().get(speaker_name))
@@ -270,7 +269,7 @@ def api_speaker_preview() -> ResponseReturnValue:
use_gpu=use_gpu
,
tts_provider=resolved_provider,
supertonic_total_steps=supertonic_total_steps or int(settings.get("supertonic_total_steps") or 5),
supertonic_total_steps=supertonic_total_steps or int(settings.get("supertonic_total_steps") or 8),
pronunciation_overrides=pronunciation_overrides,
manual_overrides=manual_overrides,
speakers=speakers,
+5 -1
View File
@@ -43,8 +43,12 @@ def update_settings() -> ResponseReturnValue:
current["language"] = (form.get("language") or "en").strip()
current["default_speaker"] = (form.get("default_speaker") or "").strip()
current["default_voice"] = (form.get("default_voice") or "").strip()
provider = str(form.get("tts_provider") or "kokoro").strip().lower()
if provider in {"kokoro", "supertonic"}:
current["tts_provider"] = provider
try:
current["supertonic_total_steps"] = max(2, min(15, int(form.get("supertonic_total_steps", current.get("supertonic_total_steps", 5)))))
total_steps = int(form.get("supertonic_total_steps", current.get("supertonic_total_steps", 8)))
current["supertonic_total_steps"] = max(2, min(15, total_steps))
except (TypeError, ValueError):
pass
try:
+18 -8
View File
@@ -7,7 +7,6 @@ from flask.typing import ResponseReturnValue
from abogen.webui.service import PendingJob, JobStatus
from abogen.webui.routes.utils.service import get_service
from abogen.tts_plugin.utils import is_plugin_registered
from abogen.webui.routes.utils.settings import (
load_settings,
coerce_bool,
@@ -33,7 +32,7 @@ from abogen.webui.routes.utils.common import split_profile_spec
from abogen.utils import calculate_text_length
from abogen.voice_profiles import serialize_profiles, normalize_profile_entry
from abogen.chunking import ChunkLevel, build_chunks_for_chapters
from abogen.tts_plugin.utils import get_default_voice
from abogen.constants import VOICES_INTERNAL
from abogen.speaker_configs import get_config
from abogen.kokoro_text_normalization import normalize_roman_numeral_titles
from dataclasses import dataclass
@@ -578,9 +577,9 @@ def apply_book_step_form(
# NOTE: Do not auto-set a global TTS provider at the book level based on the
# narrator defaults. Provider is resolved per-speaker/per-chunk from the voice
# spec (e.g. "speaker:Name" for saved speakers, or a Kokoro mix formula).
# This enables mixed-provider conversions (e.g. narrator=SuperTonic, characters=Kokoro).
# This enables mixed-provider conversions (e.g. narrator=Supertonic, characters=Kokoro).
provider_value = str(form.get("tts_provider") or "").strip().lower()
if is_plugin_registered(provider_value):
if provider_value in {"kokoro", "supertonic"}:
pending.tts_provider = provider_value
# Determine the base speaker selection (saved speaker ref or raw voice).
@@ -617,8 +616,8 @@ def apply_book_step_form(
custom_formula = ""
base_voice_spec = resolved_default_voice or narrator_voice_raw
if not base_voice_spec:
base_voice_spec = get_default_voice("kokoro")
if not base_voice_spec and VOICES_INTERNAL:
base_voice_spec = VOICES_INTERNAL[0]
voice_choice, resolved_language, selected_profile = resolve_voice_choice(
pending.language,
@@ -797,8 +796,8 @@ def build_pending_job_from_extraction(
profile_selection = inferred_profile
base_voice = base_voice_input or resolved_default_voice or str(default_voice_setting).strip()
if not base_voice:
base_voice = get_default_voice("kokoro")
if not base_voice and VOICES_INTERNAL:
base_voice = VOICES_INTERNAL[0]
selected_speaker_config = (form.get("speaker_config") or "").strip()
speaker_config_payload = get_config(selected_speaker_config) if selected_speaker_config else None
@@ -914,6 +913,15 @@ def build_pending_job_from_extraction(
else:
normalization_overrides[key] = default_val
provider_value = str(form.get("tts_provider") or "").strip().lower()
if provider_value not in {"kokoro", "supertonic"}:
provider_value = settings.get("tts_provider", "kokoro")
try:
total_steps = int(form.get("supertonic_total_steps", settings.get("supertonic_total_steps", 8)))
supertonic_steps = max(2, min(15, total_steps))
except (TypeError, ValueError):
supertonic_steps = int(settings.get("supertonic_total_steps", 8))
pending = PendingJob(
id=uuid.uuid4().hex,
original_filename=original_name,
@@ -929,6 +937,8 @@ def build_pending_job_from_extraction(
replace_single_newlines=replace_single_newlines,
subtitle_format=subtitle_format,
total_characters=total_chars,
tts_provider=provider_value,
supertonic_total_steps=supertonic_steps,
save_chapters_separately=save_chapters_separately,
merge_chapters_at_end=merge_chapters_at_end,
separate_chapters_format=separate_chapters_format,
+7 -17
View File
@@ -14,17 +14,6 @@ _preview_pipelines: Dict[Tuple[str, str], Any] = {}
_preview_pipeline_lock = threading.Lock()
def clear_preview_pipelines() -> None:
"""Dispose all cached preview pipelines and clear the cache."""
with _preview_pipeline_lock:
for pipeline in _preview_pipelines.values():
try:
pipeline.dispose()
except Exception:
pass
_preview_pipelines.clear()
def _select_device() -> str:
import platform
@@ -89,9 +78,10 @@ def get_preview_pipeline(language: str, device: str) -> Any:
pipeline = _preview_pipelines.get(key)
if pipeline is not None:
return pipeline
from abogen.tts_plugin.utils import create_pipeline
from abogen.utils import load_numpy_kpipeline
pipeline = create_pipeline("kokoro", lang_code=language, device=device)
_, KPipeline = load_numpy_kpipeline()
pipeline = KPipeline(lang_code=language, repo_id="hexgrad/Kokoro-82M", device=device)
_preview_pipelines[key] = pipeline
return pipeline
@@ -102,7 +92,7 @@ def generate_preview_audio(
speed: float,
use_gpu: bool,
tts_provider: str = "kokoro",
supertonic_total_steps: int = 5,
supertonic_total_steps: int = 8,
max_seconds: float = 8.0,
pronunciation_overrides: Optional[Iterable[Mapping[str, Any]]] = None,
manual_overrides: Optional[Iterable[Mapping[str, Any]]] = None,
@@ -147,9 +137,9 @@ def generate_preview_audio(
normalized_text = source_text
if provider == "supertonic":
from abogen.tts_plugin.utils import create_pipeline
from abogen.tts_supertonic import SupertonicPipeline
pipeline = create_pipeline("supertonic")
pipeline = SupertonicPipeline(sample_rate=SAMPLE_RATE, auto_download=True, total_steps=supertonic_total_steps)
segments = pipeline(
normalized_text,
voice=voice_spec,
@@ -211,7 +201,7 @@ def synthesize_preview(
speed: float,
use_gpu: bool,
tts_provider: str = "kokoro",
supertonic_total_steps: int = 5,
supertonic_total_steps: int = 8,
max_seconds: float = 8.0,
pronunciation_overrides: Optional[Iterable[Mapping[str, Any]]] = None,
manual_overrides: Optional[Iterable[Mapping[str, Any]]] = None,
+1 -1
View File
@@ -25,7 +25,7 @@ def submit_job(pending: PendingJob) -> str:
tts_provider=getattr(pending, "tts_provider", "kokoro"),
voice=pending.voice,
speed=pending.speed,
supertonic_total_steps=getattr(pending, "supertonic_total_steps", 5),
supertonic_total_steps=getattr(pending, "supertonic_total_steps", 8),
use_gpu=pending.use_gpu,
subtitle_mode=pending.subtitle_mode,
output_format=pending.output_format,
+4 -3
View File
@@ -6,8 +6,8 @@ from abogen.constants import (
LANGUAGE_DESCRIPTIONS,
SUBTITLE_FORMATS,
SUPPORTED_SOUND_FORMATS,
VOICES_INTERNAL,
)
from abogen.tts_plugin.utils import get_default_voice
from abogen.normalization_settings import (
DEFAULT_LLM_PROMPT,
environment_llm_defaults,
@@ -174,8 +174,9 @@ def settings_defaults() -> Dict[str, Any]:
"subtitle_format": "srt",
"save_mode": "default_output" if has_output_override() else "save_next_to_input",
"default_speaker": "",
"default_voice": get_default_voice("kokoro"),
"supertonic_total_steps": 5,
"default_voice": VOICES_INTERNAL[0] if VOICES_INTERNAL else "",
"tts_provider": "kokoro",
"supertonic_total_steps": 8,
"supertonic_speed": 1.0,
"replace_single_newlines": False,
"use_gpu": True,
+7 -6
View File
@@ -17,10 +17,10 @@ from abogen.constants import (
SUPPORTED_SOUND_FORMATS,
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
SAMPLE_VOICE_TEXTS,
VOICES_INTERNAL,
)
from abogen.tts_plugin.utils import get_voices
from abogen.speaker_configs import list_configs
from abogen.tts_plugin.utils import create_pipeline
from abogen.utils import load_numpy_kpipeline
from abogen.webui.conversion_runner import _select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN
_preview_pipeline_lock = threading.RLock()
@@ -285,7 +285,7 @@ def filter_voice_catalog(
def build_voice_catalog() -> List[Dict[str, str]]:
catalog: List[Dict[str, str]] = []
gender_map = {"f": "Female", "m": "Male"}
for voice_id in get_voices("kokoro"):
for voice_id in VOICES_INTERNAL:
prefix, _, rest = voice_id.partition("_")
language_code = prefix[0] if prefix else "a"
gender_code = prefix[1] if len(prefix) > 1 else ""
@@ -590,7 +590,7 @@ def template_options() -> Dict[str, Any]:
voice_catalog = build_voice_catalog()
return {
"languages": LANGUAGE_DESCRIPTIONS,
"voices": get_voices("kokoro"),
"voices": VOICES_INTERNAL,
"subtitle_formats": SUBTITLE_FORMATS,
"supported_langs_for_subs": SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
"output_formats": SUPPORTED_SOUND_FORMATS,
@@ -666,7 +666,7 @@ def resolve_voice_choice(
# Provider-aware behavior:
# - Kokoro profiles typically represent mixes (formula strings).
# - SuperTonic profiles represent a discrete voice id + settings.
# - Supertonic profiles represent a discrete voice id + settings.
# In that case, we return a speaker reference so downstream can
# resolve provider per-speaker and allow mixed-provider casting.
if provider == "supertonic":
@@ -741,7 +741,8 @@ def get_preview_pipeline(language: str, device: str):
pipeline = _preview_pipelines.get(key)
if pipeline is not None:
return pipeline
pipeline = create_pipeline("kokoro", lang_code=language, device=device)
_, KPipeline = load_numpy_kpipeline()
pipeline = KPipeline(lang_code=language, repo_id="hexgrad/Kokoro-82M", device=device)
_preview_pipelines[key] = pipeline
return pipeline
+1 -1
View File
@@ -17,7 +17,7 @@ from abogen.speaker_configs import (
save_configs,
delete_config,
)
from abogen.constants import VOICES_INTERNAL
voices_bp = Blueprint("voices", __name__)
+7 -9
View File
@@ -111,7 +111,7 @@ class Job:
subtitle_format: str
created_at: float
tts_provider: str = "kokoro"
supertonic_total_steps: int = 5
supertonic_total_steps: int = 8
save_chapters_separately: bool = False
merge_chapters_at_end: bool = True
separate_chapters_format: str = "wav"
@@ -204,7 +204,7 @@ class Job:
"queue_position": self.queue_position,
"options": {
"tts_provider": getattr(self, "tts_provider", "kokoro"),
"supertonic_total_steps": getattr(self, "supertonic_total_steps", 5),
"supertonic_total_steps": getattr(self, "supertonic_total_steps", 8),
"save_chapters_separately": self.save_chapters_separately,
"merge_chapters_at_end": self.merge_chapters_at_end,
"separate_chapters_format": self.separate_chapters_format,
@@ -552,7 +552,7 @@ class PendingJob:
normalization_overrides: Dict[str, Any]
created_at: float
tts_provider: str = "kokoro"
supertonic_total_steps: int = 5
supertonic_total_steps: int = 8
cover_image_path: Optional[Path] = None
cover_image_mime: Optional[str] = None
chapter_intro_delay: float = 0.5
@@ -621,7 +621,7 @@ class ConversionService:
voice: str,
speed: float,
tts_provider: str = "kokoro",
supertonic_total_steps: int = 5,
supertonic_total_steps: int = 8,
use_gpu: bool,
subtitle_mode: str,
output_format: str,
@@ -674,7 +674,7 @@ class ConversionService:
voice=voice,
speed=speed,
tts_provider=tts_provider,
supertonic_total_steps=int(supertonic_total_steps or 5),
supertonic_total_steps=int(supertonic_total_steps or 8),
use_gpu=use_gpu,
subtitle_mode=subtitle_mode,
output_format=output_format,
@@ -1147,7 +1147,7 @@ class ConversionService:
"tts_provider": getattr(job, "tts_provider", "kokoro"),
"voice": job.voice,
"speed": job.speed,
"supertonic_total_steps": getattr(job, "supertonic_total_steps", 5),
"supertonic_total_steps": getattr(job, "supertonic_total_steps", 8),
"use_gpu": job.use_gpu,
"subtitle_mode": job.subtitle_mode,
"output_format": job.output_format,
@@ -1275,7 +1275,7 @@ class ConversionService:
replace_single_newlines=bool(payload.get("replace_single_newlines", False)),
subtitle_format=payload.get("subtitle_format", "srt"),
created_at=float(payload.get("created_at", time.time())),
supertonic_total_steps=int(payload.get("supertonic_total_steps", 5)),
supertonic_total_steps=int(payload.get("supertonic_total_steps", 8)),
save_chapters_separately=bool(payload.get("save_chapters_separately", False)),
merge_chapters_at_end=bool(payload.get("merge_chapters_at_end", True)),
separate_chapters_format=payload.get("separate_chapters_format", "wav"),
@@ -1609,12 +1609,10 @@ def build_service(
output_root: Optional[Path] = None,
uploads_root: Optional[Path] = None,
) -> ConversionService:
global _service_instance
output_root = output_root or default_storage_root()
service = ConversionService(
output_root=output_root,
uploads_root=uploads_root,
runner=runner,
)
_service_instance = service
return service
@@ -26,6 +26,26 @@
{% set subtitle_value = settings_dict.get('subtitle_mode', 'Disabled') %}
{% endif %}
{% endif %}
{% set tts_provider_value = form_values.get('tts_provider') if form_values else None %}
{% if not tts_provider_value %}
{% if pending and pending.tts_provider %}
{% set tts_provider_value = pending.tts_provider %}
{% else %}
{% set tts_provider_value = settings_dict.get('tts_provider', 'kokoro') %}
{% endif %}
{% endif %}
{% set supertonic_steps_value = form_values.get('supertonic_total_steps') if form_values else None %}
{% if supertonic_steps_value is none %}
{% if pending and pending.supertonic_total_steps is not none %}
{% set supertonic_steps_value = pending.supertonic_total_steps %}
{% else %}
{% set supertonic_steps_value = settings_dict.get('supertonic_total_steps', 8) %}
{% endif %}
{% endif %}
{% if supertonic_steps_value is not none and supertonic_steps_value is string %}
{% set supertonic_steps_value = supertonic_steps_value|int %}
{% endif %}
{% set is_supertonic = tts_provider_value == 'supertonic' %}
{% set generate_flag = form_values.get('generate_epub3') if form_values else None %}
{% if generate_flag is not none %}
{% set generate_epub3 = True %}
@@ -273,6 +293,13 @@
<div class="form-section__layout form-section__layout--split">
<div class="form-section__group">
<div class="field">
<label for="tts_provider">TTS Engine</label>
<select id="tts_provider" name="tts_provider" data-role="tts-provider" {{ 'disabled' if readonly else '' }}>
<option value="kokoro" {% if tts_provider_value == 'kokoro' %}selected{% endif %}>Kokoro</option>
<option value="supertonic" {% if tts_provider_value == 'supertonic' %}selected{% endif %}>Supertonic</option>
</select>
</div>
<div class="field" data-role="voice-profile-field">
<label for="voice_profile">Voice profile</label>
<select id="voice_profile" name="voice_profile" data-role="voice-profile" {{ 'disabled' if readonly else '' }}>
<option value="__standard" {% if profile_value == '__standard' %}selected{% endif %}>Standard voice</option>
@@ -280,13 +307,13 @@
{% if options.voice_profile_options %}
<optgroup label="Saved mixes">
{% for profile in options.voice_profile_options %}
<option value="{{ profile.name }}" data-language="{{ profile.language }}" data-formula="{{ profile.formula|e }}" {% if profile_value == profile.name %}selected{% endif %}>{{ profile.name }}{% if profile.language %} ({{ profile.language|upper }}){% endif %}</option>
<option value="{{ profile.name }}" data-language="{{ profile.language }}" data-formula="{{ profile.formula|e }}" data-provider="{{ profile.provider|default('kokoro')|lower }}" {% if profile_value == profile.name %}selected{% endif %}>{{ profile.name }}{% if profile.language %} ({{ profile.language|upper }}){% endif %}{% if profile.provider and profile.provider|lower != 'kokoro' %} · {{ profile.provider|capitalize }}{% endif %}</option>
{% endfor %}
</optgroup>
{% endif %}
</select>
</div>
<div class="field" data-role="voice-field" {% if profile_value != '__standard' %}hidden aria-hidden="true"{% endif %}>
<div class="field" data-role="voice-field" data-provider="kokoro" {% if profile_value != '__standard' or is_supertonic %}hidden aria-hidden="true"{% endif %}>
<label for="voice">Voice</label>
<select id="voice" name="voice" data-role="voice-select" data-default="{{ narrator_voice or settings_dict.get('default_voice', '') }}" {{ 'disabled' if readonly else '' }}>
{% for voice in options.voices %}
@@ -294,10 +321,23 @@
{% endfor %}
</select>
</div>
<div class="field" data-conditional="formula" data-role="formula-field" {% if profile_value != '__formula' %}hidden aria-hidden="true"{% endif %}>
<div class="field" data-role="voice-field" data-provider="supertonic" {% if profile_value != '__standard' or not is_supertonic %}hidden aria-hidden="true"{% endif %}>
<label for="voice_st">Supertonic voice</label>
<select id="voice_st" name="voice" data-role="voice-select" data-default="{{ narrator_voice or 'M1' }}" {{ 'disabled' if readonly else '' }}>
{% for voice in ['M1','M2','M3','M4','M5','F1','F2','F3','F4','F5'] %}
<option value="{{ voice }}" {% if narrator_voice == voice and profile_value == '__standard' %}selected{% endif %}>{{ voice }}</option>
{% endfor %}
</select>
</div>
<div class="field" data-conditional="formula" data-role="formula-field" data-provider="kokoro" {% if profile_value != '__formula' or is_supertonic %}hidden aria-hidden="true"{% endif %}>
<label for="voice_formula">Custom voice formula</label>
<input type="text" id="voice_formula" name="voice_formula" placeholder="af_nova*0.4+am_liam*0.6" data-role="voice-formula" value="{{ voice_formula_value }}" {{ 'disabled' if readonly else '' }}>
</div>
<div class="field" data-role="supertonic-steps-field" {% if not is_supertonic %}hidden aria-hidden="true"{% endif %}>
<label for="supertonic_total_steps">Supertonic quality (total steps)</label>
<input type="number" id="supertonic_total_steps" name="supertonic_total_steps" min="2" max="15" value="{{ supertonic_steps_value }}" {{ 'disabled' if readonly else '' }}>
<p class="hint">2 = fastest/lowest quality, 15 = slowest/highest quality.</p>
</div>
</div>
<div class="form-section__group">
<div class="field field--slider">
@@ -423,3 +463,54 @@
</div>
</footer>
</form>
<script nonce="{{ csp_nonce() if csp_nonce else '' }}">
(function() {
const form = document.querySelector('[data-wizard-form="true"][data-step="book"]');
if (!form) return;
const providerSelect = form.querySelector('[data-role="tts-provider"]');
if (!providerSelect) return;
function filterProfilesByProvider(provider) {
const profileSelect = form.querySelector('[data-role="voice-profile"]');
if (!profileSelect) return;
const options = profileSelect.querySelectorAll('option[data-provider]');
options.forEach(function(opt) {
const matches = !opt.dataset.provider || opt.dataset.provider === provider;
opt.hidden = !matches;
if (opt.selected && opt.hidden) {
opt.selected = false;
}
});
if (!profileSelect.value || profileSelect.selectedOptions[0]?.hidden) {
const firstVisible = profileSelect.querySelector('option:not([hidden])');
if (firstVisible) profileSelect.value = firstVisible.value;
}
profileSelect.dispatchEvent(new Event('change', { bubbles: true }));
}
function syncProviderUI(provider) {
var isSupertonic = provider === 'supertonic';
form.querySelectorAll('[data-role="voice-field"]').forEach(function(el) {
el.hidden = el.dataset.provider !== provider;
el.setAttribute('aria-hidden', el.hidden ? 'true' : 'false');
});
var formulaField = form.querySelector('[data-role="formula-field"]');
if (formulaField) {
formulaField.hidden = isSupertonic;
formulaField.setAttribute('aria-hidden', isSupertonic ? 'true' : 'false');
}
var stepsField = form.querySelector('[data-role="supertonic-steps-field"]');
if (stepsField) {
stepsField.hidden = !isSupertonic;
stepsField.setAttribute('aria-hidden', isSupertonic ? 'false' : 'true');
}
filterProfilesByProvider(provider);
}
providerSelect.addEventListener('change', function() {
syncProviderUI(providerSelect.value);
});
syncProviderUI(providerSelect.value);
})();
</script>
+9
View File
@@ -61,6 +61,15 @@
<p class="hint">Pick a saved speaker from Speaker Studio to use by default for new jobs.</p>
</div>
<div class="field">
<label for="tts_provider">Default TTS Engine</label>
<select id="tts_provider" name="tts_provider">
<option value="kokoro" {% if settings.tts_provider == 'kokoro' %}selected{% endif %}>Kokoro</option>
<option value="supertonic" {% if settings.tts_provider == 'supertonic' %}selected{% endif %}>Supertonic</option>
</select>
<p class="hint">Select the default TTS engine for new jobs.</p>
</div>
<div class="field field--wide">
<p class="tag">Kokoro settings</p>
</div>
-55
View File
@@ -1,55 +0,0 @@
# Contributing to Abogen
We welcome contributions to Abogen!
## How to Contribute
1. Fork the repository
2. Create a branch for your feature
3. Make your changes
4. Write tests
5. Submit a pull request
## Code Standards
- Follow PEP 8 for Python
- Use TypeScript for JavaScript
- Type hints required for new Python code
- Document complex logic with comments
## Plugin Architecture
When contributing TTS engines, implement the **Plugin Architecture** contract.
See [Developer Guide](developer-guide.md#5-adding-a-new-plugin) for:
- Required exports (`PLUGIN_MANIFEST`, `MODEL_REQUIREMENTS`, `create_engine`)
- Engine / EngineSession contracts
- Capability interfaces (`VoiceLister`, `PreviewGenerator`, etc.)
- Step-by-step plugin creation guide
## Testing
```bash
# All tests
pytest
# Contract tests (architectural compliance)
pytest tests/contracts/
# Behavioral regression tests
pytest tests/test_behavioral_regression.py
```
## Documentation
- Update relevant docs in `docs/` when changing architecture or APIs
- Add docstrings to all public functions/classes
- Follow existing documentation style
## Pull Request Checklist
- [ ] Tests pass (`pytest`)
- [ ] Code follows style guide (`ruff check`, `ruff format`)
- [ ] Documentation updated
- [ ] No legacy architecture references (`TTSBackend`, `register_backend`, `TTSBackendRegistry`)
- [ ] Uses new Plugin Architecture patterns
-270
View File
@@ -1,270 +0,0 @@
# TTS Plugin Architecture — Architectural Reference
This document describes the **stable architectural contracts** of the TTS Plugin Architecture. It documents invariants that only change when the architecture itself changes.
---
## 1. Architecture Overview
```
┌─────────────────────────────────────────────────────────────────┐
│ Host Application │
│ ┌─────────────┐ ┌──────────────┐ ┌────────────────────────┐ │
│ │ Plugin │ │ HostContext │ │ Plugin Discovery │ │
│ │ Manager │──│ (config_dir, │ │ (plugin directories) │ │
│ │ │ │ logger, │ │ │ │
│ │ - discover │ │ http_client)│ └────────────────────────┘ │
│ │ - validate │ └──────────────┘ │ │
│ │ - activate │ ▼ │
│ │ - dispose │ ┌─────────────────────────────────────────┐ │
│ └──────┬──────┘ │ Plugin Package │ │
│ │ │ ┌──────────────┐ ┌─────────────────┐ │ │
│ ▼ │ │ PLUGIN_ │ │ MODEL_ │ │ │
│ ┌────────────┐ │ │ MANIFEST │ │ REQUIREMENTS │ │ │
│ │ Engine │◄──┤ │ create_engine│ │ │ │ │
│ └──────┬─────┘ │ └──────────────┘ └─────────────────┘ │ │
│ │ └─────────────────────────────────────────┘ │
│ │ createSession() │
│ ▼ │
│ ┌─────────────┐ │
│ │EngineSession│ │
│ └──────┬──────┘ │
│ │ synthesize() │
│ ▼ │
│ ┌────────────────┐ │
│ │SynthesizedAudio│ │
│ └────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
### Core Components
| Component | Responsibility |
|-----------|----------------|
| **PluginManifest** | Static metadata: id, name, version, api_version, capabilities, engine manifest |
| **EngineManifest** | Voice sources, parameters, audio formats |
| **HostContext** | Minimal host services: config_dir, logger, http_client |
| **Engine** | Stateless factory for sessions; thread-safe `createSession()` |
| **EngineSession** | Owns mutable execution state; not thread-safe |
| **PluginManager** | Discovers, validates, and manages plugin lifecycle |
| **Capabilities** | Optional interfaces: VoiceLister, PreviewGenerator, StreamingSynthesizer, CancelableSession |
---
## 2. Ownership Model
### Engine Ownership
```
PluginManager.create_engine() → Engine
```
- **PluginManager** creates and caches engines
- **Caller** receives `Engine` instance
- **Caller** must dispose all sessions **before** disposing engine
- **Engine.dispose()** releases engine resources
- After `Engine.dispose()`: all methods except `dispose()` raise `EngineError`
### Session Ownership
```
Engine.createSession() → EngineSession
```
- **Engine** creates session
- **Ownership transfers to caller** immediately
- **Caller** is responsible for `session.dispose()`
- **Engine does NOT track sessions** — no registry, no callbacks
- After `session.dispose()`: all methods except `dispose()` raise `EngineError`
### Disposal Order (Invariant)
```python
# Correct
engine = manager.create_engine("id")
session = engine.createSession()
try:
audio = session.synthesize(request)
finally:
session.dispose() # 1. Sessions FIRST
engine.dispose() # 2. Then engine
# INCORRECT — violates contract (undefined behavior)
engine.dispose()
session.synthesize(request) # EngineError
```
---
## 3. Lifecycle State Machine
```
DISCOVERY
PluginManager.discover(plugin_dirs)
→ Loads PLUGIN_MANIFEST, MODEL_REQUIREMENTS
→ Validates api_version (major must match)
→ Validates declared capabilities are implemented
MODEL_DOWNLOAD (if MODEL_REQUIREMENTS non-empty)
Host reads MODEL_REQUIREMENTS
Downloads/caches models
Resolves model_path
ACTIVATION
create_engine(context, model_path, config)
→ Atomic: succeeds fully or raises EngineError
→ Returns Engine
SESSION_CREATION
engine.createSession() → EngineSession
→ Ownership transfers to caller
→ Raises EngineError on failure
→ Never returns partial session
SYNTHESIS
session.synthesize(request)
→ Returns SynthesizedAudio
→ Raises EngineError on failure
→ Session remains usable after error
SESSION_DISPOSAL
session.dispose()
→ Idempotent, never raises
→ After: all methods raise EngineError
DEACTIVATION
engine.dispose()
→ Caller MUST dispose all sessions first
→ Idempotent, never raises
→ After: all methods raise EngineError
```
---
## 4. Protocol Contracts
### Engine (Protocol)
```python
@runtime_checkable
class Engine(Protocol):
def createSession(self) -> EngineSession:
"""Create a new session. Thread-safe. Transfers ownership."""
...
def dispose(self) -> None:
"""Release engine resources.
Caller must dispose all sessions first.
Idempotent, never raises.
After: all methods except dispose() raise EngineError."""
...
```
### EngineSession (Protocol)
```python
@runtime_checkable
class EngineSession(Protocol):
def synthesize(self, request: SynthesisRequest) -> SynthesizedAudio:
"""Synthesize audio.
Returns SynthesizedAudio or raises EngineError.
Session remains usable after error."""
...
def dispose(self) -> None:
"""Release session resources.
Idempotent, never raises.
After: all methods except dispose() raise EngineError."""
...
```
### Capability Protocols (Optional)
- **VoiceLister**: `listVoices(source_id: str) -> list[VoiceManifest]`
- **PreviewGenerator**: `generatePreview(voice: VoiceSelection, text: str) -> SynthesizedAudio`
- **StreamingSynthesizer**: `synthesizeStream(request: SynthesisRequest) -> Iterator[bytes]`
- **CancelableSession**: `cancel() -> None` (causes in-flight synthesize to raise `CancelledError`)
---
## 5. Error Semantics
```
EngineError (base)
├── ModelNotFoundError # Required model not found
├── ModelLoadError # Model failed to load
├── NetworkError # Network operation failed
├── InvalidInputError # Request validation failed
├── ConfigurationError # Invalid configuration
├── CancelledError # Operation cancelled via CancelableSession
└── InternalError # Unexpected internal failure
```
### When Each Is Raised
| Error | Raised By | Conditions |
|-------|-----------|------------|
| `ModelNotFoundError` | `create_engine()` | Required model not found at `model_path` |
| `ModelLoadError` | `create_engine()` | Model exists but fails to load |
| `NetworkError` | `synthesize()`, `create_engine()` | Network call fails (cloud engines) |
| `InvalidInputError` | `synthesize()` | Request validation fails (empty text, invalid voice, etc.) |
| `ConfigurationError` | `create_engine()` | Config values invalid for this engine |
| `CancelledError` | `synthesize()`, `synthesizeStream()` | `CancelableSession.cancel()` called |
| `InternalError` | Any | Unexpected internal failure (bug) |
### Dispose Contract
- `dispose()` is **idempotent** and **never raises**
- After `dispose()`: all methods except `dispose()` raise `EngineError`
- Engine: caller must dispose all sessions first; violating this is undefined behavior
---
## 6. Capabilities
| Capability | Interface | Enables |
|------------|-----------|---------|
| `voice_list` | `VoiceLister` | `listVoices(source_id)` — enumerate available voices |
| `preview` | `PreviewGenerator` | `generatePreview(voice, text)` — preview without session |
| `streaming` | `StreamingSynthesizer` | `synthesizeStream(request)` — chunked audio output |
| `cancel` | `CancelableSession` | `cancel()` — interrupt in-flight synthesis |
Plugins declare capabilities in `PluginManifest.capabilities`. Host validates at load time.
---
## 7. Contract Tests
**Location**: `tests/contracts/`
**Purpose**: Verify every plugin satisfies the architectural contracts.
**Guarantees**:
- Required exports exist (`PLUGIN_MANIFEST`, `MODEL_REQUIREMENTS`, `create_engine`)
- `create_engine` is atomic
- `Engine.createSession()` transfers ownership, never returns partial
- `dispose()` is idempotent on Engine and EngineSession
- After `dispose()`, methods raise `EngineError`
- `synthesize()` raises typed `EngineError` subtypes, session remains usable
- Declared capabilities are actually implemented
- Plugin loader validates manifest, api_version, capabilities
**Run**: `pytest tests/contracts/ -v`
---
## 8. Behavioral Tests
**Location**: `tests/test_behavioral_regression.py`
**Purpose**: Verify user-facing behavior via public API only (`create_pipeline`, `Engine`, `EngineSession`, `PluginManager`).
**Scope**:
- Synthesis with various inputs (short, long, empty, unicode, mixed scripts)
- Voice selection and listing
- Parameter handling (speed, etc.)
- Error scenarios (unknown plugin, disposal, etc.)
- Resource cleanup (dispose idempotency, no leaks)
- Pipeline utility (`create_pipeline`)
**Run**: `pytest tests/test_behavioral_regression.py -v`
---
## 9. Reference
- **Architecture Spec**: `docs/architecture-final-v2.md`
- **Amendment (lang_code)**: `docs/architecture-amendment-001.md`
- **Migration Roadmap**: `docs/migration-roadmap.md`
- **Plugin Examples**: `plugins/kokoro/`, `plugins/supertonic/`
- **Protocol Definitions**: `abogen/tts_plugin/engine.py`, `abogen/tts_plugin/capabilities.py`
-68
View File
@@ -1,68 +0,0 @@
# Getting Started
Quickstart for developers working on Abogen.
## Prerequisites
- Python 3.10+
- Node.js 20+
- npm 10+
- Git
- Docker (optional)
## Installation
```bash
# Development install with all extras
pip install -e .[dev]
# Or with uv
uv pip install -e .[dev]
```
## Running the Application
```bash
# Desktop GUI
abogen
# Web UI
abogen-web
# CLI
abogen-cli
```
## Project Structure
```
abogen/
├── pyqt/ - PyQt6 desktop GUI
├── webui/ - Flask web UI
├── tts_plugin/ - Plugin Architecture (Engine, EngineSession, Manifest)
└── plugins/ - Built-in plugins (kokoro, supertonic)
tests/
├── contracts/ - Contract compliance tests
└── ...
```
## Testing
```bash
# All tests
pytest
# Contract tests (architectural compliance)
pytest tests/contracts/
# Behavioral regression tests
pytest tests/test_behavioral_regression.py
```
## Architecture
See [Developer Guide](developer-guide.md) for Plugin Architecture details:
- Engine / EngineSession lifecycle
- Plugin contract (PLUGIN_MANIFEST, create_engine)
- Adding new plugins
- Capability interfaces
-269
View File
@@ -1,269 +0,0 @@
# Testing Guide
This document describes the testing strategy for Abogen's Plugin Architecture.
## Test Categories
### 0. Auto-Discovery Plugin Tests (`tests/plugins/`)
**Purpose**: Automatically test every plugin in `plugins/` directory without manual test creation. These tests use discovery to find all plugins and run generic tests against each one.
**What They Test**:
- **Manifest structure**: Required fields, API version format, voices field
- **Engine lifecycle**: `create_engine`, `dispose` idempotency, post-dispose behavior
- **Capability implementation**: Declared capabilities are implemented (e.g., `voice_list``VoiceLister`)
**How Auto-Discovery Works**:
```python
# tests/plugins/conftest.py
@pytest.fixture(scope="module")
def plugin_ids(plugins_dir: Path) -> list[str]:
"""Discovers all plugin directories with __init__.py"""
return [item.name for item in plugins_dir.iterdir()
if item.is_dir() and (item / "__init__.py").exists()]
```
**Test Structure**:
```
tests/plugins/
├── conftest.py # Fixtures: plugin_ids, loaded_plugin, host_context
└── test_all_plugins.py # Generic tests for every plugin
├── TestAllPluginsManifest
├── TestAllPluginsEngine
└── TestAllPluginsCapabilities
```
**Running Auto-Discovery Tests**:
```bash
# Test all plugins automatically
pytest tests/plugins/ -v
# Test specific plugin
pytest tests/plugins/ -v -k "kokoro"
# See which plugins were discovered
pytest tests/plugins/ --collect-only
```
**Adding a New Plugin**:
1. Create plugin directory: `plugins/my_plugin/`
2. Add `__init__.py` with `PLUGIN_MANIFEST`, `MODEL_REQUIREMENTS`, `create_engine`
3. Run `pytest tests/plugins/` — tests automatically discover and test your plugin!
**When to Add Plugin-Specific Tests**:
Auto-discovery tests cover generic contract validation. Create plugin-specific tests in `tests/test_<plugin>_plugin.py` for:
- Integration with real dependencies (e.g., KPipeline for Kokoro)
- Specific voice IDs and behavior
- Plugin-specific parameters and features
---
### 1. Contract Tests (`tests/contracts/`)
**Purpose**: Verify that every plugin satisfies the architectural contract. These tests ensure the Plugin Architecture's invariants are maintained.
**What They Guarantee**:
- Every plugin exports `PLUGIN_MANIFEST`, `MODEL_REQUIREMENTS`, `create_engine`
- `create_engine` is atomic (succeeds fully or raises and cleans up)
- `Engine.createSession()` returns valid `EngineSession`, transfers ownership
- `Engine.dispose()` is idempotent, never raises
- After `dispose()`, all methods raise `EngineError`
- `EngineSession.synthesize()` returns `SynthesizedAudio` or raises `EngineError` (session remains usable)
- `EngineSession.dispose()` is idempotent, never raises
- Capability interfaces (`VoiceLister`, `PreviewGenerator`, etc.) are correctly implemented
- Plugin Loader discovers, validates, and loads plugins correctly
- Plugin Manager creates, caches, and disposes engines correctly
- Value objects are immutable and have correct equality semantics
- Error hierarchy is preserved (`EngineError` base with subtypes)
**Why They Exist**:
- Provide **compile-time-like guarantees** for a dynamic plugin system
- Enable **safe plugin ecosystem** — host can trust any loaded plugin
- Catch **architectural violations** early (missing dispose, wrong return types, etc.)
- Document the **contract** in executable form
**What Every New Plugin Must Pass**:
```bash
pytest tests/contracts/ -v
# All tests must pass
```
**Running Contract Tests**:
```bash
# All contract tests
pytest tests/contracts/
# Specific contract
pytest tests/contracts/test_engine_contract.py
# With coverage
pytest tests/contracts/ --cov=abogen.tts_plugin
```
---
### 2. Behavioral Tests (`tests/test_behavioral_regression.py`)
**Purpose**: Verify external user-facing behavior using only public API. These tests are **not coupled to internal implementation**.
**What They Test**:
- Synthesis with various inputs (short, long, empty, unicode, mixed scripts)
- Voice selection and listing
- Parameter handling (speed, etc.)
- Error scenarios (unknown plugin, disposal, etc.)
- Resource cleanup (dispose idempotency, no leaks)
- Pipeline utility (`create_pipeline`)
**Why They Test Public Behavior Only**:
- **Refactoring safety**: Internal changes don't break tests
- **Real-world usage**: Tests match how consumers actually use the API
- **Plugin agnostic**: Parametrized across Kokoro, SuperTonic, and mock plugins
- **Regression detection**: Catch behavioral regressions regardless of implementation
**What They Don't Test**:
- Internal class structure
- Private methods
- Implementation details (how audio is generated, model loading internals)
**Running Behavioral Tests**:
```bash
# All behavioral tests
pytest tests/test_behavioral_regression.py -v
# With specific plugin (if installed)
pytest tests/test_behavioral_regression.py -v -k "kokoro"
```
---
### 4. Unit Tests (`tests/`)
**Purpose**: Test individual modules in isolation.
**Examples**:
- `test_book_parser.py` — EPUB/PDF/text parsing
- `test_text_normalization.py` — Text preprocessing
- `test_chunk_helpers.py` — Text chunking logic
- `test_voice_cache.py` — Voice caching
---
### 5. Integration Tests
**Purpose**: Test cross-component interactions.
**Examples**:
- `test_kokoro_plugin.py` — Full Kokoro plugin integration
- `test_supertonic_plugin.py` — Full SuperTonic plugin integration
- `test_conversion_series.py` — End-to-end conversion pipeline
---
## Test Architecture
```
tests/
├── contracts/ # Contract tests (architectural compliance)
│ ├── conftest.py # Shared fixtures (FakeEngine, FakeSession)
│ ├── test_manifest_contract.py
│ ├── test_plugin_contract.py
│ ├── test_engine_contract.py
│ ├── test_session_contract.py
│ ├── test_capabilities_contract.py
│ ├── test_loader_contract.py
│ ├── test_plugin_manager_contract.py
│ ├── test_types_contract.py
│ ├── test_errors_contract.py
│ ├── test_host_context_contract.py
│ └── test_integration.py
├── test_behavioral_regression.py # Behavioral tests (public API)
├── test_kokoro_plugin.py # Kokoro integration
├── test_supertonic_plugin.py # SuperTonic integration
└── ... # Other unit/integration tests
```
---
## Adding Tests for a New Plugin
### Auto-Discovery Tests (Automatic!)
**No manual test creation required!** When you add a new plugin to `plugins/`:
1. Create plugin directory: `plugins/my_plugin/`
2. Add `__init__.py` with required exports:
```python
PLUGIN_MANIFEST = PluginManifest(...)
MODEL_REQUIREMENTS = [...]
def create_engine(...): ...
```
3. Run `pytest tests/plugins/` — auto-discovery tests automatically find and test your plugin!
**What's Tested Automatically**:
- Manifest structure and required fields
- API version compatibility
- Engine creation and dispose contract
- Capability implementation (if declared)
### Plugin-Specific Tests (Optional)
Create `tests/test_my_plugin_plugin.py` for:
- Integration with real backend (e.g., KPipeline for Kokoro)
- Specific voice IDs and behavior
- Plugin-specific parameters and features
### Contract Tests (Deprecated for New Plugins)
**Note**: Auto-discovery tests (`tests/plugins/`) now cover contract validation for all plugins. Manual contract tests in `tests/contracts/` are only needed for testing internal architecture components.
### Behavioral Tests (Recommended)
Add parametrized tests to `tests/test_behavioral_regression.py`:
```python
# In _plugin_ids list, add your plugin
_plugin_ids = ["kokoro", "supertonic", "my_plugin"]
_plugin_engines["my_plugin"] = _YourMockEngine
_plugin_default_voices["my_plugin"] = "voice1"
_plugin_all_voices["my_plugin"] = ["voice1", "voice2"]
```
All existing behavioral tests will automatically run against your plugin.
---
## Continuous Integration
```yaml
# .github/workflows/test.yml
- name: Contract Tests
run: pytest tests/contracts/ -v
- name: Behavioral Tests
run: pytest tests/test_behavioral_regression.py -v
- name: Unit & Integration Tests
run: pytest tests/ -v --ignore=tests/test_behavioral_regression.py
```
---
## Test Design Principles
### Contract Tests
- **No mocks** for the system under test (test real plugin loading)
- **Strict assertions** on types and behavior
- **Document architecture** in test names and docstrings
- **Fail fast** on architectural violations
### Behavioral Tests
- **Only public API** (`create_pipeline`, `Engine`, `EngineSession`, `PluginManager`)
- **Parametrized** across plugins
- **Realistic scenarios** (long text, unicode, mixed scripts)
- **No implementation coupling** (test behavior, not internals)
### General
- **Fast**: Unit tests < 1s, Contract tests < 5s, Behavioral < 30s
- **Isolated**: No shared state between tests
- **Deterministic**: Same input → same output
- **Descriptive names**: `test_<component>_<scenario>_<expected>`
-689
View File
@@ -1,689 +0,0 @@
# TTS Plugin Architecture — Final Specification
## 1. Core Domain
Zero dependencies. Pure business logic.
### 1.1 Engine
Factory for sessions. Stateless. Thread-safe for createSession().
```
interface Engine:
createSession() -> EngineSession
dispose() -> void
```
**createSession() contract**:
- Returns: EngineSession
- Raises: EngineError on failure
- Ownership: Transfers to caller
- Thread-safe: Yes
**dispose() contract**:
- Releases engine resources
- Caller must ensure all sessions created by this engine are disposed before calling dispose()
- Disposing an engine while any session is still alive violates the API contract; behavior is undefined
- Idempotent: Safe to call multiple times
- Never raises: Catches and logs internally
- After dispose(): All methods except dispose() raise EngineError
### 1.2 EngineSession
Owns mutable execution state isolated from other concurrent work. NOT thread-safe.
```
interface EngineSession:
synthesize(request: SynthesisRequest) -> SynthesizedAudio
dispose() -> void
```
**synthesize() contract**:
- Returns: SynthesizedAudio
- Raises: EngineError on failure (session remains usable)
- Thread-safe: No
**dispose() contract**:
- Releases session resources
- Idempotent: Safe to call multiple times
- Never raises: Catches and logs internally
- After dispose(): All methods except dispose() raise EngineError
### 1.3 SynthesisRequest
Immutable value object.
```
SynthesisRequest:
text: string
voice: VoiceSelection
parameters: ParameterValues
format: AudioFormat
```
### 1.4 SynthesizedAudio
Immutable value object.
```
SynthesizedAudio:
data: bytes
format: AudioFormat
duration: Duration
```
### 1.5 VoiceSelection
Immutable value object. Opaque to engine.
```
VoiceSelection:
source: string
key: string
payload: any = None # Optional; required for clone/blend sources
```
### 1.6 ParameterValues
Immutable value object. Behaves like Mapping[str, Any].
```
ParameterValues:
values: Mapping[str, Any]
```
### 1.7 AudioFormat
Immutable value object.
```
AudioFormat:
mime: string
extension: string
```
### 1.8 Duration
Immutable value object.
```
Duration:
seconds: number
```
### 1.9 EngineConfig
Engine initialization settings only. No resource references.
```
EngineConfig:
device: string # "cpu", "cuda:0", etc.
# Engine-specific settings (if any)
# Unknown keys are ignored (no error)
```
---
## 2. Error Hierarchy
Typed exceptions. Engines raise EngineError or subtypes. Never raw exceptions.
```
EngineError (base)
├── ModelNotFoundError
├── ModelLoadError
├── NetworkError
├── InvalidInputError
├── ConfigurationError
├── CancelledError
└── InternalError
```
**Contract**:
- synthesize() raises EngineError on failure, session remains usable
- dispose() never raises (catches and logs internally)
- create_engine() raises EngineError on failure, cleans up partially created resources
- createSession() raises EngineError on failure, no partially initialized session returned
- cancel() causes synthesize() to raise CancelledError
---
## 3. Capability Interfaces (Optional)
Engines implement only what they support. Capabilities are additive.
### 3.1 VoiceLister
```
interface VoiceLister:
listVoices(sourceId: string) -> list[VoiceManifest]
```
### 3.2 PreviewGenerator
```
interface PreviewGenerator:
generatePreview(voice: VoiceSelection, text: string) -> SynthesizedAudio
```
### 3.3 ModelRequirements
Static at plugin level, not engine level. Host reads before creating engine.
```
MODEL_REQUIREMENTS = list[ModelManifest]
```
### 3.4 StreamingSynthesizer
Optional capability of EngineSession, not Engine.
```
interface StreamingSynthesizer:
synthesizeStream(request: SynthesisRequest) -> Iterator[bytes]
```
**Iterator contract**:
- Yields audio chunks as they become available
- Raises CancelledError if cancel() is called during iteration
- Raises EngineError on synthesis failure
- Iterator exhaustion = synthesis complete
- Session remains usable after iterator completes
### 3.5 CancelableSession
Optional capability for engines that support cancellation.
```
interface CancelableSession:
cancel() -> void
```
**cancel() contract**:
- Cancels in-progress synthesize()
- synthesize() raises CancelledError (subtype of EngineError)
- EngineSession remains usable after cancellation (unless implementation documents otherwise)
---
## 4. Plugin Manifest
Static metadata. Immutable. No dependencies.
### 4.1 PluginManifest
```
PluginManifest:
id: string
name: string
version: string
api_version: string # semver format: MAJOR.MINOR
description: string
author: string
capabilities: list[string]
requires: RequirementManifest
engine: EngineManifest
```
**api_version contract**:
- Format: semver (MAJOR.MINOR)
- Compatibility: Host rejects plugin if major version differs
- Minor version: backward compatible, Host accepts higher minor
### 4.2 EngineManifest
```
EngineManifest:
voiceSources: list[VoiceSourceManifest]
parameters: list[ParameterManifest]
audioFormats: list[AudioFormatManifest]
```
### 4.3 VoiceSourceManifest
```
VoiceSourceManifest:
id: string
name: string
type: string # "list", "speaker_id", "clone", "blend", "generate", "none"
config: any
```
### 4.4 VoiceManifest
```
VoiceManifest:
id: string
name: string
tags: list[string]
```
### 4.5 ParameterManifest
```
ParameterManifest:
id: string
name: string
description: string
type: string # "float", "int", "string", "boolean", "enum"
default: any
min: number (optional)
max: number (optional)
step: number (optional)
options: list[EnumOption] (optional)
unit: string (optional)
group: string (optional)
```
### 4.6 AudioFormatManifest
```
AudioFormatManifest:
mime: string
extension: string
```
### 4.7 EnumOption
```
EnumOption:
value: string
label: string
```
### 4.8 RequirementManifest
```
RequirementManifest:
gpu: GpuRequirement (optional)
memory: number (optional)
internet: boolean (optional)
```
### 4.9 GpuRequirement
```
GpuRequirement:
required: boolean
type: string (optional)
memory: number (optional)
```
### 4.10 ModelManifest
```
ModelManifest:
id: string
name: string
size: string
```
---
## 5. Host Services
### 5.1 HostContext
Minimal. 3 fields maximum. No business logic.
```
HostContext:
config_dir: Path # For API keys, preferences
logger: Logger # For logging
http_client: HttpClient # For network requests
```
---
## 6. Plugin Contract
### 6.1 Plugin Exports
```python
# plugins/kokoro/__init__.py
PLUGIN_MANIFEST = PluginManifest(...)
MODEL_REQUIREMENTS = [...] # Static at plugin level
def create_engine(
context: HostContext,
model_path: Path | None,
config: EngineConfig
) -> Engine:
"""Create engine. Atomic: succeeds fully or raises and cleans up."""
...
```
### 6.2 create_engine() Contract
- Parameters:
- context: HostContext (host services)
- model_path: Path | None (resolved model path, or None for cloud/no-model engines)
- config: EngineConfig (engine initialization settings)
- Returns: Engine
- Raises: EngineError on failure
- Atomic: Succeeds fully or cleans up and raises
- Thread-safe: Can be called from any thread
---
## 7. Object Lifecycle
### 7.1 Engine Lifecycle
```
1. DISCOVERY
Host scans plugin directories
Loads PLUGIN_MANIFEST and MODEL_REQUIREMENTS
2. MODEL DOWNLOAD (if MODEL_REQUIREMENTS non-empty)
Host reads MODEL_REQUIREMENTS
Downloads/caches required models
Resolves model_path for create_engine()
3. ACTIVATION
Host calls create_engine(context, model_path, config)
Engine created, ready to use
Raises EngineError on failure
4. SESSION CREATION
Client calls engine.createSession()
Returns EngineSession
Ownership transfers to caller
Raises EngineError on failure
No partially initialized session returned
5. SYNTHESIS
Client calls session.synthesize(request)
Returns SynthesizedAudio
Raises EngineError on failure (session remains usable)
6. SESSION DISPOSAL
Client calls session.dispose()
Releases session resources
7. DEACTIVATION
Client calls engine.dispose()
Caller must ensure all sessions are disposed first
Disposing engine while sessions are alive is undefined behavior
Releases engine resources
```
### 7.2 EngineSession Lifecycle
```
1. CREATION
Created by Engine.createSession()
Ownership transfers to caller
Raises EngineError on failure
2. USAGE
Client calls synthesize() one or more times
Each call returns SynthesizedAudio or raises EngineError
Session remains usable after synthesize() failure
If CancelableSession: cancel() causes synthesize() to raise CancelledError
If StreamingSynthesizer: iterator raises CancelledError on cancel(), EngineError on failure
3. DISPOSAL
Client calls dispose()
Releases session resources
After dispose(), all methods except dispose() raise EngineError
```
### 7.3 Ownership Rules
- Engine.createSession() transfers ownership of the returned session to the caller
- Caller is responsible for disposing all sessions before disposing the engine
- Engine does not track sessions; it has no lifecycle registry
- Disposing an engine while any session is still alive violates the API contract; behavior is undefined
- This design avoids coupling, synchronization overhead, and lifecycle registry complexity
### 7.4 Concurrent Operations
**Engine.dispose() concurrent with Engine.createSession()**:
- createSession() must either succeed with fully initialized EngineSession or raise EngineError
- Partially initialized EngineSession must never be returned
- After dispose() completes, subsequent createSession() calls must raise EngineError
**EngineSession.dispose() concurrent with EngineSession.synthesize()**:
- Not thread-safe. Caller must ensure synthesize() completes before dispose().
**EngineSession.dispose() concurrent with StreamingSynthesizer.synthesizeStream()**:
- Not thread-safe. Caller must ensure stream iteration completes before dispose().
---
## 8. Thread Safety Contract
| Component | Thread-safe | Notes |
|-----------|-------------|-------|
| Engine | Yes | createSession() can be called from any thread |
| EngineSession | No | synthesize() must be called from one thread at a time |
| HostContext | Yes | Provides shared services |
| VoiceSelection | Yes | Immutable value object |
| ParameterValues | Yes | Immutable value object |
| AudioFormat | Yes | Immutable value object |
| EngineConfig | Yes | Immutable value object |
---
## 9. dispose() Contract
**General rules**:
- Calling dispose() multiple times is safe (no-op on second call)
- dispose() never raises exceptions (catches and logs internally)
- After dispose(), all methods except dispose() raise EngineError
**Engine.dispose()**:
- Caller must ensure all sessions are disposed first
- Disposing engine while sessions are alive violates API contract; behavior is undefined
- Releases engine resources
**EngineSession.dispose()**:
- Releases session resources
---
## 10. Dependency Rules
```
Core Domain (Engine, EngineSession, Value Objects)
-> No dependencies
Plugin Manifest (PluginManifest, ModelManifest, etc.)
-> No dependencies
Host Context (HostContext)
-> Depends on: Core Domain (for types)
Plugin Implementation
-> Depends on: Core Domain, Host Context
Host
-> Depends on: Core Domain, Plugin Manifest
```
**Forbidden**:
- Core Domain -> anything else
- Plugin Manifest -> anything else
- Plugin Implementation -> Host (only receives HostContext)
- Host -> Plugin Implementation (only via create_engine function)
---
## 11. Architectural Invariants
1. Core Domain has zero dependencies
2. Plugins receive HostContext at creation, not via global state
3. Model requirements are static (plugin level), not dynamic (engine level)
4. Host validates capability implementation at load time: each capability declared in PluginManifest.capabilities must be implemented by the exported object via the corresponding interface
5. synthesize() raises typed exceptions, not returns Result
6. dispose() is idempotent and never raises
7. No global state, no service locator
8. VoiceSelection and ParameterValues are opaque to engine
9. Display information comes from VoiceManifest
10. HostContext is minimal (3 fields max)
11. EngineConfig contains only engine settings, not resource references
12. EngineSession owns mutable execution state isolated from other concurrent work
13. Engine.createSession() transfers ownership to caller
14. Caller must dispose all sessions before disposing engine
15. After dispose(), all methods except dispose() raise EngineError
16. create_engine() is atomic (all-or-nothing)
17. Garbage collection without dispose() may leak (documented)
18. Capabilities are additive (new capabilities don't break old plugins)
19. api_version enables compatibility checking
20. createSession() returns fully initialized session or raises, never partial
21. cancel() causes synthesize() to raise CancelledError
22. EngineSession remains usable after cancellation
23. Engine does not track sessions; no lifecycle registry
---
## 12. Validation Examples
### 12.1 Kokoro
```python
PLUGIN_MANIFEST = PluginManifest(
id="kokoro",
api_version="1.0",
capabilities=["voice_list", "preview", "voice_blend"],
engine=EngineManifest(
voiceSources=[
VoiceSourceManifest(id="builtin", type="list", config={"voices": [...]}),
VoiceSourceManifest(id="formula", type="blend", config={"syntax": "{a}*0.5+{b}*0.5"}),
],
parameters=[ParameterManifest(id="speed", type="float", default=1.0, min=0.5, max=2.0)],
audioFormats=[AudioFormatManifest(mime="audio/wav", extension="wav")],
),
)
MODEL_REQUIREMENTS = []
def create_engine(context: HostContext, model_path: Path | None, config: EngineConfig) -> Engine:
model = load_kokoro(model_path)
return KokoroEngine(model, config.device)
```
### 12.2 SuperTonic
```python
PLUGIN_MANIFEST = PluginManifest(
id="supertonic",
api_version="1.0",
capabilities=["voice_list", "preview"],
engine=EngineManifest(
voiceSources=[VoiceSourceManifest(id="builtin", type="list", config={"voices": [...]})],
parameters=[
ParameterManifest(id="speed", type="float", default=1.0, min=0.5, max=2.0),
ParameterManifest(id="steps", type="int", default=20, min=5, max=50),
],
audioFormats=[AudioFormatManifest(mime="audio/wav", extension="wav")],
),
)
MODEL_REQUIREMENTS = []
def create_engine(context: HostContext, model_path: Path | None, config: EngineConfig) -> Engine:
model = load_supertonic(model_path)
return SuperTonicEngine(model, config.device)
```
### 12.3 ElevenLabs
```python
PLUGIN_MANIFEST = PluginManifest(
id="elevenlabs",
api_version="1.0",
capabilities=["voice_list"],
requires=RequirementManifest(internet=True),
engine=EngineManifest(
voiceSources=[VoiceSourceManifest(id="cloud", type="list", config={"speakers": [...]})],
parameters=[ParameterManifest(id="stability", type="float", default=0.5, min=0.0, max=1.0)],
audioFormats=[AudioFormatManifest(mime="audio/mpeg", extension="mp3")],
),
)
MODEL_REQUIREMENTS = []
def create_engine(context: HostContext, model_path: Path | None, config: EngineConfig) -> Engine:
api_key = (context.config_dir / "elevenlabs_key").read_text()
return ElevenLabsEngine(api_key)
```
### 12.4 Piper
```python
PLUGIN_MANIFEST = PluginManifest(
id="piper",
api_version="1.0",
capabilities=[],
engine=EngineManifest(
voiceSources=[VoiceSourceManifest(id="downloadable", type="list", config={"models": [...]})],
parameters=[ParameterManifest(id="speed", type="float", default=1.0, min=0.5, max=2.0)],
audioFormats=[AudioFormatManifest(mime="audio/wav", extension="wav")],
),
)
MODEL_REQUIREMENTS = [
ModelManifest(id="en_US-lessac-medium", name="English Lessac Medium", size="100MB"),
]
def create_engine(context: HostContext, model_path: Path | None, config: EngineConfig) -> Engine:
return PiperEngine(model_path, config.device)
```
### 12.5 XTTS (with streaming and cancellation)
```python
PLUGIN_MANIFEST = PluginManifest(
id="xtts",
api_version="1.0",
capabilities=["voice_list", "preview", "voice_clone", "streaming", "cancel"],
requires=RequirementManifest(gpu=GpuRequirement(required=True, type="cuda")),
engine=EngineManifest(
voiceSources=[
VoiceSourceManifest(id="speakers", type="speaker_id", config={"speakers": [...]}),
VoiceSourceManifest(id="clone", type="clone", config={"requiresAudio": True, "maxDuration": 30}),
],
parameters=[ParameterManifest(id="temperature", type="float", default=0.7, min=0.1, max=1.0)],
audioFormats=[AudioFormatManifest(mime="audio/wav", extension="wav")],
),
)
MODEL_REQUIREMENTS = [
ModelManifest(id="xtts_v2", name="XTTS v2", size="2GB"),
]
def create_engine(context: HostContext, model_path: Path | None, config: EngineConfig) -> Engine:
return XTTSEngine(model_path, config.device)
```
XTTS session implements: EngineSession, StreamingSynthesizer, CancelableSession.
---
## 13. Summary of All Decisions
| Aspect | Decision |
|--------|----------|
| Engine | Factory, stateless, thread-safe for createSession() |
| EngineSession | Owns mutable execution state, not thread-safe |
| EngineSession ownership | Caller owns (transferred from createSession) |
| Engine session tracking | None; engine does not track sessions |
| StreamingSynthesizer | Optional capability of EngineSession |
| CancelableSession | Optional capability, cancel() raises CancelledError |
| dispose() | Idempotent, never raises |
| Engine.dispose() | Caller must dispose sessions first; undefined if violated |
| createSession() | Raises EngineError on failure, no partial sessions |
| create_engine() | Atomic, takes context, model_path, config |
| EngineConfig | Engine settings only, no resource references |
| model_path | Separate argument, not in EngineConfig |
| MODEL_REQUIREMENTS | Static at plugin level |
| HostContext | Minimal (3 fields) |
| Error handling | Typed exceptions (EngineError hierarchy) |
| Thread safety | Documented per component |
| Capabilities | Additive, optional interfaces |
| API versioning | api_version in manifest |
| Concurrent dispose/createSession | Fully initialized session or EngineError |
| Concurrent dispose/synthesizeStream | Not thread-safe; caller must complete iteration first |

Some files were not shown because too many files have changed in this diff Show More