Compare commits

...
Author SHA1 Message Date
Artem Akymenko 146000886d ci: add uv cache prune to optimize cache size 2026-07-08 21:14:15 +00:00
Artem Akymenko 31f95137dd ci: replace pip with uv for faster dependency installation 2026-07-08 18:34:35 +00:00
Artem Akymenko 6f02fda41c fix(ci): set QT_QPA_PLATFORM=offscreen for headless PyQt6 tests 2026-07-08 17:36:59 +00:00
Artem Akymenko a3c3462348 fix(ci): install libegl1 on Ubuntu and normalize line endings in epub test
- Add system dependency step for libegl1 to fix PyQt6 import on headless CI
- Normalize CRLF to LF in epub exporter whitespace test for Windows CI
2026-07-08 17:16:33 +00:00
Artem AkymenkoandGitHub 79332204d3 Merge pull request #189 from denizsafak/feat/registry-voice-resolution
refactor: Move backend resolution by voice spec into registry
2026-07-08 20:03:19 +03:00
Artem Akymenko 6deec3b9b6 refactor: move backend resolution by voice spec into registry
- Add resolve_backend_for_voice() to TTSBackendRegistry
- Add module-level wrapper resolve_backend_for_voice()
- Simplify _infer_provider_from_spec() to use registry API
- Simplify _supertonic_voice_from_spec() to only normalize
- Add 11 test cases for the new method

Resolution rules:
1. Empty spec -> fallback
2. Kokoro formula (* or +) -> kokoro
3. Exact voice ID match -> backend id
4. Unknown voice -> fallback
2026-07-08 17:02:33 +00:00
Artem Akymenko c4d14112d4 refactor: replace hardcoded backend ID sets with registry checks
Add TTSBackendRegistry.is_registered() and module-level
is_registered_backend() to validate backend IDs dynamically.
Replace all Category A hardcoded sets (validation-only) in
voice_profiles, api routes, conversion_runner, and form utils.
2026-07-08 16:33:16 +00:00
Artem Akymenko f4cb2c2329 ci: add pytest, use actions/cache@v6 2026-07-08 19:26:42 +03:00
Artem AkymenkoandGitHub 783738882f Merge pull request #188 from denizsafak/refactor/move-kokoro-voices-into-backend
refactor: move VOICES_INTERNAL into KokoroBackend module
2026-07-08 19:23:33 +03:00
Artem Akymenko e94ba5257e refactor: move VOICES_INTERNAL into KokoroBackend module
Make the Kokoro voice list an internal implementation detail of the
backend instead of a shared constant. The rest of the project already
accesses voices via get_metadata('kokoro').voices.

- Move VOICES_INTERNAL from constants.py to kokoro.py as _VOICES_INTERNAL
- Update tests to use get_metadata('kokoro').voices instead of importing
  the constant directly
2026-07-08 16:19:34 +00:00
Artem AkymenkoandGitHub 49d66839dc Merge pull request #186 from denizsafak/refactor/migrate-remaining-voice-metadata-consumers
refactor: migrate remaining consumers to get_metadata API
2026-07-08 19:01:20 +03:00
Artem AkymenkoandGitHub d0e316ea7b Merge pull request #187 from denizsafak/refactor/migrate-pyqt-to-backend-metadata
refactor(pyqt): migrate from VOICES_INTERNAL to get_metadata API
2026-07-08 19:01:02 +03:00
Artem Akymenko bb96ae502c refactor: migrate remaining consumers to get_metadata API
Replace direct VOICES_INTERNAL imports with get_metadata('kokoro').voices:
- abogen/predownload_gui.py
- abogen/subtitle_utils.py
2026-07-08 15:58:51 +00:00
Artem Akymenko a4d25accc1 refactor(pyqt): migrate from VOICES_INTERNAL to get_metadata API
Replace direct VOICES_INTERNAL imports with get_metadata('kokoro').voices
from tts_backend_registry in all PyQt modules:
- abogen/pyqt/gui.py
- abogen/pyqt/predownload_gui.py
- abogen/pyqt/voice_formula_gui.py
2026-07-08 15:57:18 +00:00
Artem AkymenkoandGitHub 66964bfd0b Merge pull request #185 from denizsafak/refactor/use-backend-metadata-in-webui
refactor(webui): replace direct VOICES_INTERNAL/DEFAULT_SUPERTONIC_VOICES with get_metadata API
2026-07-08 18:49:21 +03:00
Artem Akymenko f8f72624f8 refactor(webui): replace direct VOICES_INTERNAL/DEFAULT_SUPERTONIC_VOICES with get_metadata API
- Add get_default_voice() helper to tts_backend_registry
- Replace all VOICES_INTERNAL imports in WebUI with get_metadata().voices
- Replace all DEFAULT_SUPERTONIC_VOICES imports in conversion_runner with get_metadata().voices
- Remove unused VOICES_INTERNAL import from voices.py

Core modules (voice_profiles, voice_formulas, voice_cache) already used
get_metadata(). This completes the WebUI migration.
2026-07-08 15:42:49 +00:00
Artem Akymenko e7a88a513a ci: fix duplicate triggers, pin macos-14 to avoid migration warning 2026-07-08 16:52:26 +03:00
Artem AkymenkoandGitHub 2277f16d0a Merge pull request #184 from denizsafak/refactor/use-backend-metadata-for-voice-lists
refactor: migrate core modules to use TTSBackendMetadata.voices via registry
2026-07-08 16:51:58 +03:00
Artem Akymenko 1d50429b87 refactor: migrate core modules to use TTSBackendMetadata.voices via registry
Replace direct imports of VOICES_INTERNAL and DEFAULT_SUPERTONIC_VOICES
in voice_profiles, voice_formulas, and voice_cache with get_metadata()
from TTSBackendRegistry. Adds get_metadata() top-level function to
tts_backend_registry as symmetric counterpart to register_backend() and
create_backend().
2026-07-08 13:43:52 +00:00
Artem Akymenko 29681a5fbb ci: update actions to v7/v6, add pip caching, optimize Dockerfile layer order 2026-07-08 16:22:52 +03:00
Artem AkymenkoandGitHub 50fa2e5b9e Merge pull request #183 from denizsafak/refactor/store-supported-voices-in-backend-metadata
feat: store supported voices in TTSBackendMetadata
2026-07-08 16:02:57 +03:00
Artem Akymenko 5816feb6da feat: store supported voices in TTSBackendMetadata
Add voices field to TTSBackendMetadata so each backend's supported
voice list is part of its metadata rather than external constants.

- Add voices: tuple[str, ...] = () to TTSBackendMetadata
- Create _KOKORO_METADATA / _SUPERTONIC_METADATA as single source
  of truth for both metadata property and registry registration
- Update KokoroBackend.get_available_voices() to use self.metadata.voices
- Update SupertonicBackend.get_available_voices() to use self.metadata.voices
- Add tests for voices field, metadata voice content, and unified instance identity
2026-07-06 17:40:49 +00:00
Artem AkymenkoandGitHub b95df8f217 Merge pull request #182 from denizsafak/refactor/add-kokoro-backend
feat: add KokoroBackend implementing TTSBackend protocol
2026-07-06 17:29:49 +03:00
Artem AkymenkoandGitHub 245e67284e Merge pull request #181 from denizsafak/refactor/add-supertonic-backend
feat: add SupertonicBackend implementing TTSBackend protocol
2026-07-06 17:29:22 +03:00
Artem Akymenko e2557d961b feat: add KokoroBackend implementing TTSBackend protocol
- Create KokoroBackend class implementing TTSBackend protocol
- Move all KPipeline interaction inside KokoroBackend
- Update LoadPipelineThread to create backend via create_backend()
- Update ConversionThread and VoicePreviewThread to accept backend
- Replace np_module/kpipeline_class parameters with single backend
- Add 24 unit tests for KokoroBackend
- KPipeline is now an internal implementation detail of KokoroBackend
2026-07-06 14:10:54 +00:00
Artem Akymenko 9c6b3774b4 feat: add SupertonicBackend implementing TTSBackend protocol
Encapsulate SupertonicPipeline as an internal detail of
SupertonicBackend. The factory create_supertonic_backend() now
returns a SupertonicBackend instance instead of a raw
SupertonicPipeline, satisfying the TTSBackend protocol with
metadata, synthesize, get_available_voices, get_supported_formats,
and get_info methods. Backward-compatible __call__ delegates to
the internal pipeline.
2026-07-06 14:09:30 +00:00
Artem AkymenkoandGitHub fd9fe5579a Merge pull request #180 from k0sm0naft/refactor/use-registry-for-preview
refactor: migrate preview and conversion code to use TTSBackendRegistry
2026-07-06 16:53:28 +03:00
Artem Akymenko f079373821 refactor: migrate preview and conversion code to use TTSBackendRegistry
Migrate all preview/debug/conversion pipeline creation to use
TTSBackendRegistry.create_backend() instead of direct imports:

- debug_tts_runner._load_pipeline(): Kokoro via registry
- preview.get_preview_pipeline(): Kokoro via registry
- preview.generate_preview_audio(): Supertonic via registry
- voice.get_preview_pipeline(): Kokoro via registry
- conversion_runner._load_pipeline(): both backends via registry
- conversion_runner inline pipeline creation: both via registry
- test: update mock to target tts_backend_registry.create_backend
2026-07-06 15:59:22 +03:00
Deniz ŞafakandGitHub fbb5d4e368 Merge pull request #179 from k0sm0naft/refactor/backend-registry
Add TTS backend registry and automatic backend registration
2026-07-06 15:14:47 +03:00
Artem Akymenko 57fec453e2 feat: auto-register existing TTS backends
- Add create_kokoro_backend() factory in kokoro.py
- Add create_supertonic_backend() factory in supertonic.py
- Auto-discover backend modules in __init__.py via pkgutil
- Both backends register themselves on import
- Tests verify registration and factory callables
2026-07-06 15:04:49 +03:00
Artem Akymenko 58fe22e3d5 feat: add TTSBackendRegistry for backend registration and creation
- TTSBackendRegistry class with register(), list_backends(), get_metadata(), create_backend()
- Global registry singleton with register_backend() and create_backend() convenience functions
- Unit tests for registry operations
2026-07-06 15:04:49 +03:00
Deniz ŞafakandGitHub ab8cbc4911 Merge pull request #178 from k0sm0naft/refactor/backend-package
refactor: move backend implementations to tts_backends package
2026-07-06 14:50:16 +03:00
Deniz ŞafakandGitHub 5e2048072a Merge pull request #177 from k0sm0naft/refactor/tts-backend-interface
feat: Add TTSBackendMetadata model
2026-07-06 14:49:35 +03:00
Artem Akymenko 66ed2a202d refactor: move backend implementations to tts_backends package
Moved SupertonicPipeline from abogen/tts_supertonic.py to
abogen/tts_backends/supertonic.py and load_numpy_kpipeline from
abogen/utils.py to abogen/tts_backends/kokoro.py.

Git correctly detects the Supertonic file as a rename (R),
preserving full commit history.

- New package: abogen/tts_backends/
  - __init__.py (package marker)
  - supertonic.py (SupertonicPipeline, moved from tts_supertonic.py)
  - kokoro.py (load_numpy_kpipeline, moved from utils.py)
- abogen/utils.py: re-exports load_numpy_kpipeline for backward compat
- All imports updated to new canonical paths
2026-07-06 14:04:51 +03:00
Artem Akymenko 45e859dac4 feat: Add TTSBackendMetadata model 2026-07-06 12:58:06 +03:00
Deniz ŞafakandGitHub 56d3e414b3 Merge pull request #176 from k0sm0naft/feat/voice-metadata
feat: add VoiceMetadata data model for TTS backends
2026-07-06 11:01:22 +03:00
Deniz ŞafakandGitHub b942bcb820 Merge pull request #175 from k0sm0naft/refactor/tts-backend-interface
refactor: Switch TTSBackend from ABC to Protocol
2026-07-06 11:00:46 +03:00
Artem Akymenko 47efcb4420 feat: add VoiceMetadata data model for TTS backends 2026-07-05 19:07:57 +00:00
Artem Akymenko 7b3f9d8615 Merge branch 'main' into refactor/tts-backend-interface 2026-07-05 16:53:40 +03:00
Artem Akymenko 9833bb0843 refactor: Switch TTSBackend from ABC to Protocol 2026-07-05 13:47:31 +00:00
Deniz ŞafakandGitHub cbc05ead42 Merge pull request #173 from k0sm0naft/refactor/tts-backend-interface
refactor: introduce TTS backend abstraction
2026-07-03 21:04:47 +03:00
Artem Akymenko 50b4d6872a feat: Add minimal TTSBackend interface for future extensibility
- Create TTSBackend abstract base class with minimal contract
- Implement KokoroTTSBackend that maintains existing behavior
- Update conversion_runner.py to use new interface
- No behavioral changes, GUI unchanged, no new features
2026-07-03 01:25:41 +03:00
37 changed files with 1932 additions and 576 deletions
+33 -12
View File
@@ -1,7 +1,9 @@
name: pip install
run-name: pip install
on:
name: CI
run-name: CI
on:
push:
branches: [main]
paths:
- '**.py'
- 'pyproject.toml'
@@ -11,23 +13,42 @@ on:
- 'pyproject.toml'
- '.github/workflows/**'
workflow_dispatch:
jobs:
install-and-run:
test:
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
os: [ubuntu-latest, macos-14, windows-latest]
python-version: ['3.12']
fail-fast: false
continue-on-error: true
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v7
- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
- name: Install from repository
run: python -m pip install .
#- name: Run abogen
# run: abogen
- name: Install uv
uses: astral-sh/setup-uv@v8.3.1
with:
enable-cache: true
cache-dependency-glob: pyproject.toml
- name: Install system dependencies (Ubuntu)
if: runner.os == 'Linux'
run: sudo apt-get update && sudo apt-get install -y libegl1
- name: Install dependencies
run: uv pip install --system .[dev]
- name: Run tests
env:
QT_QPA_PLATFORM: offscreen
run: pytest tests/ -v --tb=short
- name: Minimize uv cache
if: always()
run: uv cache prune --ci
+63 -63
View File
@@ -1,63 +1,63 @@
name: Build multi-arch Docker Image
on:
# Build and push
#release:
# types: [published]
# Build only
#push: it
# branches: [main]
# TODO - enable build on pull requests if build times can be reduced
# pull_request:
workflow_dispatch:
env:
IMAGE_REPOSITORY: ghcr.io/denizsafak/abogen
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Login to Github Container Registry
# Only if we need to push an image
# if: ${{ github.event_name == 'release' && github.event.action == 'published' }}
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Setup for buildx
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
# Debugging information
- name: Docker info
run: docker info
- name: Buildx inspect
run: docker buildx inspect
# Build and (optionally) push the image
- name: Build image
uses: docker/build-push-action@v6
with:
context: ./abogen
file: ./abogen/Dockerfile
# platforms: linux/amd64,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x
# platforms: linux/amd64,linux/arm64
platforms: linux/amd64 # using the solution mentioned in https://github.com/denizsafak/abogen/issues/46
# Only push if we are publishing a release
# push: ${{ github.event_name == 'release' && github.event.action == 'published' }}
push: true
# Use a 'temp' tag, that won't be pushed, for non-release builds
tags: ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name || 'latest' }}
# Use a cache to reduce build times
cache-to: type=gha,mode=max
cache-from: type=gha
name: Build multi-arch Docker Image
on:
# Build and push
#release:
# types: [published]
# Build only
#push: it
# branches: [main]
# TODO - enable build on pull requests if build times can be reduced
# pull_request:
workflow_dispatch:
env:
IMAGE_REPOSITORY: ghcr.io/denizsafak/abogen
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Login to Github Container Registry
# Only if we need to push an image
# if: ${{ github.event_name == 'release' && github.event.action == 'published' }}
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# Setup for buildx
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
# Debugging information
- name: Docker info
run: docker info
- name: Buildx inspect
run: docker buildx inspect
# Build and (optionally) push the image
- name: Build image
uses: docker/build-push-action@v6
with:
context: ./abogen
file: ./abogen/Dockerfile
# platforms: linux/amd64,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x
# platforms: linux/amd64,linux/arm64
platforms: linux/amd64 # using the solution mentioned in https://github.com/denizsafak/abogen/issues/46
# Only push if we are publishing a release
# push: ${{ github.event_name == 'release' && github.event.action == 'published' }}
push: true
# Use a 'temp' tag, that won't be pushed, for non-release builds
tags: ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name || 'latest' }}
# Use a cache to reduce build times
cache-to: type=gha,mode=max
cache-from: type=gha
-58
View File
@@ -63,64 +63,6 @@ SUPPORTED_INPUT_FORMATS = [
# 384 if self.lang_code in 'ab':
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION = list(LANGUAGE_DESCRIPTIONS.keys())
# Voice and sample text constants
VOICES_INTERNAL = [
"af_alloy",
"af_aoede",
"af_bella",
"af_heart",
"af_jessica",
"af_kore",
"af_nicole",
"af_nova",
"af_river",
"af_sarah",
"af_sky",
"am_adam",
"am_echo",
"am_eric",
"am_fenrir",
"am_liam",
"am_michael",
"am_onyx",
"am_puck",
"am_santa",
"bf_alice",
"bf_emma",
"bf_isabella",
"bf_lily",
"bm_daniel",
"bm_fable",
"bm_george",
"bm_lewis",
"ef_dora",
"em_alex",
"em_santa",
"ff_siwis",
"hf_alpha",
"hf_beta",
"hm_omega",
"hm_psi",
"if_sara",
"im_nicola",
"jf_alpha",
"jf_gongitsune",
"jf_nezumi",
"jf_tebukuro",
"jm_kumo",
"pf_dora",
"pm_alex",
"pm_santa",
"zf_xiaobei",
"zf_xiaoni",
"zf_xiaoxiao",
"zf_xiaoyi",
"zm_yunjian",
"zm_yunxi",
"zm_yunxia",
"zm_yunyang",
]
# Voice and sample text mapping
SAMPLE_VOICE_TEXTS = {
"a": "This is a sample of the selected voice.",
+5 -4
View File
@@ -21,7 +21,8 @@ from PyQt6.QtWidgets import (
)
from PyQt6.QtCore import QThread, pyqtSignal
from abogen.constants import COLORS, VOICES_INTERNAL
from abogen.constants import COLORS
from abogen.tts_backend_registry import get_metadata
from abogen.spacy_utils import SPACY_MODELS
import abogen.hf_tracker
@@ -114,7 +115,7 @@ class PreDownloadWorker(QThread):
self._voices_success = False
return
voice_list = VOICES_INTERNAL
voice_list = get_metadata("kokoro").voices
for idx, voice in enumerate(voice_list, start=1):
if self._cancelled:
self._voices_success = False
@@ -462,14 +463,14 @@ class PreDownloadDialog(QDialog):
try:
from huggingface_hub import try_to_load_from_cache
for voice in VOICES_INTERNAL:
for voice in get_metadata("kokoro").voices:
if not try_to_load_from_cache(
repo_id="hexgrad/Kokoro-82M", filename=f"voices/{voice}.pt"
):
missing.append(voice)
except Exception:
# If HF missing, report all as missing
return False, list(VOICES_INTERNAL)
return False, list(get_metadata("kokoro").voices)
return (len(missing) == 0), missing
def _check_kokoro_model(self) -> bool:
+28 -56
View File
@@ -5,6 +5,7 @@ import hashlib # For generating unique cache filenames
from platformdirs import user_desktop_dir
from PyQt6.QtCore import QThread, pyqtSignal, Qt, QTimer
from PyQt6.QtWidgets import QCheckBox, QVBoxLayout, QDialog, QLabel, QDialogButtonBox
import numpy as np
import soundfile as sf
from abogen.utils import (
create_process,
@@ -259,8 +260,7 @@ class ConversionThread(QThread):
output_folder,
subtitle_mode,
output_format,
np_module,
kpipeline_class,
backend,
start_time,
total_char_count,
use_gpu=True,
@@ -270,8 +270,7 @@ class ConversionThread(QThread):
super().__init__()
self._chapter_options_event = threading.Event()
self._timestamp_response_event = threading.Event()
self.np = np_module
self.KPipeline = kpipeline_class
self.backend = backend
self.file_name = file_name
self.lang_code = lang_code
self.speed = speed
@@ -490,19 +489,6 @@ class ConversionThread(QThread):
self.log_updated.emit(("\nInitializing TTS pipeline...", "grey"))
# Set device based on use_gpu setting and platform
if self.use_gpu:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps" # Use MPS for Apple Silicon
else:
device = "cuda" # Use CUDA for other platforms
else:
device = "cpu"
tts = self.KPipeline(
lang_code=self.lang_code, repo_id="hexgrad/Kokoro-82M", device=device
)
# Check if the input is a subtitle file or timestamp text file
is_subtitle_file = False
is_timestamp_text = False
@@ -538,7 +524,7 @@ class ConversionThread(QThread):
# Process subtitle files separately
if is_subtitle_file or is_timestamp_text:
self._process_subtitle_file(tts, base_path, is_timestamp_text)
self._process_subtitle_file(self.backend, base_path, is_timestamp_text)
return
if self.is_direct_text:
@@ -1071,7 +1057,7 @@ class ConversionThread(QThread):
for segment_idx, (voice_name, segment_text) in enumerate(voice_segments):
# Load voice for this segment (with caching)
try:
loaded_voice = self.load_voice_cached(voice_name, tts)
loaded_voice = self.load_voice_cached(voice_name, self.backend)
if segment_idx > 0:
voice_display = voice_name if len(voice_name) < 50 else voice_name[:47] + "..."
self.log_updated.emit((f" → Voice: {voice_display}", "grey"))
@@ -1080,7 +1066,7 @@ class ConversionThread(QThread):
(f"⚠ Voice loading error for '{voice_name}', continuing with previous", "orange")
)
if segment_idx == 0:
loaded_voice = self.load_voice_cached(self.voice, tts)
loaded_voice = self.load_voice_cached(self.voice, self.backend)
# Determine if spaCy segmentation should be used for PRE-TTS segmentation
# Only non-English languages use spaCy for pre-segmentation
@@ -1166,7 +1152,7 @@ class ConversionThread(QThread):
print("Using split pattern: (unprintable)")
for text_segment in text_segments:
for result in tts(
for result in self.backend(
text_segment,
voice=loaded_voice,
speed=self.speed,
@@ -1368,7 +1354,7 @@ class ConversionThread(QThread):
silence_samples = int(
self.silence_duration * 24000
) # Silence duration at 24,000 Hz
silence_audio = self.np.zeros(silence_samples, dtype="float32")
silence_audio = np.zeros(silence_samples, dtype="float32")
silence_bytes = silence_audio.tobytes()
if merged_out_file:
@@ -1707,7 +1693,7 @@ class ConversionThread(QThread):
max_end_time = max(
(end for _, end, _ in subtitles if end is not None), default=0
)
audio_buffer = self.np.zeros(
audio_buffer = np.zeros(
int(max_end_time * rate) + rate, dtype="float32"
)
@@ -1771,7 +1757,7 @@ class ConversionThread(QThread):
# Generate TTS audio
tts_results = [
r
for r in tts(
for r in self.backend(
processed_text,
voice=loaded_voice,
speed=self.speed,
@@ -1789,11 +1775,11 @@ class ConversionThread(QThread):
# Concatenate audio and determine duration
full_audio = (
self.np.concatenate(
np.concatenate(
[a.numpy() if hasattr(a, "numpy") else a for a in audio_chunks]
)
if audio_chunks
else self.np.zeros(
else np.zeros(
int((subtitle_duration or 0) * rate), dtype="float32"
)
)
@@ -1827,8 +1813,8 @@ class ConversionThread(QThread):
num_stages = max(
1,
int(
self.np.ceil(
self.np.log(speed_factor) / self.np.log(2.0)
np.ceil(
np.log(speed_factor) / np.log(2.0)
)
),
)
@@ -1861,7 +1847,7 @@ class ConversionThread(QThread):
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
full_audio = self.np.frombuffer(
full_audio = np.frombuffer(
speed_proc.communicate(input=full_audio.tobytes())[0],
dtype="float32",
)
@@ -1875,7 +1861,7 @@ class ConversionThread(QThread):
tts_results = [
r
for r in tts(
for r in self.backend(
processed_text,
voice=loaded_voice,
speed=new_speed,
@@ -1886,14 +1872,14 @@ class ConversionThread(QThread):
audio_chunks = [r.audio for r in tts_results]
full_audio = (
self.np.concatenate(
np.concatenate(
[
a.numpy() if hasattr(a, "numpy") else a
for a in audio_chunks
]
)
if audio_chunks
else self.np.zeros(
else np.zeros(
int(subtitle_duration * rate), dtype="float32"
)
)
@@ -1910,10 +1896,10 @@ class ConversionThread(QThread):
# Pad or trim to subtitle duration
target_samples = int(subtitle_duration * rate)
if len(full_audio) < target_samples:
full_audio = self.np.concatenate(
full_audio = np.concatenate(
[
full_audio,
self.np.zeros(
np.zeros(
target_samples - len(full_audio), dtype="float32"
),
]
@@ -1926,10 +1912,10 @@ class ConversionThread(QThread):
end_sample = start_sample + len(full_audio)
if end_sample > len(audio_buffer):
# Extend buffer if needed
audio_buffer = self.np.concatenate(
audio_buffer = np.concatenate(
[
audio_buffer,
self.np.zeros(
np.zeros(
end_sample - len(audio_buffer), dtype="float32"
),
]
@@ -1971,7 +1957,7 @@ class ConversionThread(QThread):
self.progress_updated.emit(percent, etr_str)
# Normalize audio buffer to prevent clipping from mixed overlaps
max_amplitude = self.np.abs(audio_buffer).max()
max_amplitude = np.abs(audio_buffer).max()
if max_amplitude > 1.0:
self.log_updated.emit(
f"\n -> Normalizing audio (peak: {max_amplitude:.2f})"
@@ -2440,8 +2426,7 @@ class VoicePreviewThread(QThread):
def __init__(
self,
np_module,
kpipeline_class,
backend,
lang_code,
voice,
speed,
@@ -2449,8 +2434,7 @@ class VoicePreviewThread(QThread):
parent=None,
):
super().__init__(parent)
self.np_module = np_module
self.kpipeline_class = kpipeline_class
self.backend = backend
self.lang_code = lang_code
self.voice = voice
self.speed = speed
@@ -2484,31 +2468,19 @@ class VoicePreviewThread(QThread):
# Generate the preview and save to cache
try:
# Set device based on use_gpu setting and platform
if self.use_gpu:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps" # Use MPS for Apple Silicon
else:
device = "cuda" # Use CUDA for other platforms
else:
device = "cpu"
tts = self.kpipeline_class(
lang_code=self.lang_code, repo_id="hexgrad/Kokoro-82M", device=device
)
# Enable voice formula support for preview
if "*" in self.voice:
loaded_voice = get_new_voice(tts, self.voice, self.use_gpu)
loaded_voice = get_new_voice(self.backend, self.voice, self.use_gpu)
else:
loaded_voice = self.voice
sample_text = get_sample_voice_text(self.lang_code)
audio_segments = []
for result in tts(
for result in self.backend(
sample_text, voice=loaded_voice, speed=self.speed, split_pattern=None
):
audio_segments.append(result.audio)
if audio_segments:
audio = self.np_module.concatenate(audio_segments)
audio = np.concatenate(audio_segments)
# Save directly to the cache path
sf.write(self.cache_path, audio, 24000)
self.temp_wav = self.cache_path
+34 -13
View File
@@ -82,11 +82,11 @@ from abogen.constants import (
GITHUB_URL,
PROGRAM_DESCRIPTION,
LANGUAGE_DESCRIPTIONS,
VOICES_INTERNAL,
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
COLORS,
SUBTITLE_FORMATS,
)
from abogen.tts_backend_registry import get_metadata
import threading
from abogen.pyqt.voice_formula_gui import VoiceFormulaDialog
from abogen.voice_profiles import load_profiles
@@ -1873,7 +1873,7 @@ class abogen(QWidget):
for pname in load_profiles().keys():
self.voice_combo.addItem(profile_icon, pname, f"profile:{pname}")
# re-add voices
for v in VOICES_INTERNAL:
for v in get_metadata("kokoro").voices:
icon = QIcon()
flag_path = get_resource_path("abogen.assets.flags", f"{v[0]}.png")
if flag_path and os.path.exists(flag_path):
@@ -2316,9 +2316,9 @@ class abogen(QWidget):
file_size_str = "Unknown"
# pipeline_loaded_callback remains unchanged
def pipeline_loaded_callback(np_module, kpipeline_class, error):
def pipeline_loaded_callback(backend, error):
if error:
self.update_log((f"Error loading numpy or KPipeline: {error}", "red"))
self.update_log((f"Error loading TTS backend: {error}", "red"))
prevent_sleep_end()
return
@@ -2341,8 +2341,7 @@ class abogen(QWidget):
self.selected_output_folder,
subtitle_mode=actual_subtitle_mode,
output_format=self.selected_format,
np_module=np_module,
kpipeline_class=kpipeline_class,
backend=backend,
start_time=self.start_time,
total_char_count=self.char_count,
use_gpu=self.gpu_ok,
@@ -2426,7 +2425,20 @@ class abogen(QWidget):
self.gpu_ok = gpu_ok
self.update_log((gpu_msg, gpu_ok))
self.update_log("Loading modules...")
load_thread = LoadPipelineThread(pipeline_loaded_callback)
# Determine device based on GPU availability
if gpu_ok:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps"
else:
device = "cuda"
else:
device = "cpu"
lang_code = self.selected_lang or "a"
load_thread = LoadPipelineThread(
pipeline_loaded_callback, lang_code=lang_code, device=device
)
load_thread.start()
threading.Thread(target=gpu_and_load, daemon=True).start()
@@ -2863,18 +2875,27 @@ class abogen(QWidget):
)
self.loading_movie.start()
def pipeline_loaded_callback(np_module, kpipeline_class, error):
self._on_pipeline_loaded_for_preview(np_module, kpipeline_class, error)
# Determine device based on GPU availability
if self.gpu_ok:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps"
else:
device = "cuda"
else:
device = "cpu"
load_thread = LoadPipelineThread(pipeline_loaded_callback)
lang = self.selected_lang or "a"
load_thread = LoadPipelineThread(
self._on_pipeline_loaded_for_preview, lang_code=lang, device=device
)
load_thread.start()
def _on_pipeline_loaded_for_preview(self, np_module, kpipeline_class, error):
def _on_pipeline_loaded_for_preview(self, backend, error):
# stop loading animation and restore icon on error
if error:
self.loading_movie.stop()
self._show_error_message_box(
"Loading Error", f"Error loading numpy or KPipeline: {error}"
"Loading Error", f"Error loading TTS backend: {error}"
)
self.btn_preview.setIcon(self.play_icon)
self.btn_preview.setEnabled(True)
@@ -2912,7 +2933,7 @@ class abogen(QWidget):
gpu_msg, gpu_ok = get_gpu_acceleration(self.use_gpu)
self.preview_thread = VoicePreviewThread(
np_module, kpipeline_class, lang, voice, speed, gpu_ok
backend, lang, voice, speed, gpu_ok
)
self.preview_thread.finished.connect(self._play_preview_audio)
self.preview_thread.error.connect(self._preview_error)
+5 -4
View File
@@ -21,7 +21,8 @@ from PyQt6.QtWidgets import (
)
from PyQt6.QtCore import QThread, pyqtSignal
from abogen.constants import COLORS, VOICES_INTERNAL
from abogen.constants import COLORS
from abogen.tts_backend_registry import get_metadata
from abogen.spacy_utils import SPACY_MODELS
import abogen.hf_tracker
@@ -114,7 +115,7 @@ class PreDownloadWorker(QThread):
self._voices_success = False
return
voice_list = VOICES_INTERNAL
voice_list = get_metadata("kokoro").voices
for idx, voice in enumerate(voice_list, start=1):
if self._cancelled:
self._voices_success = False
@@ -462,14 +463,14 @@ class PreDownloadDialog(QDialog):
try:
from huggingface_hub import try_to_load_from_cache
for voice in VOICES_INTERNAL:
for voice in get_metadata("kokoro").voices:
if not try_to_load_from_cache(
repo_id="hexgrad/Kokoro-82M", filename=f"voices/{voice}.pt"
):
missing.append(voice)
except Exception:
# If HF missing, report all as missing
return False, list(VOICES_INTERNAL)
return False, list(get_metadata("kokoro").voices)
return (len(missing) == 0), missing
def _check_kokoro_model(self) -> bool:
+3 -3
View File
@@ -28,11 +28,11 @@ from PyQt6.QtWidgets import (
from PyQt6.QtCore import Qt, QTimer, QPoint, QRect, QSize
from PyQt6.QtGui import QPixmap, QIcon, QAction
from abogen.constants import (
VOICES_INTERNAL,
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
LANGUAGE_DESCRIPTIONS,
COLORS,
)
from abogen.tts_backend_registry import get_metadata
import re
import platform
from abogen.utils import get_resource_path
@@ -179,7 +179,7 @@ class VoiceMixer(QWidget):
layout.addWidget(QLabel(name), alignment=Qt.AlignmentFlag.AlignCenter)
# Voice name label with gender icon
is_female = self.voice_name in VOICES_INTERNAL and self.voice_name[1] == "f"
is_female = self.voice_name in get_metadata("kokoro").voices and self.voice_name[1] == "f"
# Icons layout (flag and gender)
icons_layout = QHBoxLayout()
@@ -772,7 +772,7 @@ class VoiceFormulaDialog(QDialog):
def add_voices(self, initial_state):
first_enabled_voice = None
for voice in VOICES_INTERNAL:
for voice in get_metadata("kokoro").voices:
language_code = voice[0] # First character is the language code
matching_voice = next(
(item for item in initial_state if item[0] == voice), None
+7 -7
View File
@@ -466,7 +466,7 @@ def sanitize_name_for_os(name, is_folder=True):
def validate_voice_name(voice_name):
"""Validate voice name against VOICES_INTERNAL list (case-insensitive).
"""Validate voice name against available voices (case-insensitive).
Handles both single voices and formulas like 'af_heart*0.5 + am_echo*0.5'.
Args:
@@ -477,10 +477,10 @@ def validate_voice_name(voice_name):
- is_valid: True if all voices in the name/formula are valid
- invalid_voice_name: The first invalid voice found, or None if all valid
"""
from abogen.constants import VOICES_INTERNAL
from abogen.tts_backend_registry import get_metadata
# Create case-insensitive lookup set (done once per call)
voice_lookup_lower = {v.lower() for v in VOICES_INTERNAL}
voice_lookup_lower = {v.lower() for v in get_metadata("kokoro").voices}
voice_name = voice_name.strip()
# Check if it's a formula (contains *)
@@ -505,7 +505,7 @@ def split_text_by_voice_markers(text, default_voice):
"""Split text by voice markers, returning list of (voice, text) tuples.
IMPORTANT: Returns the last voice used so it can persist across chapters.
Voice names are normalized to lowercase to match VOICES_INTERNAL.
Voice names are normalized to lowercase to match canonical voice names.
Args:
text: Text potentially containing <<VOICE:name>> markers
@@ -518,7 +518,7 @@ def split_text_by_voice_markers(text, default_voice):
- valid_count: Number of valid voice markers processed
- invalid_count: Number of invalid voice markers skipped
"""
from abogen.constants import VOICES_INTERNAL
from abogen.tts_backend_registry import get_metadata
voice_splits = list(_VOICE_MARKER_SEARCH_PATTERN.finditer(text))
@@ -560,7 +560,7 @@ def split_text_by_voice_markers(text, default_voice):
# Find the canonical (lowercase) voice name
voice_part_lower = voice_part.strip().lower()
canonical_voice = next(
(v for v in VOICES_INTERNAL if v.lower() == voice_part_lower),
(v for v in get_metadata("kokoro").voices if v.lower() == voice_part_lower),
voice_part.strip()
)
normalized_parts.append(f"{canonical_voice}*{weight.strip()}")
@@ -569,7 +569,7 @@ def split_text_by_voice_markers(text, default_voice):
# Find the canonical (lowercase) voice name
voice_name_lower = voice_name.lower()
current_voice = next(
(v for v in VOICES_INTERNAL if v.lower() == voice_name_lower),
(v for v in get_metadata("kokoro").voices if v.lower() == voice_name_lower),
voice_name
)
valid_markers += 1
+89
View File
@@ -0,0 +1,89 @@
"""
TTS Backend Interface
This module defines the protocol for TTS backends and the
metadata model that describes a backend implementation.
"""
from dataclasses import dataclass
from typing import Protocol, List, Dict, Any
@dataclass(frozen=True)
class TTSBackendMetadata:
"""
Immutable metadata describing a TTS backend implementation.
Attributes:
id: Unique backend identifier (e.g. ``"kokoro"``, ``"supertonic"``).
name: Human-readable display name.
description: Short description of the backend.
voices: Tuple of supported voice identifiers.
"""
id: str
name: str
description: str
voices: tuple[str, ...] = ()
class TTSBackend(Protocol):
"""
Protocol for TTS backends.
All TTS backends must implement this interface to be compatible
with the application.
"""
@property
def metadata(self) -> TTSBackendMetadata:
...
def __init__(self, **kwargs) -> None:
"""
Initialize the TTS backend.
Args:
**kwargs: Backend-specific configuration parameters
"""
...
def synthesize(self, text: str, **kwargs) -> bytes:
"""
Synthesize speech from text.
Args:
text: Text to synthesize
**kwargs: Additional parameters for synthesis
Returns:
Audio data as bytes
"""
...
def get_available_voices(self) -> List[str]:
"""
Get list of available voices.
Returns:
List of voice identifiers
"""
...
def get_supported_formats(self) -> List[str]:
"""
Get list of supported audio formats.
Returns:
List of supported audio formats
"""
...
def get_info(self) -> Dict[str, Any]:
"""
Get backend information.
Returns:
Dictionary with backend information
"""
...
+146
View File
@@ -0,0 +1,146 @@
"""
TTS Backend Registry
Provides a global registry for TTS backend factories.
Backends register themselves with metadata and a factory callable.
The registry is universal and does not know about backend constructors.
"""
from typing import Callable, Any
from abogen.tts_backend import TTSBackend, TTSBackendMetadata
class TTSBackendRegistry:
"""Registry of TTS backend factories.
Stores metadata and factory callables for registered backends.
"""
def __init__(self) -> None:
self._backends: dict[str, TTSBackendMetadata] = {}
self._factories: dict[str, Callable[..., TTSBackend]] = {}
def register(
self,
metadata: TTSBackendMetadata,
factory: Callable[..., TTSBackend],
) -> None:
"""Register a backend with its metadata and factory callable."""
self._backends[metadata.id] = metadata
self._factories[metadata.id] = factory
def is_registered(self, backend_id: str) -> bool:
"""Return True if a backend with the given id is registered."""
return backend_id in self._backends
def list_backends(self) -> list[TTSBackendMetadata]:
"""Return metadata for all registered backends."""
return list(self._backends.values())
def get_metadata(self, backend_id: str) -> TTSBackendMetadata:
"""Get metadata for a specific backend.
Raises:
KeyError: If backend with given id is not registered.
"""
if backend_id not in self._backends:
raise KeyError(f"Unknown backend: {backend_id}")
return self._backends[backend_id]
def create_backend(self, backend_id: str, **kwargs: Any) -> TTSBackend:
"""Create a backend instance by id.
Raises:
KeyError: If backend with given id is not registered.
"""
if backend_id not in self._factories:
raise KeyError(f"Unknown backend: {backend_id}")
return self._factories[backend_id](**kwargs)
def resolve_backend_for_voice(
self,
spec: str,
fallback: str = "kokoro",
) -> str:
"""Determine which backend owns the given voice specification.
Resolution rules:
1. Empty spec -> fallback
2. Kokoro formula (contains '*' or '+') -> "kokoro"
3. Exact voice ID match against registered backends -> backend id
4. Unknown voice -> fallback
"""
raw = str(spec or "").strip()
if not raw:
return fallback
if "*" in raw or "+" in raw:
return "kokoro"
upper = raw.upper()
for metadata in self._backends.values():
if upper in metadata.voices:
return metadata.id
return fallback
_registry = TTSBackendRegistry()
def register_backend(
metadata: TTSBackendMetadata,
factory: Callable[..., TTSBackend],
) -> None:
"""Register a TTS backend in the global registry."""
_registry.register(metadata, factory)
def get_metadata(backend_id: str) -> TTSBackendMetadata:
"""Get metadata for a specific backend by id.
Ensures all backends are registered by importing the tts_backends
package on first access.
Raises:
KeyError: If backend with given id is not registered.
"""
import abogen.tts_backends # noqa: F401 — triggers backend registration
return _registry.get_metadata(backend_id)
def get_default_voice(backend_id: str, fallback: str = "") -> str:
"""Return the first voice of a backend, or *fallback* if none."""
voices = get_metadata(backend_id).voices
return voices[0] if voices else fallback
def create_backend(backend_id: str, **kwargs: Any) -> TTSBackend:
"""Create a TTS backend instance by provider id."""
return _registry.create_backend(backend_id, **kwargs)
def is_registered_backend(backend_id: str) -> bool:
"""Return True if *backend_id* is a registered TTS backend."""
import abogen.tts_backends # noqa: F401 — triggers backend registration
return _registry.is_registered(backend_id)
def resolve_backend_for_voice(
spec: str,
fallback: str = "kokoro",
) -> str:
"""Determine which backend owns the given voice specification.
Ensures all backends are registered by importing the tts_backends
package on first access.
Resolution rules:
1. Empty spec -> fallback
2. Kokoro formula (contains '*' or '+') -> "kokoro"
3. Exact voice ID match against registered backends -> backend id
4. Unknown voice -> fallback
"""
import abogen.tts_backends # noqa: F401 — triggers backend registration
return _registry.resolve_backend_for_voice(spec, fallback=fallback)
+20
View File
@@ -0,0 +1,20 @@
"""TTS backends package.
Backend modules are auto-discovered and imported here.
Each backend module registers itself with the global registry
when imported.
"""
import importlib
import pkgutil
def _discover_backends():
"""Import all modules in this package to trigger their registration."""
package = __name__
for _importer, modname, _ispkg in pkgutil.iter_modules(path=__path__):
importlib.import_module(f"{package}.{modname}")
_discover_backends()
+179
View File
@@ -0,0 +1,179 @@
"""
Kokoro TTS Backend
Encapsulates the Kokoro KPipeline as a TTSBackend implementation.
"""
from __future__ import annotations
from typing import Any, Dict, Iterator, List, Optional
import numpy as np
from abogen.tts_backend import TTSBackendMetadata
# Internal voice list — source of truth for Kokoro voices.
# The rest of the project accesses voices via get_metadata("kokoro").voices.
_VOICES_INTERNAL = [
"af_alloy",
"af_aoede",
"af_bella",
"af_heart",
"af_jessica",
"af_kore",
"af_nicole",
"af_nova",
"af_river",
"af_sarah",
"af_sky",
"am_adam",
"am_echo",
"am_eric",
"am_fenrir",
"am_liam",
"am_michael",
"am_onyx",
"am_puck",
"am_santa",
"bf_alice",
"bf_emma",
"bf_isabella",
"bf_lily",
"bm_daniel",
"bm_fable",
"bm_george",
"bm_lewis",
"ef_dora",
"em_alex",
"em_santa",
"ff_siwis",
"hf_alpha",
"hf_beta",
"hm_omega",
"hm_psi",
"if_sara",
"im_nicola",
"jf_alpha",
"jf_gongitsune",
"jf_nezumi",
"jf_tebukuro",
"jm_kumo",
"pf_dora",
"pm_alex",
"pm_santa",
"zf_xiaobei",
"zf_xiaoni",
"zf_xiaoxiao",
"zf_xiaoyi",
"zm_yunjian",
"zm_yunxi",
"zm_yunxia",
"zm_yunyang",
]
_KOKORO_METADATA = TTSBackendMetadata(
id="kokoro",
name="Kokoro",
description="Kokoro TTS engine",
voices=tuple(_VOICES_INTERNAL),
)
def _load_kpipeline():
"""Lazy-load Kokoro dependencies."""
from kokoro import KPipeline # type: ignore[import-not-found]
return KPipeline
class KokoroBackend:
"""TTSBackend implementation wrapping the Kokoro KPipeline.
All interaction with KPipeline is encapsulated here.
The rest of the project depends only on this class.
"""
def __init__(self, **kwargs: Any) -> None:
lang_code = kwargs["lang_code"]
repo_id = kwargs.get("repo_id", "hexgrad/Kokoro-82M")
device = kwargs.get("device", "cpu")
KPipeline = _load_kpipeline()
self._pipeline = KPipeline(
lang_code=lang_code,
repo_id=repo_id,
device=device,
)
self._lang_code = lang_code
@property
def metadata(self) -> TTSBackendMetadata:
return _KOKORO_METADATA
def __call__(
self,
text: str,
*,
voice: Any,
speed: float = 1.0,
split_pattern: Optional[str] = None,
) -> Iterator[Any]:
"""Delegate to KPipeline's __call__."""
return self._pipeline(
text,
voice=voice,
speed=speed,
split_pattern=split_pattern,
)
def load_single_voice(self, voice_name: str) -> Any:
"""Load a single voice tensor. Used by voice formula system."""
return self._pipeline.load_single_voice(voice_name)
def synthesize(self, text: str, **kwargs: Any) -> bytes:
"""Synthesize speech from text. Returns raw audio bytes."""
voice = kwargs.get("voice", "")
speed = kwargs.get("speed", 1.0)
split_pattern = kwargs.get("split_pattern", None)
audio_parts: list[np.ndarray] = []
for segment in self(text, voice=voice, speed=speed, split_pattern=split_pattern):
audio = segment.audio
if hasattr(audio, "numpy"):
audio = audio.numpy()
audio_parts.append(np.asarray(audio, dtype="float32"))
if not audio_parts:
return b""
combined = np.concatenate(audio_parts).astype("float32", copy=False)
return combined.tobytes()
def get_available_voices(self) -> List[str]:
"""Return known Kokoro voice identifiers."""
return list(self.metadata.voices)
def get_supported_formats(self) -> List[str]:
"""Kokoro outputs raw PCM float32 audio."""
return ["pcm_float32"]
def get_info(self) -> Dict[str, Any]:
return {
"id": "kokoro",
"name": "Kokoro",
"lang_code": self._lang_code,
}
def create_kokoro_backend(**kwargs: Any) -> KokoroBackend:
"""Factory callable registered with TTSBackendRegistry."""
return KokoroBackend(**kwargs)
# --- Registration ---
from abogen.tts_backend_registry import register_backend # noqa: E402
register_backend(
metadata=_KOKORO_METADATA,
factory=create_kokoro_backend,
)
@@ -5,7 +5,7 @@ from dataclasses import dataclass
import logging
import math
import re
from typing import Any, Iterable, Iterator, Optional
from typing import Any, Dict, Iterable, Iterator, List, Optional
import numpy as np
@@ -15,6 +15,15 @@ logger = logging.getLogger(__name__)
DEFAULT_SUPERTONIC_VOICES = ("M1", "M2", "M3", "M4", "M5", "F1", "F2", "F3", "F4", "F5")
from abogen.tts_backend import TTSBackendMetadata
_SUPERTONIC_METADATA = TTSBackendMetadata(
id="supertonic",
name="SuperTonic",
description="SuperTonic TTS engine",
voices=DEFAULT_SUPERTONIC_VOICES,
)
@dataclass
class SupertonicSegment:
@@ -273,3 +282,111 @@ class SupertonicPipeline:
audio = _resample_linear(audio, src_rate, self.sample_rate)
yield SupertonicSegment(graphemes=chunk_to_speak, audio=audio)
class SupertonicBackend:
"""Supertonic TTS backend implementing the TTSBackend protocol.
Encapsulates ``SupertonicPipeline`` as an internal implementation detail.
"""
@property
def metadata(self) -> TTSBackendMetadata:
return _SUPERTONIC_METADATA
def __init__(self, **kwargs: Any) -> None:
self._pipeline = SupertonicPipeline(
sample_rate=kwargs.get("sample_rate", 24000),
auto_download=kwargs.get("auto_download", True),
total_steps=kwargs.get("total_steps", 5),
)
def synthesize(self, text: str, **kwargs: Any) -> bytes:
"""Synthesize speech and return raw audio bytes (WAV).
Delegates to the internal :class:`SupertonicPipeline` and concatenates
all produced segments into a single byte buffer.
"""
import io
import soundfile as sf
voice = kwargs.get("voice", "M1")
speed = float(kwargs.get("speed", 1.0))
split_pattern = kwargs.get("split_pattern")
total_steps = kwargs.get("total_steps")
segments = self._pipeline(
text,
voice=voice,
speed=speed,
split_pattern=split_pattern,
total_steps=total_steps,
)
audio_parts: list[np.ndarray] = []
for seg in segments:
audio_parts.append(seg.audio)
if not audio_parts:
return b""
combined = np.concatenate(audio_parts)
buf = io.BytesIO()
sf.write(buf, combined, self._pipeline.sample_rate, format="WAV")
return buf.getvalue()
def get_available_voices(self) -> List[str]:
"""Return the list of built-in SuperTonic voice identifiers."""
return list(self.metadata.voices)
def get_supported_formats(self) -> List[str]:
return ["wav"]
def get_info(self) -> Dict[str, Any]:
return {
"sample_rate": self._pipeline.sample_rate,
"total_steps": self._pipeline.total_steps,
"max_chunk_length": self._pipeline.max_chunk_length,
"voices": list(DEFAULT_SUPERTONIC_VOICES),
}
def __call__(
self,
text: str,
*,
voice: str,
speed: float,
split_pattern: Optional[str] = None,
total_steps: Optional[int] = None,
) -> Iterator[SupertonicSegment]:
"""Backward-compatible call interface, delegates to the pipeline."""
return self._pipeline(
text,
voice=voice,
speed=speed,
split_pattern=split_pattern,
total_steps=total_steps,
)
def create_supertonic_backend(**kwargs: Any) -> SupertonicBackend:
"""Create a SuperTonic TTS backend instance.
Args:
sample_rate: Audio sample rate. Defaults to 24000.
auto_download: Auto-download models. Defaults to True.
total_steps: Inference steps. Defaults to 5.
Returns:
SupertonicBackend instance.
"""
return SupertonicBackend(**kwargs)
from abogen.tts_backend_registry import register_backend # noqa: E402
register_backend(
metadata=_SUPERTONIC_METADATA,
factory=create_supertonic_backend,
)
+10 -11
View File
@@ -529,21 +529,20 @@ def prevent_sleep_end():
_sleep_procs[system] = None
def load_numpy_kpipeline():
import numpy as np
from kokoro import KPipeline # type: ignore[import-not-found]
return np, KPipeline
class LoadPipelineThread(Thread):
def __init__(self, callback):
def __init__(self, callback, lang_code="a", device="cpu"):
super().__init__()
self.callback = callback
self.lang_code = lang_code
self.device = device
def run(self):
try:
np_module, kpipeline_class = load_numpy_kpipeline()
self.callback(np_module, kpipeline_class, None)
from abogen.tts_backend_registry import create_backend
backend = create_backend(
"kokoro", lang_code=self.lang_code, device=self.device
)
self.callback(backend, None)
except Exception as e:
self.callback(None, None, str(e))
self.callback(None, str(e))
+4 -3
View File
@@ -17,7 +17,7 @@ if LocalEntryNotFoundError is None: # pragma: no cover - fallback for tests
pass
from abogen.constants import VOICES_INTERNAL
from abogen.tts_backend_registry import get_metadata
_CACHE_LOCK = threading.Lock()
_CACHED_VOICES: Set[str] = set()
@@ -26,8 +26,9 @@ _BOOTSTRAPPED = False
def _normalize_targets(voices: Optional[Iterable[str]]) -> Set[str]:
kokoro_voices = get_metadata("kokoro").voices
if not voices:
return set(VOICES_INTERNAL)
return set(kokoro_voices)
normalized: Set[str] = set()
for voice in voices:
if not voice:
@@ -35,7 +36,7 @@ def _normalize_targets(voices: Optional[Iterable[str]]) -> Set[str]:
voice_id = str(voice).strip()
if not voice_id:
continue
if voice_id in VOICES_INTERNAL:
if voice_id in kokoro_voices:
normalized.add(voice_id)
return normalized
+3 -2
View File
@@ -1,7 +1,7 @@
import re
from typing import List, Tuple
from abogen.constants import VOICES_INTERNAL
from abogen.tts_backend_registry import get_metadata
# Calls parsing and loads the voice to gpu or cpu
@@ -22,6 +22,7 @@ def parse_formula_terms(formula: str) -> List[Tuple[str, float]]:
raise ValueError("Empty voice formula")
terms: List[Tuple[str, float]] = []
kokoro_voices = get_metadata("kokoro").voices
for segment in formula.split("+"):
part = segment.strip()
if not part:
@@ -30,7 +31,7 @@ def parse_formula_terms(formula: str) -> List[Tuple[str, float]]:
raise ValueError("Each component must be in the form voice*weight")
voice_name, raw_weight = part.split("*", 1)
voice_name = voice_name.strip()
if voice_name not in VOICES_INTERNAL:
if voice_name not in kokoro_voices:
raise ValueError(f"Unknown voice: {voice_name}")
try:
weight = float(raw_weight.strip())
+33
View File
@@ -0,0 +1,33 @@
from dataclasses import dataclass
@dataclass(frozen=True)
class VoiceMetadata:
"""
Immutable metadata describing a voice from a TTS backend.
This model describes a voice independently of any backend implementation.
Backends populate these objects; the application consumes them.
The ``backend_id`` field is set by the backend itself (via
``self.metadata.id``) the application never hardcodes it.
This ensures renaming a backend does not require touching voice definitions.
"""
id: str
"""Unique voice identifier within the backend (e.g. ``"af_alloy"``, ``"M1"``)."""
display_name: str
"""Human-readable display name (e.g. ``"Alloy"``, ``"Male 1"``)."""
language: str
"""Language code — backend-specific format is acceptable (e.g. ``"a"``, ``"en"``)."""
gender: str
"""Gender category: ``"female"``, ``"male"``, or ``"unknown"``."""
backend_id: str
"""Identifier of the backend that owns this voice (e.g. ``"kokoro"``).
Set automatically by the backend never hardcoded in voice definitions.
"""
+230 -229
View File
@@ -1,229 +1,230 @@
import json
import os
from typing import Any, Dict, Iterable, List, Tuple
from abogen.constants import VOICES_INTERNAL
from abogen.tts_supertonic import DEFAULT_SUPERTONIC_VOICES
from abogen.utils import get_user_config_path
def _get_profiles_path():
config_path = get_user_config_path()
config_dir = os.path.dirname(config_path)
return os.path.join(config_dir, "voice_profiles.json")
def load_profiles():
"""Load all voice profiles from JSON file."""
path = _get_profiles_path()
if os.path.exists(path):
try:
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
# always expect abogen_voice_profiles wrapper
if isinstance(data, dict) and "abogen_voice_profiles" in data:
return data["abogen_voice_profiles"]
# fallback: treat as profiles dict
if isinstance(data, dict):
return data
except Exception:
return {}
return {}
def save_profiles(profiles):
"""Save all voice profiles to JSON file."""
path = _get_profiles_path()
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
# always save with abogen_voice_profiles wrapper
json.dump({"abogen_voice_profiles": profiles}, f, indent=2)
def delete_profile(name):
"""Remove a profile by name."""
profiles = load_profiles()
if name in profiles:
del profiles[name]
save_profiles(profiles)
def duplicate_profile(src, dest):
"""Duplicate an existing profile."""
profiles = load_profiles()
if src in profiles and dest:
profiles[dest] = profiles[src]
save_profiles(profiles)
def export_profiles(export_path):
"""Export all profiles to specified JSON file."""
profiles = load_profiles()
with open(export_path, "w", encoding="utf-8") as f:
json.dump({"abogen_voice_profiles": profiles}, f, indent=2)
def serialize_profiles() -> Dict[str, Dict[str, Iterable[Tuple[str, float]]]]:
"""Return profiles in canonical dictionary form."""
return load_profiles()
def _normalize_supertonic_voice(value: Any) -> str:
raw = str(value or "").strip().upper()
return raw if raw in DEFAULT_SUPERTONIC_VOICES else "M1"
def _coerce_supertonic_steps(value: Any) -> int:
try:
steps = int(value)
except (TypeError, ValueError):
return 5
return max(2, min(15, steps))
def _coerce_supertonic_speed(value: Any) -> float:
try:
speed = float(value)
except (TypeError, ValueError):
return 1.0
return max(0.7, min(2.0, speed))
def normalize_profile_entry(entry: Any) -> Dict[str, Any]:
"""Normalize a stored profile entry.
Backwards compatible:
- Legacy Kokoro-only entries: {language, voices}
- New entries: include provider.
"""
if not isinstance(entry, dict):
return {}
provider = str(entry.get("provider") or "kokoro").strip().lower()
if provider not in {"kokoro", "supertonic"}:
provider = "kokoro"
language = str(entry.get("language") or "a").strip().lower() or "a"
if provider == "supertonic":
return {
"provider": "supertonic",
"language": language,
"voice": _normalize_supertonic_voice(
entry.get("voice") or entry.get("voice_name") or entry.get("name")
),
"total_steps": _coerce_supertonic_steps(
entry.get("total_steps")
or entry.get("supertonic_total_steps")
or entry.get("quality")
),
"speed": _coerce_supertonic_speed(
entry.get("speed") or entry.get("supertonic_speed")
),
}
voices = _normalize_voice_entries(entry.get("voices", []))
if not voices:
return {}
return {
"provider": "kokoro",
"language": language,
"voices": voices,
}
def _normalize_voice_entries(entries: Iterable) -> List[Tuple[str, float]]:
normalized: List[Tuple[str, float]] = []
for item in entries or []:
if isinstance(item, dict):
voice = item.get("id") or item.get("voice")
weight = item.get("weight")
elif isinstance(item, (list, tuple)) and len(item) >= 2:
voice, weight = item[0], item[1]
else:
continue
if voice not in VOICES_INTERNAL:
continue
if weight is None:
continue
try:
weight_val = float(weight)
except (TypeError, ValueError):
continue
if weight_val <= 0:
continue
normalized.append((voice, weight_val))
return normalized
def normalize_voice_entries(entries: Iterable) -> List[Tuple[str, float]]:
"""Public helper to normalize voice-weight pairs from arbitrary payloads."""
return _normalize_voice_entries(entries)
def save_profile(name: str, *, language: str, voices: Iterable) -> None:
"""Persist a single profile after validating its data."""
name = (name or "").strip()
if not name:
raise ValueError("Profile name is required")
normalized = _normalize_voice_entries(voices)
if not normalized:
raise ValueError("At least one voice with a weight above zero is required")
if not language:
language = "a"
profiles = load_profiles()
profiles[name] = {"provider": "kokoro", "language": language, "voices": normalized}
save_profiles(profiles)
def remove_profile(name: str) -> None:
delete_profile(name)
def import_profiles_data(data: Dict, *, replace_existing: bool = False) -> List[str]:
"""Merge profiles from a dictionary structure and persist them.
Returns the list of profile names that were added or updated.
"""
if not isinstance(data, dict):
raise ValueError("Invalid profile payload")
if "abogen_voice_profiles" in data:
data = data["abogen_voice_profiles"]
if not isinstance(data, dict):
raise ValueError("Invalid profile payload")
current = load_profiles()
updated: List[str] = []
for name, entry in data.items():
normalized = normalize_profile_entry(entry)
if not normalized:
continue
if name in current and not replace_existing:
# skip duplicates unless explicit replacement is requested
continue
current[name] = normalized
updated.append(name)
if updated:
save_profiles(current)
return updated
def export_profiles_payload(names: Iterable[str] | None = None) -> Dict[str, Dict]:
"""Return profiles limited to the provided names for download/export."""
profiles = load_profiles()
if names is None:
subset = profiles
else:
subset = {name: profiles[name] for name in names if name in profiles}
return {"abogen_voice_profiles": subset}
import json
import os
from typing import Any, Dict, Iterable, List, Tuple
from abogen.tts_backend_registry import get_metadata, is_registered_backend
from abogen.utils import get_user_config_path
def _get_profiles_path():
config_path = get_user_config_path()
config_dir = os.path.dirname(config_path)
return os.path.join(config_dir, "voice_profiles.json")
def load_profiles():
"""Load all voice profiles from JSON file."""
path = _get_profiles_path()
if os.path.exists(path):
try:
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
# always expect abogen_voice_profiles wrapper
if isinstance(data, dict) and "abogen_voice_profiles" in data:
return data["abogen_voice_profiles"]
# fallback: treat as profiles dict
if isinstance(data, dict):
return data
except Exception:
return {}
return {}
def save_profiles(profiles):
"""Save all voice profiles to JSON file."""
path = _get_profiles_path()
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
# always save with abogen_voice_profiles wrapper
json.dump({"abogen_voice_profiles": profiles}, f, indent=2)
def delete_profile(name):
"""Remove a profile by name."""
profiles = load_profiles()
if name in profiles:
del profiles[name]
save_profiles(profiles)
def duplicate_profile(src, dest):
"""Duplicate an existing profile."""
profiles = load_profiles()
if src in profiles and dest:
profiles[dest] = profiles[src]
save_profiles(profiles)
def export_profiles(export_path):
"""Export all profiles to specified JSON file."""
profiles = load_profiles()
with open(export_path, "w", encoding="utf-8") as f:
json.dump({"abogen_voice_profiles": profiles}, f, indent=2)
def serialize_profiles() -> Dict[str, Dict[str, Iterable[Tuple[str, float]]]]:
"""Return profiles in canonical dictionary form."""
return load_profiles()
def _normalize_supertonic_voice(value: Any) -> str:
raw = str(value or "").strip().upper()
supertonic_voices = get_metadata("supertonic").voices
return raw if raw in supertonic_voices else "M1"
def _coerce_supertonic_steps(value: Any) -> int:
try:
steps = int(value)
except (TypeError, ValueError):
return 5
return max(2, min(15, steps))
def _coerce_supertonic_speed(value: Any) -> float:
try:
speed = float(value)
except (TypeError, ValueError):
return 1.0
return max(0.7, min(2.0, speed))
def normalize_profile_entry(entry: Any) -> Dict[str, Any]:
"""Normalize a stored profile entry.
Backwards compatible:
- Legacy Kokoro-only entries: {language, voices}
- New entries: include provider.
"""
if not isinstance(entry, dict):
return {}
provider = str(entry.get("provider") or "kokoro").strip().lower()
if not is_registered_backend(provider):
provider = "kokoro"
language = str(entry.get("language") or "a").strip().lower() or "a"
if provider == "supertonic":
return {
"provider": "supertonic",
"language": language,
"voice": _normalize_supertonic_voice(
entry.get("voice") or entry.get("voice_name") or entry.get("name")
),
"total_steps": _coerce_supertonic_steps(
entry.get("total_steps")
or entry.get("supertonic_total_steps")
or entry.get("quality")
),
"speed": _coerce_supertonic_speed(
entry.get("speed") or entry.get("supertonic_speed")
),
}
voices = _normalize_voice_entries(entry.get("voices", []))
if not voices:
return {}
return {
"provider": "kokoro",
"language": language,
"voices": voices,
}
def _normalize_voice_entries(entries: Iterable) -> List[Tuple[str, float]]:
normalized: List[Tuple[str, float]] = []
kokoro_voices = get_metadata("kokoro").voices
for item in entries or []:
if isinstance(item, dict):
voice = item.get("id") or item.get("voice")
weight = item.get("weight")
elif isinstance(item, (list, tuple)) and len(item) >= 2:
voice, weight = item[0], item[1]
else:
continue
if voice not in kokoro_voices:
continue
if weight is None:
continue
try:
weight_val = float(weight)
except (TypeError, ValueError):
continue
if weight_val <= 0:
continue
normalized.append((voice, weight_val))
return normalized
def normalize_voice_entries(entries: Iterable) -> List[Tuple[str, float]]:
"""Public helper to normalize voice-weight pairs from arbitrary payloads."""
return _normalize_voice_entries(entries)
def save_profile(name: str, *, language: str, voices: Iterable) -> None:
"""Persist a single profile after validating its data."""
name = (name or "").strip()
if not name:
raise ValueError("Profile name is required")
normalized = _normalize_voice_entries(voices)
if not normalized:
raise ValueError("At least one voice with a weight above zero is required")
if not language:
language = "a"
profiles = load_profiles()
profiles[name] = {"provider": "kokoro", "language": language, "voices": normalized}
save_profiles(profiles)
def remove_profile(name: str) -> None:
delete_profile(name)
def import_profiles_data(data: Dict, *, replace_existing: bool = False) -> List[str]:
"""Merge profiles from a dictionary structure and persist them.
Returns the list of profile names that were added or updated.
"""
if not isinstance(data, dict):
raise ValueError("Invalid profile payload")
if "abogen_voice_profiles" in data:
data = data["abogen_voice_profiles"]
if not isinstance(data, dict):
raise ValueError("Invalid profile payload")
current = load_profiles()
updated: List[str] = []
for name, entry in data.items():
normalized = normalize_profile_entry(entry)
if not normalized:
continue
if name in current and not replace_existing:
# skip duplicates unless explicit replacement is requested
continue
current[name] = normalized
updated.append(name)
if updated:
save_profiles(current)
return updated
def export_profiles_payload(names: Iterable[str] | None = None) -> Dict[str, Dict]:
"""Return profiles limited to the provided names for download/export."""
profiles = load_profiles()
if names is None:
subset = profiles
else:
subset = {name: profiles[name] for name in names if name in profiles}
return {"abogen_voice_profiles": subset}
+8 -9
View File
@@ -2,7 +2,6 @@ FROM nvidia/cuda:12.6.3-cudnn-runtime-ubuntu22.04
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \
VIRTUAL_ENV=/opt/venv \
PATH=/opt/venv/bin:$PATH
@@ -27,22 +26,22 @@ RUN python3 -m venv "$VIRTUAL_ENV"
WORKDIR /app
COPY pyproject.toml README.md ./
COPY abogen ./abogen
RUN pip install --upgrade pip \
RUN pip install uv \
&& if [ -n "$TORCH_VERSION" ]; then \
pip install torch=="$TORCH_VERSION" torchvision=="$TORCH_VERSION" torchaudio=="$TORCH_VERSION" --index-url "$TORCH_INDEX_URL"; \
uv pip install --system torch=="$TORCH_VERSION" torchvision=="$TORCH_VERSION" torchaudio=="$TORCH_VERSION" --index-url "$TORCH_INDEX_URL"; \
else \
pip install torch torchvision torchaudio --index-url "$TORCH_INDEX_URL"; \
uv pip install --system torch torchvision torchaudio --index-url "$TORCH_INDEX_URL"; \
fi \
&& pip install --no-cache-dir . \
&& uv pip install --system . \
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl \
&& pip install --no-cache-dir "mutagen>=1.47.0"
&& uv pip install --system "mutagen>=1.47.0"
COPY abogen ./abogen
# Install onnxruntime-gpu for CUDA acceleration (supertonic uses ONNX Runtime)
# Set USE_GPU=false to skip this for CPU-only deployments
RUN if [ "$USE_GPU" = "true" ]; then \
pip install --no-cache-dir onnxruntime-gpu; \
uv pip install --system onnxruntime-gpu; \
fi
ENV ABOGEN_HOST=0.0.0.0 \
+47 -56
View File
@@ -20,7 +20,7 @@ import numpy as np
import soundfile as sf
import static_ffmpeg
from abogen.constants import VOICES_INTERNAL
from abogen.tts_backend_registry import get_metadata, is_registered_backend, resolve_backend_for_voice
from abogen.epub3.exporter import build_epub3_package
from abogen.kokoro_text_normalization import ApostropheConfig, normalize_for_pipeline, HAS_NUM2WORDS
from abogen.normalization_settings import (
@@ -39,14 +39,15 @@ from abogen.utils import (
get_user_cache_path,
get_user_output_path,
load_config,
load_numpy_kpipeline,
)
from abogen.tts_backend_registry import create_backend
from abogen.tts_backend import TTSBackend
from abogen.voice_cache import ensure_voice_assets
from abogen.voice_formulas import extract_voice_ids, get_new_voice
from abogen.voice_profiles import load_profiles, normalize_profile_entry
from abogen.pronunciation_store import increment_usage
from abogen.llm_client import LLMClientError
from abogen.tts_supertonic import DEFAULT_SUPERTONIC_VOICES, SupertonicPipeline
from .service import Job, JobStatus
@@ -56,25 +57,26 @@ SAMPLE_RATE = 24000
def _supertonic_voice_from_spec(spec: Any, fallback: str) -> str:
"""Normalize a voice specification for Supertonic.
This function only performs Supertonic-specific normalization (uppercase conversion
and fallback handling). Backend resolution is handled by the registry.
"""
raw = str(spec or "").strip()
fallback_raw = str(fallback or "").strip()
# SuperTonic voices are discrete IDs (M1/F3/...). If we see a Kokoro mix
# formula (contains '*' or '+'), ignore it and fall back to a safe voice.
if not raw or "*" in raw or "+" in raw:
raw = fallback_raw
if not raw or "*" in raw or "+" in raw:
raw = "M1"
# Normalize to uppercase for Supertonic voice IDs
upper = raw.upper() if raw else ""
upper = raw.upper()
if upper in DEFAULT_SUPERTONIC_VOICES:
return upper
# If empty or contains formula characters, use fallback
if not upper or "*" in upper or "+" in upper:
upper = fallback_raw.upper() if fallback_raw else ""
fallback_upper = fallback_raw.upper() if fallback_raw else ""
if fallback_upper in DEFAULT_SUPERTONIC_VOICES:
return fallback_upper
# If still empty, use default Supertonic voice
if not upper or "*" in upper or "+" in upper:
upper = "M1"
return "M1"
return upper
def _split_speaker_reference(value: Any) -> tuple[Optional[str], str]:
@@ -118,15 +120,7 @@ def _formula_from_kokoro_entry(entry: Mapping[str, Any]) -> str:
def _infer_provider_from_spec(value: Any, fallback: str = "kokoro") -> str:
raw = str(value or "").strip()
if not raw:
return fallback
upper = raw.upper()
if upper in DEFAULT_SUPERTONIC_VOICES:
return "supertonic"
if "*" in raw or "+" in raw:
return "kokoro"
return fallback
return resolve_backend_for_voice(str(value or ""), fallback=fallback)
class _JobCancelled(Exception):
@@ -575,7 +569,7 @@ def _spec_to_voice_ids(spec: Any) -> Set[str]:
return set(extract_voice_ids(text))
except ValueError:
return set()
if text in VOICES_INTERNAL:
if text in get_metadata("kokoro").voices:
return {text}
return set()
@@ -639,7 +633,7 @@ def _collect_required_voice_ids(job: Job) -> Set[str]:
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(_spec_to_voice_ids(payload.get(key)))
voices.update(VOICES_INTERNAL)
voices.update(get_metadata("kokoro").voices)
return voices
@@ -1573,7 +1567,7 @@ def run_conversion_job(job: Job) -> None:
def get_pipeline(provider: str) -> Any:
nonlocal kokoro_cache_ready
provider_norm = str(provider or "kokoro").strip().lower() or "kokoro"
if provider_norm not in {"kokoro", "supertonic"}:
if not is_registered_backend(provider_norm):
provider_norm = "kokoro"
existing = pipelines.get(provider_norm)
@@ -1581,7 +1575,8 @@ def run_conversion_job(job: Job) -> None:
return existing
if provider_norm == "supertonic":
pipelines[provider_norm] = SupertonicPipeline(
pipelines[provider_norm] = create_backend(
"supertonic",
sample_rate=SAMPLE_RATE,
auto_download=True,
total_steps=int(getattr(job, "supertonic_total_steps", 5) or 5),
@@ -1594,16 +1589,12 @@ def run_conversion_job(job: Job) -> None:
device = "cpu"
if not disable_gpu:
device = _select_device()
_np, KPipeline = load_numpy_kpipeline()
# Try to initialize with the selected device; fall back to CPU if CUDA fails
try:
pipelines[provider_norm] = KPipeline(lang_code=job.language, repo_id="hexgrad/Kokoro-82M", device=device)
except RuntimeError as e:
if "CUDA" in str(e) and device != "cpu":
job.add_log(f"CUDA initialization failed, falling back to CPU: {e}", level="warning")
pipelines[provider_norm] = KPipeline(lang_code=job.language, repo_id="hexgrad/Kokoro-82M", device="cpu")
else:
raise
# Create KPipeline instance directly (conforms to TTSBackend protocol)
pipelines[provider_norm] = create_backend(
"kokoro",
lang_code=job.language,
device=device
)
if not kokoro_cache_ready:
_initialize_voice_cache(job)
kokoro_cache_ready = True
@@ -1644,8 +1635,8 @@ def run_conversion_job(job: Job) -> None:
return provider, resolved, cached, speed, steps
if provider == "kokoro":
kokoro_pipeline = get_pipeline("kokoro")
choice = _resolve_voice(kokoro_pipeline, resolved, job.use_gpu)
kokoro_backend = get_pipeline("kokoro")
choice = _resolve_voice(kokoro_backend, resolved, job.use_gpu)
else:
choice = resolved
@@ -1774,8 +1765,8 @@ def run_conversion_job(job: Job) -> None:
voice_cache: Dict[str, Any] = {}
base_provider, base_voice_resolved, _, _ = resolve_voice_target(base_voice_spec)
if base_provider == "kokoro" and base_voice_resolved and "*" not in base_voice_resolved:
kokoro_pipeline = get_pipeline("kokoro")
voice_cache[f"kokoro:{base_voice_resolved}"] = _resolve_voice(kokoro_pipeline, base_voice_resolved, job.use_gpu)
kokoro_backend = get_pipeline("kokoro")
voice_cache[f"kokoro:{base_voice_resolved}"] = _resolve_voice(kokoro_backend, base_voice_resolved, job.use_gpu)
processed_chars = 0
subtitle_index = 1
current_time = 0.0
@@ -1805,8 +1796,8 @@ def run_conversion_job(job: Job) -> None:
fallback_key = next(iter(voice_cache.keys()), "")
if fallback_key and fallback_key != "__custom_mix":
intro_voice_spec = fallback_key.split(":", 1)[-1]
if not intro_voice_spec and VOICES_INTERNAL:
intro_voice_spec = VOICES_INTERNAL[0]
if not intro_voice_spec:
intro_voice_spec = get_default_voice("kokoro")
if intro_voice_spec:
intro_provider, _, intro_voice_choice, intro_speed, intro_steps = resolve_voice_choice(
@@ -1860,8 +1851,8 @@ def run_conversion_job(job: Job) -> None:
total_steps=int(supertonic_steps_override if supertonic_steps_override is not None else getattr(job, "supertonic_total_steps", 5)),
)
else:
kokoro_pipeline = get_pipeline("kokoro")
segment_iter = kokoro_pipeline(
kokoro_backend = get_pipeline("kokoro")
segment_iter = kokoro_backend(
normalized,
voice=voice_choice,
speed=float(speed_override if speed_override is not None else job.speed),
@@ -1950,8 +1941,8 @@ def run_conversion_job(job: Job) -> None:
if chapter_provider == "kokoro":
voice_choice = voice_cache.get(chapter_cache_key)
if voice_choice is None:
kokoro_pipeline = get_pipeline("kokoro")
voice_choice = _resolve_voice(kokoro_pipeline, chapter_voice_resolved, job.use_gpu)
kokoro_backend = get_pipeline("kokoro")
voice_choice = _resolve_voice(kokoro_backend, chapter_voice_resolved, job.use_gpu)
voice_cache[chapter_cache_key] = voice_choice
else:
voice_choice = chapter_voice_resolved
@@ -2095,9 +2086,9 @@ def run_conversion_job(job: Job) -> None:
if chunk_provider == "kokoro":
chunk_voice_choice = voice_cache.get(chunk_cache_key)
if chunk_voice_choice is None:
kokoro_pipeline = get_pipeline("kokoro")
kokoro_backend = get_pipeline("kokoro")
chunk_voice_choice = _resolve_voice(
kokoro_pipeline,
kokoro_backend,
chunk_voice_resolved,
job.use_gpu,
)
@@ -2239,8 +2230,8 @@ def run_conversion_job(job: Job) -> None:
if fallback_key and fallback_key != "__custom_mix":
# `voice_cache` keys are internal and include provider prefixes.
outro_voice_spec = fallback_key.split(":", 1)[-1]
if not outro_voice_spec and VOICES_INTERNAL:
outro_voice_spec = VOICES_INTERNAL[0]
if not outro_voice_spec:
outro_voice_spec = get_default_voice("kokoro")
if outro_text and outro_voice_spec:
outro_start_time = current_time
@@ -2445,7 +2436,8 @@ def _load_pipeline(job: Job):
disable_gpu = not job.use_gpu or not cfg.get("use_gpu", True)
provider = str(getattr(job, "tts_provider", "kokoro") or "kokoro").strip().lower()
if provider == "supertonic":
return SupertonicPipeline(
return create_backend(
"supertonic",
sample_rate=SAMPLE_RATE,
auto_download=True,
total_steps=int(getattr(job, "supertonic_total_steps", 5) or 5),
@@ -2454,8 +2446,7 @@ def _load_pipeline(job: Job):
device = "cpu"
if not disable_gpu:
device = _select_device()
_np, KPipeline = load_numpy_kpipeline()
return KPipeline(lang_code=job.language, repo_id="hexgrad/Kokoro-82M", device=device)
return create_backend("kokoro", lang_code=job.language, device=device)
def _select_device() -> str:
+2 -3
View File
@@ -15,7 +15,7 @@ from abogen.normalization_settings import build_apostrophe_config
from abogen.text_extractor import extract_from_path
from abogen.voice_cache import ensure_voice_assets
from abogen.webui.conversion_runner import SAMPLE_RATE, SPLIT_PATTERN, _select_device, _to_float32, _resolve_voice, _spec_to_voice_ids
from abogen.utils import load_numpy_kpipeline
from abogen.tts_backend_registry import create_backend
_MARKER_RE = re.compile(re.escape(MARKER_PREFIX) + r"(?P<code>[A-Z0-9_]+)" + re.escape(MARKER_SUFFIX))
@@ -45,8 +45,7 @@ def _load_pipeline(language: str, use_gpu: bool) -> Any:
device = "cpu"
if use_gpu:
device = _select_device()
_np, KPipeline = load_numpy_kpipeline()
return KPipeline(lang_code=language, repo_id="hexgrad/Kokoro-82M", device=device)
return create_backend("kokoro", lang_code=language, device=device)
def _extract_cases_from_text(text: str) -> List[Tuple[str, str]]:
+3 -2
View File
@@ -34,6 +34,7 @@ from abogen.normalization_settings import (
)
from abogen.llm_client import list_models, LLMClientError
from abogen.kokoro_text_normalization import normalize_for_pipeline
from abogen.tts_backend_registry import is_registered_backend
from abogen.integrations.audiobookshelf import AudiobookshelfClient, AudiobookshelfConfig
from abogen.integrations.calibre_opds import (
CalibreOPDSClient,
@@ -63,7 +64,7 @@ def api_save_voice_profile() -> ResponseReturnValue:
if profile is None:
# Speaker Studio payload format
provider = str(payload.get("provider") or "kokoro").strip().lower()
if provider not in {"kokoro", "supertonic"}:
if not is_registered_backend(provider):
provider = "kokoro"
if provider == "supertonic":
profile = {
@@ -230,7 +231,7 @@ def api_speaker_preview() -> ResponseReturnValue:
use_gpu = settings.get("use_gpu", False)
base_spec, speaker_name = split_profile_spec(voice)
resolved_provider = tts_provider if tts_provider in {"kokoro", "supertonic"} else ""
resolved_provider = tts_provider if is_registered_backend(tts_provider) else ""
if speaker_name:
entry = normalize_profile_entry(load_profiles().get(speaker_name))
+7 -6
View File
@@ -7,6 +7,7 @@ from flask.typing import ResponseReturnValue
from abogen.webui.service import PendingJob, JobStatus
from abogen.webui.routes.utils.service import get_service
from abogen.tts_backend_registry import is_registered_backend
from abogen.webui.routes.utils.settings import (
load_settings,
coerce_bool,
@@ -32,7 +33,7 @@ from abogen.webui.routes.utils.common import split_profile_spec
from abogen.utils import calculate_text_length
from abogen.voice_profiles import serialize_profiles, normalize_profile_entry
from abogen.chunking import ChunkLevel, build_chunks_for_chapters
from abogen.constants import VOICES_INTERNAL
from abogen.tts_backend_registry import get_default_voice
from abogen.speaker_configs import get_config
from abogen.kokoro_text_normalization import normalize_roman_numeral_titles
from dataclasses import dataclass
@@ -579,7 +580,7 @@ def apply_book_step_form(
# spec (e.g. "speaker:Name" for saved speakers, or a Kokoro mix formula).
# This enables mixed-provider conversions (e.g. narrator=SuperTonic, characters=Kokoro).
provider_value = str(form.get("tts_provider") or "").strip().lower()
if provider_value in {"kokoro", "supertonic"}:
if is_registered_backend(provider_value):
pending.tts_provider = provider_value
# Determine the base speaker selection (saved speaker ref or raw voice).
@@ -616,8 +617,8 @@ def apply_book_step_form(
custom_formula = ""
base_voice_spec = resolved_default_voice or narrator_voice_raw
if not base_voice_spec and VOICES_INTERNAL:
base_voice_spec = VOICES_INTERNAL[0]
if not base_voice_spec:
base_voice_spec = get_default_voice("kokoro")
voice_choice, resolved_language, selected_profile = resolve_voice_choice(
pending.language,
@@ -796,8 +797,8 @@ def build_pending_job_from_extraction(
profile_selection = inferred_profile
base_voice = base_voice_input or resolved_default_voice or str(default_voice_setting).strip()
if not base_voice and VOICES_INTERNAL:
base_voice = VOICES_INTERNAL[0]
if not base_voice:
base_voice = get_default_voice("kokoro")
selected_speaker_config = (form.get("speaker_config") or "").strip()
speaker_config_payload = get_config(selected_speaker_config) if selected_speaker_config else None
+4 -5
View File
@@ -78,10 +78,9 @@ def get_preview_pipeline(language: str, device: str) -> Any:
pipeline = _preview_pipelines.get(key)
if pipeline is not None:
return pipeline
from abogen.utils import load_numpy_kpipeline
from abogen.tts_backend_registry import create_backend
_, KPipeline = load_numpy_kpipeline()
pipeline = KPipeline(lang_code=language, repo_id="hexgrad/Kokoro-82M", device=device)
pipeline = create_backend("kokoro", lang_code=language, device=device)
_preview_pipelines[key] = pipeline
return pipeline
@@ -137,9 +136,9 @@ def generate_preview_audio(
normalized_text = source_text
if provider == "supertonic":
from abogen.tts_supertonic import SupertonicPipeline
from abogen.tts_backend_registry import create_backend
pipeline = SupertonicPipeline(sample_rate=SAMPLE_RATE, auto_download=True, total_steps=supertonic_total_steps)
pipeline = create_backend("supertonic", sample_rate=SAMPLE_RATE, auto_download=True, total_steps=supertonic_total_steps)
segments = pipeline(
normalized_text,
voice=voice_spec,
+2 -2
View File
@@ -6,8 +6,8 @@ from abogen.constants import (
LANGUAGE_DESCRIPTIONS,
SUBTITLE_FORMATS,
SUPPORTED_SOUND_FORMATS,
VOICES_INTERNAL,
)
from abogen.tts_backend_registry import get_default_voice
from abogen.normalization_settings import (
DEFAULT_LLM_PROMPT,
environment_llm_defaults,
@@ -174,7 +174,7 @@ def settings_defaults() -> Dict[str, Any]:
"subtitle_format": "srt",
"save_mode": "default_output" if has_output_override() else "save_next_to_input",
"default_speaker": "",
"default_voice": VOICES_INTERNAL[0] if VOICES_INTERNAL else "",
"default_voice": get_default_voice("kokoro"),
"supertonic_total_steps": 5,
"supertonic_speed": 1.0,
"replace_single_newlines": False,
+5 -6
View File
@@ -17,10 +17,10 @@ from abogen.constants import (
SUPPORTED_SOUND_FORMATS,
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
SAMPLE_VOICE_TEXTS,
VOICES_INTERNAL,
)
from abogen.tts_backend_registry import get_metadata
from abogen.speaker_configs import list_configs
from abogen.utils import load_numpy_kpipeline
from abogen.tts_backend_registry import create_backend
from abogen.webui.conversion_runner import _select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN
_preview_pipeline_lock = threading.RLock()
@@ -285,7 +285,7 @@ def filter_voice_catalog(
def build_voice_catalog() -> List[Dict[str, str]]:
catalog: List[Dict[str, str]] = []
gender_map = {"f": "Female", "m": "Male"}
for voice_id in VOICES_INTERNAL:
for voice_id in get_metadata("kokoro").voices:
prefix, _, rest = voice_id.partition("_")
language_code = prefix[0] if prefix else "a"
gender_code = prefix[1] if len(prefix) > 1 else ""
@@ -590,7 +590,7 @@ def template_options() -> Dict[str, Any]:
voice_catalog = build_voice_catalog()
return {
"languages": LANGUAGE_DESCRIPTIONS,
"voices": VOICES_INTERNAL,
"voices": get_metadata("kokoro").voices,
"subtitle_formats": SUBTITLE_FORMATS,
"supported_langs_for_subs": SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
"output_formats": SUPPORTED_SOUND_FORMATS,
@@ -741,8 +741,7 @@ def get_preview_pipeline(language: str, device: str):
pipeline = _preview_pipelines.get(key)
if pipeline is not None:
return pipeline
_, KPipeline = load_numpy_kpipeline()
pipeline = KPipeline(lang_code=language, repo_id="hexgrad/Kokoro-82M", device=device)
pipeline = create_backend("kokoro", lang_code=language, device=device)
_preview_pipelines[key] = pipeline
return pipeline
+1 -1
View File
@@ -17,7 +17,7 @@ from abogen.speaker_configs import (
save_configs,
delete_config,
)
from abogen.constants import VOICES_INTERNAL
voices_bp = Blueprint("voices", __name__)
+2 -2
View File
@@ -1,7 +1,7 @@
from types import SimpleNamespace
from typing import cast
from abogen.constants import VOICES_INTERNAL
from abogen.tts_backend_registry import get_metadata
from abogen.webui.conversion_runner import (
_chapter_voice_spec,
_chunk_voice_spec,
@@ -49,4 +49,4 @@ def test_voice_collection_includes_formula_components():
voices = _collect_required_voice_ids(job)
assert {"af_nova", "am_liam"}.issubset(voices)
assert voices.issuperset(VOICES_INTERNAL)
assert voices.issuperset(get_metadata("kokoro").voices)
+1 -1
View File
@@ -197,7 +197,7 @@ def test_epub3_preserves_original_whitespace(tmp_path) -> None:
)
assert match is not None
original_text = html.unescape(match.group(1))
assert "Second line\n\nThird paragraph." in original_text
assert "Second line\n\nThird paragraph." in original_text.replace("\r\n", "\n")
def test_epub3_sentence_chunks_render_as_paragraphs(tmp_path) -> None:
+216
View File
@@ -0,0 +1,216 @@
"""Tests for KokoroBackend class."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Iterator, List
from unittest.mock import MagicMock, patch
import numpy as np
import pytest
from abogen.tts_backend import TTSBackendMetadata
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
@dataclass
class _FakeSegment:
graphemes: str
audio: Any # np.ndarray or torch-like tensor
class _FakePipeline:
"""Minimal mock for kokoro.KPipeline."""
def __init__(self, *, lang_code: str, repo_id: str, device: str):
self.lang_code = lang_code
self.repo_id = repo_id
self.device = device
self._voices: dict[str, np.ndarray] = {}
def __call__(
self,
text: str,
*,
voice: Any = "",
speed: float = 1.0,
split_pattern: str | None = None,
) -> Iterator[_FakeSegment]:
yield _FakeSegment(graphemes=text, audio=np.zeros(100, dtype="float32"))
def load_single_voice(self, name: str) -> np.ndarray:
if name not in self._voices:
self._voices[name] = np.ones((1, 256), dtype="float32") * 0.5
return self._voices[name]
def _make_backend(**kwargs: Any):
"""Create KokoroBackend with mocked KPipeline."""
with patch("abogen.tts_backends.kokoro._load_kpipeline") as load:
load.return_value = _FakePipeline
from abogen.tts_backends.kokoro import KokoroBackend
return KokoroBackend(**kwargs)
# ---------------------------------------------------------------------------
# Tests
# ---------------------------------------------------------------------------
class TestKokoroBackendMetadata:
def test_metadata_returns_tts_backend_metadata(self):
backend = _make_backend(lang_code="a")
meta = backend.metadata
assert isinstance(meta, TTSBackendMetadata)
def test_metadata_fields(self):
backend = _make_backend(lang_code="a")
meta = backend.metadata
assert meta.id == "kokoro"
assert meta.name == "Kokoro"
assert "Kokoro" in meta.description
class TestKokoroBackendInit:
def test_stores_lang_code(self):
backend = _make_backend(lang_code="b")
assert backend._lang_code == "b"
def test_default_repo_id(self):
with patch("abogen.tts_backends.kokoro._load_kpipeline") as load:
load.return_value = _FakePipeline
from abogen.tts_backends.kokoro import KokoroBackend
b = KokoroBackend(lang_code="a")
assert b._pipeline.repo_id == "hexgrad/Kokoro-82M"
def test_custom_repo_id(self):
backend = _make_backend(lang_code="a", repo_id="custom/repo")
assert backend._pipeline.repo_id == "custom/repo"
def test_default_device(self):
backend = _make_backend(lang_code="a")
assert backend._pipeline.device == "cpu"
def test_custom_device(self):
backend = _make_backend(lang_code="a", device="cuda")
assert backend._pipeline.device == "cuda"
class TestKokoroBackendCall:
def test_call_delegates_to_pipeline(self):
backend = _make_backend(lang_code="a")
results = list(backend("hello", voice="af_heart", speed=1.2, split_pattern=r"\n"))
assert len(results) == 1
assert results[0].graphemes == "hello"
def test_call_returns_iterator(self):
backend = _make_backend(lang_code="a")
result = backend("test", voice="af_heart")
assert hasattr(result, "__iter__")
def test_call_with_voice_tensor(self):
backend = _make_backend(lang_code="a")
voice_tensor = np.ones((1, 256), dtype="float32")
results = list(backend("test", voice=voice_tensor))
assert len(results) == 1
def test_call_default_speed(self):
backend = _make_backend(lang_code="a")
# Should not raise with default speed
list(backend("text", voice="af_heart"))
def test_call_default_split_pattern_is_none(self):
backend = _make_backend(lang_code="a")
# split_pattern defaults to None
list(backend("text", voice="af_heart"))
class TestLoadSingleVoice:
def test_load_single_voice_delegates(self):
backend = _make_backend(lang_code="a")
tensor = backend.load_single_voice("af_heart")
assert isinstance(tensor, np.ndarray)
assert tensor.shape == (1, 256)
def test_load_single_voice_caches(self):
backend = _make_backend(lang_code="a")
t1 = backend.load_single_voice("af_heart")
t2 = backend.load_single_voice("af_heart")
assert t1 is t2 # same object
class TestSynthesize:
def test_synthesize_returns_bytes(self):
backend = _make_backend(lang_code="a")
result = backend.synthesize("hello", voice="af_heart")
assert isinstance(result, bytes)
def test_synthesize_nonempty(self):
backend = _make_backend(lang_code="a")
result = backend.synthesize("hello", voice="af_heart")
assert len(result) > 0
def test_synthesize_with_speed(self):
backend = _make_backend(lang_code="a")
result = backend.synthesize("hello", voice="af_heart", speed=1.5)
assert isinstance(result, bytes)
def test_synthesize_empty_text(self):
backend = _make_backend(lang_code="a")
# Empty text produces no segments
result = backend.synthesize("", voice="af_heart")
assert isinstance(result, bytes)
class TestProtocolMethods:
def test_get_available_voices(self):
backend = _make_backend(lang_code="a")
voices = backend.get_available_voices()
assert isinstance(voices, list)
assert len(voices) > 0
assert all(isinstance(v, str) for v in voices)
def test_get_supported_formats(self):
backend = _make_backend(lang_code="a")
formats = backend.get_supported_formats()
assert "pcm_float32" in formats
def test_get_info(self):
backend = _make_backend(lang_code="a")
info = backend.get_info()
assert info["id"] == "kokoro"
assert info["name"] == "Kokoro"
assert info["lang_code"] == "a"
class TestRegistration:
def test_factory_creates_kokoro_backend(self):
from abogen.tts_backends.kokoro import create_kokoro_backend, KokoroBackend
with patch("abogen.tts_backends.kokoro._load_kpipeline") as load:
load.return_value = _FakePipeline
backend = create_kokoro_backend(lang_code="a")
assert isinstance(backend, KokoroBackend)
def test_registry_has_kokoro(self):
import abogen.tts_backends # noqa: F401
from abogen.tts_backend_registry import _registry
meta = _registry.get_metadata("kokoro")
assert meta.id == "kokoro"
assert meta.name == "Kokoro"
def test_registry_factory_returns_kokoro_backend(self):
import abogen.tts_backends # noqa: F401
from abogen.tts_backend_registry import _registry
from abogen.tts_backends.kokoro import KokoroBackend
factory = _registry._factories["kokoro"]
with patch("abogen.tts_backends.kokoro._load_kpipeline") as load:
load.return_value = _FakePipeline
backend = factory(lang_code="a")
assert isinstance(backend, KokoroBackend)
+11 -6
View File
@@ -19,7 +19,7 @@ def test_preview_applies_manual_override_before_normalization(monkeypatch):
# And stub the kokoro pipeline path so generate_preview_audio won't proceed.
# We'll instead validate by calling the override logic through generate_preview_audio
# with provider=supertonic and stub SupertonicPipeline to capture input.
# with provider=supertonic and stub create_backend to capture input.
captured = {}
class DummyPipeline:
@@ -30,11 +30,16 @@ def test_preview_applies_manual_override_before_normalization(monkeypatch):
captured["text"] = text
return iter(())
monkeypatch.setitem(
__import__("sys").modules,
"abogen.tts_supertonic",
type("M", (), {"SupertonicPipeline": DummyPipeline}),
)
from abogen import tts_backend_registry
original_create_backend = tts_backend_registry.create_backend
def _mock_create_backend(backend_id, **kwargs):
if backend_id == "supertonic":
return DummyPipeline(**kwargs)
return original_create_backend(backend_id, **kwargs)
monkeypatch.setattr(tts_backend_registry, "create_backend", _mock_create_backend)
try:
preview.generate_preview_audio(
+314
View File
@@ -0,0 +1,314 @@
from dataclasses import dataclass
from abogen.tts_backend import TTSBackendMetadata
from abogen.tts_backend_registry import TTSBackendRegistry
class TestTTSBackendMetadata:
def test_is_frozen_dataclass(self):
assert dataclass(TTSBackendMetadata)
def test_fields_are_present(self):
meta = TTSBackendMetadata(
id="test",
name="Test Backend",
description="A test backend",
)
assert meta.id == "test"
assert meta.name == "Test Backend"
assert meta.description == "A test backend"
def test_voices_field_default_empty(self):
meta = TTSBackendMetadata(
id="test",
name="Test",
description="Test backend",
)
assert meta.voices == ()
def test_voices_field_stored(self):
meta = TTSBackendMetadata(
id="test",
name="Test",
description="Test backend",
voices=("v1", "v2"),
)
assert meta.voices == ("v1", "v2")
def test_is_immutable(self):
import pytest
meta = TTSBackendMetadata(
id="kokoro",
name="Kokoro",
description="Test",
)
with pytest.raises(Exception):
meta.id = "changed"
class TestTTSBackendRegistry:
def test_register_and_list(self):
registry = TTSBackendRegistry()
meta = TTSBackendMetadata(id="a", name="A", description="Backend A")
registry.register(metadata=meta, factory=lambda: None)
backends = registry.list_backends()
assert len(backends) == 1
assert backends[0].id == "a"
def test_list_multiple(self):
registry = TTSBackendRegistry()
meta_a = TTSBackendMetadata(id="a", name="A", description="A")
meta_b = TTSBackendMetadata(id="b", name="B", description="B")
registry.register(metadata=meta_a, factory=lambda: None)
registry.register(metadata=meta_b, factory=lambda: None)
backends = registry.list_backends()
ids = [b.id for b in backends]
assert "a" in ids
assert "b" in ids
def test_get_metadata(self):
registry = TTSBackendRegistry()
meta = TTSBackendMetadata(id="x", name="X", description="X backend")
registry.register(metadata=meta, factory=lambda: None)
result = registry.get_metadata("x")
assert result.id == "x"
assert result.name == "X"
def test_get_metadata_unknown_raises(self):
import pytest
registry = TTSBackendRegistry()
with pytest.raises(KeyError, match="Unknown backend: nope"):
registry.get_metadata("nope")
def test_create_backend(self):
registry = TTSBackendRegistry()
meta = TTSBackendMetadata(id="test", name="Test", description="Test backend")
def factory(**kwargs):
return {"created": True, "kwargs": kwargs}
registry.register(metadata=meta, factory=factory)
result = registry.create_backend("test", foo="bar")
assert result == {"created": True, "kwargs": {"foo": "bar"}}
def test_create_backend_unknown_raises(self):
import pytest
registry = TTSBackendRegistry()
with pytest.raises(KeyError, match="Unknown backend: missing"):
registry.create_backend("missing")
def test_register_overwrites(self):
registry = TTSBackendRegistry()
meta1 = TTSBackendMetadata(id="x", name="V1", description="First")
meta2 = TTSBackendMetadata(id="x", name="V2", description="Second")
registry.register(metadata=meta1, factory=lambda: "v1")
registry.register(metadata=meta2, factory=lambda: "v2")
result = registry.get_metadata("x")
assert result.name == "V2"
assert registry.create_backend("x") == "v2"
class TestBackendRegistration:
"""Tests that existing backends are auto-registered."""
def test_import_triggers_registration(self):
import abogen.tts_backends # noqa: F401
from abogen.tts_backend_registry import _registry
backends = _registry.list_backends()
ids = [b.id for b in backends]
assert "kokoro" in ids
assert "supertonic" in ids
def test_kokoro_metadata(self):
import abogen.tts_backends # noqa: F401
from abogen.tts_backend_registry import _registry
meta = _registry.get_metadata("kokoro")
assert meta.id == "kokoro"
assert meta.name == "Kokoro"
assert "Kokoro" in meta.description
def test_supertonic_metadata(self):
import abogen.tts_backends # noqa: F401
from abogen.tts_backend_registry import _registry
meta = _registry.get_metadata("supertonic")
assert meta.id == "supertonic"
assert meta.name == "SuperTonic"
assert "SuperTonic" in meta.description
def test_kokoro_metadata_has_voices(self):
import abogen.tts_backends # noqa: F401
from abogen.tts_backend_registry import _registry
meta = _registry.get_metadata("kokoro")
assert isinstance(meta.voices, tuple)
assert len(meta.voices) > 0
assert all(isinstance(v, str) for v in meta.voices)
def test_supertonic_metadata_has_voices(self):
import abogen.tts_backends # noqa: F401
from abogen.tts_backend_registry import _registry
meta = _registry.get_metadata("supertonic")
assert isinstance(meta.voices, tuple)
assert len(meta.voices) == 10
assert meta.voices == ("M1", "M2", "M3", "M4", "M5", "F1", "F2", "F3", "F4", "F5")
def test_kokoro_factory_callable(self):
import abogen.tts_backends # noqa: F401
from abogen.tts_backend_registry import _registry
factory = _registry._factories["kokoro"]
assert callable(factory)
def test_supertonic_factory_callable(self):
import abogen.tts_backends # noqa: F401
from abogen.tts_backend_registry import _registry
factory = _registry._factories["supertonic"]
assert callable(factory)
def test_kokoro_metadata_voices_match_registry(self):
"""Ensure the metadata property on the instance shares voices with registry."""
from abogen.tts_backends.kokoro import _KOKORO_METADATA
from abogen.tts_backend_registry import _registry
registry_meta = _registry.get_metadata("kokoro")
assert _KOKORO_METADATA is registry_meta
assert _KOKORO_METADATA.voices == registry_meta.voices
def test_supertonic_metadata_voices_match_registry(self):
"""Ensure the metadata property on the instance shares voices with registry."""
from abogen.tts_backends.supertonic import _SUPERTONIC_METADATA
from abogen.tts_backend_registry import _registry
registry_meta = _registry.get_metadata("supertonic")
assert _SUPERTONIC_METADATA is registry_meta
assert _SUPERTONIC_METADATA.voices == registry_meta.voices
class TestResolveBackendForVoice:
"""Tests for the resolve_backend_for_voice method."""
def test_empty_spec_returns_fallback(self):
registry = TTSBackendRegistry()
assert registry.resolve_backend_for_voice("", fallback="kokoro") == "kokoro"
assert registry.resolve_backend_for_voice("", fallback="supertonic") == "supertonic"
def test_none_spec_returns_fallback(self):
registry = TTSBackendRegistry()
assert registry.resolve_backend_for_voice(None, fallback="kokoro") == "kokoro"
def test_kokoro_formula_with_star_returns_kokoro(self):
registry = TTSBackendRegistry()
assert registry.resolve_backend_for_voice("af_nova*0.7") == "kokoro"
def test_kokoro_formula_with_plus_returns_kokoro(self):
registry = TTSBackendRegistry()
assert registry.resolve_backend_for_voice("af_nova*0.7+am_liam*0.3") == "kokoro"
def test_kokoro_voice_id_resolves_to_kokoro(self):
registry = TTSBackendRegistry()
meta = TTSBackendMetadata(
id="kokoro",
name="Kokoro",
description="Kokoro TTS",
voices=("af_nova", "am_liam"),
)
registry.register(metadata=meta, factory=lambda: None)
assert registry.resolve_backend_for_voice("af_nova") == "kokoro"
assert registry.resolve_backend_for_voice("am_liam") == "kokoro"
def test_supertonic_voice_id_resolves_to_supertonic(self):
registry = TTSBackendRegistry()
meta = TTSBackendMetadata(
id="supertonic",
name="SuperTonic",
description="SuperTonic TTS",
voices=("M1", "M2", "F1", "F2"),
)
registry.register(metadata=meta, factory=lambda: None)
assert registry.resolve_backend_for_voice("M1") == "supertonic"
assert registry.resolve_backend_for_voice("F2") == "supertonic"
def test_unknown_voice_returns_fallback(self):
registry = TTSBackendRegistry()
meta = TTSBackendMetadata(
id="kokoro",
name="Kokoro",
description="Kokoro TTS",
voices=("af_nova",),
)
registry.register(metadata=meta, factory=lambda: None)
assert registry.resolve_backend_for_voice("unknown_voice") == "kokoro"
assert registry.resolve_backend_for_voice("unknown_voice", fallback="supertonic") == "supertonic"
def test_case_insensitive_matching(self):
registry = TTSBackendRegistry()
meta = TTSBackendMetadata(
id="supertonic",
name="SuperTonic",
description="SuperTonic TTS",
voices=("M1", "F1"),
)
registry.register(metadata=meta, factory=lambda: None)
assert registry.resolve_backend_for_voice("m1") == "supertonic"
assert registry.resolve_backend_for_voice("f1") == "supertonic"
def test_default_fallback_is_kokoro(self):
registry = TTSBackendRegistry()
assert registry.resolve_backend_for_voice("unknown") == "kokoro"
def test_multiple_backends_resolution(self):
registry = TTSBackendRegistry()
kokoro_meta = TTSBackendMetadata(
id="kokoro",
name="Kokoro",
description="Kokoro TTS",
voices=("af_nova",),
)
supertonic_meta = TTSBackendMetadata(
id="supertonic",
name="SuperTonic",
description="SuperTonic TTS",
voices=("M1",),
)
registry.register(metadata=kokoro_meta, factory=lambda: None)
registry.register(metadata=supertonic_meta, factory=lambda: None)
assert registry.resolve_backend_for_voice("af_nova") == "kokoro"
assert registry.resolve_backend_for_voice("M1") == "supertonic"
def test_global_wrapper_resolve_backend_for_voice(self):
from abogen.tts_backend_registry import resolve_backend_for_voice
# Test with empty spec
assert resolve_backend_for_voice("") == "kokoro"
# Test with formula
assert resolve_backend_for_voice("af_nova*0.7") == "kokoro"
# Test with a registered voice
assert resolve_backend_for_voice("af_nova") == "kokoro"
assert resolve_backend_for_voice("M1") == "supertonic"
+63 -8
View File
@@ -1,6 +1,6 @@
import numpy as np
from abogen.tts_supertonic import SupertonicPipeline
from abogen.tts_backends.supertonic import SupertonicBackend, SupertonicPipeline
class _DummyTTS:
@@ -26,13 +26,23 @@ class _DummyTTS:
return audio, 0.05
def test_supertonic_pipeline_strips_unsupported_characters_and_retries():
# Avoid importing/initializing real supertonic by manually constructing the pipeline.
def _make_pipeline() -> SupertonicPipeline:
pipeline = SupertonicPipeline.__new__(SupertonicPipeline)
pipeline.sample_rate = 24000
pipeline.total_steps = 5
pipeline.max_chunk_length = 1000
pipeline._tts = _DummyTTS()
return pipeline
def _make_backend() -> SupertonicBackend:
backend = SupertonicBackend.__new__(SupertonicBackend)
backend._pipeline = _make_pipeline()
return backend
def test_supertonic_pipeline_strips_unsupported_characters_and_retries():
pipeline = _make_pipeline()
segs = list(pipeline("Hello • world", voice="M1", speed=1.0))
assert len(segs) == 1
@@ -43,11 +53,56 @@ def test_supertonic_pipeline_strips_unsupported_characters_and_retries():
def test_supertonic_pipeline_drops_chunk_if_only_unsupported_characters():
pipeline = SupertonicPipeline.__new__(SupertonicPipeline)
pipeline.sample_rate = 24000
pipeline.total_steps = 5
pipeline.max_chunk_length = 1000
pipeline._tts = _DummyTTS()
pipeline = _make_pipeline()
segs = list(pipeline("", voice="M1", speed=1.0))
assert segs == []
# --- SupertonicBackend tests ---
def test_backend_metadata():
backend = _make_backend()
meta = backend.metadata
assert meta.id == "supertonic"
assert meta.name == "SuperTonic"
assert "SuperTonic" in meta.description
def test_backend_get_available_voices():
backend = _make_backend()
voices = backend.get_available_voices()
assert isinstance(voices, list)
assert "M1" in voices
assert "F1" in voices
def test_backend_get_supported_formats():
backend = _make_backend()
formats = backend.get_supported_formats()
assert "wav" in formats
def test_backend_get_info():
backend = _make_backend()
info = backend.get_info()
assert info["sample_rate"] == 24000
assert info["total_steps"] == 5
assert isinstance(info["voices"], list)
def test_backend_call_delegates_to_pipeline():
backend = _make_backend()
segs = list(backend("Hello • world", voice="M1", speed=1.0))
assert len(segs) == 1
assert segs[0].audio.size > 0
def test_backend_synthesize_returns_wav_bytes():
backend = _make_backend()
wav_bytes = backend.synthesize("Hello world", voice="M1", speed=1.0)
assert isinstance(wav_bytes, bytes)
assert len(wav_bytes) > 0
# WAV magic number
assert wav_bytes[:4] == b"RIFF"
+2 -2
View File
@@ -3,7 +3,7 @@ from typing import cast
import pytest
from abogen.constants import VOICES_INTERNAL
from abogen.tts_backend_registry import get_metadata
from abogen.voice_cache import (
LocalEntryNotFoundError,
_CACHED_VOICES,
@@ -66,4 +66,4 @@ def test_collect_required_voice_ids_includes_all():
voices = _collect_required_voice_ids(cast(Job, job))
assert {"af_nova", "am_liam", "am_michael"}.issubset(voices)
assert voices.issuperset(VOICES_INTERNAL)
assert voices.issuperset(get_metadata("kokoro").voices)
+1 -1
View File
@@ -1,7 +1,7 @@
from __future__ import annotations
from abogen.webui.conversion_runner import _resolve_voice, _supertonic_voice_from_spec
from abogen.tts_supertonic import DEFAULT_SUPERTONIC_VOICES
from abogen.tts_backends.supertonic import DEFAULT_SUPERTONIC_VOICES
def test_resolve_voice_formula_without_pipeline_does_not_crash() -> None:
+233
View File
@@ -0,0 +1,233 @@
import pytest
from abogen.voice_metadata import VoiceMetadata
class TestVoiceMetadataCreation:
def test_create_with_all_fields(self):
voice = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
assert voice.id == "af_alloy"
assert voice.display_name == "Alloy"
assert voice.language == "a"
assert voice.gender == "female"
assert voice.backend_id == "kokoro"
def test_create_supertonic_voice(self):
voice = VoiceMetadata(
id="M1",
display_name="Male 1",
language="en",
gender="male",
backend_id="supertonic",
)
assert voice.id == "M1"
assert voice.backend_id == "supertonic"
def test_create_with_unknown_gender(self):
voice = VoiceMetadata(
id="custom_voice",
display_name="Custom",
language="en",
gender="unknown",
backend_id="custom_backend",
)
assert voice.gender == "unknown"
class TestVoiceMetadataImmutability:
def test_frozen_dataclass(self):
voice = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
with pytest.raises(AttributeError):
voice.id = "new_id"
def test_cannot_modify_display_name(self):
voice = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
with pytest.raises(AttributeError):
voice.display_name = "New Name"
def test_cannot_modify_backend_id(self):
voice = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
with pytest.raises(AttributeError):
voice.backend_id = "new_backend"
class TestVoiceMetadataEquality:
def test_equal_voices_are_equal(self):
voice1 = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
voice2 = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
assert voice1 == voice2
def test_different_voices_are_not_equal(self):
voice1 = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
voice2 = VoiceMetadata(
id="am_adam",
display_name="Adam",
language="a",
gender="male",
backend_id="kokoro",
)
assert voice1 != voice2
def test_different_backend_id_not_equal(self):
voice1 = VoiceMetadata(
id="custom",
display_name="Custom",
language="en",
gender="unknown",
backend_id="backend_a",
)
voice2 = VoiceMetadata(
id="custom",
display_name="Custom",
language="en",
gender="unknown",
backend_id="backend_b",
)
assert voice1 != voice2
class TestVoiceMetadataHashing:
def test_hashable(self):
voice = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
assert hash(voice) is not None
def test_equal_voices_same_hash(self):
voice1 = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
voice2 = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
assert hash(voice1) == hash(voice2)
def test_usable_in_set(self):
voice1 = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
voice2 = VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id="kokoro",
)
voice3 = VoiceMetadata(
id="am_adam",
display_name="Adam",
language="a",
gender="male",
backend_id="kokoro",
)
voice_set = {voice1, voice2, voice3}
assert len(voice_set) == 2
class TestVoiceMetadataUseCases:
def test_backend_populates_backend_id(self):
"""Simulate how a backend would populate backend_id automatically."""
class MockBackend:
def __init__(self):
self._backend_id = "kokoro"
def get_voices(self):
return [
VoiceMetadata(
id="af_alloy",
display_name="Alloy",
language="a",
gender="female",
backend_id=self._backend_id,
),
]
backend = MockBackend()
voices = backend.get_voices()
assert voices[0].backend_id == "kokoro"
def test_filter_by_language(self):
voices = [
VoiceMetadata(id="af_alloy", display_name="Alloy", language="a", gender="female", backend_id="kokoro"),
VoiceMetadata(id="jf_alpha", display_name="Alpha", language="j", gender="female", backend_id="kokoro"),
VoiceMetadata(id="am_adam", display_name="Adam", language="a", gender="male", backend_id="kokoro"),
]
english_voices = [v for v in voices if v.language == "a"]
assert len(english_voices) == 2
def test_filter_by_gender(self):
voices = [
VoiceMetadata(id="af_alloy", display_name="Alloy", language="a", gender="female", backend_id="kokoro"),
VoiceMetadata(id="am_adam", display_name="Adam", language="a", gender="male", backend_id="kokoro"),
VoiceMetadata(id="am_puck", display_name="Puck", language="a", gender="male", backend_id="kokoro"),
]
male_voices = [v for v in voices if v.gender == "male"]
assert len(male_voices) == 2
def test_filter_by_backend(self):
voices = [
VoiceMetadata(id="af_alloy", display_name="Alloy", language="a", gender="female", backend_id="kokoro"),
VoiceMetadata(id="M1", display_name="Male 1", language="en", gender="male", backend_id="supertonic"),
]
kokoro_voices = [v for v in voices if v.backend_id == "kokoro"]
assert len(kokoro_voices) == 1
assert kokoro_voices[0].id == "af_alloy"