Compare commits

..
38 Commits
Author SHA1 Message Date
Artem Akymenko 94e6b3f62e refactor: move calculate_text_length to domain, add build_chapter_payload to app-layer
- Create domain/text_utils.py with canonical calculate_text_length
  (strips chapter markers, voice markers, metadata tags)
- Remove calculate_text_length from utils.py and subtitle_utils.py
- Update all imports to use domain.text_utils directly
- Create application/chapter_selection.py with build_chapter_payload()
  (orchestrates preselection + character count + safety net)
- Replace manual chapter logic in form.py with build_chapter_payload()
- 22 new tests for text_utils and chapter_selection (1550 total)
2026-07-30 14:57:10 +00:00
Artem Akymenko d334266238 refactor: remove eager voice cache bootstrap from WebUI
Voice cache is now initialized lazily by domain's initialize_voice_cache()
in PipelinePool.get() — downloads only needed voices per request instead
of all ~100 kokoro voices at service startup. Reduces redundant downloads
and unifies initialization path for all UIs.
2026-07-30 13:45:20 +00:00
Artem Akymenko 2d18501839 refactor: consolidate metadata normalization into domain layer
- Add expand_metadata_aliases() for concept fan-out (series→5, author→2,
  description→2, tags→3 keys)
- Replace _normalize_metadata_tags in webui/service.py with
  normalize_metadata_map from domain
- Replace _normalize_metadata in epub3/exporter.py with
  normalize_metadata_map from domain
- Replace manual fan-out in form.py with expand_metadata_aliases()
- Rewrite metadata_overrides.py to use expand_metadata_aliases()
- Remove unused normalize_metadata_casefold import from exporters.py
- 19 new tests for expand_metadata_aliases (1528 total)
2026-07-30 13:04:44 +00:00
Artem Akymenko 464bf8e17d refactor: move speaker metadata logic to domain layer
Move build_narrator_roster, build_speaker_roster, match_configured_speaker,
apply_speaker_config_to_roster, prepare_speaker_metadata from
webui/routes/utils/voice.py to domain/speaker_metadata.py.

- prepare_speaker_metadata now accepts optional inject_recommended callback
  for UI-specific voice enrichment
- 47 new tests in test_domain_speaker_metadata.py
- All 1509 tests pass
2026-07-30 12:00:33 +00:00
Artem Akymenko 9bf4f8e809 feat: add voice resolution to domain layer, migrate WebUI imports
- Add formula_from_profile, resolve_profile_voice, resolve_voice_setting, resolve_voice_choice to domain/voice_resolution.py
- Add build_voice_catalog, filter_voice_catalog to domain/voice_catalog.py
- Update webui/routes/utils/voice.py to import from domain
- Update webui/routes/utils/form.py and voices.py to import from domain directly
- Update synthesize.py to use domain resolve_voice
- Add 29 tests for voice resolution functions
2026-07-30 10:58:23 +00:00
Artem Akymenko f802fb2af6 refactor: centralize integration config in domain layer, fix OutputFormat enum serialization
- Move stored_integration_config(), build_audiobookshelf_config() to domain/settings_core.py
- Remove legacy fallback from stored_integration_config() (only config[integrations])
- Add load_audiobookshelf_config() as combined entry point
- PostConversionHooks reads config directly via stored_integration_config()
- WebUI imports from domain/settings_core instead of webui/routes/utils/settings
- Remove duplicate _build_abs_config() from PostConversionHooks
- Remove dead audiobookshelf code from infrastructure/exporters.py
- Remove duplicate audiobookshelf functions from webui/service.py
- Fix OutputFormat enum serialization in output_layout_service.py and conversion_executor.py
  (f'{enum}' gave 'OutputFormat.WAV' instead of '.wav')
- Mock spaCy in test_returns_at_least_one_segment instead of loading real model
- Add 28 tests for PostConversionHooks and build_audiobookshelf_config
2026-07-29 13:11:56 +00:00
Artem Akymenko c293cc90f6 chore: suppress phonemizer word-count-mismatch warnings (normal behavior) 2026-07-29 11:26:43 +00:00
Artem Akymenko 696ce1ebd0 fix: use existing to_float32 from audio_helpers, keep INFO log level
- synthesize.py: import to_float32 from domain.audio_helpers instead of duplicate
- Revert all log levels back to INFO (user controls verbosity via --log-level)
2026-07-29 11:24:01 +00:00
Artem Akymenko d3ded8af0e feat: add comprehensive logging throughout conversion pipeline
- voice_resolver: log spec resolution and cache hits
- conversion_executor: log chapter start/end, voice resolution, timing
- conversion_planner: log plan summary
- conversion_service: log entry point params and outcome
- conversion_runner: log job params and lifecycle
- form.py: log PendingJob creation
- synthesize.py: log pipeline creation and device selection
2026-07-29 11:05:52 +00:00
Artem Akymenko c706f7714a fix: language enum coercion, preview logging, file picker
- form.py: convert language string to Language enum at all entry points
- conversion_request.py: add _coerce_enums() for defense-in-depth
- synthesize.py: fix NameError (lang -> language) in preview pipeline
- voice.py: render LANGUAGE_DESCRIPTIONS keys as .value strings for Jinja
- dashboard.js: open file picker on dropzone click
- api.py: add logging to preview endpoint for debugging
2026-07-29 10:57:21 +00:00
Artem Akymenko 2b70b9ca45 feat: Supertonic language + total_steps propagation
Language enum expanded from 9 to 33 languages:
- Added 24 new ISO 639-1 languages: AR, BG, CS, DA, DE, EL, ET, FI,
  HR, HU, ID, KO, LT, LV, NL, PL, RO, RU, SK, SL, SV, TR, UK, VI
- Updated display_name, is_cjk (added KO)

Supertonic language mapping (32 languages, no ZH):
- engine.py: _SUPERTONIC_LANG_MAP, engine_language(), supported_languages()
- __init__.py: create_engine() passes config.language to pipeline
- pipeline.py: __init__() accepts language, resolves to ISO code;
  __call__() passes lang= to TTS.synthesize()

total_steps propagation:
- tts_segments(): +total_steps param, conditionally passed to backend
- synthesize_text(): +total_steps param
- run_tts_segment_loop(): +total_steps param
- executor: all 5 synthesize_text() calls pass total_steps

Integration:
- pipeline_factory: create_pipeline_for_job() passes language to supertonic
- preview path: create_pipeline('supertonic', language=language)

Tests updated to accept total_steps in FakeBackend.__call__
2026-07-28 14:09:42 +03:00
Artem Akymenko 953bef1e71 refactor: group ConversionRequest fields into config objects
Domain config types (domain/config_types.py):
- PronunciationConfig: pronunciation/heteronym/normalization overrides
- SubtitleConfig: mode, format, max_words
- CoverConfig: path, mime

Domain functions now accept config objects:
- build_tts_context(subtitle=, pronunciation=) instead of 9 individual params
- make_subtitle_writer(subtitle=) instead of 3 params
- process_and_write_subtitles(subtitle=) instead of 2 params
- embed_m4b_metadata(cover=) instead of 2 params
- build_epub3_package(cover=) instead of 2 params

ConversionRequest: 18 flat fields + 8 config objects
Application/config.py re-exports domain types
All tests updated to new API
2026-07-28 13:41:45 +03:00
Artem Akymenko 146cc81271 refactor: centralize cleanup in app layer
- New: application/cleanup.py — flush_cuda(), dispose_engines(), cleanup(), register_ui_cleanup()
- conversion_service.py finally: pool.dispose_all() + voice_cache.clear() + flush_cuda()
- webui/conversion_runner.py: removed gc/cuda finally block (cleanup in run_conversion)
- shutdown.py: 160→120 lines, 5 inline cleanups → 4 process-level + app_cleanup() delegation
- Fixed bugs: _PIPELINES (didn't exist), PluginManager.dispose_all() (never called),
  VoiceCache.clear() (never called in finally), duplicate cleanup removed
2026-07-28 13:41:28 +03:00
Artem Akymenko 61204cc389 refactor: move pool/cache/resolver into run_conversion()
- New: application/voice_resolver.py — AppVoiceResolver (app-layer, takes ConversionRequest)
- conversion_service.py: run_conversion(request, events) creates PipelinePool, VoiceCache, AppVoiceResolver internally
- conversion_runner.py: 268→165 lines, removed WebUIVoiceResolver, pool/cache creation
- Tests: mock fixture for pool/resolver, removed unused variables
- Added pool.dispose_all() in finally block for cleanup
2026-07-27 13:37:19 +03:00
Artem Akymenko f7a224cc46 chore: remove dead test files and deleted conversion_adapter
All 8 deleted test files were duplicates of existing domain tests:
- test_chapter_overrides → covered by test_chapter_merge_normalize
- test_conversion_chapter_titles → covered by test_chapter_titles
- test_conversion_series → covered by test_title_builder
- test_conversion_voice_resolution → covered by test_voice_resolution
- test_voice_cache → covered by test_voice_resolution
- test_manual_overrides_applied_first → covered by test_pronunciation
- test_conversion_adapters → tested deleted conversion_adapter module
- test_import_layering → tested deleted conversion_adapter module

Also removed deleted conversion_adapter.py (logic moved to conversion_runner).
2026-07-27 09:56:51 +00:00
Artem Akymenko 98ab2d925e refactor: simplify _build_request, restore epub3_export as config object
Simplify _build_request() in WebUI
- Remove profiles loading (load_profiles, normalize_profile_entry)
- Remove PronunciationConfig/Epub3ExportConfig building
- Pass raw fields directly to ConversionRequest
- Clean up unused imports

Bugfix: epub3_export as Optional[Epub3ExportConfig] (None = disabled)
- Reverted generate_epub3: bool + epub3_book_id: str back to object pattern
- Consistent with word_substitution, subtitle_input, chapter_chunk
- Updated conversion_service.py _finalize() to use request.epub3_export
2026-07-27 12:41:29 +03:00
Artem Akymenko 625b6610e2 refactor: remove _prepare_tts_context, use build_tts_context directly
- Deleted _prepare_tts_context() (104 lines of duplicated logic)
- Replaced with direct build_tts_context() call in run_conversion()
- Fixed _finalize() to use raw fields (generate_epub3, epub3_book_id)
- Removed _MockJob antipattern
- Updated tests to use raw normalization_overrides field
- ConversionRequest uses raw fields instead of PronunciationConfig/Epub3ExportConfig
2026-07-27 12:19:59 +03:00
Artem Akymenko 0dc491e420 refactor: unify Language enum across all layers
- EngineConfig.language: Language (was lang_code: str = 'a')
- Engine owns _KOKORO_LANG_MAP, engine_language(), supported_languages()
- Engine provides language_for_voice_id() for voice catalog
- Plugins/kokoro/__init__.py calls engine_language() internally
- create_pipeline(plugin_id, language=Language) — no kokoro codes
- pipeline_factory.py clean of kokoro-specific code
- Domain functions raise TypeError if non-enum passed
- WebUI api.py: _parse_language() helper at API boundary
- Voice catalog returns ISO codes (lang.value)
- Constants: LANGUAGE_DESCRIPTIONS keyed by Language enum
- All tests updated for Language enum
- 1414 tests pass
2026-07-27 07:36:36 +00:00
Artem Akymenko 713abdfd73 refactor: voice resolution on ConversionRequest, PipelinePool without job param
- Added speakers field to ConversionRequest
- Rewrote collect_required_voice_ids(), initialize_voice_cache(),
  job_voice_fallback(), chapter_voice_spec(), chunk_voice_spec()
  to accept ConversionRequest instead of job
- PipelinePool.get() now takes request= instead of job=
- Updated all tests to use ConversionRequest interface
- 1493 tests passing
2026-07-26 14:11:55 +03:00
Artem Akymenko 73f42e9563 refactor: voice fallback logging + base voice validation 2026-07-26 11:42:56 +03:00
Artem Akymenko 2c61f55f81 refactor: spaCy pre-TTS segmentation moved to shared layer
- conversion_pipeline.py: new spacy_pre_tts_segmentation() function
- Handles: condition checks, language exclusion, split_pattern override
- Executor: integrated spaCy pre-TTS before each synthesize_text() call
- 6 new tests for condition checks and fallback behavior
- English excluded from pre-TTS (spaCy only for post-TTS subtitles)
- Fallback to regex when spaCy unavailable
2026-07-26 11:42:43 +03:00
Artem Akymenko 6497e8c47a refactor: metadata + markers unified in shared layer
- MarkerCollector: SRP extraction from executor (observation, not execution)
- Outro marker: executor now records outro as chapter marker
- Voice format: chapter voices [{provider, voice}], chunk voice {provider, voice}
- _finalize(): build_metadata_payload() + metadata.json + record_override_usage()
- ffmetadata: voices list → comma-separated string
- EPUB3: ChunkOverlay.voice = dict, _render_chunk_inline handles dict format
- 8 new tests: multi-speaker, ffmetadata format, EPUB3 format
- Updated existing tests for new voice format
2026-07-26 11:42:27 +03:00
Artem Akymenko 0b953d48e8 refactor: heading transforms + dedup moved to shared layer
- conversion_planner.py: caps normalization in _build_chapters()
- conversion_executor.py: heading dedup + state machine via headings_equivalent()
- Fixed seg_start_time → chapter_body_start bug in executor
- Removed getattr fallback defaults in both adapters
- Added 4 tests for caps normalization and heading dedup
- Updated ARCHITECTURE_REFACTOR_PLAN.md with deferred WebUI cleanup
2026-07-26 11:41:59 +03:00
Artem Akymenko f516cf1985 merge: resolve conflicts with origin/main
- Import run_tts_segment_loop (used in pyqt/conversion.py)
- Use build_tts_context() instead of manual _MergeJob + TTSContext
- Drop unused AudioSink import
2026-07-24 18:34:49 +00:00
Artem Akymenko 3857c27aae refactor: clean unused imports (ruff F401/F811), fix build_tts_context defaults
- Remove 77 unused imports across domain/application/runner files via ruff
- Add # noqa: F401 to re-exports used by tests and debug_tts_runner
- Fix build_tts_context: usage_counter uses 'is not None' instead of truthiness
- Fix test assertions: compiled rules use 'replacement' key not 'pronunciation'
- Add re-exports: _compile_pronunciation_rules, _merge_pronunciation_overrides
2026-07-24 21:26:36 +03:00
Artem Akymenko 7ed2addb11 refactor: add build_metadata_payload() to domain, unify metadata assembly in both UIs 2026-07-24 21:26:21 +03:00
Artem Akymenko cfc7de7abf refactor: unify PUNCTUATION constants in domain/split_pattern.py 2026-07-24 19:17:39 +03:00
Artem Akymenko 654c395943 refactor: move sanitize_name_for_os to domain/output_paths.py 2026-07-24 19:17:39 +03:00
Artem Akymenko d1a84cfb8b refactor: move voice marker functions to domain/voice_markers.py 2026-07-24 19:17:38 +03:00
Artem Akymenko 332934c0cf test: tests for apply_overrides, LLM mode, usage_counter, chunk_groups 2026-07-24 19:17:38 +03:00
Artem Akymenko c79838a5a1 fix: chunk_groups_by_chapter in planner, chunks assigned to correct chapter 2026-07-24 19:17:38 +03:00
Artem Akymenko d51a9118e4 fix: apply_overrides, LLM mode check, usage_counter in service 2026-07-24 19:17:38 +03:00
Artem Akymenko 3311bef2f7 feat: EPUB3 finalizer in service, extraction in ConversionPlan 2026-07-24 19:17:37 +03:00
Artem Akymenko 4123cadd87 feat: m4b finalizer in service (embed_m4b_metadata) 2026-07-24 19:17:37 +03:00
Artem Akymenko 2f83d10a1e feat: per-chapter subtitle writer in executor 2026-07-24 19:17:37 +03:00
Artem Akymenko a1241ee9ca refactor: word substitution in planner, add planner tests 2026-07-24 19:17:37 +03:00
Artem Akymenko 0ee5bb0496 refactor: config objects for feature toggles in ConversionRequest 2026-07-24 19:17:36 +03:00
Artem Akymenko 7d28b7eb52 test: mock subprocess.Popen in test_executor_m4b_forces_merge to eliminate ffmpeg dependency 2026-07-24 19:17:36 +03:00
112 changed files with 6362 additions and 4795 deletions
+62
View File
@@ -0,0 +1,62 @@
"""Chapter selection helpers for the application layer.
Builds chapter payloads with smart defaults (preselection based on
supplement score) and character counts. Used by both WebUI and PyQt.
"""
from __future__ import annotations
from typing import Any, Dict, List
from abogen.domain.chapter_classification import (
ensure_at_least_one_chapter_enabled,
should_preselect_chapter,
)
from abogen.domain.text_utils import calculate_text_length
def build_chapter_payload(
chapters: List[Any],
source_name: str = "",
) -> List[Dict[str, Any]]:
"""Build a chapter payload with preselection and character counts.
Args:
chapters: List of chapter-like objects with ``title`` and ``text`` attributes.
source_name: Fallback title for the placeholder chapter when *chapters* is empty.
Returns:
List of chapter dicts ready for ``PendingJob.chapters`` or ``ChapterChunkConfig``.
"""
total = len(chapters)
payload: List[Dict[str, Any]] = []
for index, chapter in enumerate(chapters):
title = getattr(chapter, "title", "") or ""
text = getattr(chapter, "text", "") or ""
enabled = should_preselect_chapter(title, text, index, total)
payload.append(
{
"id": f"{index:04d}",
"index": index,
"title": title,
"text": text,
"characters": calculate_text_length(text),
"enabled": enabled,
}
)
if not payload:
payload.append(
{
"id": "0000",
"index": 0,
"title": source_name,
"text": "",
"characters": 0,
"enabled": True,
}
)
ensure_at_least_one_chapter_enabled(payload)
return payload
+73
View File
@@ -0,0 +1,73 @@
"""Application-layer cleanup — global resource disposal.
Handles:
- GPU/CUDA memory flush
- TTS engine disposal (PluginManager)
- UI-specific cleanup callbacks (registered by entry points)
Called by shutdown.py at process exit and by run_conversion() per-conversion.
"""
from __future__ import annotations
import gc
from typing import Callable
_UI_CLEANUPS: list[Callable[[], None]] = []
def flush_cuda() -> None:
"""Run GC and release CUDA cache. Safe to call multiple times."""
gc.collect()
try:
import torch
if torch.cuda.is_available():
torch.cuda.empty_cache()
torch.cuda.ipc_collect()
except Exception:
pass
def dispose_engines() -> None:
"""Dispose all cached TTS engines via PluginManager."""
try:
from abogen.tts_plugin.plugin_manager import get_plugin_manager
get_plugin_manager().dispose_all()
except Exception:
pass
def _clear_global_voice_cache() -> None:
"""Reset the global voice download cache state."""
try:
from abogen.voice_cache import clear_voice_cache
clear_voice_cache()
except Exception:
pass
def register_ui_cleanup(fn: Callable[[], None]) -> None:
"""Register a UI-specific cleanup callback (e.g. preview threads, temp files)."""
_UI_CLEANUPS.append(fn)
def cleanup() -> None:
"""Run all application-level cleanups. Idempotent."""
dispose_engines()
flush_cuda()
_clear_global_voice_cache()
for fn in _UI_CLEANUPS:
try:
fn()
except Exception:
pass
_UI_CLEANUPS.clear()
__all__ = [
"flush_cuda",
"dispose_engines",
"register_ui_cleanup",
"cleanup",
]
+95
View File
@@ -0,0 +1,95 @@
"""Feature config objects for ConversionRequest.
Each config object groups parameters for a specific feature.
If the object is None, the feature is disabled.
This keeps ConversionRequest clean: no boolean flags for feature toggles,
no scattered parameters across unrelated fields.
Domain config types (PronunciationConfig, SubtitleConfig) live in
domain/config_types.py — domain defines the contract, app fills them.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional
from abogen.domain.config_types import CoverConfig, PronunciationConfig, SubtitleConfig
from abogen.domain.enums import OutputFormat, SaveMode
@dataclass(frozen=True)
class WordSubstitutionConfig:
"""Word substitution settings.
When present on ConversionRequest, word substitution is applied
to the source text before chapter parsing.
"""
substitutions_list: str = ""
case_sensitive: bool = False
replace_caps: bool = False
replace_numerals: bool = False
fix_punctuation: bool = False
@dataclass(frozen=True)
class SubtitleInputConfig:
"""Subtitle file input settings.
When present on ConversionRequest, the source is treated as a
subtitle file (.srt/.ass/.vtt) or timestamp text, and the
subtitle-to-audio pipeline is used instead of normal text conversion.
"""
is_timestamp_text: bool = False
@dataclass(frozen=True)
class Epub3ExportConfig:
"""EPUB3 export settings.
When present on ConversionRequest, an EPUB3 package with
synchronized audio narration is generated after conversion.
"""
book_id: str = ""
@dataclass(frozen=True)
class ChapterChunkConfig:
"""Chapter and chunk configuration.
Groups chapter overrides, chunk data, and speaker settings
used by the planner to build segments.
"""
chapter_overrides: List[Dict[str, Any]] = field(default_factory=list)
chunks: List[Dict[str, Any]] = field(default_factory=list)
chunk_level: str = "paragraph"
speaker_mode: str = "single"
speakers: Dict[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
_VALID_CHUNK_LEVELS = ("paragraph", "sentence")
_VALID_SPEAKER_MODES = ("single", "multi")
if self.chunk_level not in _VALID_CHUNK_LEVELS:
raise ValueError(
f"chunk_level must be one of {_VALID_CHUNK_LEVELS}, got {self.chunk_level!r}"
)
if self.speaker_mode not in _VALID_SPEAKER_MODES:
raise ValueError(
f"speaker_mode must be one of {_VALID_SPEAKER_MODES}, got {self.speaker_mode!r}"
)
@dataclass(frozen=True)
class SaveConfig:
"""Save/output settings.
Groups save mode, output folder, chapter splitting, and merge options.
"""
mode: SaveMode = SaveMode.SAVE_NEXT_TO_INPUT
output_folder: Optional[Path] = None
save_chapters_separately: bool = False
merge_chapters_at_end: bool = True
separate_chapters_format: OutputFormat = OutputFormat.WAV
save_as_project: bool = False
+290 -64
View File
@@ -8,15 +8,13 @@ This is Stage 6 of the conversion flow unification plan.
from __future__ import annotations from __future__ import annotations
import logging
import time import time
from contextlib import ExitStack from contextlib import ExitStack
from typing import Any, Callable, Dict, List, Optional, Tuple from typing import Any, Callable, Dict, List, Optional, Set, Tuple
from abogen.application.conversion_models import ( from abogen.application.conversion_models import (
ChapterPlan,
ConversionPlan, ConversionPlan,
IntroOutroSpec,
SegmentPlan,
) )
from abogen.application.conversion_ports import ( from abogen.application.conversion_ports import (
AudioSink, AudioSink,
@@ -35,10 +33,117 @@ from abogen.domain.conversion_engine import (
) )
from abogen.domain.enums import OutputFormat, SubtitleMode from abogen.domain.enums import OutputFormat, SubtitleMode
from abogen.domain.normalization import TTSContext from abogen.domain.normalization import TTSContext
from abogen.domain.chapter_titles import (
apply_chapter_text_transforms,
headings_equivalent as _headings_equivalent,
)
from abogen.domain.output_paths import sanitize_filename_for_chapter from abogen.domain.output_paths import sanitize_filename_for_chapter
from abogen.infrastructure.subtitle_writer import make_subtitle_writer from abogen.infrastructure.subtitle_writer import make_subtitle_writer
# ─── MarkerCollector ───
class MarkerCollector:
"""Observes execution events and accumulates chapter/chunk markers.
Separates marker collection from synthesis logic.
"""
def __init__(self) -> None:
self._chapter_markers: List[Dict[str, Any]] = []
self._chunk_markers: List[Dict[str, Any]] = []
self._current_chapter_voices: Set[Tuple[str, str]] = set()
self._current_chapter_index: int = 0
self._current_chapter_title: str = ""
self._current_chapter_start: float = 0.0
def on_chapter_start(
self, index: int, title: str, start_time: float
) -> None:
"""Record chapter start."""
self._current_chapter_index = index
self._current_chapter_title = title
self._current_chapter_start = start_time
self._current_chapter_voices.clear()
def on_segment(
self,
provider: str,
voice: Any,
voice_spec: str,
speaker_id: str = "narrator",
) -> None:
"""Record a voice used in this chapter (for multi-speaker tracking)."""
self._current_chapter_voices.add((provider, voice_spec))
def on_chunk(
self,
chunk_id: str,
chapter_index: int,
chunk_index: int,
start: float,
end: float,
speaker_id: str,
provider: str,
voice_spec: str,
level: str,
characters: int,
) -> None:
"""Record a chunk marker."""
self._chunk_markers.append({
"id": chunk_id,
"chapter_index": chapter_index,
"chunk_index": chunk_index,
"start": start,
"end": end,
"speaker_id": speaker_id,
"voice": {"provider": provider, "voice": voice_spec},
"level": level,
"characters": characters,
})
def on_chapter_end(self, end_time: float) -> None:
"""Record chapter end and build chapter marker."""
voices = [
{"provider": p, "voice": v}
for p, v in sorted(self._current_chapter_voices)
]
self._chapter_markers.append({
"chapter_index": self._current_chapter_index,
"index": self._current_chapter_index + 1,
"title": self._current_chapter_title,
"start": self._current_chapter_start,
"end": end_time,
"voices": voices,
})
def on_outro(
self,
start_time: float,
end_time: float,
provider: str,
voice_spec: str,
) -> None:
"""Record outro chapter marker."""
self._chapter_markers.append({
"chapter_index": len(self._chapter_markers),
"index": len(self._chapter_markers) + 1,
"title": "Outro",
"start": start_time,
"end": end_time,
"voices": [{"provider": provider, "voice": voice_spec}],
})
@property
def chapter_markers(self) -> List[Dict[str, Any]]:
return self._chapter_markers
@property
def chunk_markers(self) -> List[Dict[str, Any]]:
return self._chunk_markers
def execute_conversion( def execute_conversion(
plan: ConversionPlan, plan: ConversionPlan,
events: ConversionEvents, events: ConversionEvents,
@@ -66,6 +171,15 @@ def execute_conversion(
""" """
request = plan.request request = plan.request
result = ConversionResult(metadata=plan.metadata) result = ConversionResult(metadata=plan.metadata)
collector = MarkerCollector()
logging.info(
"[executor] Starting: chapters=%d intro=%s outro=%s merge=%s",
len(plan.chapters),
bool(plan.intro and plan.intro.enabled),
bool(plan.outro and plan.outro.enabled),
request.save.merge_chapters_at_end,
)
# Determine cancellation checker # Determine cancellation checker
if check_cancelled is None: if check_cancelled is None:
@@ -88,7 +202,7 @@ def execute_conversion(
) )
# Compute subtitle flag once (used in every synthesize_text call) # Compute subtitle flag once (used in every synthesize_text call)
use_spacy = request.subtitle_mode not in (SubtitleMode.DISABLED, SubtitleMode.LINE) use_spacy = request.subtitle.mode not in (SubtitleMode.DISABLED, SubtitleMode.LINE)
# Output paths # Output paths
output_layout = plan.output_layout output_layout = plan.output_layout
@@ -96,15 +210,18 @@ def execute_conversion(
raise ValueError("ConversionPlan must have an output_layout") raise ValueError("ConversionPlan must have an output_layout")
# Determine if merged output is needed # Determine if merged output is needed
merge_chapters = request.merge_chapters_at_end or not request.save_chapters_separately merge_chapters = request.save.merge_chapters_at_end or not request.save.save_chapters_separately
if request.output_format == OutputFormat.M4B: if request.output_format == OutputFormat.M4B:
merge_chapters = True merge_chapters = True
# Resolve voices # Resolve voices
base_voice_spec = request.voice or "M1" base_voice_spec = request.voice or "M1"
logging.info("[executor] Resolving base voice: spec=%s", base_voice_spec)
base_provider, base_voice_choice, base_speed, base_steps = _resolve_voice( base_provider, base_voice_choice, base_speed, base_steps = _resolve_voice(
voice_resolver, base_voice_spec, request voice_resolver, base_voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
) )
logging.info("[executor] Base voice resolved: provider=%s voice=%s speed=%.2f", base_provider, base_voice_choice, base_speed)
# Use ExitStack for resource management # Use ExitStack for resource management
with ExitStack() as stack: with ExitStack() as stack:
@@ -112,7 +229,7 @@ def execute_conversion(
audio_sink: Optional[AudioSink] = None audio_sink: Optional[AudioSink] = None
audio_path = None audio_path = None
if merge_chapters: if merge_chapters:
audio_path = output_layout.audio_dir / f"{_base_name(request)}.{request.output_format}" audio_path = output_layout.audio_dir / f"{_base_name(request)}{request.output_format.dot_ext}"
meta = plan.metadata if plan.metadata else None meta = plan.metadata if plan.metadata else None
audio_sink = stack.enter_context( audio_sink = stack.enter_context(
open_audio_sink( open_audio_sink(
@@ -126,19 +243,17 @@ def execute_conversion(
# Open subtitle writer if needed # Open subtitle writer if needed
subtitle_writer: Optional[SubtitleWriter] = None subtitle_writer: Optional[SubtitleWriter] = None
if request.subtitle_mode != SubtitleMode.DISABLED and audio_sink: if request.subtitle.mode != SubtitleMode.DISABLED and audio_sink:
subtitle_writer = make_subtitle_writer( subtitle_writer = make_subtitle_writer(
audio_path, audio_path,
request.subtitle_format, request.subtitle,
request.subtitle_mode,
max_words=request.max_subtitle_words,
) )
if subtitle_writer: if subtitle_writer:
subtitle_writer.open() subtitle_writer.open()
stack.callback(subtitle_writer.close) stack.callback(subtitle_writer.close)
result.subtitle_paths.append(subtitle_writer.path) result.subtitle_paths.append(subtitle_writer.path)
effective_subtitle_mode = request.subtitle_mode if subtitle_writer else SubtitleMode.DISABLED effective_subtitle_mode = request.subtitle.mode if subtitle_writer else SubtitleMode.DISABLED
synth = SynthParams( synth = SynthParams(
tts_context=tts_context, tts_context=tts_context,
@@ -147,14 +262,14 @@ def execute_conversion(
on_progress=lambda pct, etr: events.progress(pct, etr), on_progress=lambda pct, etr: events.progress(pct, etr),
audio_sink=audio_sink, audio_sink=audio_sink,
subtitle_mode=effective_subtitle_mode, subtitle_mode=effective_subtitle_mode,
max_subtitle_words=request.max_subtitle_words, max_subtitle_words=request.subtitle.max_words,
lang_code=request.language, language=request.language,
use_spacy_segmentation=use_spacy, use_spacy_segmentation=use_spacy,
) )
# Chapter directory # Chapter directory
chapter_dir = None chapter_dir = None
if request.save_chapters_separately and len(plan.chapters) > 1: if request.save.save_chapters_separately and len(plan.chapters) > 1:
chapter_dir = output_layout.audio_dir / "chapters" chapter_dir = output_layout.audio_dir / "chapters"
chapter_dir.mkdir(parents=True, exist_ok=True) chapter_dir.mkdir(parents=True, exist_ok=True)
@@ -163,7 +278,8 @@ def execute_conversion(
if plan.intro and plan.intro.enabled and merge_chapters: if plan.intro and plan.intro.enabled and merge_chapters:
events.log(f"Title intro: {plan.intro.text[:80]}") events.log(f"Title intro: {plan.intro.text[:80]}")
intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice( intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice(
voice_resolver, plan.intro.voice_spec, request voice_resolver, plan.intro.voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
) )
intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu) intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu)
synthesize_text( synthesize_text(
@@ -172,6 +288,7 @@ def execute_conversion(
backend=intro_backend, backend=intro_backend,
voice=intro_voice, voice=intro_voice,
speed=intro_speed or request.speed, speed=intro_speed or request.speed,
total_steps=intro_steps,
chapter_sink=None, chapter_sink=None,
preview_callback=lambda text: events.log(f" {text[:80]}"), preview_callback=lambda text: events.log(f" {text[:80]}"),
) )
@@ -184,33 +301,58 @@ def execute_conversion(
chapter_display = f"Chapter {chapter_idx}/{len(plan.chapters)}: {chapter.title}" chapter_display = f"Chapter {chapter_idx}/{len(plan.chapters)}: {chapter.title}"
events.log(f"Processing {chapter_display}") events.log(f"Processing {chapter_display}")
logging.info("[executor] Chapter %d/%d: %s", chapter_idx, len(plan.chapters), chapter.title)
# Resolve chapter voice # Resolve chapter voice
chapter_provider, chapter_voice, chapter_speed, chapter_steps = _resolve_voice( chapter_provider, chapter_voice, chapter_speed, chapter_steps = _resolve_voice(
voice_resolver, chapter.voice_spec, request voice_resolver, chapter.voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
) )
logging.info("[executor] Chapter %d voice: provider=%s voice=%s speed=%.2f", chapter_idx, chapter_provider, chapter_voice, chapter_speed)
chapter_backend = pipeline_provider.get(chapter_provider, request.language, request.use_gpu) chapter_backend = pipeline_provider.get(chapter_provider, request.language, request.use_gpu)
# Record chapter start for markers
collector.on_chapter_start(chapter_idx - 1, chapter.title, stats.current_time)
# Per-chapter sink # Per-chapter sink
chapter_sink: Optional[AudioSink] = None chapter_sink: Optional[AudioSink] = None
chapter_path = None chapter_path = None
if chapter_dir: if chapter_dir:
chapter_filename = sanitize_filename_for_chapter(chapter.title, chapter_idx) chapter_filename = sanitize_filename_for_chapter(chapter.title, chapter_idx)
chapter_path = chapter_dir / f"{chapter_filename}.{request.separate_chapters_format}" chapter_path = chapter_dir / f"{chapter_filename}.{request.save.separate_chapters_format}"
chapter_sink = stack.enter_context( chapter_sink = stack.enter_context(
open_audio_sink( open_audio_sink(
chapter_path, chapter_path,
request.separate_chapters_format, request.save.separate_chapters_format,
cancel_check=check_cancelled, cancel_check=check_cancelled,
) )
) )
result.chapter_paths.append(chapter_path) result.chapter_paths.append(chapter_path)
# Per-chapter subtitle writer
chapter_subtitle_writer: Optional[SubtitleWriter] = None
if chapter_dir and request.subtitle.mode != SubtitleMode.DISABLED and chapter_sink:
from abogen.infrastructure.subtitle_writer import resolve_subtitle_format
chapter_filename = sanitize_filename_for_chapter(chapter.title, chapter_idx)
subtitle_ext, _ = resolve_subtitle_format(
request.subtitle
)
chapter_subtitle_path = chapter_dir / f"{chapter_filename}.{subtitle_ext}"
chapter_subtitle_writer = make_subtitle_writer(
chapter_subtitle_path,
request.subtitle,
)
if chapter_subtitle_writer:
chapter_subtitle_writer.open()
result.subtitle_paths.append(chapter_subtitle_writer.path)
# Intro delay before first chapter # Intro delay before first chapter
if not intro_emitted and plan.intro and plan.intro.enabled: if not intro_emitted and plan.intro and plan.intro.enabled:
# Intro will be emitted with first chapter # Intro will be emitted with first chapter
intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice( intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice(
voice_resolver, plan.intro.voice_spec, request voice_resolver, plan.intro.voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
) )
intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu) intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu)
synthesize_text( synthesize_text(
@@ -219,6 +361,7 @@ def execute_conversion(
backend=intro_backend, backend=intro_backend,
voice=intro_voice, voice=intro_voice,
speed=intro_speed or request.speed, speed=intro_speed or request.speed,
total_steps=intro_steps,
chapter_sink=chapter_sink, chapter_sink=chapter_sink,
preview_callback=lambda text: events.log(f" Intro: {text[:80]}"), preview_callback=lambda text: events.log(f" Intro: {text[:80]}"),
) )
@@ -232,6 +375,7 @@ def execute_conversion(
) )
# Process heading # Process heading
heading_text = ""
if chapter.title: if chapter.title:
heading_text = _format_heading(chapter.title, chapter_idx, request) heading_text = _format_heading(chapter.title, chapter_idx, request)
if heading_text: if heading_text:
@@ -252,59 +396,120 @@ def execute_conversion(
stats=stats, stats=stats,
) )
# Heading dedup: check if first line of body matches heading
pending_heading_strip = False
if heading_text and chapter.body_text:
first_line = next(
(line.strip() for line in chapter.body_text.splitlines() if line.strip()),
"",
)
if first_line and _headings_equivalent(first_line, heading_text):
pending_heading_strip = True
# Process body segments # Process body segments
chapter_chunk_markers: List[Dict[str, Any]] = []
for seg_idx, segment in enumerate(chapter.segments): for seg_idx, segment in enumerate(chapter.segments):
check_cancelled() check_cancelled()
# Apply heading dedup to first segment (consume-once)
seg_text = segment.text
if pending_heading_strip and seg_text.strip():
seg_text, heading_removed, _ = apply_chapter_text_transforms(
seg_text,
heading_text=heading_text,
raw_title=chapter.title,
strip_heading=True,
normalize_caps=False,
)
if heading_removed:
pending_heading_strip = False
if not seg_text.strip():
continue
# Resolve segment voice (may differ from chapter voice) # Resolve segment voice (may differ from chapter voice)
if segment.voice_spec != chapter.voice_spec: if segment.voice_spec != chapter.voice_spec:
seg_provider, seg_voice, seg_speed, seg_steps = _resolve_voice( seg_provider, seg_voice, seg_speed, seg_steps = _resolve_voice(
voice_resolver, segment.voice_spec, request voice_resolver, segment.voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
) )
seg_backend = pipeline_provider.get(seg_provider, request.language, request.use_gpu) seg_backend = pipeline_provider.get(seg_provider, request.language, request.use_gpu)
else: else:
seg_provider = chapter_provider seg_provider = chapter_provider
seg_voice = chapter_voice seg_voice = chapter_voice
seg_speed = chapter_speed seg_speed = chapter_speed
seg_steps = chapter_steps
seg_backend = chapter_backend seg_backend = chapter_backend
seg_start_time = stats.current_time # Track voice for chapter marker
local_segments, accumulated_tokens = synthesize_text( collector.on_segment(seg_provider, seg_voice, segment.voice_spec)
text=segment.text,
params=synth, # spaCy pre-TTS segmentation
backend=seg_backend, from abogen.domain.conversion_pipeline import spacy_pre_tts_segmentation
voice=seg_voice,
speed=seg_speed or request.speed, is_subtitle_input = bool(
chapter_sink=chapter_sink, request.subtitle_input
preview_callback=lambda text: events.log(f" {text[:80]}"), )
spacy_segments, active_split = spacy_pre_tts_segmentation(
seg_text,
request.language,
request.subtitle.mode,
is_subtitle_input=is_subtitle_input,
use_spacy_segmentation=use_spacy,
log_callback=lambda msg: events.log(msg),
) )
# Process subtitles seg_start_time = stats.current_time
if subtitle_writer and audio_sink and accumulated_tokens: accumulated_tokens: List[Dict[str, Any]] = []
process_and_write_subtitles( for spacy_seg in spacy_segments:
accumulated_tokens, if not spacy_seg.strip():
subtitle_writer, continue
subtitle_mode=request.subtitle_mode, _, seg_tokens = synthesize_text(
max_subtitle_words=request.max_subtitle_words, text=spacy_seg,
lang_code=request.language, params=synth,
use_spacy_segmentation=use_spacy, backend=seg_backend,
fallback_end_time=stats.current_time, voice=seg_voice,
speed=seg_speed or request.speed,
total_steps=seg_steps,
chapter_sink=chapter_sink,
preview_callback=lambda text: events.log(f" {text[:80]}"),
split_pattern_override=active_split,
) )
accumulated_tokens.extend(seg_tokens)
# Process subtitles
if audio_sink and accumulated_tokens:
if subtitle_writer:
process_and_write_subtitles(
accumulated_tokens,
subtitle_writer,
subtitle=request.subtitle,
language=request.language,
use_spacy_segmentation=use_spacy,
fallback_end_time=stats.current_time,
)
if chapter_subtitle_writer:
process_and_write_subtitles(
accumulated_tokens,
chapter_subtitle_writer,
subtitle=request.subtitle,
language=request.language,
use_spacy_segmentation=use_spacy,
fallback_end_time=stats.current_time,
)
# Record chunk marker # Record chunk marker
if segment.source in ("chunk", "voice_marker"): if segment.source in ("chunk", "voice_marker"):
chapter_chunk_markers.append({ collector.on_chunk(
"id": segment.chunk_id, chunk_id=segment.chunk_id or "",
"chapter_index": chapter_idx - 1, chapter_index=chapter_idx - 1,
"chunk_index": segment.chunk_index or seg_idx, chunk_index=segment.chunk_index or seg_idx,
"start": seg_start_time, start=seg_start_time,
"end": stats.current_time, end=stats.current_time,
"speaker_id": segment.speaker_id, speaker_id=segment.speaker_id or "narrator",
"voice": segment.voice_spec, provider=seg_provider,
"level": segment.level or request.chunk_level, voice_spec=segment.voice_spec,
"characters": len(segment.text), level=segment.level or (request.chapter_chunk.chunk_level if request.chapter_chunk else "paragraph"),
}) characters=len(segment.text),
)
# Silence between chapters # Silence between chapters
if chapter_idx < len(plan.chapters) and request.silence_between_chapters > 0: if chapter_idx < len(plan.chapters) and request.silence_between_chapters > 0:
@@ -319,21 +524,22 @@ def execute_conversion(
if chapter_sink: if chapter_sink:
chapter_sink.close() chapter_sink.close()
# Add chapter marker # Close chapter subtitle writer
result.chapter_markers.append({ if chapter_subtitle_writer:
"chapter_index": chapter_idx - 1, chapter_subtitle_writer.close()
"title": chapter.title,
"start": stats.current_time - (stats.current_time - seg_start_time) if chapter.segments else stats.current_time,
"end": stats.current_time,
})
result.chunk_markers.extend(chapter_chunk_markers) # Record chapter end for markers
collector.on_chapter_end(stats.current_time)
logging.info("[executor] Chapter %d/%d done: time=%.1fs", chapter_idx, len(plan.chapters), stats.current_time)
logging.info("[executor] All chapters done: total=%.1fs", stats.current_time)
# Process outro # Process outro
if plan.outro and plan.outro.enabled and merge_chapters: if plan.outro and plan.outro.enabled and merge_chapters:
events.log(f"Closing outro: {plan.outro.text[:80]}") events.log(f"Closing outro: {plan.outro.text[:80]}")
outro_provider, outro_voice, outro_speed, outro_steps = _resolve_voice( outro_provider, outro_voice, outro_speed, outro_steps = _resolve_voice(
voice_resolver, plan.outro.voice_spec, request voice_resolver, plan.outro.voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
) )
outro_backend = pipeline_provider.get(outro_provider, request.language, request.use_gpu) outro_backend = pipeline_provider.get(outro_provider, request.language, request.use_gpu)
@@ -346,18 +552,24 @@ def execute_conversion(
stats=stats, stats=stats,
) )
outro_start = stats.current_time
synthesize_text( synthesize_text(
text=plan.outro.text, text=plan.outro.text,
params=synth, params=synth,
backend=outro_backend, backend=outro_backend,
voice=outro_voice, voice=outro_voice,
speed=outro_speed or request.speed, speed=outro_speed or request.speed,
total_steps=outro_steps,
chapter_sink=None, chapter_sink=None,
preview_callback=lambda text: events.log(f" {text[:80]}"), preview_callback=lambda text: events.log(f" {text[:80]}"),
) )
# Record outro marker
collector.on_outro(outro_start, stats.current_time, outro_provider, plan.outro.voice_spec)
events.log("Outro synthesized.") events.log("Outro synthesized.")
# Set result metadata # Set result metadata
result.chapter_markers = collector.chapter_markers
result.chunk_markers = collector.chunk_markers
result.total_chapters = len(plan.chapters) result.total_chapters = len(plan.chapters)
result.total_segments = sum(len(ch.segments) for ch in plan.chapters) result.total_segments = sum(len(ch.segments) for ch in plan.chapters)
result.total_characters = total_characters result.total_characters = total_characters
@@ -375,6 +587,8 @@ def _resolve_voice(
resolver: VoiceResolver, resolver: VoiceResolver,
voice_spec: str, voice_spec: str,
request: Any, request: Any,
*,
log_callback: Optional[Callable[[str], None]] = None,
) -> Tuple[str, Any, Optional[float], Optional[int]]: ) -> Tuple[str, Any, Optional[float], Optional[int]]:
"""Resolve a voice spec and return (provider, voice, speed, steps).""" """Resolve a voice spec and return (provider, voice, speed, steps)."""
try: try:
@@ -385,9 +599,21 @@ def _resolve_voice(
resolved.speed, resolved.speed,
resolved.supertonic_steps, resolved.supertonic_steps,
) )
except Exception: except Exception as exc:
# Fallback to base voice # Fallback to base voice
resolved = resolver.resolve(request.voice or "M1") base_spec = request.voice or "M1"
if log_callback:
log_callback(
f"Voice '{voice_spec}' failed to resolve: {exc}. "
f"Falling back to '{base_spec}'."
)
try:
resolved = resolver.resolve(base_spec)
except Exception as fallback_exc:
raise RuntimeError(
f"Both voice '{voice_spec}' and fallback '{base_spec}' failed to resolve. "
f"Primary error: {exc}; Fallback error: {fallback_exc}"
) from fallback_exc
return ( return (
resolved.provider, resolved.provider,
resolved.voice, resolved.voice,
+3 -1
View File
@@ -15,12 +15,13 @@ The planning flow:
from __future__ import annotations from __future__ import annotations
from dataclasses import dataclass, field from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING, Any, Dict, List, Optional from typing import TYPE_CHECKING, Any, Dict, List, Optional
if TYPE_CHECKING: if TYPE_CHECKING:
from abogen.application.conversion_request import ConversionRequest from abogen.application.conversion_request import ConversionRequest
from abogen.text_extractor import ExtractionResult
@dataclass @dataclass
@@ -93,3 +94,4 @@ class ConversionPlan:
intro: Optional[IntroOutroSpec] = None intro: Optional[IntroOutroSpec] = None
outro: Optional[IntroOutroSpec] = None outro: Optional[IntroOutroSpec] = None
output_layout: Optional[OutputLayout] = None output_layout: Optional[OutputLayout] = None
extraction: Optional[ExtractionResult] = None
+70 -25
View File
@@ -8,14 +8,13 @@ This is Stage 2 of the conversion flow unification plan.
from __future__ import annotations from __future__ import annotations
from pathlib import Path import logging
from typing import Any, Dict, List, Optional, Tuple from typing import Any, Dict, List, Optional, Tuple
from abogen.application.conversion_models import ( from abogen.application.conversion_models import (
ChapterPlan, ChapterPlan,
ConversionPlan, ConversionPlan,
IntroOutroSpec, IntroOutroSpec,
OutputLayout,
SegmentPlan, SegmentPlan,
) )
from abogen.application.conversion_request import ConversionRequest from abogen.application.conversion_request import ConversionRequest
@@ -25,7 +24,7 @@ from abogen.domain.file_type import auto_select_relevant_chapters
from abogen.domain.intro_outro import resolve_intro, resolve_outro from abogen.domain.intro_outro import resolve_intro, resolve_outro
from abogen.domain.metadata_extraction import extract_metadata_for_file from abogen.domain.metadata_extraction import extract_metadata_for_file
from abogen.domain.metadata_merge import merge_metadata from abogen.domain.metadata_merge import merge_metadata
from abogen.subtitle_utils import split_text_by_voice_markers from abogen.domain.voice_markers import split_text_by_voice_markers
def build_conversion_plan(request: ConversionRequest) -> ConversionPlan: def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
@@ -50,7 +49,7 @@ def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
raise ValueError("No text content to convert") raise ValueError("No text content to convert")
# 2. Extract metadata # 2. Extract metadata
metadata = _extract_metadata(request) metadata, extraction = _extract_metadata(request)
# 3. Parse chapters # 3. Parse chapters
raw_chapters = _parse_chapters(source_text, request) raw_chapters = _parse_chapters(source_text, request)
@@ -67,6 +66,13 @@ def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
# 7. Resolve output layout # 7. Resolve output layout
output_layout = resolve_output_layout(request) output_layout = resolve_output_layout(request)
logging.info(
"[planner] Plan built: chapters=%d intro=%s outro=%s",
len(chapters),
bool(intro and intro.enabled),
bool(outro and outro.enabled),
)
return ConversionPlan( return ConversionPlan(
request=request, request=request,
metadata=metadata, metadata=metadata,
@@ -74,6 +80,7 @@ def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
intro=intro, intro=intro,
outro=outro, outro=outro,
output_layout=output_layout, output_layout=output_layout,
extraction=extraction,
) )
@@ -82,22 +89,44 @@ def _extract_source_text(request: ConversionRequest) -> Optional[str]:
from abogen.subtitle_utils import clean_text from abogen.subtitle_utils import clean_text
if request.direct_text: if request.direct_text:
return clean_text(request.direct_text) text = clean_text(request.direct_text)
if request.source_path and request.source_path.exists(): elif request.source_path and request.source_path.exists():
encoding = "utf-8" encoding = "utf-8"
try: try:
with open(request.source_path, "r", encoding=encoding, errors="replace") as f: with open(request.source_path, "r", encoding=encoding, errors="replace") as f:
text = f.read() text = f.read()
except Exception: except Exception:
return None return None
return clean_text(text) text = clean_text(text)
return None else:
return None
# Apply word substitutions if configured
if request.word_substitution:
from abogen.word_substitution import apply_word_substitutions
ws = request.word_substitution
text = apply_word_substitutions(
text,
ws.substitutions_list,
ws.case_sensitive,
ws.replace_caps,
ws.replace_numerals,
ws.fix_punctuation,
)
return text
def _extract_metadata(request: ConversionRequest) -> Dict[str, Any]: def _extract_metadata(
"""Extract metadata from source file.""" request: ConversionRequest,
) -> Tuple[Dict[str, Any], Optional[Any]]:
"""Extract metadata from source file.
Returns (metadata, extraction) tuple.
"""
if request.direct_text: if request.direct_text:
return dict(request.metadata_tags) return dict(request.metadata_tags), None
if request.source_path and request.source_path.exists(): if request.source_path and request.source_path.exists():
try: try:
@@ -106,11 +135,12 @@ def _extract_metadata(request: ConversionRequest) -> Dict[str, Any]:
) )
metadata = dict(extraction.metadata) if extraction.metadata else {} metadata = dict(extraction.metadata) if extraction.metadata else {}
except Exception: except Exception:
extraction = None
metadata = {} metadata = {}
metadata = merge_metadata(metadata, request.metadata_tags) metadata = merge_metadata(metadata, request.metadata_tags)
return metadata return metadata, extraction
return dict(request.metadata_tags) return dict(request.metadata_tags), None
def _parse_chapters( def _parse_chapters(
@@ -144,8 +174,9 @@ def _apply_selection(
] ]
# If user specified chapters, apply overrides # If user specified chapters, apply overrides
if request.chapter_overrides: chapter_chunk = request.chapter_chunk
selected, _, diagnostics = apply_chapter_overrides(extracted, request.chapter_overrides) if chapter_chunk and chapter_chunk.chapter_overrides:
selected, _, diagnostics = apply_chapter_overrides(extracted, chapter_chunk.chapter_overrides)
if selected: if selected:
# Map back to (title, text, voice) tuples # Map back to (title, text, voice) tuples
result = [] result = []
@@ -187,11 +218,17 @@ def _build_chapters(
selected_chapters: List[Tuple[str, str, str]], request: ConversionRequest selected_chapters: List[Tuple[str, str, str]], request: ConversionRequest
) -> List[ChapterPlan]: ) -> List[ChapterPlan]:
"""Build ChapterPlan with SegmentPlan for each chapter.""" """Build ChapterPlan with SegmentPlan for each chapter."""
from abogen.domain.chapter_titles import normalize_chapter_opening_caps
chapters = [] chapters = []
for idx, (title, body_text, default_voice) in enumerate(selected_chapters, 1): for idx, (title, body_text, default_voice) in enumerate(selected_chapters, 1):
# Build segments for this chapter # Apply caps normalization to body text if enabled
segments = _build_segments(body_text, default_voice, request) if request.normalize_chapter_opening_caps and body_text:
body_text, _ = normalize_chapter_opening_caps(body_text)
# Build segments for this chapter (idx is 1-based, chunks use 0-based)
segments = _build_segments(body_text, default_voice, request, chapter_index=idx - 1)
chapter = ChapterPlan( chapter = ChapterPlan(
index=idx, index=idx,
@@ -207,7 +244,8 @@ def _build_chapters(
def _build_segments( def _build_segments(
body_text: str, default_voice: str, request: ConversionRequest body_text: str, default_voice: str, request: ConversionRequest,
chapter_index: int = 0,
) -> List[SegmentPlan]: ) -> List[SegmentPlan]:
"""Build SegmentPlan list for a chapter's body text. """Build SegmentPlan list for a chapter's body text.
@@ -216,9 +254,15 @@ def _build_segments(
segments = [] segments = []
# Check for chunks (WebUI style) # Check for chunks (WebUI style)
if request.chunks: chapter_chunk = request.chapter_chunk
# Group chunks by chapter (simplified — assume chunks are for current chapter) if chapter_chunk and chapter_chunk.chunks:
for chunk_idx, chunk in enumerate(request.chunks): # Group chunks by chapter index
from abogen.domain.chunk_utils import group_chunks_by_chapter
chunk_groups = group_chunks_by_chapter(chapter_chunk.chunks)
chunks_for_chapter = chunk_groups.get(chapter_index, [])
for chunk_idx, chunk in enumerate(chunks_for_chapter):
chunk_text = chunk.get("normalized_text") or chunk.get("text", "") chunk_text = chunk.get("normalized_text") or chunk.get("text", "")
if not chunk_text or not chunk_text.strip(): if not chunk_text or not chunk_text.strip():
continue continue
@@ -234,7 +278,7 @@ def _build_segments(
speaker_id=speaker_id, speaker_id=speaker_id,
chunk_id=chunk.get("id"), chunk_id=chunk.get("id"),
chunk_index=chunk.get("chunk_index", chunk_idx), chunk_index=chunk.get("chunk_index", chunk_idx),
level=chunk.get("level", request.chunk_level), level=chunk.get("level", chapter_chunk.chunk_level),
source="chunk", source="chunk",
) )
) )
@@ -242,7 +286,7 @@ def _build_segments(
# Check for voice markers (PyQt style) # Check for voice markers (PyQt style)
# Detect markers even if validation fails (voice names may not be loaded yet) # Detect markers even if validation fails (voice names may not be loaded yet)
from abogen.subtitle_utils import _VOICE_MARKER_SEARCH_PATTERN from abogen.domain.voice_markers import _VOICE_MARKER_SEARCH_PATTERN
has_voice_markers = bool(_VOICE_MARKER_SEARCH_PATTERN.search(body_text)) has_voice_markers = bool(_VOICE_MARKER_SEARCH_PATTERN.search(body_text))
voice_segments, last_voice, valid_count, invalid_count = split_text_by_voice_markers( voice_segments, last_voice, valid_count, invalid_count = split_text_by_voice_markers(
@@ -284,8 +328,9 @@ def _resolve_chunk_voice(
"""Resolve voice for a chunk.""" """Resolve voice for a chunk."""
# Check for speaker-based voice # Check for speaker-based voice
speaker_id = chunk.get("speaker_id", "narrator") speaker_id = chunk.get("speaker_id", "narrator")
if speaker_id and speaker_id != "narrator" and request.speakers: speakers = request.chapter_chunk.speakers if request.chapter_chunk else {}
speaker_config = request.speakers.get(speaker_id, {}) if speaker_id and speaker_id != "narrator" and speakers:
speaker_config = speakers.get(speaker_id, {})
if isinstance(speaker_config, dict): if isinstance(speaker_config, dict):
voice = speaker_config.get("voice") voice = speaker_config.get("voice")
if voice: if voice:
+1 -1
View File
@@ -10,7 +10,7 @@ implementations (PyQt signals, Flask Job, etc.).
from __future__ import annotations from __future__ import annotations
from dataclasses import dataclass from dataclasses import dataclass
from typing import Any, List, Optional, Protocol, runtime_checkable from typing import Any, Protocol
class ConversionCancelled(Exception): class ConversionCancelled(Exception):
+47 -48
View File
@@ -14,7 +14,17 @@ from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import Any, Dict, List, Optional from typing import Any, Dict, List, Optional
from abogen.domain.enums import Language, OutputFormat, SaveMode, SubtitleFormat, SubtitleMode from abogen.application.conversion_config import (
ChapterChunkConfig,
CoverConfig,
Epub3ExportConfig,
PronunciationConfig,
SaveConfig,
SubtitleConfig,
SubtitleInputConfig,
WordSubstitutionConfig,
)
from abogen.domain.enums import Language, OutputFormat
class ConversionRequestError(ValueError): class ConversionRequestError(ValueError):
@@ -23,19 +33,12 @@ class ConversionRequestError(ValueError):
# Numeric field constraints: attr -> (min, max) # Numeric field constraints: attr -> (min, max)
_NUMERIC_CONSTRAINTS: dict[str, tuple[float, float | None]] = { _NUMERIC_CONSTRAINTS: dict[str, tuple[float, float | None]] = {
"max_subtitle_words": (1, 500),
"speed": (0.5, 3.0), "speed": (0.5, 3.0),
"supertonic_total_steps": (2, 15), "supertonic_total_steps": (2, 15),
"silence_between_chapters": (0.0, None), "silence_between_chapters": (0.0, None),
"chapter_intro_delay": (0.0, None), "chapter_intro_delay": (0.0, None),
} }
# Enum-like fields that must be in allowed set
_ENUM_CONSTRAINTS: dict[str, tuple[str, ...]] = {
"chunk_level": ("paragraph", "sentence"),
"speaker_mode": ("single", "multi"),
}
@dataclass @dataclass
class ConversionRequest: class ConversionRequest:
@@ -44,10 +47,14 @@ class ConversionRequest:
Only contains fields that describe the conversion task itself. Only contains fields that describe the conversion task itself.
UI-only fields (display, logging, user prompts) stay in adapters. UI-only fields (display, logging, user prompts) stay in adapters.
Feature toggles use config objects (None = disabled):
- word_substitution, subtitle_input, chapter_chunk, epub3_export
- pronunciation (raw data, compiled by app layer)
- subtitle, save, cover (grouped parameters)
Validation runs on creation via __post_init__: Validation runs on creation via __post_init__:
- None values → replaced with field default (from declaration) - None values → replaced with field default (from declaration)
- Numeric fields → clamped to valid range - Numeric fields → clamped to valid range
- String enums → validated against allowed set
""" """
# --- Source --- # --- Source ---
@@ -66,17 +73,6 @@ class ConversionRequest:
# --- Output Format --- # --- Output Format ---
output_format: OutputFormat = OutputFormat.WAV output_format: OutputFormat = OutputFormat.WAV
subtitle_mode: SubtitleMode = SubtitleMode.DISABLED
subtitle_format: SubtitleFormat = SubtitleFormat.SRT
max_subtitle_words: int = 50
# --- Save Options ---
save_mode: SaveMode = SaveMode.SAVE_NEXT_TO_INPUT
output_folder: Optional[Path] = None
save_chapters_separately: bool = False
merge_chapters_at_end: bool = True
separate_chapters_format: OutputFormat = OutputFormat.WAV
save_as_project: bool = False
# --- Timing --- # --- Timing ---
silence_between_chapters: float = 2.0 silence_between_chapters: float = 2.0
@@ -89,34 +85,28 @@ class ConversionRequest:
auto_prefix_chapter_titles: bool = True auto_prefix_chapter_titles: bool = True
normalize_chapter_opening_caps: bool = False normalize_chapter_opening_caps: bool = False
# --- Pronunciation / Normalization ---
pronunciation_overrides: List[Dict[str, Any]] = field(default_factory=list)
manual_overrides: List[Dict[str, Any]] = field(default_factory=list)
heteronym_overrides: List[Dict[str, Any]] = field(default_factory=list)
normalization_overrides: Optional[Dict[str, Any]] = None
# --- Chapter/Chunk Configuration ---
chapter_overrides: List[Dict[str, Any]] = field(default_factory=list)
chunks: List[Dict[str, Any]] = field(default_factory=list)
chunk_level: str = "paragraph"
speaker_mode: str = "single"
speakers: Dict[str, Any] = field(default_factory=dict)
# --- Metadata --- # --- Metadata ---
metadata_tags: Dict[str, Any] = field(default_factory=dict) metadata_tags: Dict[str, Any] = field(default_factory=dict)
# --- Artifacts --- # --- Grouped configs ---
cover_image_path: Optional[Path] = None subtitle: SubtitleConfig = field(default_factory=SubtitleConfig)
cover_image_mime: Optional[str] = None save: SaveConfig = field(default_factory=SaveConfig)
generate_epub3: bool = False cover: CoverConfig = field(default_factory=CoverConfig)
pronunciation: PronunciationConfig = field(default_factory=PronunciationConfig)
# --- Feature configs (None = disabled) ---
epub3_export: Optional[Epub3ExportConfig] = None
word_substitution: Optional[WordSubstitutionConfig] = None
subtitle_input: Optional[SubtitleInputConfig] = None
chapter_chunk: Optional[ChapterChunkConfig] = None
def __post_init__(self) -> None: def __post_init__(self) -> None:
"""Resolve None → default, then validate and clamp.""" """Resolve None → default, then validate and clamp."""
_apply_none_defaults(self) _apply_none_defaults(self)
if not self.tts_provider: if not self.tts_provider:
self.tts_provider = "kokoro" self.tts_provider = "kokoro"
_coerce_enums(self)
_clamp_numerics(self) _clamp_numerics(self)
_validate_enums(self)
def _apply_none_defaults(obj: ConversionRequest) -> None: def _apply_none_defaults(obj: ConversionRequest) -> None:
@@ -130,6 +120,25 @@ def _apply_none_defaults(obj: ConversionRequest) -> None:
setattr(obj, f.name, f.default_factory()) setattr(obj, f.name, f.default_factory())
# Enum fields that accept string coercion: attr -> (enum_class, fallback)
_ENUM_COERCIONS: dict[str, tuple[type, Any]] = {
"language": (Language, Language.EN_US),
"output_format": (OutputFormat, OutputFormat.WAV),
}
def _coerce_enums(obj: ConversionRequest) -> None:
"""Coerce string values to their expected enum types."""
for attr, (enum_cls, fallback) in _ENUM_COERCIONS.items():
val = getattr(obj, attr)
if isinstance(val, enum_cls):
continue
try:
setattr(obj, attr, enum_cls.from_str(str(val)))
except (ValueError, AttributeError):
setattr(obj, attr, fallback)
def _clamp_numerics(obj: ConversionRequest) -> None: def _clamp_numerics(obj: ConversionRequest) -> None:
"""Clamp numeric fields to valid ranges.""" """Clamp numeric fields to valid ranges."""
for attr, (min_v, max_v) in _NUMERIC_CONSTRAINTS.items(): for attr, (min_v, max_v) in _NUMERIC_CONSTRAINTS.items():
@@ -144,13 +153,3 @@ def _clamp_numerics(obj: ConversionRequest) -> None:
if max_v is not None: if max_v is not None:
clamped = min(max_v, clamped) clamped = min(max_v, clamped)
setattr(obj, attr, clamped) setattr(obj, attr, clamped)
def _validate_enums(obj: ConversionRequest) -> None:
"""Validate string enum fields against allowed values."""
for attr, allowed in _ENUM_CONSTRAINTS.items():
val = getattr(obj, attr)
if val not in allowed:
raise ConversionRequestError(
f"{attr} must be one of {allowed}, got {val!r}"
)
+3
View File
@@ -37,6 +37,9 @@ class ConversionResult:
total_segments: int = 0 total_segments: int = 0
total_characters: int = 0 total_characters: int = 0
# --- Override usage tracking ---
usage_counter: Dict[str, int] = field(default_factory=dict)
@dataclass @dataclass
class ConversionError: class ConversionError:
+167 -89
View File
@@ -15,42 +15,35 @@ The service NEVER imports from PyQt or WebUI.
from __future__ import annotations from __future__ import annotations
from typing import Any, Callable, Dict, Optional import logging
from collections import defaultdict
from typing import Any, Dict
from abogen.application.conversion_executor import execute_conversion from abogen.application.conversion_executor import execute_conversion
from abogen.application.conversion_models import ConversionPlan from abogen.application.conversion_models import ConversionPlan
from abogen.application.conversion_planner import build_conversion_plan from abogen.application.conversion_planner import build_conversion_plan
from abogen.application.conversion_ports import ( from abogen.application.conversion_ports import ConversionEvents
ConversionEvents,
PipelineProvider,
VoiceResolver,
)
from abogen.application.conversion_request import ConversionRequest from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_result import ConversionResult from abogen.application.conversion_result import ConversionResult
from abogen.domain.enums import SubtitleMode from abogen.domain.normalization import build_tts_context
from abogen.domain.normalization import TTSContext
from abogen.domain.split_pattern import get_split_pattern
def run_conversion( def run_conversion(
request: ConversionRequest, request: ConversionRequest,
events: ConversionEvents, events: ConversionEvents,
pipeline_provider: PipelineProvider,
voice_resolver: VoiceResolver,
) -> ConversionResult: ) -> ConversionResult:
"""Execute a conversion request and return the result. """Execute a conversion request and return the result.
This is the single entry point for both UIs. It orchestrates: This is the single entry point for both UIs. It orchestrates:
1. TTS context preparation 1. Voice infrastructure setup (pool, cache, resolver)
2. Conversion planning 2. TTS context preparation
3. Conversion execution 3. Conversion planning
4. Resource cleanup 4. Conversion execution
5. Resource cleanup
Args: Args:
request: Normalized conversion request request: Normalized conversion request
events: UI-specific callbacks (log, progress, check_cancelled) events: UI-specific callbacks (log, progress, check_cancelled)
pipeline_provider: Provides TTS backends
voice_resolver: Resolves voice specs into loaded voices
Returns: Returns:
ConversionResult with paths and markers ConversionResult with paths and markers
@@ -60,10 +53,30 @@ def run_conversion(
ValueError: If request is invalid ValueError: If request is invalid
Exception: On TTS or I/O errors Exception: On TTS or I/O errors
""" """
from abogen.domain.pipeline_factory import PipelinePool
from abogen.domain.voice_loader import VoiceCache
pool = PipelinePool()
voice_cache = VoiceCache()
try: try:
# Stage 1: Prepare TTS context # Stage 0: Create voice resolver
events.log("Preparing conversion pipeline") events.log("Preparing conversion pipeline")
tts_context = _prepare_tts_context(request, events) logging.info(
"[app] run_conversion: provider=%s language=%s voice=%s speed=%.2f",
request.tts_provider, request.language, request.voice, request.speed,
)
resolver = _create_voice_resolver(request, pool, voice_cache)
# Stage 1: Prepare TTS context
usage_counter: Dict[str, int] = defaultdict(int)
tts_context = build_tts_context(
language=request.language,
subtitle=request.subtitle,
pronunciation=request.pronunciation,
usage_counter=usage_counter,
log_callback=lambda level, msg: events.log(msg, level=level),
)
# Stage 2: Build conversion plan # Stage 2: Build conversion plan
events.log("Building conversion plan") events.log("Building conversion plan")
@@ -74,99 +87,164 @@ def run_conversion(
result = execute_conversion( result = execute_conversion(
plan=plan, plan=plan,
events=events, events=events,
pipeline_provider=pipeline_provider, pipeline_provider=pool,
voice_resolver=voice_resolver, voice_resolver=resolver,
tts_context=tts_context, tts_context=tts_context,
) )
# Stage 4: Finalize # Propagate usage counter to result
result.usage_counter = dict(usage_counter)
# Stage 4: Finalize (m4b metadata embedding, EPUB3 generation)
_finalize(request, result, plan, events)
events.log("Conversion complete") events.log("Conversion complete")
logging.info("[app] run_conversion completed successfully")
return result return result
except Exception as e: except Exception as e:
events.log(f"Conversion failed: {e}", level="error") events.log(f"Conversion failed: {e}", level="error")
logging.exception("[app] run_conversion failed: %s", e)
raise raise
finally:
pool.dispose_all()
voice_cache.clear()
from abogen.application.cleanup import flush_cuda
flush_cuda()
def _prepare_tts_context( def _create_voice_resolver(
request: ConversionRequest, request: ConversionRequest,
events: ConversionEvents, pool: Any,
) -> TTSContext: cache: Any,
"""Prepare TTSContext with normalization settings. ) -> Any:
"""Create AppVoiceResolver with loaded profiles.
This compiles pronunciation/heteronym rules and creates the Loads voice profiles from disk, normalizes them, and creates
normalization context used during conversion. an AppVoiceResolver that can resolve voice specs into loaded voices.
Args:
request: Conversion request with override settings
events: For logging warnings about missing features
Returns:
TTSContext ready for text normalization
""" """
from abogen.domain.normalization import ( from abogen.application.voice_resolver import AppVoiceResolver
build_apostrophe_config, from abogen.voice_profiles import load_profiles, normalize_profile_entry
get_runtime_settings,
)
from abogen.domain.pronunciation import (
compile_heteronym_sentence_rules,
compile_pronunciation_rules,
merge_pronunciation_overrides,
)
# Get runtime normalization settings try:
normalization_settings = get_runtime_settings() profiles = load_profiles()
except Exception:
profiles = {}
# Build apostrophe config normalized_profiles: Dict[str, Dict[str, Any]] = {}
apostrophe_config = build_apostrophe_config( for name, entry in (profiles or {}).items():
settings=normalization_settings, normalized = normalize_profile_entry(entry)
) if normalized:
normalized_profiles[str(name)] = normalized
return AppVoiceResolver(request, normalized_profiles, pool, cache)
def _finalize(
request: ConversionRequest,
result: ConversionResult,
plan: ConversionPlan,
events: ConversionEvents,
) -> None:
"""Post-conversion finalization (m4b metadata embedding, EPUB3 generation, etc.)."""
from abogen.domain.enums import OutputFormat
# m4b metadata embedding
if (
result.audio_path
and request.output_format == OutputFormat.M4B
):
from abogen.infrastructure.exporters import ExportService
export_svc = ExportService()
# Check for num2words availability
if apostrophe_config.convert_numbers:
try: try:
import num2words # noqa: F401 export_svc.embed_m4b_metadata(
except ImportError: audio_path=result.audio_path,
events.log( metadata=result.metadata or {},
"Number normalization is enabled but 'num2words' library is not available. " chapters=result.chapter_markers or [],
"Numbers will NOT be converted to words.", cover=request.cover,
level="warning", log_callback=lambda msg, level="info": events.log(msg, level=level),
) )
except Exception as exc:
events.log(f"Failed to embed m4b metadata: {exc}", level="error")
raise RuntimeError(f"Failed to embed m4b metadata: {exc}") from exc
# Compute split pattern # EPUB3 generation
split_pattern = get_split_pattern( if request.epub3_export and plan.extraction:
request.language or Language.EN_US, audio_asset = result.audio_path
request.subtitle_mode or SubtitleMode.DISABLED, if not audio_asset and result.chapter_paths:
) audio_asset = result.chapter_paths[0]
# Merge pronunciation overrides (manual + pronunciation) if audio_asset:
# Create a mock job-like object for merge_pronunciation_overrides try:
class _MockJob:
def __init__(self, req):
self.pronunciation_overrides = req.pronunciation_overrides
self.manual_overrides = req.manual_overrides
self.heteronym_overrides = req.heteronym_overrides
merged_overrides = merge_pronunciation_overrides(_MockJob(request)) from abogen.epub3.exporter import build_epub3_package
# Compile rules epub_root = result.project_root or plan.output_layout.parent_dir
pronunciation_rules = compile_pronunciation_rules(merged_overrides) from abogen.domain.output_paths import build_output_path
heteronym_rules = compile_heteronym_sentence_rules(request.heteronym_overrides)
if heteronym_rules: epub_output_path = build_output_path(epub_root, request.original_filename, "epub")
events.log( events.log("Generating EPUB 3 package...")
f"Applying {len(heteronym_rules)} heteronym override(s) during conversion.", epub_path = build_epub3_package(
level="debug", output_path=epub_output_path,
) book_id=request.epub3_export.book_id,
if pronunciation_rules: extraction=plan.extraction,
events.log( metadata_tags=result.metadata or {},
f"Applying {len(pronunciation_rules)} pronunciation override(s) during conversion.", chapter_markers=result.chapter_markers or [],
level="debug", chunk_markers=result.chunk_markers or [],
chunks=request.chapter_chunk.chunks if request.chapter_chunk else [],
audio_path=audio_asset,
speaker_mode=request.chapter_chunk.speaker_mode if request.chapter_chunk else "single",
cover=request.cover,
)
result.epub_path = epub_path
result.artifacts["epub3"] = epub_path
events.log(f"EPUB 3 package created at {epub_path}")
except Exception as exc:
events.log(f"Failed to generate EPUB 3: {exc}", level="error")
else:
events.log("Skipped EPUB 3 generation: audio output unavailable.", level="warning")
# Build metadata payload and write metadata.json
if plan.output_layout and plan.output_layout.metadata_dir:
from abogen.domain.metadata_helpers import build_metadata_payload
metadata_payload = build_metadata_payload(
metadata=result.metadata,
chapter_markers=result.chapter_markers,
chunk_markers=result.chunk_markers,
chunk_level=request.chapter_chunk.chunk_level if request.chapter_chunk else None,
speaker_mode=request.chapter_chunk.speaker_mode if request.chapter_chunk else None,
speakers=request.chapter_chunk.speakers if request.chapter_chunk else None,
generate_epub3=bool(request.epub3_export),
) )
return TTSContext( metadata_dir = plan.output_layout.metadata_dir
split_pattern=split_pattern, metadata_dir.mkdir(parents=True, exist_ok=True)
pronunciation_rules=pronunciation_rules, metadata_file = metadata_dir / "metadata.json"
heteronym_rules=heteronym_rules,
normalization_overrides=request.normalization_overrides, import json
)
metadata_file.write_text(json.dumps(metadata_payload, indent=2), encoding="utf-8")
result.artifacts["metadata"] = metadata_file
events.log(f"Metadata written to {metadata_file}")
# Record override usage
if result.usage_counter:
try:
from abogen.normalization_settings import record_override_usage
record_override_usage(result.usage_counter)
except Exception as exc:
events.log(f"Failed to record override usage: {exc}", level="debug")
# Post-conversion hooks (Audiobookshelf, etc.)
from abogen.application.integration_hooks import PostConversionHooks
hooks = PostConversionHooks()
hooks.run(request, result, events)
+164
View File
@@ -0,0 +1,164 @@
"""Post-conversion integration hooks.
Called by ConversionService after finalization.
Each integration is a method on PostConversionHooks — isolated, testable,
and easy to extend with new hooks (Plex, Navidrome, etc.).
The service NEVER imports from PyQt or WebUI.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any, Mapping, Optional
from abogen.application.conversion_ports import ConversionEvents
from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_result import ConversionResult
from abogen.domain.metadata_helpers import (
build_audiobookshelf_metadata as _build_abs_metadata,
load_audiobookshelf_chapters as _load_abs_chapters,
)
from abogen.domain.settings_core import (
build_audiobookshelf_config,
coerce_bool,
load_audiobookshelf_config,
stored_integration_config,
)
from abogen.integrations.audiobookshelf import (
AudiobookshelfClient,
AudiobookshelfUploadError,
)
logger = logging.getLogger(__name__)
class PostConversionHooks:
"""Runs post-conversion integrations (Audiobookshelf, etc.).
Usage::
hooks = PostConversionHooks()
hooks.run(request, result, events)
"""
def run(
self,
request: ConversionRequest,
result: ConversionResult,
events: ConversionEvents,
) -> None:
"""Run all registered post-conversion hooks."""
self._maybe_send_to_audiobookshelf(request, result, events)
# ------------------------------------------------------------------
# Audiobookshelf
# ------------------------------------------------------------------
def _maybe_send_to_audiobookshelf(
self,
request: ConversionRequest,
result: ConversionResult,
events: ConversionEvents,
) -> None:
"""Upload finished audiobook to Audiobookshelf if enabled."""
abs_settings = stored_integration_config("audiobookshelf")
if not abs_settings:
return
enabled = coerce_bool(abs_settings.get("enabled"), False)
auto_send = coerce_bool(abs_settings.get("auto_send"), False)
if not (enabled and auto_send):
return
config = build_audiobookshelf_config(abs_settings)
if config is None:
events.log(
"Audiobookshelf upload skipped: configure base URL, API token, "
"library ID, and folder ID first.",
level="warning",
)
return
audio_path = result.audio_path
if not audio_path or not audio_path.exists():
events.log(
"Audiobookshelf upload skipped: audio output not found.",
level="warning",
)
return
# Build metadata
filename = request.original_filename or "Audiobook"
lang = request.language.value if hasattr(request.language, "value") else str(request.language)
metadata = _build_abs_metadata(
result.metadata or {},
language=lang,
filename=Path(filename).stem,
)
# Load chapters from metadata artifact
chapters = None
if config.send_chapters:
metadata_artifact = result.artifacts.get("metadata")
if metadata_artifact:
metadata_path = (
metadata_artifact
if isinstance(metadata_artifact, Path)
else Path(str(metadata_artifact))
)
chapters = _load_abs_chapters(metadata_path)
# Resolve cover
cover_path = None
if config.send_cover and request.cover and request.cover.path:
candidate = request.cover.path
if isinstance(candidate, Path) and candidate.exists():
cover_path = candidate
# Resolve subtitles
subtitles = None
if config.send_subtitles and result.subtitle_paths:
subtitles = [
p for p in result.subtitle_paths
if isinstance(p, Path) and p.exists()
]
# Upload
client = AudiobookshelfClient(config)
display_title = metadata.get("title") or audio_path.stem
try:
existing_items = client.find_existing_items(
display_title, folder_id=config.folder_id,
)
except AudiobookshelfUploadError as exc:
events.log(f"Audiobookshelf lookup failed: {exc}", level="error")
return
if existing_items:
events.log(
f"Removing existing Audiobookshelf item(s) for '{display_title}'.",
level="info",
)
try:
client.delete_items(existing_items)
except Exception as exc:
events.log(
f"Failed to remove existing item(s): {exc}", level="warning",
)
try:
client.upload_audiobook(
audio_path,
metadata=metadata,
cover_path=cover_path,
chapters=chapters,
subtitles=subtitles,
)
events.log("Audiobookshelf upload queued.", level="info")
except AudiobookshelfUploadError as exc:
events.log(f"Audiobookshelf upload failed: {exc}", level="error")
except Exception as exc:
events.log(f"Audiobookshelf integration error: {exc}", level="error")
+7 -8
View File
@@ -15,7 +15,6 @@ Responsibilities:
from __future__ import annotations from __future__ import annotations
from pathlib import Path from pathlib import Path
from typing import Optional
from abogen.application.conversion_models import OutputLayout from abogen.application.conversion_models import OutputLayout
from abogen.application.conversion_request import ConversionRequest from abogen.application.conversion_request import ConversionRequest
@@ -40,8 +39,8 @@ def resolve_output_layout(request: ConversionRequest) -> OutputLayout:
OutputLayout with resolved paths OutputLayout with resolved paths
""" """
# Determine base output directory # Determine base output directory
if request.save_mode == SaveMode.CUSTOM_FOLDER and request.output_folder: if request.save.mode == SaveMode.CUSTOM_FOLDER and request.save.output_folder:
parent_dir = Path(request.output_folder) parent_dir = Path(request.save.output_folder)
elif request.source_path: elif request.source_path:
parent_dir = request.source_path.parent parent_dir = request.source_path.parent
else: else:
@@ -67,7 +66,7 @@ def resolve_output_layout(request: ConversionRequest) -> OutputLayout:
subtitle_dir = None subtitle_dir = None
metadata_dir = None metadata_dir = None
if request.save_as_project: if request.save.save_as_project:
project_root, audio_dir, subtitle_dir, metadata_dir = resolve_project_layout( project_root, audio_dir, subtitle_dir, metadata_dir = resolve_project_layout(
original_filename=request.original_filename, original_filename=request.original_filename,
save_as_project=True, save_as_project=True,
@@ -99,7 +98,7 @@ def resolve_merged_path(
base_name = sanitize_output_stem( base_name = sanitize_output_stem(
request.original_filename or "output" request.original_filename or "output"
) )
return layout.audio_dir / f"{base_name}.{request.output_format}" return layout.audio_dir / f"{base_name}{request.output_format.dot_ext}"
def resolve_chapter_path( def resolve_chapter_path(
@@ -125,7 +124,7 @@ def resolve_chapter_path(
slug = re.sub(r'[\s_]+', '_', slug).strip('_') slug = re.sub(r'[\s_]+', '_', slug).strip('_')
if not slug: if not slug:
slug = f"chapter_{chapter_index}" slug = f"chapter_{chapter_index}"
filename = f"{chapter_index:02d}_{slug}.{request.separate_chapters_format}" filename = f"{chapter_index:02d}_{slug}.{request.save.separate_chapters_format}"
return layout.audio_dir / "chapters" / filename return layout.audio_dir / "chapters" / filename
@@ -145,6 +144,6 @@ def should_merge_output(request: ConversionRequest) -> bool:
""" """
if request.output_format == OutputFormat.M4B: if request.output_format == OutputFormat.M4B:
return True return True
if not request.save_chapters_separately: if not request.save.save_chapters_separately:
return True return True
return request.merge_chapters_at_end return request.save.merge_chapters_at_end
+81
View File
@@ -0,0 +1,81 @@
"""AppVoiceResolver — voice resolution inside the application layer.
Resolves voice specs into loaded voices using profiles, pipeline pool,
and voice cache. Replaces UI-specific resolvers (WebUIVoiceResolver,
PyQtVoiceResolver) with a single app-layer implementation.
"""
from __future__ import annotations
import logging
from typing import Any, Dict, Optional
from abogen.application.conversion_ports import ResolvedVoice, VoiceResolver
from abogen.application.conversion_request import ConversionRequest
from abogen.domain.pipeline_factory import PipelinePool
from abogen.domain.voice_loader import VoiceCache, resolve_voice
from abogen.domain.voice_utils import resolve_voice_target
class AppVoiceResolver:
"""App-layer implementation of VoiceResolver protocol.
Uses ConversionRequest instead of Job. Loads profiles, creates
resolver internally — UIs don't need to manage this.
"""
def __init__(
self,
request: ConversionRequest,
normalized_profiles: Dict[str, Dict[str, Any]],
pool: PipelinePool,
cache: VoiceCache,
):
self._request = request
self._profiles = normalized_profiles
self._cache = cache
self._pool = pool
def resolve(self, voice_spec: str) -> ResolvedVoice:
"""Resolve a voice spec into a loaded voice."""
provider, resolved, speed, steps = resolve_voice_target(
voice_spec,
self._profiles,
job_voice=self._request.voice,
job_tts_provider=self._request.tts_provider,
job_supertonic_total_steps=self._request.supertonic_total_steps,
job_speed=self._request.speed,
)
cache_key = f"{provider}:{resolved}" if resolved else provider
cached = self._cache.get(cache_key)
if cached is not None:
logging.info("[resolver] Cache hit: spec=%s -> provider=%s resolved=%s", voice_spec, provider, resolved)
return ResolvedVoice(
provider=provider,
resolved_spec=resolved,
voice=cached,
speed=speed,
supertonic_steps=steps or 0,
)
if provider == "kokoro":
kokoro_backend = self._pool.get(
"kokoro", self._request.language, self._request.use_gpu,
)
loaded = resolve_voice(
resolved, kokoro_backend, self._request.use_gpu, cache=self._cache,
)
else:
loaded = resolved
self._cache.set(cache_key, loaded)
logging.info("[resolver] Resolved: spec=%s -> provider=%s resolved=%s speed=%.2f steps=%s",
voice_spec, provider, resolved, speed, steps)
return ResolvedVoice(
provider=provider,
resolved_spec=resolved,
voice=loaded,
speed=speed,
supertonic_steps=steps or 0,
)
+2 -1
View File
@@ -12,7 +12,8 @@ import fitz # PyMuPDF
import markdown import markdown
from abogen.utils import detect_encoding from abogen.utils import detect_encoding
from abogen.subtitle_utils import clean_text, calculate_text_length from abogen.subtitle_utils import clean_text
from abogen.domain.text_utils import calculate_text_length
# Pre-compile frequently used regex patterns # Pre-compile frequently used regex patterns
_BRACKETED_NUMBERS_PATTERN = re.compile(r"\[\s*\d+\s*\]") _BRACKETED_NUMBERS_PATTERN = re.compile(r"\[\s*\d+\s*\]")
+27 -14
View File
@@ -1,4 +1,5 @@
from abogen.utils import get_version from abogen.utils import get_version
from abogen.domain.enums import Language
# Program Information # Program Information
PROGRAM_NAME = "abogen" PROGRAM_NAME = "abogen"
@@ -16,8 +17,22 @@ SUBTITLE_FORMATS = [
("ass_centered_narrow", "ASS (centered narrow)"), ("ass_centered_narrow", "ASS (centered narrow)"),
] ]
# Language description mapping # Language description mapping (Language enum → human-readable label).
LANGUAGE_DESCRIPTIONS = { LANGUAGE_DESCRIPTIONS = {
Language.EN_US: "American English",
Language.EN_GB: "British English",
Language.ES: "Spanish",
Language.FR: "French",
Language.HI: "Hindi",
Language.IT: "Italian",
Language.JA: "Japanese",
Language.PT_BR: "Brazilian Portuguese",
Language.ZH: "Mandarin Chinese",
}
# Display-only mapping for kokoro codes → labels.
# Used by voice catalog and PyQt (legacy) where kokoro codes are still present.
KOKORO_CODE_LABELS = {
"a": "American English", "a": "American English",
"b": "British English", "b": "British English",
"e": "Spanish", "e": "Spanish",
@@ -56,24 +71,22 @@ SUPPORTED_INPUT_FORMATS = [
] ]
# Supported languages for subtitle generation # Supported languages for subtitle generation
# Currently, only 'a (American English)' and 'b (British English)' are supported for subtitle generation. # Currently, only English (EN_US, EN_GB) are supported for subtitle generation.
# This is because tokens that contain timestamps are not generated for other languages in the Kokoro pipeline. # This is because tokens that contain timestamps are not generated for other languages in the Kokoro pipeline.
# Please refer to: https://github.com/hexgrad/kokoro/blob/6d87f4ae7abc2d14dbc4b3ef2e5f19852e861ac2/kokoro/pipeline.py # Please refer to: https://github.com/hexgrad/kokoro/blob/6d87f4ae7abc2d14dbc4b3ef2e5f19852e861ac2/kokoro/pipeline.py
# 383 English processing (unchanged) SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION = [Language.EN_US, Language.EN_GB]
# 384 if self.lang_code in 'ab':
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION = list(LANGUAGE_DESCRIPTIONS.keys())
# Voice and sample text mapping # Voice and sample text mapping
SAMPLE_VOICE_TEXTS = { SAMPLE_VOICE_TEXTS = {
"a": "This is a sample of the selected voice.", Language.EN_US: "This is a sample of the selected voice.",
"b": "This is a sample of the selected voice.", Language.EN_GB: "This is a sample of the selected voice.",
"e": "Este es una muestra de la voz seleccionada.", Language.ES: "Este es una muestra de la voz seleccionada.",
"f": "Ceci est un exemple de la voix sélectionnée.", Language.FR: "Ceci est un exemple de la voix sélectionnée.",
"h": "यह चयनित आवाज़ का एक नमूना है।", Language.HI: "यह चयनित आवाज़ का एक नमूना है।",
"i": "Questo è un esempio della voce selezionata.", Language.IT: "Questo è un esempio della voce selezionata.",
"j": "これは選択した声のサンプルです。", Language.JA: "これは選択した声のサンプルです。",
"p": "Este é um exemplo da voz selecionada.", Language.PT_BR: "Este é um exemplo da voz selecionada.",
"z": "这是所选语音的示例。", Language.ZH: "这是所选语音的示例。",
} }
COLORS = { COLORS = {
+3 -2
View File
@@ -7,8 +7,9 @@ text for TTS synthesis.
from __future__ import annotations from __future__ import annotations
from collections import defaultdict from collections import defaultdict
from typing import Any, Dict, Iterable, Mapping, Optional from typing import Any, Dict, Iterable, Mapping
from abogen.domain.enums import Language
from abogen.pronunciation_store import increment_usage from abogen.pronunciation_store import increment_usage
@@ -44,7 +45,7 @@ def record_override_usage(
if not usage_counter: if not usage_counter:
return return
language = getattr(job, "language", "") or "a" language = getattr(job, "language", Language.EN_US) or Language.EN_US
for normalized, amount in usage_counter.items(): for normalized, amount in usage_counter.items():
if amount <= 0: if amount <= 0:
continue continue
+52
View File
@@ -0,0 +1,52 @@
"""Domain config types — shared contracts for domain functions.
These dataclasses group parameters that domain functions receive.
Domain defines them, app layer fills them.
Why here (domain) and not application:
- build_tts_context() is in domain → needs PronunciationConfig
- make_subtitle_writer() is in infrastructure → needs SubtitleConfig
- embed_m4b_metadata() is in infrastructure → needs CoverConfig
- Domain should not depend on application layer (DIP)
"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional
from abogen.domain.enums import SubtitleFormat, SubtitleMode
@dataclass(frozen=True)
class PronunciationConfig:
"""Pronunciation and normalization override settings.
Used by build_tts_context() to compile override rules.
"""
pronunciation_overrides: List[Dict[str, Any]] = field(default_factory=list)
manual_overrides: List[Dict[str, Any]] = field(default_factory=list)
heteronym_overrides: List[Dict[str, Any]] = field(default_factory=list)
normalization_overrides: Optional[Dict[str, Any]] = None
@dataclass(frozen=True)
class SubtitleConfig:
"""Subtitle output settings.
Used by make_subtitle_writer() and process_and_write_subtitles().
"""
mode: SubtitleMode = SubtitleMode.DISABLED
format: SubtitleFormat = SubtitleFormat.SRT
max_words: int = 50
@dataclass(frozen=True)
class CoverConfig:
"""Cover image settings.
Used by embed_m4b_metadata() and build_epub3_package().
"""
path: Optional[Path] = None
mime: Optional[str] = None
+25 -10
View File
@@ -19,11 +19,11 @@ from __future__ import annotations
import time import time
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any, Callable, List, Optional, Protocol from typing import Any, Callable, Optional, Protocol
from abogen.domain.audio_sink import AudioSink from abogen.domain.audio_sink import AudioSink
from abogen.domain.conversion_pipeline import tts_segments from abogen.domain.conversion_pipeline import tts_segments
from abogen.domain.enums import SubtitleMode from abogen.domain.enums import Language, SubtitleMode
from abogen.domain.normalization import TTSContext from abogen.domain.normalization import TTSContext
from abogen.domain.progress import calc_etr_str from abogen.domain.progress import calc_etr_str
from abogen.domain.subtitle_generation import process_subtitle_tokens from abogen.domain.subtitle_generation import process_subtitle_tokens
@@ -61,6 +61,7 @@ def run_tts_segment_loop(
voice: Any, voice: Any,
speed: float, speed: float,
split_pattern: str, split_pattern: str,
total_steps: Optional[int] = None,
chapter_sink: Optional[AudioSink] = None, chapter_sink: Optional[AudioSink] = None,
preview_callback: Optional[Callable[[str], None]] = None, preview_callback: Optional[Callable[[str], None]] = None,
on_segment: Optional[Callable[[SegmentInfo], None]] = None, on_segment: Optional[Callable[[SegmentInfo], None]] = None,
@@ -74,6 +75,7 @@ def run_tts_segment_loop(
voice: Voice name/id for the backend. voice: Voice name/id for the backend.
speed: Speech speed multiplier. speed: Speech speed multiplier.
split_pattern: Regex pattern used by the TTS engine for sentence splitting. split_pattern: Regex pattern used by the TTS engine for sentence splitting.
total_steps: Inference quality steps (Supertonic only, ignored by Kokoro).
preview_callback: Called with a short preview string per segment. preview_callback: Called with a short preview string per segment.
on_segment: Called with a SegmentInfo for each segment *before* on_segment: Called with a SegmentInfo for each segment *before*
audio is written. Useful for callers that need per-segment audio is written. Useful for callers that need per-segment
@@ -95,6 +97,7 @@ def run_tts_segment_loop(
speed=speed, speed=speed,
split_pattern=split_pattern, split_pattern=split_pattern,
current_time=params.stats.current_time, current_time=params.stats.current_time,
total_steps=total_steps,
): ):
if params.check_cancel(): if params.check_cancel():
break break
@@ -151,25 +154,35 @@ def process_and_write_subtitles(
accumulated_tokens: list[dict], accumulated_tokens: list[dict],
subtitle_writer: Any, subtitle_writer: Any,
*, *,
subtitle_mode: str, subtitle: "SubtitleConfig | str",
max_subtitle_words: int, max_subtitle_words: int | None = None,
lang_code: str, language: Language,
use_spacy_segmentation: bool, use_spacy_segmentation: bool,
fallback_end_time: float, fallback_end_time: float,
) -> None: ) -> None:
"""Process accumulated subtitle tokens and write entries to a subtitle writer. """Process accumulated subtitle tokens and write entries to a subtitle writer.
This is the standard subtitle post-processing step shared by both UIs. Accepts a SubtitleConfig object or a subtitle mode string
for backward compatibility.
""" """
from abogen.domain.config_types import SubtitleConfig
if isinstance(subtitle, SubtitleConfig):
mode_str = subtitle.mode.value
words = subtitle.max_words
else:
mode_str = subtitle
words = max_subtitle_words or 50
if not accumulated_tokens or not subtitle_writer: if not accumulated_tokens or not subtitle_writer:
return return
new_entries: list[tuple] = [] new_entries: list[tuple] = []
process_subtitle_tokens( process_subtitle_tokens(
accumulated_tokens, accumulated_tokens,
new_entries, new_entries,
max_subtitle_words, words,
subtitle_mode, mode_str,
lang_code, language,
use_spacy_segmentation=use_spacy_segmentation, use_spacy_segmentation=use_spacy_segmentation,
fallback_end_time=fallback_end_time, fallback_end_time=fallback_end_time,
) )
@@ -191,7 +204,7 @@ class SynthParams:
audio_sink: Optional[AudioSink] = None audio_sink: Optional[AudioSink] = None
subtitle_mode: str = "Disabled" subtitle_mode: str = "Disabled"
max_subtitle_words: int = 50 max_subtitle_words: int = 50
lang_code: str = "a" language: Language = Language.EN_US
use_spacy_segmentation: bool = False use_spacy_segmentation: bool = False
@@ -202,6 +215,7 @@ def synthesize_text(
backend: Any, backend: Any,
voice: Any, voice: Any,
speed: float, speed: float,
total_steps: Optional[int] = None,
chapter_sink: Optional[AudioSink] = None, chapter_sink: Optional[AudioSink] = None,
preview_callback: Optional[Callable[[str], None]] = None, preview_callback: Optional[Callable[[str], None]] = None,
on_segment: Optional[Callable[[SegmentInfo], None]] = None, on_segment: Optional[Callable[[SegmentInfo], None]] = None,
@@ -219,6 +233,7 @@ def synthesize_text(
backend=backend, backend=backend,
voice=voice, voice=voice,
speed=speed, speed=speed,
total_steps=total_steps,
split_pattern=split_pattern_override or params.tts_context.split_pattern, split_pattern=split_pattern_override or params.tts_context.split_pattern,
chapter_sink=chapter_sink, chapter_sink=chapter_sink,
preview_callback=preview_callback, preview_callback=preview_callback,
+118 -5
View File
@@ -9,8 +9,8 @@ from __future__ import annotations
import logging import logging
from dataclasses import dataclass, field from dataclasses import dataclass, field
from abogen.domain.enums import SubtitleMode from abogen.domain.enums import Language, SubtitleMode
from typing import Any, Callable, Dict, Iterator, List, Optional from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple
import numpy as np import numpy as np
@@ -21,6 +21,112 @@ from abogen.domain.audio_buffer import SAMPLE_RATE
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# Languages where spaCy is used for pre-TTS segmentation
# English ("a", "b") is excluded — spaCy only used for post-TTS subtitles
_SPACY_EXCLUDED_LANGS = {Language.EN_US, Language.EN_GB}
# CJK languages — different spacing pattern
_CJK_LANGS = {Language.ZH, Language.JA}
def spacy_pre_tts_segmentation(
text: str,
lang_code: Any,
subtitle_mode: Any,
*,
is_subtitle_input: bool = False,
use_spacy_segmentation: bool = True,
log_callback: Optional[Callable[[str], None]] = None,
) -> Tuple[List[str], str]:
"""Segment text using spaCy before TTS, with split_pattern override.
For non-English languages, spaCy sentence segmentation produces better
sentence boundaries than regex. This function:
1. Checks if spaCy should be used (toggle on, not disabled mode, not subtitle input)
2. For non-English: runs spaCy segmentation, computes split_pattern override
3. For English: returns single segment with default pattern (spaCy only for subtitles)
4. If spaCy fails: falls back to default pattern
Args:
text: Text to segment.
lang_code: Language code (Language enum or string like "a", "de", "fr").
subtitle_mode: SubtitleMode enum or string.
is_subtitle_input: True if source is .srt/.ass/.vtt file.
use_spacy_segmentation: User toggle for spaCy segmentation.
log_callback: Optional logging function.
Returns:
Tuple of (text_segments, active_split_pattern).
text_segments is a list of sentences (always at least one element).
active_split_pattern is the regex to use for TTS backend splitting.
"""
from abogen.domain.split_pattern import PUNCTUATION_COMMAS, get_split_pattern
def _log(msg: str) -> None:
if log_callback:
log_callback(msg)
# Normalize language
lang_enum = _to_language_enum(lang_code)
# Default split pattern
default_split = get_split_pattern(lang_code, subtitle_mode)
# Check conditions
if not use_spacy_segmentation:
return [text], default_split
subtitle_mode_str = _to_subtitle_mode_str(subtitle_mode)
if subtitle_mode_str in ("Disabled", "Line"):
return [text], default_split
if is_subtitle_input:
return [text], default_split
# English: spaCy only for post-TTS subtitles, not pre-TTS
if lang_enum in _SPACY_EXCLUDED_LANGS:
return [text], default_split
# Non-English: run spaCy pre-TTS segmentation
from abogen.spacy_utils import segment_sentences
_log("Using spaCy for sentence segmentation (pre-TTS)...")
spacy_sentences = segment_sentences(text, lang_code, log_callback=log_callback)
if not spacy_sentences:
_log("spaCy: Fallback to default segmentation...")
return [text], default_split
_log(f"spaCy: Text segmented into {len(spacy_sentences)} sentences...")
# Compute split_pattern override based on subtitle mode
spacing_pattern = r"\s*" if lang_enum in _CJK_LANGS else r"\s+"
if subtitle_mode_str == "Sentence + Comma":
active_split = r"(?<=[{}]){}|\n+".format(PUNCTUATION_COMMAS, spacing_pattern)
else:
# Sentence mode: spaCy already split, only split on newlines
active_split = "\n"
return spacy_sentences, active_split
def _to_language_enum(lang_code: Any) -> Language:
"""Convert lang_code to Language enum."""
if isinstance(lang_code, Language):
return lang_code
try:
return Language.from_str(str(lang_code))
except (ValueError, AttributeError):
return Language.EN_US
def _to_subtitle_mode_str(subtitle_mode: Any) -> str:
"""Convert subtitle_mode to string."""
if isinstance(subtitle_mode, SubtitleMode):
return subtitle_mode.value
return str(subtitle_mode)
@dataclass @dataclass
class SegmentResult: class SegmentResult:
@@ -40,6 +146,7 @@ def tts_segments(
speed: float, speed: float,
split_pattern: str, split_pattern: str,
current_time: float = 0.0, current_time: float = 0.0,
total_steps: Optional[int] = None,
) -> Iterator[SegmentResult]: ) -> Iterator[SegmentResult]:
"""Invoke TTS backend on (already normalized) text and yield SegmentResults. """Invoke TTS backend on (already normalized) text and yield SegmentResults.
@@ -53,16 +160,20 @@ def tts_segments(
speed: TTS speed multiplier. speed: TTS speed multiplier.
split_pattern: Regex pattern for sentence splitting. split_pattern: Regex pattern for sentence splitting.
current_time: Current position in the audio timeline (seconds). current_time: Current position in the audio timeline (seconds).
total_steps: Inference quality steps (Supertonic only, ignored by Kokoro).
Yields: Yields:
SegmentResult for each non-empty TTS segment. SegmentResult for each non-empty TTS segment.
""" """
segment_iter = backend( kwargs: dict[str, Any] = dict(
text,
voice=voice, voice=voice,
speed=speed, speed=speed,
split_pattern=split_pattern, split_pattern=split_pattern,
) )
if total_steps is not None:
kwargs["total_steps"] = total_steps
segment_iter = backend(text, **kwargs)
chunk_start = current_time chunk_start = current_time
@@ -109,6 +220,7 @@ def emit_text_segments(
speed: float, speed: float,
split_pattern: str, split_pattern: str,
current_time: float = 0.0, current_time: float = 0.0,
total_steps: Optional[int] = None,
# normalization # normalization
heteronym_rules: Any = None, heteronym_rules: Any = None,
pronunciation_rules: Any = None, pronunciation_rules: Any = None,
@@ -159,6 +271,7 @@ def emit_text_segments(
speed=speed, speed=speed,
split_pattern=split_pattern, split_pattern=split_pattern,
current_time=current_time, current_time=current_time,
total_steps=total_steps,
) )
@@ -176,7 +289,7 @@ def emit_text_to_sinks(
# subtitle # subtitle
subtitle_writer: Any = None, subtitle_writer: Any = None,
subtitle_mode: str = "Disabled", subtitle_mode: str = "Disabled",
subtitle_lang: str = "a", subtitle_lang: Language = Language.EN_US,
max_subtitle_words: int = 50, max_subtitle_words: int = 50,
use_spacy_segmentation: bool = True, use_spacy_segmentation: bool = True,
# normalization # normalization
+52 -4
View File
@@ -128,8 +128,8 @@ class InputFormat(str, Enum):
class Language(str, Enum): class Language(str, Enum):
"""TTS language code (ISO 639-1 with region where needed). """TTS language code (ISO 639-1 with region where needed).
Each engine (Kokoro, Supertonic) maps these to its own Each engine maps these to its own internal language identifiers.
internal language identifiers. Engines report which languages they support via ``supported_languages()``.
""" """
EN_US = "en-US" EN_US = "en-US"
EN_GB = "en-GB" EN_GB = "en-GB"
@@ -140,6 +140,30 @@ class Language(str, Enum):
JA = "ja" JA = "ja"
PT_BR = "pt-BR" PT_BR = "pt-BR"
ZH = "zh" ZH = "zh"
AR = "ar"
BG = "bg"
CS = "cs"
DA = "da"
DE = "de"
EL = "el"
ET = "et"
FI = "fi"
HR = "hr"
HU = "hu"
ID = "id"
KO = "ko"
LT = "lt"
LV = "lv"
NL = "nl"
PL = "pl"
RO = "ro"
RU = "ru"
SK = "sk"
SL = "sl"
SV = "sv"
TR = "tr"
UK = "uk"
VI = "vi"
@property @property
def display_name(self) -> str: def display_name(self) -> str:
@@ -154,13 +178,37 @@ class Language(str, Enum):
"ja": "Japanese", "ja": "Japanese",
"pt-BR": "Brazilian Portuguese", "pt-BR": "Brazilian Portuguese",
"zh": "Mandarin Chinese", "zh": "Mandarin Chinese",
"ar": "Arabic",
"bg": "Bulgarian",
"cs": "Czech",
"da": "Danish",
"de": "German",
"el": "Greek",
"et": "Estonian",
"fi": "Finnish",
"hr": "Croatian",
"hu": "Hungarian",
"id": "Indonesian",
"ko": "Korean",
"lt": "Lithuanian",
"lv": "Latvian",
"nl": "Dutch",
"pl": "Polish",
"ro": "Romanian",
"ru": "Russian",
"sk": "Slovak",
"sl": "Slovenian",
"sv": "Swedish",
"tr": "Turkish",
"uk": "Ukrainian",
"vi": "Vietnamese",
} }
return _names[self.value] return _names[self.value]
@property @property
def is_cjk(self) -> bool: def is_cjk(self) -> bool:
"""True for CJK languages (Chinese, Japanese).""" """True for CJK languages (Chinese, Japanese, Korean)."""
return self in (self.ZH, self.JA) return self in (self.ZH, self.JA, self.KO)
@property @property
def supports_subtitle_tokens(self) -> bool: def supports_subtitle_tokens(self) -> bool:
-1
View File
@@ -11,7 +11,6 @@ import logging
import os import os
import re import re
import uuid import uuid
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple from typing import Any, Dict, List, Optional, Tuple
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
+91
View File
@@ -21,6 +21,60 @@ _SERIES_NUMBER_KEYS = (
) )
_SERIES_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?") _SERIES_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?")
_SERIES_NAME_ALIASES = ("series", "series_name", "seriesname", "series_title", "seriestitle")
_SERIES_INDEX_ALIASES = ("series_index", "series_sequence", "series_position", "book_number")
_AUTHOR_ALIASES = ("author", "authors")
_DESCRIPTION_ALIASES = ("description", "summary")
_TAGS_ALIASES = ("tags", "keywords", "genre")
def expand_metadata_aliases(tags: Mapping[str, Any]) -> Dict[str, Any]:
"""Expand concept aliases so each concept has all canonical keys set.
One input concept fans out to multiple keys so that downstream consumers
can look up any variant and find the value.
Expanded concepts:
series -> series, series_name, seriesname, series_title, seriestitle
series_index -> series_index, series_sequence, series_position, book_number
author -> author, authors
description -> description, summary
tags -> tags, keywords, genre
"""
if not tags:
return {}
result: Dict[str, Any] = {}
for key, value in tags.items():
if value is None:
continue
text = str(value).strip() if not isinstance(value, (list, tuple, set)) else value
if not text:
continue
key_lower = str(key).strip().lower()
if not key_lower:
continue
if key_lower in _SERIES_NAME_ALIASES:
for alias in _SERIES_NAME_ALIASES:
result[alias] = text
elif key_lower in _SERIES_INDEX_ALIASES:
for alias in _SERIES_INDEX_ALIASES:
result[alias] = text
elif key_lower in _AUTHOR_ALIASES:
for alias in _AUTHOR_ALIASES:
result[alias] = text
elif key_lower in _DESCRIPTION_ALIASES:
for alias in _DESCRIPTION_ALIASES:
result[alias] = text
elif key_lower in _TAGS_ALIASES:
for alias in _TAGS_ALIASES:
result[alias] = text
else:
result[key_lower] = text
return result
def normalize_metadata_map(values: Optional[Mapping[str, Any]]) -> Dict[str, str]: def normalize_metadata_map(values: Optional[Mapping[str, Any]]) -> Dict[str, str]:
normalized: Dict[str, str] = {} normalized: Dict[str, str] = {}
@@ -403,3 +457,40 @@ def load_audiobookshelf_chapters(
if title and start is not None and end is not None: if title and start is not None and end is not None:
cleaned.append({"title": str(title), "start": start, "end": end}) cleaned.append({"title": str(title), "start": start, "end": end})
return cleaned or None return cleaned or None
def build_metadata_payload(
metadata: Optional[Dict[str, Any]] = None,
chapter_markers: Optional[List[Dict[str, Any]]] = None,
chunk_markers: Optional[List[Dict[str, Any]]] = None,
chunk_level: Optional[str] = None,
speaker_mode: Optional[str] = None,
speakers: Optional[Dict[str, Any]] = None,
generate_epub3: bool = False,
) -> Dict[str, Any]:
"""Build the canonical metadata payload dict for persistence and downstream use.
This is the single source of truth for metadata assembly. Both PyQt and WebUI
runners should call this instead of building the dict manually.
Args:
metadata: Normalized metadata tags dict.
chapter_markers: List of chapter marker dicts with title/start/end.
chunk_markers: List of chunk marker dicts.
chunk_level: Chunk granularity level (e.g. 'chapter', 'chunk').
speaker_mode: Speaker mode ('single', 'multi', etc.).
speakers: Speaker profile mapping.
generate_epub3: Whether EPUB3 generation is enabled.
Returns:
Complete metadata payload dict.
"""
return {
"metadata": dict(metadata or {}),
"chapters": chapter_markers or [],
"chunks": chunk_markers or [],
"chunk_level": chunk_level,
"speaker_mode": speaker_mode,
"speakers": dict(speakers or {}),
"generate_epub3": generate_epub3,
}
+25 -68
View File
@@ -8,23 +8,22 @@ from __future__ import annotations
from typing import Any, Dict, Mapping from typing import Any, Dict, Mapping
from abogen.domain.metadata_helpers import expand_metadata_aliases
def normalize_opds_metadata(metadata_payload: Mapping[str, Any]) -> Dict[str, Any]: def normalize_opds_metadata(metadata_payload: Mapping[str, Any]) -> Dict[str, Any]:
"""Normalize OPDS/Calibre metadata into canonical override keys. """Normalize OPDS/Calibre metadata into canonical override keys.
Takes a metadata payload with various key aliases (e.g. 'series'/'series_name', Takes a metadata payload with various key aliases (e.g. 'series'/'series_name',
'tags'/'keywords', 'authors'/'creator') and returns a dict with canonical 'tags'/'keywords', 'authors'/'creator') and returns a dict with all
keys set. concept aliases expanded.
Args: Args:
metadata_payload: Raw metadata dict from OPDS/Calibre import. metadata_payload: Raw metadata dict from OPDS/Calibre import.
Returns: Returns:
Dict with canonical metadata keys (series, series_index, tags, Dict with all canonical metadata key aliases expanded.
description, subtitle, publisher, authors).
""" """
metadata_overrides: Dict[str, Any] = {}
def _stringify(value: Any) -> str: def _stringify(value: Any) -> str:
if value is None: if value is None:
return "" return ""
@@ -33,67 +32,25 @@ def normalize_opds_metadata(metadata_payload: Mapping[str, Any]) -> Dict[str, An
return ", ".join(part for part in parts if part) return ", ".join(part for part in parts if part)
return str(value).strip() return str(value).strip()
raw_series = metadata_payload.get("series") or metadata_payload.get("series_name") # Map OPDS-specific keys to common concept keys before expansion
series_name = str(raw_series or "").strip() normalized_input: Dict[str, Any] = {}
if series_name: for key, value in metadata_payload.items():
metadata_overrides["series"] = series_name if value is None:
metadata_overrides.setdefault("series_name", series_name) continue
key_lower = str(key).strip().lower()
if not key_lower:
continue
text = _stringify(value)
if not text:
continue
series_index_value = ( # Map OPDS-specific author aliases
metadata_payload.get("series_index") if key_lower in ("creator", "dc_creator"):
or metadata_payload.get("series_position") normalized_input["author"] = text
or metadata_payload.get("series_sequence") # Map OPDS-specific subtitle aliases
or metadata_payload.get("book_number") elif key_lower in ("sub_title", "calibre_subtitle"):
) normalized_input["subtitle"] = text
if series_index_value is not None: else:
series_index_text = str(series_index_value).strip() normalized_input[key_lower] = text
if series_index_text:
metadata_overrides.setdefault("series_index", series_index_text)
metadata_overrides.setdefault("series_position", series_index_text)
metadata_overrides.setdefault("series_sequence", series_index_text)
metadata_overrides.setdefault("book_number", series_index_text)
tags_value = metadata_payload.get("tags") or metadata_payload.get("keywords") return expand_metadata_aliases(normalized_input)
if tags_value:
tags_text = _stringify(tags_value)
if tags_text:
metadata_overrides.setdefault("tags", tags_text)
metadata_overrides.setdefault("keywords", tags_text)
metadata_overrides.setdefault("genre", tags_text)
description_value = metadata_payload.get("description") or metadata_payload.get("summary")
if description_value:
description_text = _stringify(description_value)
if description_text:
metadata_overrides.setdefault("description", description_text)
metadata_overrides.setdefault("summary", description_text)
subtitle_value = (
metadata_payload.get("subtitle")
or metadata_payload.get("sub_title")
or metadata_payload.get("calibre_subtitle")
)
if subtitle_value:
subtitle_text = _stringify(subtitle_value)
if subtitle_text:
metadata_overrides.setdefault("subtitle", subtitle_text)
publisher_value = metadata_payload.get("publisher")
if publisher_value:
publisher_text = _stringify(publisher_value)
if publisher_text:
metadata_overrides.setdefault("publisher", publisher_text)
authors_value = (
metadata_payload.get("authors")
or metadata_payload.get("author")
or metadata_payload.get("creator")
or metadata_payload.get("dc_creator")
)
if authors_value:
authors_text = _stringify(authors_value)
if authors_text:
metadata_overrides.setdefault("authors", authors_text)
metadata_overrides.setdefault("author", authors_text)
return metadata_overrides
+121 -1
View File
@@ -13,8 +13,9 @@ resources so they can be created once and passed as a single object.
from __future__ import annotations from __future__ import annotations
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any, Dict, List, Mapping, Optional from typing import Any, Callable, Dict, List, Mapping, Optional
from abogen.domain.enums import Language
from abogen.kokoro_text_normalization import ( from abogen.kokoro_text_normalization import (
ApostropheConfig, ApostropheConfig,
normalize_for_pipeline as _normalize_for_pipeline, normalize_for_pipeline as _normalize_for_pipeline,
@@ -123,3 +124,122 @@ def prepare_text_for_tts(
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG) apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
return _normalize_for_pipeline(result, config=apostrophe_config, settings=runtime_settings) return _normalize_for_pipeline(result, config=apostrophe_config, settings=runtime_settings)
def build_tts_context(
*,
language: Language,
subtitle: "SubtitleConfig | str" = "Disabled",
pronunciation: Optional["PronunciationConfig"] = None,
speakers: Optional[Dict[str, Any]] = None,
usage_counter: Optional[Dict[str, int]] = None,
log_callback: Optional[Callable[[str, str], None]] = None,
) -> TTSContext:
"""Build a TTSContext from raw data. Single entry point for both UIs.
Loads normalization settings, applies overrides, validates configuration,
merges pronunciation overrides, and compiles all rules.
Args:
language: Language enum value.
subtitle: SubtitleConfig object or subtitle mode string.
pronunciation: PronunciationConfig with override rules.
speakers: Speaker profile mapping.
usage_counter: Mutable dict for tracking override usage.
log_callback: Callable(level, message) for warnings.
Returns:
TTSContext ready for text normalization.
"""
from abogen.domain.config_types import PronunciationConfig, SubtitleConfig
from abogen.domain.enums import SubtitleMode
from abogen.domain.pronunciation import (
compile_heteronym_sentence_rules,
compile_pronunciation_rules,
merge_pronunciation_overrides,
)
from abogen.domain.split_pattern import get_split_pattern
def _log(msg: str, level: str = "warning") -> None:
if log_callback:
log_callback(level, msg)
# Resolve subtitle mode
if isinstance(subtitle, SubtitleConfig):
resolved_subtitle = subtitle.mode
else:
try:
resolved_subtitle = SubtitleMode.from_str(subtitle) if not isinstance(subtitle, SubtitleMode) else subtitle
except ValueError:
resolved_subtitle = SubtitleMode.DISABLED
# Resolve pronunciation config
if pronunciation is None:
pronunciation = PronunciationConfig()
# Get runtime normalization settings
runtime_settings = get_runtime_settings()
# Apply per-job normalization overrides
if pronunciation.normalization_overrides:
runtime_settings = _apply_overrides(runtime_settings, pronunciation.normalization_overrides)
# Build apostrophe config
apostrophe_config = build_apostrophe_config(settings=runtime_settings)
# Validate LLM apostrophe mode
apostrophe_mode = str(runtime_settings.get("normalization_apostrophe_mode", "spacy")).lower()
if apostrophe_mode == "llm":
from abogen.normalization_settings import build_llm_configuration
llm_config = build_llm_configuration(runtime_settings)
if not llm_config.is_configured():
raise RuntimeError(
"LLM-based apostrophe normalization is selected, but the LLM configuration is incomplete."
)
# Check for num2words availability
if apostrophe_config.convert_numbers:
try:
import num2words # noqa: F401
except ImportError:
_log(
"Number normalization is enabled but 'num2words' library is not available. "
"Numbers will NOT be converted to words."
)
# Compute split pattern
if not isinstance(language, Language):
raise TypeError(f"language must be Language enum, got {type(language).__name__}: {language!r}")
split_pattern = get_split_pattern(language, resolved_subtitle)
# Merge pronunciation overrides
source = {
"pronunciation_overrides": pronunciation.pronunciation_overrides,
"manual_overrides": pronunciation.manual_overrides,
"speakers": speakers or {},
"language": language,
}
merged_overrides = merge_pronunciation_overrides(source)
# Compile rules
pronunciation_rules = compile_pronunciation_rules(merged_overrides)
heteronym_rules = compile_heteronym_sentence_rules(pronunciation.heteronym_overrides)
if heteronym_rules:
_log(
f"Applying {len(heteronym_rules)} heteronym override(s) during conversion.",
level="debug",
)
if pronunciation_rules:
_log(
f"Applying {len(pronunciation_rules)} pronunciation override(s) during conversion.",
level="debug",
)
return TTSContext(
split_pattern=split_pattern,
pronunciation_rules=pronunciation_rules,
heteronym_rules=heteronym_rules,
normalization_overrides=pronunciation.normalization_overrides,
usage_counter=usage_counter if usage_counter is not None else {},
)
+45 -2
View File
@@ -11,9 +11,8 @@ import platform
import re import re
from datetime import datetime from datetime import datetime
from pathlib import Path from pathlib import Path
from typing import Any, Callable, List, Optional, Tuple from typing import Callable, List, Optional, Tuple
from abogen.subtitle_utils import sanitize_name_for_os
from abogen.text_extractor import ExtractedChapter from abogen.text_extractor import ExtractedChapter
@@ -21,6 +20,9 @@ _OUTPUT_SANITIZE_RE = re.compile(r"[^\w\-_.]+")
# OS-specific illegal characters for filenames # OS-specific illegal characters for filenames
_WINDOWS_ILLEGAL_CHARS_RE = re.compile(r'[<>:"/\\|?*\x00-\x1f]') _WINDOWS_ILLEGAL_CHARS_RE = re.compile(r'[<>:"/\\|?*\x00-\x1f]')
_MACOS_ILLEGAL_CHARS_RE = re.compile(r"[:]")
_LINUX_ILLEGAL_CHARS_RE = re.compile(r"[/\x00]")
_CONTROL_CHARS_RE = re.compile(r"[\x00-\x1f]")
_UNIX_CONTROL_CHARS_RE = re.compile(r'[\x00-\x1f]') _UNIX_CONTROL_CHARS_RE = re.compile(r'[\x00-\x1f]')
_RESERVED_NAMES = frozenset( _RESERVED_NAMES = frozenset(
{"CON", "PRN", "AUX", "NUL"} {"CON", "PRN", "AUX", "NUL"}
@@ -29,6 +31,47 @@ _RESERVED_NAMES = frozenset(
) )
def sanitize_name_for_os(name: str, is_folder: bool = True) -> str:
"""Sanitize a filename or folder name based on the operating system.
Args:
name: The name to sanitize
is_folder: Whether this is a folder name (default: True)
Returns:
Sanitized name safe for the current OS
"""
if not name:
return "audiobook"
system = platform.system()
if system == "Windows":
sanitized = _WINDOWS_ILLEGAL_CHARS_RE.sub("_", name)
sanitized = _CONTROL_CHARS_RE.sub("_", sanitized)
sanitized = sanitized.rstrip(". ")
if sanitized.upper() in _RESERVED_NAMES or sanitized.upper().split(".")[0] in _RESERVED_NAMES:
sanitized = f"_{sanitized}"
elif system == "Darwin":
sanitized = _MACOS_ILLEGAL_CHARS_RE.sub("_", name)
sanitized = _CONTROL_CHARS_RE.sub("_", sanitized)
if is_folder and sanitized.startswith("."):
sanitized = "_" + sanitized[1:]
else:
sanitized = _LINUX_ILLEGAL_CHARS_RE.sub("_", name)
sanitized = _UNIX_CONTROL_CHARS_RE.sub("_", sanitized)
if is_folder and sanitized.startswith("."):
sanitized = "_" + sanitized[1:]
if not sanitized or sanitized.strip() == "":
sanitized = "audiobook"
if len(sanitized) > 255:
sanitized = sanitized[:255].rstrip(". ")
return sanitized
def slugify(title: str, index: int) -> str: def slugify(title: str, index: int) -> str:
sanitized = re.sub(r"[^\w\-]+", "_", title.lower()).strip("_") sanitized = re.sub(r"[^\w\-]+", "_", title.lower()).strip("_")
if not sanitized: if not sanitized:
+21 -32
View File
@@ -2,30 +2,21 @@
Provides a unified interface for creating and managing TTS pipelines Provides a unified interface for creating and managing TTS pipelines
across all UI layers (WebUI, PyQt, CLI). across all UI layers (WebUI, PyQt, CLI).
Language handling: the engine owns the mapping between Language enum
and its internal format. Callers pass Language enum; the engine
converts internally. No engine-specific codes leak outside the engine.
""" """
from __future__ import annotations from __future__ import annotations
from typing import Any, Dict, Optional from typing import Any, Dict
from abogen.domain.device import select_device from abogen.domain.device import select_device
from abogen.domain.enums import Language from abogen.domain.enums import Language
from abogen.domain.voice_resolution import initialize_voice_cache from abogen.domain.voice_resolution import initialize_voice_cache
from abogen.tts_plugin.utils import create_pipeline, is_plugin_registered from abogen.tts_plugin.utils import create_pipeline, is_plugin_registered
# Kokoro-specific language mapping (engine's responsibility)
_KOKORO_LANG_MAP = {
Language.EN_US: "a",
Language.EN_GB: "b",
Language.ES: "e",
Language.FR: "f",
Language.HI: "h",
Language.IT: "i",
Language.JA: "j",
Language.PT_BR: "p",
Language.ZH: "z",
}
def resolve_device(use_gpu: bool) -> str: def resolve_device(use_gpu: bool) -> str:
"""Determine compute device from job and global config flags.""" """Determine compute device from job and global config flags."""
@@ -39,29 +30,25 @@ def resolve_device(use_gpu: bool) -> str:
def create_pipeline_for_job( def create_pipeline_for_job(
provider: str, provider: str,
language: str, language: Language,
use_gpu: bool, use_gpu: bool,
) -> Any: ) -> Any:
"""Create a TTS pipeline with proper device selection. """Create a TTS pipeline with proper device selection.
Handles provider validation, GPU decision, and plugin checks. Args:
provider: TTS provider name ("kokoro" or "supertonic").
language: Language enum (app-layer type, not engine-specific).
use_gpu: Whether GPU acceleration is requested.
""" """
provider = str(provider or "kokoro").strip().lower() or "kokoro" provider = str(provider or "kokoro").strip().lower() or "kokoro"
if not is_plugin_registered(provider): if not is_plugin_registered(provider):
provider = "kokoro" provider = "kokoro"
# Convert Language enum to Kokoro single-letter code
try:
lang = Language.from_str(language) if not isinstance(language, Language) else language
except ValueError:
lang = Language.EN_US # fallback for unknown languages
kokoro_code = _KOKORO_LANG_MAP.get(lang, "a")
if provider == "supertonic": if provider == "supertonic":
return create_pipeline("supertonic") return create_pipeline("supertonic", language=language)
device = resolve_device(use_gpu) device = resolve_device(use_gpu)
return create_pipeline("kokoro", lang_code=kokoro_code, device=device) return create_pipeline("kokoro", language=language, device=device)
def dispose_pipelines(pipelines: Dict[str, Any]) -> None: def dispose_pipelines(pipelines: Dict[str, Any]) -> None:
@@ -80,7 +67,7 @@ class PipelinePool:
Usage:: Usage::
pool = PipelinePool() pool = PipelinePool()
backend = pool.get("kokoro", "en", use_gpu=True) backend = pool.get("kokoro", Language.EN_US, use_gpu=True)
# ... use backend ... # ... use backend ...
pool.dispose_all() pool.dispose_all()
""" """
@@ -92,18 +79,20 @@ class PipelinePool:
def get( def get(
self, self,
provider: str, provider: str,
language: str, language: Language,
use_gpu: bool, use_gpu: bool,
*, *,
job: Any = None, request: Any = None,
events: Any = None,
) -> Any: ) -> Any:
"""Get or create a cached pipeline for the given provider. """Get or create a cached pipeline for the given provider.
Args: Args:
provider: TTS provider name ("kokoro" or "supertonic"). provider: TTS provider name ("kokoro" or "supertonic").
language: Language code (for kokoro). language: Language enum (app-layer type).
use_gpu: Whether GPU acceleration is requested. use_gpu: Whether GPU acceleration is requested.
job: Optional job object for voice cache initialization. request: ConversionRequest for voice cache initialization.
events: ConversionEvents for logging during cache init.
""" """
provider = str(provider or "kokoro").strip().lower() or "kokoro" provider = str(provider or "kokoro").strip().lower() or "kokoro"
if not is_plugin_registered(provider): if not is_plugin_registered(provider):
@@ -116,8 +105,8 @@ class PipelinePool:
pipeline = create_pipeline_for_job(provider, language, use_gpu) pipeline = create_pipeline_for_job(provider, language, use_gpu)
self._pipelines[provider] = pipeline self._pipelines[provider] = pipeline
if provider == "kokoro" and not self._voice_cache_initialized and job is not None: if provider == "kokoro" and not self._voice_cache_initialized and request is not None:
initialize_voice_cache(job) initialize_voice_cache(request, events=events)
self._voice_cache_initialized = True self._voice_cache_initialized = True
return pipeline return pipeline
+16 -7
View File
@@ -180,11 +180,20 @@ def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
we must merge manual overrides so they always apply (before TTS). we must merge manual overrides so they always apply (before TTS).
Precedence: manual overrides win over existing entries for the same normalized key. Precedence: manual overrides win over existing entries for the same normalized key.
Args:
job: Either a job-like object with attributes, or a dict with keys:
``pronunciation_overrides``, ``manual_overrides``, ``speakers``, ``language``.
""" """
collected: Dict[str, Dict[str, Any]] = {} collected: Dict[str, Dict[str, Any]] = {}
existing = getattr(job, "pronunciation_overrides", None) def _get(key: str, default: Any = None) -> Any:
if isinstance(job, Mapping):
return job.get(key, default)
return getattr(job, key, default)
existing = _get("pronunciation_overrides")
if isinstance(existing, list): if isinstance(existing, list):
for entry in existing: for entry in existing:
if not isinstance(entry, Mapping): if not isinstance(entry, Mapping):
@@ -204,10 +213,10 @@ def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
"notes": str(entry.get("notes") or "").strip() or None, "notes": str(entry.get("notes") or "").strip() or None,
"context": str(entry.get("context") or "").strip() or None, "context": str(entry.get("context") or "").strip() or None,
"source": str(entry.get("source") or "pronunciation"), "source": str(entry.get("source") or "pronunciation"),
"language": getattr(job, "language", None), "language": _get("language"),
} }
speakers = getattr(job, "speakers", None) speakers = _get("speakers")
if isinstance(speakers, dict): if isinstance(speakers, dict):
for payload in speakers.values(): for payload in speakers.values():
if not isinstance(payload, Mapping): if not isinstance(payload, Mapping):
@@ -226,16 +235,16 @@ def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
"voice": str( "voice": str(
payload.get("resolved_voice") payload.get("resolved_voice")
or payload.get("voice") or payload.get("voice")
or getattr(job, "voice", "") or _get("voice", "")
).strip() ).strip()
or None, or None,
"notes": None, "notes": None,
"context": None, "context": None,
"source": "speaker", "source": "speaker",
"language": getattr(job, "language", None), "language": _get("language"),
} }
manual = getattr(job, "manual_overrides", None) manual = _get("manual_overrides")
if isinstance(manual, list): if isinstance(manual, list):
for entry in manual: for entry in manual:
if not isinstance(entry, Mapping): if not isinstance(entry, Mapping):
@@ -255,7 +264,7 @@ def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
"notes": str(entry.get("notes") or "").strip() or None, "notes": str(entry.get("notes") or "").strip() or None,
"context": str(entry.get("context") or "").strip() or None, "context": str(entry.get("context") or "").strip() or None,
"source": str(entry.get("source") or "manual"), "source": str(entry.get("source") or "manual"),
"language": getattr(job, "language", None), "language": _get("language"),
} }
return list(collected.values()) return list(collected.values())
+66 -5
View File
@@ -9,11 +9,11 @@ from __future__ import annotations
import os import os
import re import re
from dataclasses import dataclass, field from dataclasses import dataclass
from typing import Any, Callable, Dict, Mapping, Optional, Sequence from typing import Any, Callable, Dict, Mapping, Optional
from abogen.constants import ( from abogen.constants import (
LANGUAGE_DESCRIPTIONS, KOKORO_CODE_LABELS,
SUBTITLE_FORMATS, SUBTITLE_FORMATS,
SUPPORTED_SOUND_FORMATS, SUPPORTED_SOUND_FORMATS,
) )
@@ -135,10 +135,10 @@ def _norm_speaker_spec(value: Any, default: str) -> str:
def _norm_language_list(value: Any, default: list) -> list: def _norm_language_list(value: Any, default: list) -> list:
if isinstance(value, (list, tuple, set)): if isinstance(value, (list, tuple, set)):
return [code for code in value if isinstance(code, str) and code in LANGUAGE_DESCRIPTIONS] return [code for code in value if isinstance(code, str) and code in KOKORO_CODE_LABELS]
if isinstance(value, str): if isinstance(value, str):
parts = [item.strip().lower() for item in value.split(",") if item.strip()] parts = [item.strip().lower() for item in value.split(",") if item.strip()]
return [code for code in parts if code in LANGUAGE_DESCRIPTIONS] return [code for code in parts if code in KOKORO_CODE_LABELS]
return default return default
@@ -578,3 +578,64 @@ def integration_defaults() -> Dict[str, Dict[str, Any]]:
"timeout": 30.0, "timeout": 30.0,
}, },
} }
def stored_integration_config(name: str) -> Dict[str, Any]:
"""Read raw integration config from config.json.
Reads ``config["integrations"][name]``.
"""
from abogen.utils import load_config
cfg = load_config() or {}
integrations = cfg.get("integrations")
if isinstance(integrations, Mapping):
entry = integrations.get(name)
if isinstance(entry, Mapping):
return dict(entry)
return {}
def load_audiobookshelf_config() -> Optional["AudiobookshelfConfig"]:
"""Read Audiobookshelf settings from config.json and build typed config.
Returns ``None`` when the integration is not configured or required
fields are missing.
"""
raw = stored_integration_config("audiobookshelf")
if not raw:
return None
return build_audiobookshelf_config(raw)
def build_audiobookshelf_config(
settings: Mapping[str, Any],
) -> Optional["AudiobookshelfConfig"]:
"""Build :class:`AudiobookshelfConfig` from a settings dict.
Returns ``None`` when required fields (base_url, api_token, library_id)
are missing.
"""
from abogen.integrations.audiobookshelf import AudiobookshelfConfig
base_url = str(settings.get("base_url") or "").strip()
api_token = str(settings.get("api_token") or "").strip()
library_id = str(settings.get("library_id") or "").strip()
if not (base_url and api_token and library_id):
return None
try:
timeout = float(settings.get("timeout", 3600.0))
except (TypeError, ValueError):
timeout = 3600.0
return AudiobookshelfConfig(
base_url=base_url,
api_token=api_token,
library_id=library_id,
collection_id=(str(settings.get("collection_id") or "").strip() or None),
folder_id=(str(settings.get("folder_id") or "").strip() or None),
verify_ssl=coerce_bool(settings.get("verify_ssl"), True),
send_cover=coerce_bool(settings.get("send_cover"), True),
send_chapters=coerce_bool(settings.get("send_chapters"), True),
send_subtitles=coerce_bool(settings.get("send_subtitles"), False),
timeout=timeout,
)
+381
View File
@@ -0,0 +1,381 @@
"""Speaker metadata functions for building and applying speaker rosters.
This module contains the core logic for:
- Building narrator and speaker rosters from analysis results
- Matching speakers to configured presets
- Applying speaker config presets to rosters
- Preparing full speaker metadata for conversion
Moved from webui/routes/utils/voice.py to be available across all UIs.
"""
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
from abogen.speaker_analysis import analyze_speakers
from abogen.speaker_configs import slugify_label
from abogen.domain.settings_core import load_settings
def build_narrator_roster(
voice: str,
voice_profile: Optional[str],
existing: Optional[Mapping[str, Any]] = None,
) -> Dict[str, Any]:
roster: Dict[str, Any] = {
"narrator": {
"id": "narrator",
"label": "Narrator",
"voice": voice,
}
}
if voice_profile:
roster["narrator"]["voice_profile"] = voice_profile
existing_entry: Optional[Mapping[str, Any]] = None
if existing is not None:
existing_entry = existing.get("narrator") if isinstance(existing, Mapping) else None
if isinstance(existing_entry, Mapping):
roster_entry = roster["narrator"]
for key in ("label", "voice", "voice_profile", "voice_formula", "pronunciation"):
value = existing_entry.get(key)
if value is not None and value != "":
roster_entry[key] = value
return roster
def build_speaker_roster(
analysis: Dict[str, Any],
base_voice: str,
voice_profile: Optional[str],
existing: Optional[Mapping[str, Any]] = None,
order: Optional[Iterable[str]] = None,
) -> Dict[str, Any]:
roster = build_narrator_roster(base_voice, voice_profile, existing)
existing_map: Dict[str, Any] = dict(existing) if isinstance(existing, Mapping) else {}
speakers = analysis.get("speakers", {}) if isinstance(analysis, dict) else {}
ordered_ids: Iterable[str]
if order is not None:
ordered_ids = [sid for sid in order if sid in speakers]
else:
ordered_ids = speakers.keys()
for speaker_id in ordered_ids:
payload = speakers.get(speaker_id, {})
if speaker_id == "narrator":
continue
if isinstance(payload, Mapping) and payload.get("suppressed"):
continue
previous = existing_map.get(speaker_id)
roster[speaker_id] = {
"id": speaker_id,
"label": payload.get("label") or speaker_id.replace("_", " ").title(),
"analysis_confidence": payload.get("confidence"),
"analysis_count": payload.get("count"),
"gender": payload.get("gender", "unknown"),
}
detected_gender = payload.get("detected_gender")
if detected_gender:
roster[speaker_id]["detected_gender"] = detected_gender
samples = payload.get("sample_quotes")
if isinstance(samples, list):
roster[speaker_id]["sample_quotes"] = samples
if isinstance(previous, Mapping):
for key in ("voice", "voice_profile", "voice_formula", "resolved_voice", "pronunciation"):
value = previous.get(key)
if value is not None and value != "":
roster[speaker_id][key] = value
if "sample_quotes" not in roster[speaker_id]:
prev_samples = previous.get("sample_quotes")
if isinstance(prev_samples, list):
roster[speaker_id]["sample_quotes"] = prev_samples
if "detected_gender" not in roster[speaker_id]:
prev_detected = previous.get("detected_gender")
if isinstance(prev_detected, str) and prev_detected:
roster[speaker_id]["detected_gender"] = prev_detected
return roster
def match_configured_speaker(
config_speakers: Mapping[str, Any],
roster_id: str,
roster_label: str,
) -> Optional[Mapping[str, Any]]:
if not config_speakers:
return None
entry = config_speakers.get(roster_id)
if entry:
return cast(Mapping[str, Any], entry)
slug = slugify_label(roster_label)
if slug != roster_id and slug in config_speakers:
return cast(Mapping[str, Any], config_speakers[slug])
lower_label = roster_label.strip().lower()
for record in config_speakers.values():
if not isinstance(record, Mapping):
continue
if str(record.get("label", "")).strip().lower() == lower_label:
return record
return None
def apply_speaker_config_to_roster(
roster: Mapping[str, Any],
config: Optional[Mapping[str, Any]],
*,
persist_changes: bool = False,
fallback_languages: Optional[Iterable[str]] = None,
) -> Tuple[Dict[str, Any], List[str], Optional[Dict[str, Any]]]:
if not isinstance(roster, Mapping):
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
return {}, effective_languages, None
updated_roster: Dict[str, Any] = {key: dict(value) for key, value in roster.items() if isinstance(value, Mapping)}
if not config:
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
return updated_roster, effective_languages, None
speakers_map = config.get("speakers")
if not isinstance(speakers_map, Mapping):
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
return updated_roster, effective_languages, None
config_languages = config.get("languages")
if isinstance(config_languages, list):
allowed_languages = [code for code in config_languages if isinstance(code, str) and code]
else:
allowed_languages = []
if not allowed_languages and fallback_languages:
allowed_languages = [code for code in fallback_languages if isinstance(code, str) and code]
default_voice = config.get("default_voice") if isinstance(config.get("default_voice"), str) else ""
used_voices = {entry.get("resolved_voice") or entry.get("voice") for entry in updated_roster.values()} - {None}
narrator_voice = ""
narrator_entry = updated_roster.get("narrator") if isinstance(updated_roster, Mapping) else None
if isinstance(narrator_entry, Mapping):
narrator_voice = str(
narrator_entry.get("resolved_voice")
or narrator_entry.get("default_voice")
or ""
).strip()
if narrator_voice:
used_voices.add(narrator_voice)
config_changed = False
new_config_payload: Dict[str, Any] = {
"language": config.get("language", "a"),
"languages": allowed_languages,
"default_voice": default_voice,
"speakers": dict(speakers_map),
"version": config.get("version", 1),
"notes": config.get("notes", ""),
}
speakers_payload = new_config_payload["speakers"]
for speaker_id, roster_entry in updated_roster.items():
if speaker_id == "narrator":
continue
label = str(roster_entry.get("label") or speaker_id)
config_entry = match_configured_speaker(speakers_map, speaker_id, label)
if config_entry is None:
continue
voice_id = str(config_entry.get("voice") or "").strip()
voice_profile = str(config_entry.get("voice_profile") or "").strip()
voice_formula = str(config_entry.get("voice_formula") or "").strip()
resolved_voice = str(config_entry.get("resolved_voice") or "").strip()
languages = config_entry.get("languages") if isinstance(config_entry.get("languages"), list) else []
chosen_voice = resolved_voice or voice_formula or voice_id or roster_entry.get("voice")
usable_languages = languages or allowed_languages
if chosen_voice:
roster_entry["resolved_voice"] = chosen_voice
roster_entry["voice"] = chosen_voice if not voice_profile and not voice_formula else roster_entry.get("voice", chosen_voice)
if voice_profile:
roster_entry["voice_profile"] = voice_profile
if voice_formula:
roster_entry["voice_formula"] = voice_formula
roster_entry["resolved_voice"] = voice_formula
if not voice_formula and not voice_profile and resolved_voice:
roster_entry["resolved_voice"] = resolved_voice
roster_entry["config_languages"] = usable_languages or []
if chosen_voice:
used_voices.add(chosen_voice)
# persist updates back to config payload if required
if persist_changes:
slug = config_entry.get("id") or slugify_label(label)
speakers_payload[slug] = {
"id": slug,
"label": label,
"gender": config_entry.get("gender", "unknown"),
"voice": voice_id,
"voice_profile": voice_profile,
"voice_formula": voice_formula,
"resolved_voice": roster_entry.get("resolved_voice", resolved_voice or voice_id),
"languages": usable_languages,
}
new_config = new_config_payload if (persist_changes and config_changed) else None
return updated_roster, allowed_languages, new_config
def prepare_speaker_metadata(
*,
chapters: List[Dict[str, Any]],
chunks: List[Dict[str, Any]],
analysis_chunks: Optional[List[Dict[str, Any]]] = None,
voice: str,
voice_profile: Optional[str],
threshold: int,
existing_roster: Optional[Mapping[str, Any]] = None,
run_analysis: bool = True,
speaker_config: Optional[Mapping[str, Any]] = None,
apply_config: bool = False,
persist_config: bool = False,
inject_recommended: Optional[Any] = None,
) -> tuple[List[Dict[str, Any]], Dict[str, Any], Dict[str, Any], List[str], Optional[Dict[str, Any]]]:
chunk_list = [dict(chunk) for chunk in chunks]
analysis_source = [dict(chunk) for chunk in (analysis_chunks or chunks)]
threshold_value = max(1, int(threshold))
analysis_enabled = run_analysis
settings_state = load_settings()
global_random_languages = [
code
for code in settings_state.get("speaker_random_languages", [])
if isinstance(code, str) and code
]
if not analysis_enabled:
for chunk in chunk_list:
chunk["speaker_id"] = "narrator"
chunk["speaker_label"] = "Narrator"
analysis_payload = {
"version": "1.0",
"narrator": "narrator",
"assignments": {str(chunk.get("id")): "narrator" for chunk in chunk_list},
"speakers": {
"narrator": {
"id": "narrator",
"label": "Narrator",
"count": len(chunk_list),
"confidence": "low",
"sample_quotes": [],
"suppressed": False,
}
},
"suppressed": [],
"stats": {
"total_chunks": len(chunk_list),
"explicit_chunks": 0,
"active_speakers": 0,
"unique_speakers": 1,
"suppressed": 0,
},
}
roster = build_narrator_roster(voice, voice_profile, existing_roster)
narrator_pron = roster["narrator"].get("pronunciation")
if narrator_pron:
analysis_payload["speakers"]["narrator"]["pronunciation"] = narrator_pron
return chunk_list, roster, analysis_payload, [], None
analysis_result = analyze_speakers(
chapters,
analysis_source,
threshold=threshold_value,
max_speakers=0,
)
analysis_payload = analysis_result.to_dict()
speakers_payload = analysis_payload.get("speakers", {})
ordered_ids = [
sid
for sid, meta in sorted(
(
(sid, meta)
for sid, meta in speakers_payload.items()
if sid != "narrator" and isinstance(meta, Mapping) and not meta.get("suppressed")
),
key=lambda item: item[1].get("count", 0),
reverse=True,
)
]
analysis_payload["ordered_speakers"] = ordered_ids
assignments = analysis_payload.get("assignments", {})
suppressed_ids = analysis_payload.get("suppressed", [])
suppressed_details: List[Dict[str, Any]] = []
speakers_payload = analysis_payload.get("speakers", {})
if isinstance(suppressed_ids, Iterable):
for suppressed_id in suppressed_ids:
speaker_meta = speakers_payload.get(suppressed_id) if isinstance(speakers_payload, dict) else None
if isinstance(speaker_meta, dict):
suppressed_details.append(
{
"id": suppressed_id,
"label": speaker_meta.get("label")
or str(suppressed_id).replace("_", " ").title(),
"pronunciation": speaker_meta.get("pronunciation"),
}
)
else:
suppressed_details.append(
{
"id": suppressed_id,
"label": str(suppressed_id).replace("_", " ").title(),
"pronunciation": None,
}
)
analysis_payload["suppressed_details"] = suppressed_details
roster = build_speaker_roster(
analysis_payload,
voice,
voice_profile,
existing=existing_roster,
order=analysis_payload.get("ordered_speakers"),
)
applied_languages: List[str] = []
updated_config: Optional[Dict[str, Any]] = None
if apply_config and speaker_config:
roster, applied_languages, updated_config = apply_speaker_config_to_roster(
roster,
speaker_config,
persist_changes=persist_config,
fallback_languages=global_random_languages,
)
speakers_payload = analysis_payload.get("speakers")
if isinstance(speakers_payload, dict):
for roster_id, roster_payload in roster.items():
speaker_meta = speakers_payload.get(roster_id)
if isinstance(speaker_meta, dict):
for key in ("voice", "voice_profile", "voice_formula", "resolved_voice"):
value = roster_payload.get(key)
if value:
speaker_meta[key] = value
effective_languages: List[str] = []
if applied_languages:
effective_languages = applied_languages
elif isinstance(analysis_payload.get("config_languages"), list):
effective_languages = [
code for code in analysis_payload.get("config_languages", []) if isinstance(code, str) and code
]
elif global_random_languages:
effective_languages = list(global_random_languages)
if effective_languages:
analysis_payload["config_languages"] = effective_languages
speakers_payload = analysis_payload.get("speakers")
if isinstance(speakers_payload, dict):
for roster_id, roster_payload in roster.items():
if roster_id in speakers_payload and isinstance(roster_payload, dict):
pronunciation_value = roster_payload.get("pronunciation")
if pronunciation_value:
speakers_payload[roster_id]["pronunciation"] = pronunciation_value
fallback_languages = effective_languages or []
if callable(inject_recommended):
inject_recommended(roster, fallback_languages=fallback_languages)
for chunk in chunk_list:
chunk_id = str(chunk.get("id"))
speaker_id = assignments.get(chunk_id, "narrator")
chunk["speaker_id"] = speaker_id
speaker_meta = roster.get(speaker_id)
chunk["speaker_label"] = speaker_meta.get("label") if isinstance(speaker_meta, dict) else speaker_id
return chunk_list, roster, analysis_payload, applied_languages, updated_config
+11 -12
View File
@@ -1,43 +1,42 @@
from __future__ import annotations from __future__ import annotations
"""Unified split pattern logic extracted from 3 copies.""" """Unified split pattern logic extracted from 3 copies."""
import re
from abogen.domain.enums import Language, SubtitleMode from abogen.domain.enums import Language, SubtitleMode
PUNCTUATION_SENTENCE = r".!?。!?" # Canonical punctuation sets covering all supported scripts:
PUNCTUATION_SENTENCE_COMMA = r".!?,。!?、," # ASCII (. ! ?), Arabic ؟, CJK (。!?), Devanagari ।
PUNCTUATION_SENTENCE = r".!?؟。!?।"
# Commas: ASCII , CJK fullwidth CJK ideographic 、
PUNCTUATION_SENTENCE_COMMA = r".!?,?。!?،,、।"
PUNCTUATION_COMMAS = ",,、"
def get_split_pattern(language: str, subtitle_mode: str) -> str: def get_split_pattern(language: Language, subtitle_mode: str) -> str:
"""Get the appropriate split pattern based on language and subtitle mode. """Get the appropriate split pattern based on language and subtitle mode.
Args: Args:
language: Language code (a, b, e, f, etc.) language: Language enum value.
subtitle_mode: Subtitle mode ("Sentence", "Sentence + Comma", "Line", etc.) subtitle_mode: Subtitle mode ("Sentence", "Sentence + Comma", "Line", etc.)
Returns: Returns:
Split pattern string Split pattern string
""" """
try:
lang = Language.from_str(language) if not isinstance(language, Language) else language
except ValueError:
lang = None # unknown language — treat as non-English, non-CJK
try: try:
mode = SubtitleMode.from_str(subtitle_mode) if not isinstance(subtitle_mode, SubtitleMode) else subtitle_mode mode = SubtitleMode.from_str(subtitle_mode) if not isinstance(subtitle_mode, SubtitleMode) else subtitle_mode
except ValueError: except ValueError:
mode = SubtitleMode.DISABLED mode = SubtitleMode.DISABLED
# For English, always use newline splitting only # For English, always use newline splitting only
if lang in (Language.EN_US, Language.EN_GB): if language in (Language.EN_US, Language.EN_GB):
return "\n" return "\n"
# Determine spacing pattern based on language # Determine spacing pattern based on language
spacing = r"\s*" if lang and lang.is_cjk else r"\s+" spacing = r"\s*" if language.is_cjk else r"\s+"
# For CJK languages, when subtitle mode is Disabled or Line, prefer # For CJK languages, when subtitle mode is Disabled or Line, prefer
# punctuation-based splitting instead of plain newline splitting. # punctuation-based splitting instead of plain newline splitting.
if mode in (SubtitleMode.DISABLED, SubtitleMode.LINE) and lang and lang.is_cjk: if mode in (SubtitleMode.DISABLED, SubtitleMode.LINE) and language.is_cjk:
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+" return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
if mode == SubtitleMode.LINE: if mode == SubtitleMode.LINE:
+10 -16
View File
@@ -11,11 +11,7 @@ import re
from typing import List, Optional, Tuple from typing import List, Optional, Tuple
from abogen.domain.enums import Language, SubtitleMode from abogen.domain.enums import Language, SubtitleMode
from abogen.domain.split_pattern import PUNCTUATION_SENTENCE, PUNCTUATION_SENTENCE_COMMA
# Punctuation constants for sentence splitting
PUNCTUATION_SENTENCE = ".!?\u061f\u3002\uff01\uff1f" # .!? .?. ??
PUNCTUATION_SENTENCE_COMMA = ".!?,\u3001\u061f\u3002\uff01\uff0c\uff1f" # .!?, ,. ??
def process_subtitle_tokens( def process_subtitle_tokens(
@@ -23,7 +19,7 @@ def process_subtitle_tokens(
subtitle_entries: List[Tuple[float, float, str]], subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int, max_subtitle_words: int,
subtitle_mode: str, subtitle_mode: str,
lang_code: str, language: Language,
use_spacy_segmentation: bool = False, use_spacy_segmentation: bool = False,
fallback_end_time: Optional[float] = None, fallback_end_time: Optional[float] = None,
) -> None: ) -> None:
@@ -39,7 +35,7 @@ def process_subtitle_tokens(
max_subtitle_words: Maximum number of words per subtitle entry. max_subtitle_words: Maximum number of words per subtitle entry.
subtitle_mode: One of "Disabled", "Line", "Sentence", "Sentence + Comma", subtitle_mode: One of "Disabled", "Line", "Sentence", "Sentence + Comma",
"Sentence + Highlighting", or a string like "5" for word-count mode. "Sentence + Highlighting", or a string like "5" for word-count mode.
lang_code: Language code for spaCy processing (e.g., "a" for English). language: Language enum value for spaCy processing.
use_spacy_segmentation: Whether to use spaCy for sentence boundary detection. use_spacy_segmentation: Whether to use spaCy for sentence boundary detection.
fallback_end_time: Fallback end time for the last entry if none is available. fallback_end_time: Fallback end time for the last entry if none is available.
""" """
@@ -53,7 +49,7 @@ def process_subtitle_tokens(
use_spacy_for_english = ( use_spacy_for_english = (
use_spacy_segmentation use_spacy_segmentation
and subtitle_mode not in [SubtitleMode.DISABLED, SubtitleMode.LINE] and subtitle_mode not in [SubtitleMode.DISABLED, SubtitleMode.LINE]
and lang_code in [Language.EN_US, Language.EN_GB] and language in [Language.EN_US, Language.EN_GB]
and subtitle_mode in [SubtitleMode.SENTENCE, SubtitleMode.SENTENCE_COMMA] and subtitle_mode in [SubtitleMode.SENTENCE, SubtitleMode.SENTENCE_COMMA]
) )
@@ -65,7 +61,7 @@ def process_subtitle_tokens(
if use_spacy_for_english and subtitle_mode != SubtitleMode.LINE: if use_spacy_for_english and subtitle_mode != SubtitleMode.LINE:
_process_spacy_sentences( _process_spacy_sentences(
processed_tokens, subtitle_entries, max_subtitle_words, processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, lang_code, fallback_end_time subtitle_mode, language, fallback_end_time
) )
else: else:
_process_regex_sentences( _process_regex_sentences(
@@ -87,7 +83,7 @@ def _process_karaoke_highlighting(
fallback_end_time: Optional[float], fallback_end_time: Optional[float],
) -> None: ) -> None:
"""Process tokens for Sentence + Highlighting mode (karaoke effect).""" """Process tokens for Sentence + Highlighting mode (karaoke effect)."""
separator = rf"[{re.escape(PUNCTUATION_SENTENCE)}]" separator = rf"[{PUNCTUATION_SENTENCE}]"
current_sentence = [] current_sentence = []
word_count = 0 word_count = 0
@@ -145,7 +141,7 @@ def _process_spacy_sentences(
subtitle_entries: List[Tuple[float, float, str]], subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int, max_subtitle_words: int,
subtitle_mode: str, subtitle_mode: str,
lang_code: str, language: Language,
fallback_end_time: Optional[float], fallback_end_time: Optional[float],
) -> None: ) -> None:
"""Process tokens using spaCy for sentence boundary detection.""" """Process tokens using spaCy for sentence boundary detection."""
@@ -159,7 +155,7 @@ def _process_spacy_sentences(
) )
return return
nlp = get_spacy_model(lang_code) nlp = get_spacy_model(language)
if not nlp: if not nlp:
_process_regex_sentences( _process_regex_sentences(
tokens, subtitle_entries, max_subtitle_words, tokens, subtitle_entries, max_subtitle_words,
@@ -247,11 +243,9 @@ def _process_regex_sentences(
if subtitle_mode == SubtitleMode.LINE: if subtitle_mode == SubtitleMode.LINE:
separator = r"\n" separator = r"\n"
elif subtitle_mode == SubtitleMode.SENTENCE: elif subtitle_mode == SubtitleMode.SENTENCE:
# Use punctuation without comma separator = rf"[{PUNCTUATION_SENTENCE}]"
separator = rf"[{re.escape(PUNCTUATION_SENTENCE)}]"
else: # Sentence + Comma else: # Sentence + Comma
# Use punctuation with comma separator = rf"[{PUNCTUATION_SENTENCE_COMMA}]"
separator = rf"[{re.escape(PUNCTUATION_SENTENCE_COMMA)}]"
current_sentence = [] current_sentence = []
word_count = 0 word_count = 0
-1
View File
@@ -14,7 +14,6 @@ from typing import Any, Callable, List, Optional, Tuple
import numpy as np import numpy as np
from abogen.domain.audio_buffer import ( from abogen.domain.audio_buffer import (
create_silence,
fit_audio_to_duration, fit_audio_to_duration,
ffmpeg_time_stretch, ffmpeg_time_stretch,
mix_audio, mix_audio,
+22
View File
@@ -0,0 +1,22 @@
"""Text utility functions for the domain layer."""
from __future__ import annotations
import re
# Pre-compiled patterns for calculate_text_length
_METADATA_TAG_PATTERN = re.compile(r"<<METADATA_[^:]+:[^>]*>>")
_CHAPTER_MARKER_PATTERN = re.compile(r"<<CHAPTER_MARKER:[^>]*>>")
_VOICE_MARKER_PATTERN = re.compile(r"<<VOICE:[^>]*>>")
def calculate_text_length(text: str) -> int:
"""Calculate character count, ignoring internal markers and newlines.
Strips chapter markers, voice markers, and metadata tags before counting.
"""
text = _CHAPTER_MARKER_PATTERN.sub("", text)
text = _VOICE_MARKER_PATTERN.sub("", text)
text = _METADATA_TAG_PATTERN.sub("", text)
text = text.replace("\n", "").strip()
return len(text)
+1 -1
View File
@@ -1,7 +1,7 @@
from __future__ import annotations from __future__ import annotations
from pathlib import Path from pathlib import Path
from typing import Any, Dict, List, Mapping, Optional from typing import Any, List, Mapping, Optional
from .metadata_helpers import ( from .metadata_helpers import (
ensure_sentence, ensure_sentence,
+112
View File
@@ -0,0 +1,112 @@
"""Voice catalog — shared voice metadata for all UIs.
Builds a unified catalog of available voices with metadata (language,
gender, display name). Used by both WebUI and PyQt for voice selection UIs.
"""
from __future__ import annotations
from typing import Any, Dict, Iterable, List, Mapping, Optional
from abogen.constants import LANGUAGE_DESCRIPTIONS
from abogen.tts_plugin.utils import get_voices
def build_voice_catalog() -> List[Dict[str, str]]:
"""Build voice catalog with metadata for all available voices.
Returns a list of dicts, each containing:
- id: voice ID (e.g. "af_heart")
- language: language code (e.g. "a", "e")
- language_label: human-readable language name
- gender: "Female", "Male", or "Unknown"
- gender_code: "f", "m", or ""
- display_name: human-readable voice name
"""
from plugins.kokoro.engine import language_for_voice_id
catalog: List[Dict[str, str]] = []
gender_map = {"f": "Female", "m": "Male"}
for voice_id in get_voices("kokoro"):
prefix, _, rest = voice_id.partition("_")
gender_code = prefix[1] if len(prefix) > 1 else ""
lang = language_for_voice_id(voice_id)
catalog.append(
{
"id": voice_id,
"language": lang.value,
"language_label": LANGUAGE_DESCRIPTIONS.get(lang, lang.value.upper()),
"gender": gender_map.get(gender_code, "Unknown"),
"gender_code": gender_code,
"display_name": rest.replace("_", " ").title() if rest else voice_id,
}
)
return catalog
def filter_voice_catalog(
catalog: Iterable[Mapping[str, Any]],
*,
gender: str,
allowed_languages: Optional[Iterable[str]] = None,
) -> List[str]:
"""Filter voice catalog by gender and language.
Returns voice IDs that match the criteria. Falls back to broader
matches if no exact matches are found.
Args:
catalog: Voice catalog entries (from build_voice_catalog).
gender: Gender filter ("male", "female", or "unknown").
allowed_languages: Optional list of allowed language codes.
Returns:
List of matching voice IDs.
"""
allowed_set = {code.lower() for code in (allowed_languages or []) if isinstance(code, str) and code}
gender_normalized = (gender or "unknown").lower()
gender_code = ""
if gender_normalized == "male":
gender_code = "m"
elif gender_normalized == "female":
gender_code = "f"
matches: List[str] = []
seen: set[str] = set()
def _consider(entry: Mapping[str, Any]) -> None:
voice_id = entry.get("id")
if not isinstance(voice_id, str) or not voice_id:
return
if voice_id in seen:
return
seen.add(voice_id)
matches.append(voice_id)
primary: List[Mapping[str, Any]] = []
fallback: List[Mapping[str, Any]] = []
for entry in catalog:
if not isinstance(entry, Mapping):
continue
voice_lang = str(entry.get("language", "")).lower()
voice_gender_code = str(entry.get("gender_code", "")).lower()
if allowed_set and voice_lang not in allowed_set:
continue
if gender_code and voice_gender_code != gender_code:
fallback.append(entry)
continue
primary.append(entry)
for entry in primary:
_consider(entry)
if not matches:
for entry in fallback:
_consider(entry)
if not matches:
for entry in catalog:
if isinstance(entry, Mapping):
_consider(entry)
return matches
+1 -1
View File
@@ -6,7 +6,7 @@ PyQt and WebUI interfaces.
from __future__ import annotations from __future__ import annotations
from typing import Any, Dict, Optional, Tuple from typing import Any, Dict, Optional
from abogen.voice_formulas import get_new_voice from abogen.voice_formulas import get_new_voice
+117
View File
@@ -0,0 +1,117 @@
"""Voice marker parsing and text splitting.
Handles <<VOICE:name>> markers in text, splitting text into voice-specific
segments. This is domain logic about text segmentation by voice, not subtitle
processing.
"""
from __future__ import annotations
import re
from typing import List, Tuple
_VOICE_MARKER_PATTERN = re.compile(r"<<VOICE:[^>]*>>")
_VOICE_MARKER_SEARCH_PATTERN = re.compile(r"<<VOICE:(.*?)>>")
def validate_voice_name(voice_name: str) -> Tuple[bool, str | None]:
"""Validate voice name against available voices (case-insensitive).
Handles both single voices and formulas like 'af_heart*0.5 + am_echo*0.5'.
Returns:
Tuple of (is_valid, invalid_voice_name):
- is_valid: True if all voices in the name/formula are valid
- invalid_voice_name: The first invalid voice found, or None if all valid
"""
from abogen.tts_plugin.utils import get_voices
voice_lookup_lower = {v.lower() for v in get_voices("kokoro")}
voice_name = voice_name.strip()
if "*" in voice_name:
voices = voice_name.split("+")
for term in voices:
if "*" in term:
base_voice = term.split("*")[0].strip()
if base_voice.lower() not in voice_lookup_lower:
return False, base_voice
return True, None
else:
if voice_name.lower() not in voice_lookup_lower:
return False, voice_name
return True, None
def split_text_by_voice_markers(
text: str, default_voice: str
) -> Tuple[List[Tuple[str, str]], str, int, int]:
"""Split text by voice markers, returning list of (voice, text) tuples.
Returns the last voice used so it can persist across chapters.
Voice names are normalized to lowercase to match canonical voice names.
Args:
text: Text potentially containing <<VOICE:name>> markers
default_voice: Voice to use if no markers found or before first marker
Returns:
Tuple of (segments_list, last_voice_used, valid_count, invalid_count):
- segments_list: List of (voice_name, segment_text) tuples
- last_voice_used: The voice that should continue into next chapter
- valid_count: Number of valid voice markers processed
- invalid_count: Number of invalid voice markers skipped
"""
from abogen.tts_plugin.utils import get_voices
voice_splits = list(_VOICE_MARKER_SEARCH_PATTERN.finditer(text))
if not voice_splits:
return [(default_voice, text)], default_voice, 0, 0
segments: List[Tuple[str, str]] = []
current_voice = default_voice
valid_markers = 0
invalid_markers = 0
first_start = voice_splits[0].start()
if first_start > 0:
intro_text = text[:first_start].strip()
if intro_text:
segments.append((current_voice, intro_text))
for idx, match in enumerate(voice_splits):
voice_name = match.group(1).strip()
start = match.end()
end = voice_splits[idx + 1].start() if idx + 1 < len(voice_splits) else len(text)
segment_text = text[start:end].strip()
is_valid, invalid_voice = validate_voice_name(voice_name)
if is_valid:
if "*" in voice_name:
normalized_parts = []
for part in voice_name.split("+"):
part = part.strip()
if "*" in part:
voice_part, weight = part.split("*", 1)
voice_part_lower = voice_part.strip().lower()
canonical_voice = next(
(v for v in get_voices("kokoro") if v.lower() == voice_part_lower),
voice_part.strip()
)
normalized_parts.append(f"{canonical_voice}*{weight.strip()}")
current_voice = " + ".join(normalized_parts)
else:
voice_name_lower = voice_name.lower()
current_voice = next(
(v for v in get_voices("kokoro") if v.lower() == voice_name_lower),
voice_name
)
valid_markers += 1
else:
invalid_markers += 1
if segment_text:
segments.append((current_voice, segment_text))
return segments, current_voice, valid_markers, invalid_markers
+189 -24
View File
@@ -2,14 +2,17 @@
Functions for resolving voice specifications, collecting required voice IDs, Functions for resolving voice specifications, collecting required voice IDs,
and determining the voice to use for chapters and chunks. and determining the voice to use for chapters and chunks.
All functions accept ConversionRequest (the app-layer contract) instead of
UI-specific objects. This keeps the domain layer UI-agnostic.
""" """
from __future__ import annotations from __future__ import annotations
from typing import Any, Dict, Optional, Set from typing import Any, Dict, Mapping, Optional, Set, Tuple
from abogen.tts_plugin.utils import get_voices, get_default_voice from abogen.tts_plugin.utils import get_voices, get_default_voice
from abogen.voice_formulas import extract_voice_ids from abogen.voice_formulas import extract_voice_ids, pairs_to_formula
from abogen.voice_cache import ensure_voice_assets from abogen.voice_cache import ensure_voice_assets
@@ -29,12 +32,28 @@ def spec_to_voice_ids(spec: Any) -> Set[str]:
return set() return set()
def job_voice_fallback(job: Any) -> str: def _get_chapter_overrides(request: Any) -> list:
base = str(getattr(job, "voice", "") or "").strip() """Extract chapter overrides from ConversionRequest."""
cc = getattr(request, "chapter_chunk", None)
if cc is not None:
return getattr(cc, "chapter_overrides", []) or []
return []
def _get_chunks(request: Any) -> list:
"""Extract chunks from ConversionRequest."""
cc = getattr(request, "chapter_chunk", None)
if cc is not None:
return getattr(cc, "chunks", []) or []
return []
def job_voice_fallback(request: Any) -> str:
base = str(getattr(request, "voice", "") or "").strip()
if base and base != "__custom_mix": if base and base != "__custom_mix":
return base return base
speakers = getattr(job, "speakers", None) speakers = getattr(request, "speakers", None)
if isinstance(speakers, dict): if isinstance(speakers, dict):
narrator = speakers.get("narrator") narrator = speakers.get("narrator")
if isinstance(narrator, dict): if isinstance(narrator, dict):
@@ -52,7 +71,7 @@ def job_voice_fallback(job: Any) -> str:
if candidate and candidate != "__custom_mix": if candidate and candidate != "__custom_mix":
return candidate return candidate
for chapter in getattr(job, "chapters", []) or []: for chapter in _get_chapter_overrides(request):
if not isinstance(chapter, dict): if not isinstance(chapter, dict):
continue continue
for key in ("resolved_voice", "voice_formula", "voice"): for key in ("resolved_voice", "voice_formula", "voice"):
@@ -63,24 +82,24 @@ def job_voice_fallback(job: Any) -> str:
return "" return ""
def collect_required_voice_ids(job: Any) -> Set[str]: def collect_required_voice_ids(request: Any) -> Set[str]:
voices: Set[str] = set() voices: Set[str] = set()
voices.update(spec_to_voice_ids(job.voice)) voices.update(spec_to_voice_ids(request.voice))
voices.update(spec_to_voice_ids(job_voice_fallback(job))) voices.update(spec_to_voice_ids(job_voice_fallback(request)))
for chapter in getattr(job, "chapters", []) or []: for chapter in _get_chapter_overrides(request):
if not isinstance(chapter, dict): if not isinstance(chapter, dict):
continue continue
for key in ("resolved_voice", "voice_formula", "voice"): for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(chapter.get(key))) voices.update(spec_to_voice_ids(chapter.get(key)))
for chunk in getattr(job, "chunks", []) or []: for chunk in _get_chunks(request):
if not isinstance(chunk, dict): if not isinstance(chunk, dict):
continue continue
for key in ("resolved_voice", "voice_formula", "voice"): for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(chunk.get(key))) voices.update(spec_to_voice_ids(chunk.get(key)))
speakers = getattr(job, "speakers", {}) speakers = getattr(request, "speakers", {})
if isinstance(speakers, dict): if isinstance(speakers, dict):
for payload in speakers.values() or []: for payload in speakers.values() or []:
if not isinstance(payload, dict): if not isinstance(payload, dict):
@@ -92,30 +111,38 @@ def collect_required_voice_ids(job: Any) -> Set[str]:
return voices return voices
def initialize_voice_cache(job: Any) -> None: def initialize_voice_cache(request: Any, events: Any = None) -> None:
"""Initialize voice cache by downloading required voice assets.
Args:
request: ConversionRequest with voice/chapter/chunk/speaker info.
events: ConversionEvents for logging (optional, for backward compat).
"""
log = (lambda msg, level="info": events.log(msg, level=level)) if events else (lambda msg, level="info": None)
try: try:
targets = collect_required_voice_ids(job) targets = collect_required_voice_ids(request)
downloaded, errors = ensure_voice_assets( downloaded, errors = ensure_voice_assets(
targets, targets,
on_progress=lambda message: job.add_log(message, level="debug"), on_progress=lambda message: log(message, level="debug"),
) )
except RuntimeError as exc: except RuntimeError as exc:
job.add_log(f"Voice cache unavailable: {exc}", level="warning") log(f"Voice cache unavailable: {exc}", level="warning")
return return
if downloaded: if downloaded:
job.add_log( log(
f"Cached {len(downloaded)} voice asset{'s' if len(downloaded) != 1 else ''} locally.", f"Cached {len(downloaded)} voice asset{'s' if len(downloaded) != 1 else ''} locally.",
level="info", level="info",
) )
for voice_id, error in errors.items(): for voice_id, error in errors.items():
job.add_log(f"Failed to cache voice '{voice_id}': {error}", level="warning") log(f"Failed to cache voice '{voice_id}': {error}", level="warning")
def chapter_voice_spec(job: Any, override: Optional[Dict[str, Any]]) -> str: def chapter_voice_spec(request: Any, override: Optional[Dict[str, Any]]) -> str:
if not override: if not override:
return job_voice_fallback(job) return job_voice_fallback(request)
resolved = str(override.get("resolved_voice", "")).strip() resolved = str(override.get("resolved_voice", "")).strip()
if resolved: if resolved:
@@ -129,17 +156,17 @@ def chapter_voice_spec(job: Any, override: Optional[Dict[str, Any]]) -> str:
if voice: if voice:
return voice return voice
return job_voice_fallback(job) return job_voice_fallback(request)
def chunk_voice_spec(job: Any, chunk: Dict[str, Any], fallback: str) -> str: def chunk_voice_spec(request: Any, chunk: Dict[str, Any], fallback: str) -> str:
for key in ("resolved_voice", "voice_formula", "voice"): for key in ("resolved_voice", "voice_formula", "voice"):
value = chunk.get(key) value = chunk.get(key)
if value: if value:
return str(value) return str(value)
speaker_id = chunk.get("speaker_id") speaker_id = chunk.get("speaker_id")
speakers = getattr(job, "speakers", None) speakers = getattr(request, "speakers", None)
if isinstance(speakers, dict) and speaker_id in speakers: if isinstance(speakers, dict) and speaker_id in speakers:
speaker_entry = speakers.get(speaker_id) or {} speaker_entry = speakers.get(speaker_id) or {}
if isinstance(speaker_entry, dict): if isinstance(speaker_entry, dict):
@@ -163,7 +190,7 @@ def chunk_voice_spec(job: Any, chunk: Dict[str, Any], fallback: str) -> str:
if fallback: if fallback:
return fallback return fallback
return job_voice_fallback(job) return job_voice_fallback(request)
def resolve_fallback_voice_spec( def resolve_fallback_voice_spec(
@@ -188,3 +215,141 @@ def resolve_fallback_voice_spec(
if not spec: if not spec:
spec = get_default_voice(provider) spec = get_default_voice(provider)
return spec return spec
# ---------------------------------------------------------------------------
# Voice choice resolution (shared by all UIs)
# ---------------------------------------------------------------------------
def formula_from_profile(entry: Dict[str, Any]) -> Optional[str]:
"""Convert a voice profile entry to a voice formula string.
Handles both Kokoro (voices list) and SuperTonic (single voice) profiles.
Returns None if the entry has no usable voice data.
"""
if not isinstance(entry, dict):
return None
voices = entry.get("voices") or []
if not voices:
return None
return pairs_to_formula(voices)
def resolve_profile_voice(
profile_name: Optional[str],
*,
profiles: Optional[Mapping[str, Any]] = None,
) -> Tuple[str, Optional[str]]:
"""Resolve a profile name to (formula, language).
Args:
profile_name: Name of the profile to resolve.
profiles: Pre-loaded profiles dict. If None, loads from disk.
Returns:
(formula_string, language_code) or ("", None) if not found.
"""
if not profile_name:
return "", None
source = profiles if isinstance(profiles, Mapping) else None
if source is None:
from abogen.voice_profiles import load_profiles
source = load_profiles()
entry = source.get(profile_name) if isinstance(source, Mapping) else None
if not isinstance(entry, Mapping):
return "", None
formula = formula_from_profile(dict(entry)) or ""
language = entry.get("language") if isinstance(entry.get("language"), str) else None
if isinstance(language, str):
language = language.strip().lower() or None
return formula, language
def resolve_voice_setting(
value: Any,
*,
profiles: Optional[Mapping[str, Any]] = None,
) -> Tuple[str, Optional[str], Optional[str]]:
"""Resolve a raw voice setting value into (spec, profile_name, language).
Parses 'profile:name' or 'speaker:name' prefixes and resolves
the profile to a formula string.
Args:
value: Raw voice value from user input (e.g. "af_heart", "profile:MyMix").
profiles: Pre-loaded profiles dict. If None, loads from disk.
Returns:
(resolved_spec, profile_name, language) — profile_name and language
are None when the input is a plain voice spec.
"""
from abogen.domain.settings_core import split_profile_spec
base_spec, profile_name = split_profile_spec(value)
if profile_name:
formula, language = resolve_profile_voice(profile_name, profiles=profiles)
return formula or "", profile_name, language
return base_spec, None, None
def resolve_voice_choice(
language: str,
base_voice: str,
profile_name: str,
custom_formula: str,
profiles: Dict[str, Any],
) -> Tuple[str, str, Optional[str]]:
"""Resolve a user's voice selection into (resolved_voice, resolved_language, selected_profile).
Handles three input modes:
1. Profile selection → resolves to formula (Kokoro) or speaker reference (SuperTonic)
2. Custom formula → used directly
3. Plain voice spec → passed through
Args:
language: Current language code (e.g. "a", "e").
base_voice: Base voice spec (voice ID or formula).
profile_name: Selected profile name (empty string if none).
custom_formula: Custom formula string (empty string if none).
profiles: Dict of all available profiles.
Returns:
(resolved_voice, resolved_language, selected_profile)
"""
from abogen.voice_profiles import normalize_profile_entry
resolved_voice = base_voice
resolved_language = language
selected_profile = None
if profile_name:
entry_raw = profiles.get(profile_name)
entry = normalize_profile_entry(entry_raw)
provider = str((entry or {}).get("provider") or "").strip().lower()
# Provider-aware behavior:
# - Kokoro profiles typically represent mixes (formula strings).
# - SuperTonic profiles represent a discrete voice id + settings.
# In that case, we return a speaker reference so downstream can
# resolve provider per-speaker and allow mixed-provider casting.
if provider == "supertonic":
resolved_voice = f"speaker:{profile_name}"
selected_profile = profile_name
profile_language = (entry or {}).get("language")
if profile_language:
resolved_language = str(profile_language)
else:
formula = formula_from_profile(entry or {}) if entry else None
if formula:
resolved_voice = formula
selected_profile = profile_name
profile_language = (entry or {}).get("language")
if profile_language:
resolved_language = profile_language
if custom_formula:
resolved_voice = custom_formula
selected_profile = None
return resolved_voice, resolved_language, selected_profile
+1 -2
View File
@@ -1,8 +1,7 @@
from __future__ import annotations from __future__ import annotations
from typing import Any, Dict, Mapping, Optional, Tuple, Set from typing import Any, Dict, Mapping, Optional, Tuple
from abogen.voice_formulas import extract_voice_ids, get_new_voice
from abogen.tts_plugin.utils import get_voices from abogen.tts_plugin.utils import get_voices
+15 -13
View File
@@ -12,6 +12,7 @@ from typing import Any, Dict, Iterable, List, Optional, Pattern, Sequence, Tuple
import zipfile import zipfile
from abogen.text_extractor import ExtractedChapter, ExtractionResult from abogen.text_extractor import ExtractedChapter, ExtractionResult
from abogen.domain.metadata_helpers import normalize_metadata_map
@dataclass(slots=True) @dataclass(slots=True)
@@ -22,7 +23,7 @@ class ChunkOverlay:
start: Optional[float] start: Optional[float]
end: Optional[float] end: Optional[float]
speaker_id: str speaker_id: str
voice: Optional[str] voice: Optional[Dict[str, str]]
level: Optional[str] = None level: Optional[str] = None
group_id: Optional[str] = None group_id: Optional[str] = None
@@ -59,7 +60,7 @@ class EPUB3PackageBuilder:
self.output_path = output_path self.output_path = output_path
self.book_id = book_id or str(uuid.uuid4()) self.book_id = book_id or str(uuid.uuid4())
self.extraction = extraction self.extraction = extraction
self.metadata_tags = _normalize_metadata(metadata_tags) self.metadata_tags = normalize_metadata_map(metadata_tags)
self.chapter_markers = list(chapter_markers or []) self.chapter_markers = list(chapter_markers or [])
self.chunk_markers = list(chunk_markers or []) self.chunk_markers = list(chunk_markers or [])
self.chunks = list(chunks or []) self.chunks = list(chunks or [])
@@ -273,7 +274,7 @@ class EPUB3PackageBuilder:
start=_safe_float(marker.get("start")), start=_safe_float(marker.get("start")),
end=_safe_float(marker.get("end")), end=_safe_float(marker.get("end")),
speaker_id=speaker_id, speaker_id=speaker_id,
voice=str(voice) if voice else None, voice=voice if isinstance(voice, dict) else None,
level=str(level) if level else None, level=str(level) if level else None,
group_id=normalized_group_id, group_id=normalized_group_id,
) )
@@ -516,9 +517,14 @@ def build_epub3_package(
chunks: Iterable[Dict[str, Any]], chunks: Iterable[Dict[str, Any]],
audio_path: Path, audio_path: Path,
speaker_mode: str = "single", speaker_mode: str = "single",
cover: "CoverConfig | None" = None,
cover_image_path: Optional[Path] = None, cover_image_path: Optional[Path] = None,
cover_image_mime: Optional[str] = None, cover_image_mime: Optional[str] = None,
) -> Path: ) -> Path:
from abogen.domain.config_types import CoverConfig
if isinstance(cover, CoverConfig):
cover_image_path = cover.path
cover_image_mime = cover.mime
builder = EPUB3PackageBuilder( builder = EPUB3PackageBuilder(
output_path=output_path, output_path=output_path,
book_id=book_id, book_id=book_id,
@@ -545,15 +551,6 @@ class ChunkLookup:
by_chapter: Dict[int, List[Dict[str, Any]]] by_chapter: Dict[int, List[Dict[str, Any]]]
def _normalize_metadata(metadata: Optional[Dict[str, Any]]) -> Dict[str, str]:
normalized: Dict[str, str] = {}
for key, value in (metadata or {}).items():
if value is None:
continue
normalized[str(key).lower()] = str(value)
return normalized
def _combine_metadata(*sources: Dict[str, Any]) -> Dict[str, str]: def _combine_metadata(*sources: Dict[str, Any]) -> Dict[str, str]:
combined: Dict[str, str] = {} combined: Dict[str, str] = {}
for source in sources: for source in sources:
@@ -696,7 +693,12 @@ def _group_chunks_for_render(chunks: Sequence[ChunkOverlay]) -> List[Tuple[Optio
def _render_chunk_inline(chunk: ChunkOverlay) -> str: def _render_chunk_inline(chunk: ChunkOverlay) -> str:
escaped_id = html.escape(chunk.id) escaped_id = html.escape(chunk.id)
speaker_attr = f" data-speaker=\"{html.escape(chunk.speaker_id)}\"" if chunk.speaker_id else "" speaker_attr = f" data-speaker=\"{html.escape(chunk.speaker_id)}\"" if chunk.speaker_id else ""
voice_attr = f" data-voice=\"{html.escape(chunk.voice)}\"" if chunk.voice else "" voice_str = None
if chunk.voice and isinstance(chunk.voice, dict):
name = chunk.voice.get("voice", "")
provider = chunk.voice.get("provider", "")
voice_str = f"{name}@{provider}" if name and provider else name or None
voice_attr = f" data-voice=\"{html.escape(voice_str)}\"" if voice_str else ""
level_attr = f" data-level=\"{html.escape(chunk.level)}\"" if chunk.level else "" level_attr = f" data-level=\"{html.escape(chunk.level)}\"" if chunk.level else ""
raw_text = chunk.text or "" raw_text = chunk.text or ""
escaped_text = html.escape(raw_text) escaped_text = html.escape(raw_text)
+13 -136
View File
@@ -10,22 +10,14 @@ from typing import Any, Dict, List, Optional, Mapping, Sequence
import static_ffmpeg import static_ffmpeg
from abogen.domain.metadata_helpers import ( from abogen.domain.metadata_helpers import (
normalize_metadata_casefold,
split_people_field, split_people_field,
split_simple_list, split_simple_list,
first_nonempty, first_nonempty,
extract_year, extract_year,
normalize_series_sequence, normalize_series_sequence,
build_audiobookshelf_metadata as _build_abs_metadata,
load_audiobookshelf_chapters as _load_abs_chapters,
_SERIES_SEQUENCE_TAG_KEYS, _SERIES_SEQUENCE_TAG_KEYS,
) )
from abogen.epub3.exporter import build_epub3_package from abogen.epub3.exporter import build_epub3_package
from abogen.integrations.audiobookshelf import (
AudiobookshelfClient,
AudiobookshelfConfig,
AudiobookshelfUploadError,
)
from abogen.utils import create_process from abogen.utils import create_process
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -84,9 +76,14 @@ class ExportService:
title = chapter.get("title") title = chapter.get("title")
if title: if title:
lines.append(f"title={self._escape_ffmetadata_value(title)}") lines.append(f"title={self._escape_ffmetadata_value(title)}")
voice = chapter.get("voice") voices = chapter.get("voices")
if voice: if voices and isinstance(voices, list):
lines.append(f"voice={self._escape_ffmetadata_value(voice)}") voice_str = ", ".join(
f"{v.get('voice', '')}@{v.get('provider', '')}"
for v in voices if v.get("voice")
)
if voice_str:
lines.append(f"voice={self._escape_ffmetadata_value(voice_str)}")
return "\n".join(lines) + "\n" return "\n".join(lines) + "\n"
@@ -127,11 +124,16 @@ class ExportService:
audio_path: Path, audio_path: Path,
metadata: Dict[str, Any], metadata: Dict[str, Any],
chapters: List[Dict[str, Any]], chapters: List[Dict[str, Any]],
cover: "CoverConfig | None" = None,
cover_path: Optional[Path] = None, cover_path: Optional[Path] = None,
cover_mime: Optional[str] = None, cover_mime: Optional[str] = None,
log_callback: Optional[callable] = None, log_callback: Optional[callable] = None,
) -> None: ) -> None:
"""Embed metadata and chapters into M4B file using FFmpeg + Mutagen.""" """Embed metadata and chapters into M4B file using FFmpeg + Mutagen."""
from abogen.domain.config_types import CoverConfig
if isinstance(cover, CoverConfig):
cover_path = cover.path
cover_mime = cover.mime
ffmetadata_path = self.write_ffmetadata_file(audio_path, metadata, chapters) ffmetadata_path = self.write_ffmetadata_file(audio_path, metadata, chapters)
metadata_args = self._metadata_to_ffmpeg_args(metadata) metadata_args = self._metadata_to_ffmpeg_args(metadata)
@@ -310,132 +312,7 @@ class ExportService:
cover_image_mime=cover_mime, cover_image_mime=cover_mime,
) )
# ----------------------------------------------------------------------
# Audiobookshelf Integration
# ----------------------------------------------------------------------
def build_audiobookshelf_metadata(self, job: Any) -> Dict[str, Any]:
"""Build Audiobookshelf metadata from job."""
filename = Path(getattr(job, "original_filename", "") or "").stem or "Audiobook"
return _build_abs_metadata(
getattr(job, "metadata_tags", {}),
language=getattr(job, "language", "") or "",
filename=filename,
)
def load_audiobookshelf_chapters(self, job: Any) -> Optional[List[Dict[str, Any]]]:
"""Load chapters from job artifacts for Audiobookshelf."""
metadata_ref = job.result.artifacts.get("metadata") if getattr(job, "result", None) else None
if not metadata_ref:
return None
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
return _load_abs_chapters(metadata_path)
def upload_audiobookshelf(
self,
job: Any,
audio_path: Path,
subtitle_paths: List[Path],
chapters: List[Dict[str, Any]],
metadata: Dict[str, Any],
cover_path: Optional[Path] = None,
config: Optional[AudiobookshelfConfig] = None,
log_callback: Optional[callable] = None,
) -> None:
"""Upload to Audiobookshelf."""
if config is None:
cfg = getattr(job, "_abs_config", None)
if cfg is None:
from abogen.utils import load_config
global_cfg = load_config() or {}
abs_cfg = global_cfg.get("audiobookshelf")
if isinstance(abs_cfg, Mapping):
config = AudiobookshelfConfig(
base_url=str(abs_cfg.get("base_url") or "").strip(),
api_token=str(abs_cfg.get("api_token") or "").strip(),
library_id=str(abs_cfg.get("library_id") or "").strip(),
collection_id=(str(abs_cfg.get("collection_id") or "").strip() or None),
folder_id=str(abs_cfg.get("folder_id") or "").strip(),
verify_ssl=self._coerce_bool(abs_cfg.get("verify_ssl"), True),
send_cover=self._coerce_bool(abs_cfg.get("send_cover"), True),
send_chapters=self._coerce_bool(abs_cfg.get("send_chapters"), True),
send_subtitles=self._coerce_bool(abs_cfg.get("send_subtitles"), False),
timeout=float(abs_cfg.get("timeout", 3600.0)),
)
else:
if log_callback:
log_callback("Audiobookshelf upload skipped: not configured", "warning")
return
if not config.base_url or not config.api_token or not config.library_id:
if log_callback:
log_callback("Audiobookshelf upload skipped: configure base URL, API token, and library ID first", "warning")
return
if not config.folder_id:
if log_callback:
log_callback("Audiobookshelf upload skipped: enter folder name or ID in settings", "warning")
return
if not audio_path.exists():
if log_callback:
log_callback("Audiobookshelf upload skipped: audio output not found", "warning")
return
existing_subtitles = [p for p in subtitle_paths if p.exists()] if config.send_subtitles else None
chapters_to_send = chapters if config.send_chapters else None
client = AudiobookshelfClient(config)
display_title = metadata.get("title") or audio_path.stem
try:
existing_items = client.find_existing_items(display_title, folder_id=config.folder_id)
except AudiobookshelfUploadError as exc:
if log_callback:
log_callback(f"Audiobookshelf lookup failed: {exc}", "error")
return
if existing_items:
if log_callback:
log_callback(f"Removing existing Audiobookshelf item(s) for '{display_title}' before upload.", "info")
try:
client.delete_items(existing_items)
except Exception as exc:
if log_callback:
log_callback(f"Failed to remove existing item(s): {exc}", "warning")
cover_to_send = cover_path
if config.send_cover and cover_to_send:
if isinstance(cover_to_send, str):
cover_to_send = Path(cover_to_send)
if not cover_to_send.exists():
cover_to_send = None
client.upload_audiobook(
audio_path,
metadata=metadata,
cover_path=cover_to_send,
chapters=chapters_to_send,
subtitles=existing_subtitles,
)
if log_callback:
log_callback("Audiobookshelf upload queued.", "info")
# ----------------------------------------------------------------------
# Helpers
# ----------------------------------------------------------------------
@staticmethod
def _coerce_bool(value: Any, default: bool = True) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
lowered = value.strip().lower()
if lowered in {"true", "1", "yes", "on"}:
return True
if lowered in {"false", "0", "no", "off"}:
return False
return default
if value is None: if value is None:
return default return default
return bool(value) return bool(value)
+36 -19
View File
@@ -278,24 +278,28 @@ def create_subtitle_writer(
def resolve_subtitle_format( def resolve_subtitle_format(
subtitle_format: str | None, subtitle: "SubtitleConfig | str | None",
subtitle_mode: str, subtitle_mode: str | None = None,
) -> tuple[str, str]: ) -> tuple[str, str]:
"""Resolve a subtitle_format setting string to (file_extension, alignment). """Resolve a subtitle config to (file_extension, alignment).
Handles the PyQt convention where format strings encode alignment Accepts a SubtitleConfig object or individual format/mode strings
(e.g. ``"ass_centered_narrow"`` extension ``"ass"``, alignment for backward compatibility.
``"center_narrow"``).
Also enforces that ``"Sentence + Highlighting"`` mode requires ASS.
Returns: Returns:
Tuple of (file_extension, alignment) suitable for Tuple of (file_extension, alignment) suitable for
:func:`create_subtitle_writer`. :func:`create_subtitle_writer`.
""" """
fmt = (subtitle_format or "srt").lower() from abogen.domain.config_types import SubtitleConfig
if subtitle_mode == "Sentence + Highlighting" and fmt == "srt": if isinstance(subtitle, SubtitleConfig):
fmt = subtitle.format.value.lower()
mode_str = subtitle.mode.value
else:
fmt = (subtitle or "srt").lower()
mode_str = subtitle_mode or "Disabled"
if mode_str == "Sentence + Highlighting" and fmt == "srt":
fmt = "ass" fmt = "ass"
if "ass" in fmt: if "ass" in fmt:
@@ -317,26 +321,39 @@ def resolve_subtitle_format(
def make_subtitle_writer( def make_subtitle_writer(
audio_path: Path, audio_path: Path,
subtitle_format: str | None, subtitle: "SubtitleConfig | str | None",
subtitle_mode: str, subtitle_mode: str | None = None,
max_words: int = 50, max_words: int | None = None,
) -> SubtitleWriter | None: ) -> SubtitleWriter | None:
"""Convenience: resolve format and create a writer, or return None if disabled. """Convenience: resolve format and create a writer, or return None if disabled.
Returns ``None`` when ``subtitle_mode`` is ``"Disabled"`` or the Accepts a SubtitleConfig object or individual format/mode strings
for backward compatibility.
Returns ``None`` when subtitle mode is ``"Disabled"`` or the
format is unsupported. format is unsupported.
""" """
if subtitle_mode == "Disabled": from abogen.domain.config_types import SubtitleConfig
return None
extension, alignment = resolve_subtitle_format(subtitle_format, subtitle_mode) if isinstance(subtitle, SubtitleConfig):
mode_str = subtitle.mode.value
if mode_str == "Disabled":
return None
words = subtitle.max_words
else:
mode_str = subtitle_mode or subtitle or "Disabled"
if mode_str == "Disabled":
return None
words = max_words or 50
extension, alignment = resolve_subtitle_format(subtitle, subtitle_mode)
try: try:
return create_subtitle_writer( return create_subtitle_writer(
audio_path.with_suffix(f".{extension}"), audio_path.with_suffix(f".{extension}"),
extension, extension,
subtitle_mode, mode_str,
alignment=alignment, alignment=alignment,
max_words=max_words, max_words=words,
) )
except (ValueError, KeyError): except (ValueError, KeyError):
return None return None
+2 -4
View File
@@ -36,10 +36,8 @@ from abogen.domain.metadata_extraction import (
format_metadata_tags, format_metadata_tags,
) )
from abogen.subtitle_utils import ( from abogen.subtitle_utils import clean_text
clean_text, from abogen.domain.text_utils import calculate_text_length
calculate_text_length,
)
import os import os
import logging import logging
+31 -62
View File
@@ -1,5 +1,4 @@
import os import os
import re
import time import time
import hashlib # For generating unique cache filenames import hashlib # For generating unique cache filenames
from pathlib import Path from pathlib import Path
@@ -10,7 +9,6 @@ from contextlib import ExitStack, contextmanager
import numpy as np import numpy as np
import soundfile as sf import soundfile as sf
from abogen.utils import ( from abogen.utils import (
create_process,
get_user_cache_path, get_user_cache_path,
detect_encoding, detect_encoding,
) )
@@ -30,40 +28,26 @@ from abogen.domain.subtitle_processor import (
) )
from abogen.domain.output_paths import ( from abogen.domain.output_paths import (
resolve_output_directory, resolve_output_directory,
build_output_path,
sanitize_output_stem,
sanitize_filename_for_chapter, sanitize_filename_for_chapter,
resolve_unique_path, resolve_unique_path,
) )
from abogen.domain.audio_helpers import build_ffmpeg_command, to_float32 from abogen.domain.audio_helpers import build_ffmpeg_command, to_float32
from abogen.domain.audio_sink import AudioSink, open_audio_sink from abogen.domain.audio_sink import open_audio_sink
from abogen.domain.conversion_engine import run_tts_segment_loop, synthesize_text, SynthParams, SegmentStats, SegmentInfo from abogen.domain.conversion_engine import run_tts_segment_loop, synthesize_text, SynthParams, SegmentStats, SegmentInfo
from abogen.domain.intro_outro import resolve_intro, resolve_outro from abogen.domain.intro_outro import resolve_intro, resolve_outro
from abogen.domain.audio_buffer import ( from abogen.domain.audio_buffer import (
create_silence, create_silence,
mix_audio,
normalize_audio,
SAMPLE_RATE,
) )
from abogen.domain.subtitle_generation import process_subtitle_tokens from abogen.domain.subtitle_generation import process_subtitle_tokens
from abogen.domain.voice_loader import VoiceCache, load_voice_cached, resolve_voice from abogen.domain.voice_loader import VoiceCache, load_voice_cached, resolve_voice
from abogen.domain.progress import calc_etr_str
from abogen.domain.normalization import TTSContext
from abogen.domain.pronunciation import (
compile_pronunciation_rules,
compile_heteronym_sentence_rules,
merge_pronunciation_overrides,
)
from abogen.domain.metadata_extraction import ( from abogen.domain.metadata_extraction import (
extract_metadata_and_build_args, extract_metadata_from_text,
extract_metadata_for_file,
) )
from abogen.domain.text_chapters import parse_chapters_from_text from abogen.domain.text_chapters import parse_chapters_from_text
from abogen.infrastructure.exporters import ExportService from abogen.infrastructure.exporters import ExportService
import abogen.hf_tracker as hf_tracker import abogen.hf_tracker as hf_tracker
import static_ffmpeg import static_ffmpeg
import threading # for efficient waiting import threading # for efficient waiting
import subprocess
@@ -79,6 +63,7 @@ from abogen.subtitle_utils import (
sanitize_name_for_os, sanitize_name_for_os,
split_text_by_voice_markers split_text_by_voice_markers
) )
from abogen.domain.split_pattern import PUNCTUATION_COMMAS
class CountdownDialog(QDialog): class CountdownDialog(QDialog):
"""Base dialog with auto-accept countdown functionality""" """Base dialog with auto-accept countdown functionality"""
@@ -242,11 +227,6 @@ class ConversionThread(QThread):
log_updated = pyqtSignal(object) # Updated signal for log updates log_updated = pyqtSignal(object) # Updated signal for log updates
chapters_detected = pyqtSignal(int) # Signal for chapter detection chapters_detected = pyqtSignal(int) # Signal for chapter detection
# Punctuation constants for unified handling across languages
PUNCTUATION_SENTENCE = ".!?।。!?"
PUNCTUATION_SENTENCE_COMMA = ".!?,।。!?、,"
PUNCTUATION_COMMAS = ",,、"
def __init__( def __init__(
self, self,
file_name, file_name,
@@ -525,6 +505,9 @@ class ConversionThread(QThread):
) as file: ) as file:
text = file.read() text = file.read()
# Extract metadata BEFORE clean_text strips the tags
self._extracted_metadata = extract_metadata_from_text(text)
# Clean up text using utility function # Clean up text using utility function
text = clean_text(text) text = clean_text(text)
@@ -550,22 +533,18 @@ class ConversionThread(QThread):
) )
# --- Compile normalization rules (heteronym + pronunciation) --- # --- Compile normalization rules (heteronym + pronunciation) ---
from abogen.domain.normalization import TTSContext from abogen.domain.config_types import PronunciationConfig
from abogen.domain.normalization import build_tts_context
class _MergeJob: self._tts_context = build_tts_context(
pronunciation_overrides = getattr(self, "pronunciation_overrides", None) language=self.lang_code,
manual_overrides = getattr(self, "manual_overrides", None) subtitle=self.subtitle_mode,
heteronym_overrides = getattr(self, "heteronym_overrides", None) pronunciation=PronunciationConfig(
language = self.lang_code pronunciation_overrides=getattr(self, "pronunciation_overrides", None) or [],
manual_overrides=getattr(self, "manual_overrides", None) or [],
pronunciation_overrides = merge_pronunciation_overrides(_MergeJob()) heteronym_overrides=getattr(self, "heteronym_overrides", None) or [],
self._tts_context = TTSContext( normalization_overrides=getattr(self, "normalization_overrides", None),
split_pattern=self.split_pattern,
pronunciation_rules=compile_pronunciation_rules(pronunciation_overrides),
heteronym_rules=compile_heteronym_sentence_rules(
getattr(self, "heteronym_overrides", None)
), ),
normalization_overrides=getattr(self, "normalization_overrides", None), log_callback=lambda level, msg: self.log_updated.emit((msg, "grey" if level == "info" else "orange")),
) )
# --- Chapter splitting logic --- # --- Chapter splitting logic ---
@@ -764,7 +743,7 @@ class ConversionThread(QThread):
intro_emitted = False intro_emitted = False
if merge_chapters_at_end: if merge_chapters_at_end:
intro_spec = resolve_intro( intro_spec = resolve_intro(
extract_metadata_for_file(self.file_name, self.is_direct_text), self._extracted_metadata,
os.path.basename(self.file_name) if self.file_name else "", os.path.basename(self.file_name) if self.file_name else "",
getattr(self, "read_title_intro", False), getattr(self, "read_title_intro", False),
self.voice, self.voice, list(self.voice_cache._cache.keys()), self.voice, self.voice, list(self.voice_cache._cache.keys()),
@@ -937,7 +916,7 @@ class ConversionThread(QThread):
# For Sentence + Comma mode, still split on commas within spaCy sentences # For Sentence + Comma mode, still split on commas within spaCy sentences
if self.subtitle_mode == "Sentence + Comma": if self.subtitle_mode == "Sentence + Comma":
active_split_pattern = r"(?<=[{}]){}|\n+".format( active_split_pattern = r"(?<=[{}]){}|\n+".format(
self.PUNCTUATION_COMMAS, spacing_pattern PUNCTUATION_COMMAS, spacing_pattern
) )
else: else:
active_split_pattern = ( active_split_pattern = (
@@ -1100,7 +1079,7 @@ class ConversionThread(QThread):
# --- Outro synthesis --- # --- Outro synthesis ---
if merge_chapters_at_end: if merge_chapters_at_end:
outro_spec = resolve_outro( outro_spec = resolve_outro(
extract_metadata_for_file(self.file_name, self.is_direct_text), self._extracted_metadata,
os.path.basename(self.file_name) if self.file_name else "", os.path.basename(self.file_name) if self.file_name else "",
getattr(self, "read_closing_outro", True), getattr(self, "read_closing_outro", True),
self.voice, self.voice, list(self.voice_cache._cache.keys()), self.voice, self.voice, list(self.voice_cache._cache.keys()),
@@ -1141,12 +1120,7 @@ class ConversionThread(QThread):
# Add chapters via ExportService (unified with WebUI) # Add chapters via ExportService (unified with WebUI)
if total_chapters > 1: if total_chapters > 1:
export_svc = ExportService() export_svc = ExportService()
metadata_text = read_text_for_metadata( metadata = dict(getattr(self, "_extracted_metadata", {}))
file_path=self.file_name,
is_direct_text=self.is_direct_text,
direct_text=self.file_name if self.is_direct_text else None,
)
metadata = extract_metadata_from_text(metadata_text) if metadata_text else {}
# Convert cover_path from metadata to Path if present # Convert cover_path from metadata to Path if present
cover_path_raw = metadata.pop("cover_path", None) cover_path_raw = metadata.pop("cover_path", None)
cover_path = Path(cover_path_raw) if cover_path_raw and os.path.exists(cover_path_raw) else None cover_path = Path(cover_path_raw) if cover_path_raw and os.path.exists(cover_path_raw) else None
@@ -1385,33 +1359,28 @@ class ConversionThread(QThread):
raise ValueError(f"Unsupported output format: {self.output_format}") raise ValueError(f"Unsupported output format: {self.output_format}")
def _extract_and_add_metadata_tags_to_ffmpeg_cmd(self): def _extract_and_add_metadata_tags_to_ffmpeg_cmd(self):
"""Extract metadata tags from text content and add them to ffmpeg command""" """Build ffmpeg metadata args from previously extracted metadata."""
# Read text for metadata extraction metadata = getattr(self, "_extracted_metadata", None)
text = read_text_for_metadata( if not metadata or not any(metadata.values()):
file_path=self.file_name,
is_direct_text=self.is_direct_text,
direct_text=self.file_name if self.is_direct_text else None,
)
if not text:
self.log_updated.emit( self.log_updated.emit(
("Warning: Could not read file for metadata extraction", "orange") ("Warning: No metadata tags found in text", "orange")
) )
return [], None return [], None
# Extract metadata and build ffmpeg args
filename = self.file_name if self.is_direct_text else ( filename = self.file_name if self.is_direct_text else (
self.display_path if self.display_path else self.file_name self.display_path if self.display_path else self.file_name
) )
try: try:
metadata_options, cover_path = extract_metadata_and_build_args( from abogen.domain.metadata_extraction import build_ffmpeg_metadata_args, get_filename_from_path
text=text, actual_filename = get_filename_from_path(
filename=filename, file_path=filename,
display_path=getattr(self, "display_path", None), display_path=getattr(self, "display_path", None),
from_queue=getattr(self, "from_queue", False), from_queue=getattr(self, "from_queue", False),
) )
return metadata_options, cover_path args = build_ffmpeg_metadata_args(metadata, actual_filename)
cover_path = metadata.get("cover_path")
return args, cover_path
except Exception as e: except Exception as e:
self.log_updated.emit( self.log_updated.emit(
(f"Warning: Metadata extraction error: {e}", "orange") (f"Warning: Metadata extraction error: {e}", "orange")
+30 -13
View File
@@ -17,6 +17,12 @@ import os
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Dict, List, Optional from typing import Any, Callable, Dict, List, Optional
from abogen.application.conversion_config import (
ChapterChunkConfig,
Epub3ExportConfig,
PronunciationConfig,
WordSubstitutionConfig,
)
from abogen.application.conversion_request import ConversionRequest from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_ports import ConversionCancelled, ResolvedVoice from abogen.application.conversion_ports import ConversionCancelled, ResolvedVoice
@@ -54,6 +60,25 @@ def build_conversion_request_from_thread(thread: Any) -> ConversionRequest:
if thread.output_folder: if thread.output_folder:
output_folder = Path(thread.output_folder) output_folder = Path(thread.output_folder)
# Build pronunciation config
pronunciation = None
pron_overrides = getattr(thread, "pronunciation_overrides", []) or []
manual_overrides = getattr(thread, "manual_overrides", []) or []
heteronym_overrides = getattr(thread, "heteronym_overrides", []) or []
norm_overrides = getattr(thread, "normalization_overrides", None)
if pron_overrides or manual_overrides or heteronym_overrides or norm_overrides:
pronunciation = PronunciationConfig(
pronunciation_overrides=pron_overrides,
manual_overrides=manual_overrides,
heteronym_overrides=heteronym_overrides,
normalization_overrides=norm_overrides,
)
# Build epub3 config
epub3_export = None
if getattr(thread, "generate_epub3", False):
epub3_export = Epub3ExportConfig()
return ConversionRequest( return ConversionRequest(
# Source # Source
source_path=source_path, source_path=source_path,
@@ -87,24 +112,16 @@ def build_conversion_request_from_thread(thread: Any) -> ConversionRequest:
read_title_intro=getattr(thread, "read_title_intro", False), read_title_intro=getattr(thread, "read_title_intro", False),
read_closing_outro=getattr(thread, "read_closing_outro", True), read_closing_outro=getattr(thread, "read_closing_outro", True),
auto_prefix_chapter_titles=getattr(thread, "auto_prefix_chapter_titles", True), auto_prefix_chapter_titles=getattr(thread, "auto_prefix_chapter_titles", True),
normalize_chapter_opening_caps=getattr(thread, "normalize_chapter_opening_caps", False), normalize_chapter_opening_caps=thread.normalize_chapter_opening_caps,
# Pronunciation / Normalization
pronunciation_overrides=getattr(thread, "pronunciation_overrides", []) or [],
manual_overrides=getattr(thread, "manual_overrides", []) or [],
heteronym_overrides=getattr(thread, "heteronym_overrides", []) or [],
normalization_overrides=getattr(thread, "normalization_overrides", None),
# Chapter/Chunk Configuration
chapter_overrides=[], # PyQt doesn't use chapter overrides from GUI
chunks=[], # PyQt doesn't use chunks from GUI
chunk_level="paragraph",
speaker_mode="single",
speakers={},
# Metadata # Metadata
metadata_tags=getattr(thread, "metadata_tags", {}) or {}, metadata_tags=getattr(thread, "metadata_tags", {}) or {},
# Artifacts # Artifacts
cover_image_path=getattr(thread, "cover_image_path", None), cover_image_path=getattr(thread, "cover_image_path", None),
cover_image_mime=getattr(thread, "cover_image_mime", None), cover_image_mime=getattr(thread, "cover_image_mime", None),
generate_epub3=getattr(thread, "generate_epub3", False), # Feature configs
pronunciation=pronunciation,
epub3_export=epub3_export,
chapter_chunk=ChapterChunkConfig(), # PyQt doesn't use chapter overrides from GUI
) )
+2 -4
View File
@@ -70,10 +70,8 @@ from abogen.utils import (
LoadPipelineThread, LoadPipelineThread,
) )
from abogen.subtitle_utils import ( from abogen.subtitle_utils import clean_text
clean_text, from abogen.domain.text_utils import calculate_text_length
calculate_text_length,
)
from abogen.pyqt.conversion import ConversionThread, VoicePreviewThread, PlayAudioThread, ChapterOptionsDialog, TimestampDetectionDialog from abogen.pyqt.conversion import ConversionThread, VoicePreviewThread, PlayAudioThread, ChapterOptionsDialog, TimestampDetectionDialog
from abogen.pyqt.book_handler import HandlerDialog from abogen.pyqt.book_handler import HandlerDialog
+1 -1
View File
@@ -523,7 +523,7 @@ class QueueManager(QDialog):
return attrs return attrs
def add_files_from_paths(self, file_paths): def add_files_from_paths(self, file_paths):
from abogen.subtitle_utils import calculate_text_length from abogen.domain.text_utils import calculate_text_length
from PyQt6.QtWidgets import QMessageBox from PyQt6.QtWidgets import QMessageBox
import os import os
+36 -52
View File
@@ -1,8 +1,19 @@
"""Graceful shutdown - single module, no over-engineering.""" """Graceful shutdown — process-level hooks and orchestration.
Responsibilities:
- Install atexit/signal/Qt hooks
- Stop WebUI ConversionService (worker thread)
- Restore sleep prevention
- Terminate child processes (ffmpeg, etc.)
- Delegate GPU/engine/UI cleanup to application.cleanup
App-layer cleanup (GPU, engines, UI callbacks) lives in application/cleanup.py.
Per-conversion cleanup lives in run_conversion() finally block.
"""
from __future__ import annotations from __future__ import annotations
import atexit import atexit
import gc
import signal import signal
import sys import sys
from typing import Callable from typing import Callable
@@ -28,20 +39,11 @@ def _run_cleanups() -> None:
pass pass
# ---- Register built-in cleanup functions ---- # ---- Process-level cleanup functions ----
# 1. Restore sleep prevention
def _restore_sleep() -> None:
try:
from abogen.utils import prevent_sleep_end
prevent_sleep_end()
except Exception:
pass
register_cleanup(_restore_sleep) def _stop_conversion_service() -> None:
"""Stop WebUI ConversionService worker thread."""
# 2. Shutdown web UI ConversionService
def _shutdown_conversion_service() -> None:
try: try:
from abogen.webui.service import get_service from abogen.webui.service import get_service
svc = get_service() svc = get_service()
@@ -50,50 +52,18 @@ def _shutdown_conversion_service() -> None:
except Exception: except Exception:
pass pass
register_cleanup(_shutdown_conversion_service)
# 3. Clear TTS pipelines and GPU memory def _restore_sleep() -> None:
def _cleanup_tts_pipelines() -> None: """Restore system sleep prevention (caffeinate/systemd-inhibit/Windows)."""
# Clear web UI pipeline cache
try: try:
from abogen.webui.conversion_runner import _PIPELINES from abogen.utils import prevent_sleep_end
_PIPELINES.clear() prevent_sleep_end()
except Exception: except Exception:
pass pass
# Clear PyQt conversion thread voice cache
try:
from abogen.pyqt.conversion import ConversionThread
if hasattr(ConversionThread, "voice_cache"):
ConversionThread.voice_cache.clear()
except Exception:
pass
gc.collect()
# Release CUDA cache
try:
import torch
if torch.cuda.is_available():
torch.cuda.empty_cache()
torch.cuda.ipc_collect()
except Exception:
pass
register_cleanup(_cleanup_tts_pipelines)
# 4. Clear global voice cache
def _clear_voice_cache() -> None:
try:
from abogen.voice_cache import clear_voice_cache
clear_voice_cache()
except Exception:
pass
register_cleanup(_clear_voice_cache)
# 5. Terminate child processes (ffmpeg, etc.)
def _terminate_subprocesses() -> None: def _terminate_subprocesses() -> None:
"""Terminate all child processes (ffmpeg, etc.)."""
try: try:
import psutil import psutil
except Exception: except Exception:
@@ -115,6 +85,20 @@ def _terminate_subprocesses() -> None:
except Exception: except Exception:
pass pass
def _app_cleanup() -> None:
"""Delegate to application-layer cleanup (engines, GPU, UI callbacks)."""
try:
from abogen.application.cleanup import cleanup
cleanup()
except Exception:
pass
# Register in execution order
register_cleanup(_stop_conversion_service)
register_cleanup(_app_cleanup)
register_cleanup(_restore_sleep)
register_cleanup(_terminate_subprocesses) register_cleanup(_terminate_subprocesses)
@@ -133,7 +117,7 @@ def register_shutdown() -> None:
except Exception: except Exception:
pass pass
# Qt hook # Qt hook — connect AFTER QApplication is created
try: try:
from PyQt6.QtWidgets import QApplication from PyQt6.QtWidgets import QApplication
+14 -40
View File
@@ -21,20 +21,6 @@ SPACY_MODELS = {
Language.HI: "xx_sent_ud_sm", Language.HI: "xx_sent_ud_sm",
} }
# Kokoro single-letter codes -> Language enum (inverse of pipeline_factory._KOKORO_LANG_MAP)
_KOKORO_TO_LANGUAGE = {
"a": Language.EN_US,
"b": Language.EN_GB,
"e": Language.ES,
"f": Language.FR,
"h": Language.HI,
"i": Language.IT,
"j": Language.JA,
"p": Language.PT_BR,
"z": Language.ZH,
}
def _load_spacy(): def _load_spacy():
"""Lazy load spaCy module.""" """Lazy load spaCy module."""
global _spacy global _spacy
@@ -48,12 +34,12 @@ def _load_spacy():
return _spacy return _spacy
def get_spacy_model(lang_code, log_callback=None): def get_spacy_model(language: Language, log_callback=None):
""" """
Get or load a spaCy model for the given language code. Get or load a spaCy model for the given language.
Args: Args:
lang_code: Language code or Language enum (e.g., "a", "en-US", Language.EN_US) language: Language enum value.
log_callback: Optional function to log messages log_callback: Optional function to log messages
Returns: Returns:
@@ -61,36 +47,24 @@ def get_spacy_model(lang_code, log_callback=None):
""" """
def log(msg, is_error=False): def log(msg, is_error=False):
# Prefer GUI log callback when provided to avoid spamming stdout.
if log_callback: if log_callback:
color = "red" if is_error else "grey" color = "red" if is_error else "grey"
try: try:
log_callback((msg, color)) log_callback((msg, color))
except Exception: except Exception:
# Fallback to printing if callback misbehaves
print(msg) print(msg)
else: else:
print(msg) print(msg)
# Normalize to Language enum if not isinstance(language, Language):
if not isinstance(lang_code, Language): raise TypeError(f"language must be Language enum, got {type(language).__name__}: {language!r}")
if isinstance(lang_code, str) and lang_code in _KOKORO_TO_LANGUAGE:
lang_code = _KOKORO_TO_LANGUAGE[lang_code]
else:
try:
lang_code = Language.from_str(lang_code)
except ValueError:
log(f"\nspaCy: Unknown language '{lang_code}'...")
return None
# Check if model is cached if language in _nlp_cache:
if lang_code in _nlp_cache: return _nlp_cache[language]
return _nlp_cache[lang_code]
# Check if language is supported model_name = SPACY_MODELS.get(language)
model_name = SPACY_MODELS.get(lang_code)
if not model_name: if not model_name:
log(f"\nspaCy: No model mapping for language '{lang_code}'...") log(f"\nspaCy: No model mapping for language '{language}'...")
return None return None
# Lazy load spaCy # Lazy load spaCy
@@ -114,7 +88,7 @@ def get_spacy_model(lang_code, log_callback=None):
if "parser" not in nlp.pipe_names and "sentencizer" not in nlp.pipe_names: if "parser" not in nlp.pipe_names and "sentencizer" not in nlp.pipe_names:
nlp.add_pipe("sentencizer") nlp.add_pipe("sentencizer")
_nlp_cache[lang_code] = nlp _nlp_cache[language] = nlp
return nlp return nlp
except OSError: except OSError:
# Model not found, attempt download # Model not found, attempt download
@@ -131,7 +105,7 @@ def get_spacy_model(lang_code, log_callback=None):
if "parser" not in nlp.pipe_names and "sentencizer" not in nlp.pipe_names: if "parser" not in nlp.pipe_names and "sentencizer" not in nlp.pipe_names:
nlp.add_pipe("sentencizer") nlp.add_pipe("sentencizer")
_nlp_cache[lang_code] = nlp _nlp_cache[language] = nlp
log(f"spaCy model '{model_name}' downloaded and loaded") log(f"spaCy model '{model_name}' downloaded and loaded")
return nlp return nlp
except Exception as e: except Exception as e:
@@ -145,19 +119,19 @@ def get_spacy_model(lang_code, log_callback=None):
return None return None
def segment_sentences(text, lang_code, log_callback=None): def segment_sentences(text, language: Language, log_callback=None):
""" """
Segment text into sentences using spaCy. Segment text into sentences using spaCy.
Args: Args:
text: Text to segment text: Text to segment
lang_code: Language code language: Language enum value
log_callback: Optional function to log messages log_callback: Optional function to log messages
Returns: Returns:
List of sentence strings, or None if spaCy unavailable List of sentence strings, or None if spaCy unavailable
""" """
nlp = get_spacy_model(lang_code, log_callback) nlp = get_spacy_model(language, log_callback)
if nlp is None: if nlp is None:
return None return None
+2 -2
View File
@@ -4,7 +4,7 @@ import json
import os import os
from typing import Any, Dict, List, Optional from typing import Any, Dict, List, Optional
from abogen.constants import LANGUAGE_DESCRIPTIONS from abogen.constants import KOKORO_CODE_LABELS
from abogen.utils import get_user_config_path from abogen.utils import get_user_config_path
_CONFIG_WRAPPER_KEY = "abogen_speaker_configs" _CONFIG_WRAPPER_KEY = "abogen_speaker_configs"
@@ -163,4 +163,4 @@ def list_configs() -> List[Dict[str, Any]]:
def describe_language(code: str) -> str: def describe_language(code: str) -> str:
code = (code or "a").lower() code = (code or "a").lower()
return LANGUAGE_DESCRIPTIONS.get(code, code.upper()) return KOKORO_CODE_LABELS.get(code, code.upper())
+19 -199
View File
@@ -1,7 +1,7 @@
import re import re
import platform
from abogen.utils import detect_encoding, load_config from abogen.utils import detect_encoding, load_config
from abogen.constants import SAMPLE_VOICE_TEXTS from abogen.constants import SAMPLE_VOICE_TEXTS
from abogen.domain.enums import Language
# Pre-compile frequently used regex patterns for better performance # Pre-compile frequently used regex patterns for better performance
_METADATA_TAG_PATTERN = re.compile(r"<<METADATA_[^:]+:[^>]*>>") _METADATA_TAG_PATTERN = re.compile(r"<<METADATA_[^:]+:[^>]*>>")
@@ -23,13 +23,6 @@ _VTT_NOTE_PATTERN = re.compile(r"NOTE\s*\n.*?(?=\n\n|$)", re.DOTALL)
_DOUBLE_NEWLINE_SPLIT_PATTERN = re.compile(r"\n\s*\n") _DOUBLE_NEWLINE_SPLIT_PATTERN = re.compile(r"\n\s*\n")
_VTT_TIMESTAMP_PATTERN = re.compile(r"([\d:.]+)\s*-->\s*([\d:.]+)") _VTT_TIMESTAMP_PATTERN = re.compile(r"([\d:.]+)\s*-->\s*([\d:.]+)")
_TIMESTAMP_ONLY_PATTERN = re.compile(r"^(\d{1,2}:\d{2}:\d{2}(?:[.,]\d{1,3})?)$") _TIMESTAMP_ONLY_PATTERN = re.compile(r"^(\d{1,2}:\d{2}:\d{2}(?:[.,]\d{1,3})?)$")
_WINDOWS_ILLEGAL_CHARS_PATTERN = re.compile(r'[<>:"/\\|?*]')
_CONTROL_CHARS_PATTERN = re.compile(r"[\x00-\x1f]")
_LINUX_CONTROL_CHARS_PATTERN = re.compile(
r"[\x01-\x1f]"
) # Linux: exclude \x00 for separate handling
_MACOS_ILLEGAL_CHARS_PATTERN = re.compile(r"[:]")
_LINUX_ILLEGAL_CHARS_PATTERN = re.compile(r"[/\x00]")
def clean_subtitle_text(text): def clean_subtitle_text(text):
@@ -41,17 +34,6 @@ def clean_subtitle_text(text):
return text.strip() return text.strip()
def calculate_text_length(text):
# Use pre-compiled patterns for better performance
# Ignore chapter markers, voice markers, and metadata patterns in a single pass
text = _CHAPTER_MARKER_PATTERN.sub("", text)
text = _VOICE_MARKER_PATTERN.sub("", text)
text = _METADATA_TAG_PATTERN.sub("", text)
# Ignore newlines and leading/trailing spaces
text = text.replace("\n", "").strip()
# Calculate character count
char_count = len(text)
return char_count
def clean_text(text, *args, **kwargs): def clean_text(text, *args, **kwargs):
@@ -396,189 +378,27 @@ def parse_ass_file(file_path):
return subtitles return subtitles
def get_sample_voice_text(lang_code): def get_sample_voice_text(language):
return SAMPLE_VOICE_TEXTS.get(lang_code, SAMPLE_VOICE_TEXTS["a"]) """Get sample voice text for a language.
def sanitize_name_for_os(name, is_folder=True):
"""
Sanitize a filename or folder name based on the operating system.
Args: Args:
name: The name to sanitize language: Language enum value or string (for backward compatibility).
is_folder: Whether this is a folder name (default: True)
Returns:
Sanitized name safe for the current OS
""" """
if not name: if isinstance(language, str):
return "audiobook" try:
language = Language.from_str(language)
system = platform.system() except (ValueError, AttributeError):
language = Language.EN_US
if system == "Windows": return SAMPLE_VOICE_TEXTS.get(language, SAMPLE_VOICE_TEXTS[Language.EN_US])
# Windows illegal characters: < > : " / \ | ? *
# Also can't end with space or dot
# Use pre-compiled pattern for better performance
sanitized = _WINDOWS_ILLEGAL_CHARS_PATTERN.sub("_", name)
# Remove control characters (0-31)
sanitized = _CONTROL_CHARS_PATTERN.sub("_", sanitized)
# Remove trailing spaces and dots
sanitized = sanitized.rstrip(". ")
# Windows reserved names (CON, PRN, AUX, NUL, COM1-9, LPT1-9)
reserved = (
["CON", "PRN", "AUX", "NUL"]
+ [f"COM{i}" for i in range(1, 10)]
+ [f"LPT{i}" for i in range(1, 10)]
)
if sanitized.upper() in reserved or sanitized.upper().split(".")[0] in reserved:
sanitized = f"_{sanitized}"
elif system == "Darwin": # macOS
# macOS illegal characters: : (colon is converted to / by the system)
# Also can't start with dot (hidden file) for folders typically
# Use pre-compiled pattern for better performance
sanitized = _MACOS_ILLEGAL_CHARS_PATTERN.sub("_", name)
# Remove control characters
sanitized = _CONTROL_CHARS_PATTERN.sub("_", sanitized)
# Avoid leading dot for folders (creates hidden folders)
if is_folder and sanitized.startswith("."):
sanitized = "_" + sanitized[1:]
else: # Linux and others
# Linux illegal characters: / and null character
# Though / is illegal, most other chars are technically allowed
# Use pre-compiled pattern for better performance
sanitized = _LINUX_ILLEGAL_CHARS_PATTERN.sub("_", name)
# Remove other control characters for safety (excluding \x00 which is already handled)
sanitized = _LINUX_CONTROL_CHARS_PATTERN.sub("_", sanitized)
# Avoid leading dot for folders (creates hidden folders)
if is_folder and sanitized.startswith("."):
sanitized = "_" + sanitized[1:]
# Ensure the name is not empty after sanitization
if not sanitized or sanitized.strip() == "":
sanitized = "audiobook"
# Limit length to 255 characters (common limit across filesystems)
if len(sanitized) > 255:
sanitized = sanitized[:255].rstrip(". ")
return sanitized
def validate_voice_name(voice_name): # Backward-compatible re-exports — canonical location is domain/output_paths.py
"""Validate voice name against available voices (case-insensitive). from abogen.domain.output_paths import sanitize_name_for_os # noqa: E402, F401
Handles both single voices and formulas like 'af_heart*0.5 + am_echo*0.5'.
Args: # Backward-compatible re-exports — canonical location is domain/voice_markers.py
voice_name: Voice name or formula string to validate from abogen.domain.voice_markers import ( # noqa: E402, F401
validate_voice_name,
Returns: split_text_by_voice_markers,
Tuple of (is_valid, invalid_voice_name): _VOICE_MARKER_PATTERN,
- is_valid: True if all voices in the name/formula are valid _VOICE_MARKER_SEARCH_PATTERN,
- invalid_voice_name: The first invalid voice found, or None if all valid )
"""
from abogen.tts_plugin.utils import get_voices
# Create case-insensitive lookup set (done once per call)
voice_lookup_lower = {v.lower() for v in get_voices("kokoro")}
voice_name = voice_name.strip()
# Check if it's a formula (contains *)
if "*" in voice_name:
# Extract voice names from formula
voices = voice_name.split("+")
for term in voices:
if "*" in term:
base_voice = term.split("*")[0].strip()
# Case-insensitive comparison
if base_voice.lower() not in voice_lookup_lower:
return False, base_voice
return True, None
else:
# Single voice - case-insensitive comparison
if voice_name.lower() not in voice_lookup_lower:
return False, voice_name
return True, None
def split_text_by_voice_markers(text, default_voice):
"""Split text by voice markers, returning list of (voice, text) tuples.
IMPORTANT: Returns the last voice used so it can persist across chapters.
Voice names are normalized to lowercase to match canonical voice names.
Args:
text: Text potentially containing <<VOICE:name>> markers
default_voice: Voice to use if no markers found or before first marker
Returns:
Tuple of (segments_list, last_voice_used, valid_count, invalid_count):
- segments_list: List of (voice_name, segment_text) tuples
- last_voice_used: The voice that should continue into next chapter
- valid_count: Number of valid voice markers processed
- invalid_count: Number of invalid voice markers skipped
"""
from abogen.tts_plugin.utils import get_voices
voice_splits = list(_VOICE_MARKER_SEARCH_PATTERN.finditer(text))
if not voice_splits:
# No voice markers, return entire text with default voice
return [(default_voice, text)], default_voice, 0, 0
segments = []
current_voice = default_voice
valid_markers = 0
invalid_markers = 0
# Text before first marker uses default voice
first_start = voice_splits[0].start()
if first_start > 0:
intro_text = text[:first_start].strip()
if intro_text:
segments.append((current_voice, intro_text))
# Process each voice marker
for idx, match in enumerate(voice_splits):
voice_name = match.group(1).strip()
start = match.end()
end = voice_splits[idx + 1].start() if idx + 1 < len(voice_splits) else len(text)
segment_text = text[start:end].strip()
# Validate voice name
is_valid, invalid_voice = validate_voice_name(voice_name)
if is_valid:
# Normalize to lowercase to match canonical form
# Handle both single voices and formulas
if "*" in voice_name:
# Normalize each voice in the formula
normalized_parts = []
for part in voice_name.split("+"):
part = part.strip()
if "*" in part:
voice_part, weight = part.split("*", 1)
# Find the canonical (lowercase) voice name
voice_part_lower = voice_part.strip().lower()
canonical_voice = next(
(v for v in get_voices("kokoro") if v.lower() == voice_part_lower),
voice_part.strip()
)
normalized_parts.append(f"{canonical_voice}*{weight.strip()}")
current_voice = " + ".join(normalized_parts)
else:
# Find the canonical (lowercase) voice name
voice_name_lower = voice_name.lower()
current_voice = next(
(v for v in get_voices("kokoro") if v.lower() == voice_name_lower),
voice_name
)
valid_markers += 1
else:
# Invalid voice - stay with previous voice
invalid_markers += 1
if segment_text:
segments.append((current_voice, segment_text))
# Return segments, last voice, and counts
return segments, current_voice, valid_markers, invalid_markers
+2 -1
View File
@@ -16,7 +16,8 @@ import markdown # type: ignore[import]
from bs4 import BeautifulSoup, NavigableString # type: ignore[import] from bs4 import BeautifulSoup, NavigableString # type: ignore[import]
from ebooklib import epub # type: ignore[import] from ebooklib import epub # type: ignore[import]
from .utils import calculate_text_length, clean_text, detect_encoding from .utils import clean_text, detect_encoding
from .domain.text_utils import calculate_text_length
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
+1 -1
View File
@@ -8,7 +8,7 @@ Usage:
from abogen.tts_plugin.plugin_manager import get_plugin_manager from abogen.tts_plugin.plugin_manager import get_plugin_manager
manager = get_plugin_manager() manager = get_plugin_manager()
engine = manager.create_engine("kokoro", lang_code="a", device="cpu") engine = manager.create_engine("kokoro", language=Language.EN_US, device="cpu")
session = engine.create_session() session = engine.create_session()
try: try:
result = session.synthesize("Hello world") result = session.synthesize("Hello world")
+5 -3
View File
@@ -9,6 +9,8 @@ from __future__ import annotations
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any, Mapping from typing import Any, Mapping
from abogen.domain.enums import Language
@dataclass(frozen=True) @dataclass(frozen=True)
class AudioFormat: class AudioFormat:
@@ -103,9 +105,9 @@ class EngineConfig:
Attributes: Attributes:
device: Device to use (e.g., "cpu", "cuda:0"). device: Device to use (e.g., "cpu", "cuda:0").
lang_code: Language code for the engine (e.g., "a" for Kokoro English). language: Language enum value. The engine converts to its internal
Plugins that do not require a language code ignore this field. format internally callers never see engine-specific codes.
""" """
device: str = "cpu" device: str = "cpu"
lang_code: str = "a" language: Language = Language.EN_US
+5 -4
View File
@@ -10,6 +10,7 @@ from typing import Any, Iterator
import numpy as np import numpy as np
from abogen.domain.enums import Language
from abogen.tts_plugin.plugin_manager import get_plugin_manager from abogen.tts_plugin.plugin_manager import get_plugin_manager
@@ -123,7 +124,7 @@ class Pipeline:
Presents the same interface that old callers expect:: Presents the same interface that old callers expect::
pipeline = create_pipeline("kokoro", lang_code="a", device="cpu") pipeline = create_pipeline("kokoro", language=Language.EN_US, device="cpu")
for segment in pipeline(text, voice="af_nova", speed=1.0): for segment in pipeline(text, voice="af_nova", speed=1.0):
audio = segment.audio audio = segment.audio
""" """
@@ -200,7 +201,7 @@ class Pipeline:
def create_pipeline( def create_pipeline(
plugin_id: str, plugin_id: str,
*, *,
lang_code: str = "a", language: Language = Language.EN_US,
device: str = "cpu", device: str = "cpu",
) -> Pipeline: ) -> Pipeline:
"""Create a callable TTS pipeline via the Plugin Architecture. """Create a callable TTS pipeline via the Plugin Architecture.
@@ -211,7 +212,7 @@ def create_pipeline(
Args: Args:
plugin_id: Plugin identifier (e.g., "kokoro", "supertonic"). plugin_id: Plugin identifier (e.g., "kokoro", "supertonic").
lang_code: Language code for the engine. language: Language enum value (app-layer type, not engine-specific).
device: Device to use (e.g., "cpu", "cuda:0"). device: Device to use (e.g., "cpu", "cuda:0").
Returns: Returns:
@@ -235,7 +236,7 @@ def create_pipeline(
})(), })(),
) )
config = EngineConfig(device=device, lang_code=lang_code) config = EngineConfig(device=device, language=language)
engine = manager.create_engine(plugin_id, context=ctx, model_path=None, config=config) engine = manager.create_engine(plugin_id, context=ctx, model_path=None, config=config)
return Pipeline(engine) return Pipeline(engine)
-13
View File
@@ -428,19 +428,6 @@ def save_config(config):
pass pass
def calculate_text_length(text):
# Ignore chapter markers
text = re.sub(r"<<CHAPTER_MARKER:.*?>>", "", text)
# Ignore metadata patterns
text = re.sub(r"<<METADATA_[^:]+:[^>]*>>", "", text)
# Ignore newlines
text = text.replace("\n", "")
# Ignore leading/trailing spaces
text = text.strip()
# Calculate character count
char_count = len(text)
return char_count
def get_gpu_acceleration(enabled): def get_gpu_acceleration(enabled):
try: try:
+8
View File
@@ -29,6 +29,13 @@ class _SuppressSuccessfulAccessFilter(logging.Filter):
return " 200 " not in message and " 201 " not in message and " 204 " not in message return " 200 " not in message and " 201 " not in message and " 204 " not in message
class _SuppressPhonemizerWarnings(logging.Filter):
"""Suppress phonemizer word-count-mismatch warnings (normal behavior)."""
def filter(self, record: logging.LogRecord) -> bool: # pragma: no cover - small utility
return "words count mismatch" not in record.getMessage()
_access_log_filter_attached = False _access_log_filter_attached = False
@@ -123,6 +130,7 @@ def create_app(config: Optional[dict[str, Any]] = None) -> Flask:
global _access_log_filter_attached global _access_log_filter_attached
if not _access_log_filter_attached: if not _access_log_filter_attached:
logging.getLogger("werkzeug").addFilter(_SuppressSuccessfulAccessFilter()) logging.getLogger("werkzeug").addFilter(_SuppressSuccessfulAccessFilter())
logging.getLogger("phonemizer").addFilter(_SuppressPhonemizerWarnings())
_access_log_filter_attached = True _access_log_filter_attached = True
return app return app
-162
View File
@@ -1,162 +0,0 @@
"""WebUI adapter: Job -> ConversionRequest.
Converts a WebUI Job into a ConversionRequest that the application layer can process.
This adapter is the bridge between the WebUI layer and the application/domain layer.
The adapter is responsible for:
- Mapping Job fields to ConversionRequest fields
- Handling UI-specific state (logs, progress, cancellation)
- Providing PipelineProvider and VoiceResolver implementations
All conversions happen through this adapter the application layer
never accesses Job directly.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional
from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_ports import ConversionCancelled, ResolvedVoice
def build_conversion_request_from_job(job: Any) -> ConversionRequest:
"""Convert a WebUI Job into a ConversionRequest.
This is the primary function that maps Job fields to ConversionRequest.
All fields are copied the request is independent of the Job.
Args:
job: WebUI Job instance
Returns:
ConversionRequest with all Job data mapped
"""
return ConversionRequest(
# Source
source_path=Path(job.stored_path) if job.stored_path else None,
original_filename=job.original_filename,
# TTS Settings
language=job.language,
tts_provider=job.tts_provider,
voice=job.voice,
voice_profile=job.voice_profile,
speed=job.speed,
use_gpu=job.use_gpu,
supertonic_total_steps=job.supertonic_total_steps,
# Output Format
output_format=job.output_format,
subtitle_mode=job.subtitle_mode,
subtitle_format=job.subtitle_format,
max_subtitle_words=job.max_subtitle_words,
# Save Options
save_mode=job.save_mode,
output_folder=Path(job.output_folder) if job.output_folder else None,
save_chapters_separately=job.save_chapters_separately,
merge_chapters_at_end=job.merge_chapters_at_end,
separate_chapters_format=job.separate_chapters_format,
save_as_project=job.save_as_project,
# Timing
silence_between_chapters=job.silence_between_chapters,
chapter_intro_delay=job.chapter_intro_delay,
# Content Processing
replace_single_newlines=job.replace_single_newlines,
read_title_intro=job.read_title_intro,
read_closing_outro=job.read_closing_outro,
auto_prefix_chapter_titles=job.auto_prefix_chapter_titles,
normalize_chapter_opening_caps=job.normalize_chapter_opening_caps,
# Pronunciation / Normalization
pronunciation_overrides=job.pronunciation_overrides or [],
manual_overrides=job.manual_overrides or [],
heteronym_overrides=job.heteronym_overrides or [],
normalization_overrides=job.normalization_overrides or {},
# Chapter/Chunk Configuration
chapter_overrides=job.chapters or [],
chunks=job.chunks or [],
chunk_level=job.chunk_level,
speaker_mode=job.speaker_mode,
speakers=job.speakers or {},
# Metadata
metadata_tags=job.metadata_tags or {},
# Artifacts
cover_image_path=Path(job.cover_image_path) if job.cover_image_path else None,
cover_image_mime=job.cover_image_mime,
generate_epub3=job.generate_epub3,
)
class WebJobEvents:
"""WebUI implementation of ConversionEvents protocol.
Wraps a Job to provide logging, progress, and cancellation.
"""
def __init__(self, job: Any):
self._job = job
def log(self, message: str, level: str = "info") -> None:
"""Log a message to the Job."""
self._job.add_log(message, level=level)
def progress(self, pct: int, etr: str) -> None:
"""Update progress on the Job."""
self._job.progress = pct / 100.0
self._job.etr_str = etr
def check_cancelled(self) -> None:
"""Check if the Job was cancelled.
Raises:
ConversionCancelled: If cancellation was requested
"""
if self._job.cancel_requested:
raise ConversionCancelled("Job cancelled by user")
class WebPipelineProvider:
"""WebUI implementation of PipelineProvider protocol.
Wraps PipelinePool to provide TTS backends.
"""
def __init__(self, pipeline_pool: Any):
self._pool = pipeline_pool
def get(self, provider: str, language: str, use_gpu: bool) -> Any:
"""Get a TTS backend instance."""
return self._pool.get(provider, language, use_gpu)
def dispose_all(self) -> None:
"""Dispose all backend resources."""
self._pool.dispose_all()
class WebVoiceResolver:
"""WebUI implementation of VoiceResolver protocol.
Wraps the voice resolution logic from conversion_runner.py.
"""
def __init__(
self,
resolve_fn: Callable[[str], tuple[str, str, Any, Optional[float], Optional[int]]],
):
"""Initialize with a voice resolution function.
Args:
resolve_fn: Function that takes a voice_spec and returns
(provider, resolved_spec, voice_choice, speed, steps)
"""
self._resolve_fn = resolve_fn
def resolve(self, voice_spec: str) -> ResolvedVoice:
"""Resolve a voice spec into a loaded voice."""
provider, resolved_spec, voice, speed, steps = self._resolve_fn(voice_spec)
return ResolvedVoice(
provider=provider,
resolved_spec=resolved_spec,
voice=voice,
speed=speed or 1.0,
supertonic_steps=steps or 5,
)
File diff suppressed because it is too large Load Diff
+16 -28
View File
@@ -14,9 +14,12 @@ from abogen.kokoro_text_normalization import normalize_for_pipeline
from abogen.normalization_settings import build_apostrophe_config from abogen.normalization_settings import build_apostrophe_config
from abogen.text_extractor import extract_from_path from abogen.text_extractor import extract_from_path
from abogen.voice_cache import ensure_voice_assets from abogen.voice_cache import ensure_voice_assets
from abogen.webui.conversion_runner import SAMPLE_RATE, _select_device, _to_float32, _spec_to_voice_ids from abogen.domain.device import select_device as _select_device
from abogen.domain.audio_helpers import to_float32 as _to_float32, SAMPLE_RATE
from abogen.domain.voice_resolution import spec_to_voice_ids as _spec_to_voice_ids
from abogen.domain.voice_loader import resolve_voice from abogen.domain.voice_loader import resolve_voice
from abogen.domain.split_pattern import get_split_pattern from abogen.domain.split_pattern import get_split_pattern
from abogen.domain.enums import Language
from abogen.tts_plugin.utils import create_pipeline from abogen.tts_plugin.utils import create_pipeline
@@ -43,11 +46,11 @@ def _resolve_voice_setting(value: str) -> tuple[str, Optional[str], Optional[str
return resolve_voice_setting(value) return resolve_voice_setting(value)
def _load_pipeline(language: str, use_gpu: bool) -> Any: def _load_pipeline(language: Language, use_gpu: bool) -> Any:
device = "cpu" device = "cpu"
if use_gpu: if use_gpu:
device = _select_device() device = _select_device()
return create_pipeline("kokoro", lang_code=language, device=device) return create_pipeline("kokoro", language=language, device=device)
def _extract_cases_from_text(text: str) -> List[Tuple[str, str]]: def _extract_cases_from_text(text: str) -> List[Tuple[str, str]]:
@@ -127,32 +130,14 @@ def run_debug_tts_wavs(
if missing: if missing:
raise RuntimeError(f"Debug EPUB missing expected codes: {', '.join(missing)}") raise RuntimeError(f"Debug EPUB missing expected codes: {', '.join(missing)}")
language = str(settings.get("language") or "a").strip() or "a" raw_language = str(settings.get("language") or "en-US").strip() or "en-US"
# Kokoro's KPipeline expects short language codes like "a" (American English), try:
# but older settings may store ISO-like values such as "en". language = Language.from_str(raw_language)
language_aliases = { except ValueError:
"en": "a", language = Language.EN_US
"en-us": "a",
"en_us": "a",
"en-gb": "b",
"en_gb": "b",
"es": "e",
"es-es": "e",
"fr": "f",
"fr-fr": "f",
"hi": "h",
"it": "i",
"pt": "p",
"pt-br": "p",
"ja": "j",
"jp": "j",
"zh": "z",
"zh-cn": "z",
}
language = language_aliases.get(language.lower(), language)
voice_spec = str(settings.get("default_voice") or "").strip() voice_spec = str(settings.get("default_voice") or "").strip()
use_gpu = bool(settings.get("use_gpu", False)) use_gpu = bool(settings.get("use_gpu", False))
speed = float(settings.get("default_speed", 1.0) or 1.0) speed = float(settings.get("default_speed") or 1.0)
# Settings may store "profile:<name>" which is not a Kokoro voice ID. # Settings may store "profile:<name>" which is not a Kokoro voice ID.
# Resolve it to a concrete voice formula (e.g. "af_heart*0.5+...") so Kokoro # Resolve it to a concrete voice formula (e.g. "af_heart*0.5+...") so Kokoro
@@ -162,7 +147,10 @@ def run_debug_tts_wavs(
if resolved_voice: if resolved_voice:
voice_spec = resolved_voice voice_spec = resolved_voice
if profile_language: if profile_language:
language = str(profile_language).strip() or language try:
language = Language.from_str(str(profile_language).strip()) or language
except (ValueError, AttributeError):
pass
except Exception: except Exception:
# Voice profile resolution is best-effort; fall back to raw voice_spec. # Voice profile resolution is best-effort; fall back to raw voice_spec.
pass pass
+32 -4
View File
@@ -6,6 +6,7 @@ from pathlib import Path
from flask import Blueprint, request, jsonify, send_file, url_for, current_app from flask import Blueprint, request, jsonify, send_file, url_for, current_app
from flask.typing import ResponseReturnValue from flask.typing import ResponseReturnValue
from abogen.domain.enums import Language
from abogen.webui.routes.utils.settings import ( from abogen.webui.routes.utils.settings import (
load_settings, load_settings,
load_integration_settings, load_integration_settings,
@@ -47,6 +48,21 @@ from werkzeug.utils import secure_filename
api_bp = Blueprint("api", __name__) api_bp = Blueprint("api", __name__)
def _parse_language(value: Any) -> Language:
"""Parse a frontend language value to Language enum.
This is the API boundary frontend sends strings, backend parses
to Language enum. No engine-specific codes leak outside the engine.
"""
if isinstance(value, Language):
return value
try:
return Language.from_str(str(value or "").strip())
except (ValueError, AttributeError):
return Language.EN_US
# --- Voice Profile Routes --- # --- Voice Profile Routes ---
@api_bp.get("/voice-profiles") @api_bp.get("/voice-profiles")
@@ -152,7 +168,7 @@ def api_export_voice_profiles() -> ResponseReturnValue:
def api_voice_profiles_preview() -> ResponseReturnValue: def api_voice_profiles_preview() -> ResponseReturnValue:
payload = request.get_json(force=True, silent=True) or {} payload = request.get_json(force=True, silent=True) or {}
text = str(payload.get("text") or "").strip() or "Hello world" text = str(payload.get("text") or "").strip() or "Hello world"
language = str(payload.get("language") or "a").strip().lower() or "a" language = _parse_language(payload.get("language"))
speed = coerce_float(payload.get("speed"), 1.0) speed = coerce_float(payload.get("speed"), 1.0)
max_seconds = coerce_float(payload.get("max_seconds"), 8.0) max_seconds = coerce_float(payload.get("max_seconds"), 8.0)
@@ -168,6 +184,11 @@ def api_voice_profiles_preview() -> ResponseReturnValue:
voice_spec = "" voice_spec = ""
resolved_provider = provider or "kokoro" resolved_provider = provider or "kokoro"
current_app.logger.info(
"[preview] provider=%s language=%s speed=%.2f profile=%s formula=%s",
resolved_provider, language, speed, profile_name or "-", formula or "-",
)
profiles = load_profiles() profiles = load_profiles()
if resolved_provider == "supertonic" and not profile_name: if resolved_provider == "supertonic" and not profile_name:
voice_spec = str(payload.get("voice") or payload.get("supertonic_voice") or "M1").strip() or "M1" voice_spec = str(payload.get("voice") or payload.get("supertonic_voice") or "M1").strip() or "M1"
@@ -186,7 +207,7 @@ def api_voice_profiles_preview() -> ResponseReturnValue:
speed = float(normalized_entry.get("speed") or speed) speed = float(normalized_entry.get("speed") or speed)
else: else:
voice_spec = formula_from_profile(normalized_entry) or "" voice_spec = formula_from_profile(normalized_entry) or ""
language = str(normalized_entry.get("language") or language) language = _parse_language(normalized_entry.get("language") or language)
elif formula: elif formula:
voice_spec = formula voice_spec = formula
resolved_provider = "kokoro" resolved_provider = "kokoro"
@@ -198,7 +219,13 @@ def api_voice_profiles_preview() -> ResponseReturnValue:
voice_spec = formula_from_profile(normalized_entry) or "" voice_spec = formula_from_profile(normalized_entry) or ""
resolved_provider = "kokoro" resolved_provider = "kokoro"
current_app.logger.info(
"[preview] resolved: provider=%s voice_spec=%s",
resolved_provider, voice_spec[:80] if voice_spec else "-",
)
if not voice_spec: if not voice_spec:
current_app.logger.warning("[preview] empty voice_spec, returning 400")
return jsonify({"error": "Unable to resolve preview voice"}), 400 return jsonify({"error": "Unable to resolve preview voice"}), 400
try: try:
@@ -213,6 +240,7 @@ def api_voice_profiles_preview() -> ResponseReturnValue:
max_seconds=max_seconds, max_seconds=max_seconds,
) )
except Exception as exc: except Exception as exc:
current_app.logger.exception("[preview] synthesis failed: %s", exc)
return jsonify({"error": str(exc)}), 500 return jsonify({"error": str(exc)}), 500
@api_bp.post("/speaker-preview") @api_bp.post("/speaker-preview")
@@ -221,7 +249,7 @@ def api_speaker_preview() -> ResponseReturnValue:
pending_id = str(payload.get("pending_id") or "").strip() pending_id = str(payload.get("pending_id") or "").strip()
text = payload.get("text", "Hello world") text = payload.get("text", "Hello world")
voice = payload.get("voice", "af_heart") voice = payload.get("voice", "af_heart")
language = payload.get("language", "a") language = _parse_language(payload.get("language"))
speed_value = payload.get("speed") speed_value = payload.get("speed")
speed = coerce_float(speed_value, 1.0) speed = coerce_float(speed_value, 1.0)
tts_provider = str(payload.get("tts_provider") or "").strip().lower() tts_provider = str(payload.get("tts_provider") or "").strip().lower()
@@ -576,7 +604,7 @@ def api_entity_pronunciation_preview() -> ResponseReturnValue:
token = payload.get("token", "").strip() token = payload.get("token", "").strip()
pronunciation = payload.get("pronunciation", "").strip() pronunciation = payload.get("pronunciation", "").strip()
voice = payload.get("voice", "").strip() voice = payload.get("voice", "").strip()
language = payload.get("language", "a").strip() language = _parse_language(payload.get("language"))
if not token and not pronunciation: if not token and not pronunciation:
return jsonify({"error": "Token or pronunciation required"}), 400 return jsonify({"error": "Token or pronunciation required"}), 400
+4 -2
View File
@@ -8,6 +8,8 @@ from flask.typing import ResponseReturnValue
from abogen.webui.service import ( from abogen.webui.service import (
JobStatus, JobStatus,
)
from abogen.domain.metadata_helpers import (
build_audiobookshelf_metadata, build_audiobookshelf_metadata,
load_audiobookshelf_chapters, load_audiobookshelf_chapters,
) )
@@ -19,9 +21,9 @@ from abogen.webui.routes.utils.epub import (
locate_job_epub, locate_job_epub,
locate_job_audio, locate_job_audio,
) )
from abogen.webui.routes.utils.settings import ( from abogen.domain.settings_core import (
stored_integration_config,
build_audiobookshelf_config, build_audiobookshelf_config,
stored_integration_config,
) )
from abogen.webui.routes.utils.common import existing_paths from abogen.webui.routes.utils.common import existing_paths
from abogen.infrastructure.exporters import ExportService from abogen.infrastructure.exporters import ExportService
+44 -61
View File
@@ -1,14 +1,12 @@
import logging
import time import time
import uuid import uuid
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
from flask import request, render_template, jsonify from flask import request, render_template, jsonify
from flask.typing import ResponseReturnValue from flask.typing import ResponseReturnValue
from abogen.domain.chapter_classification import ( from abogen.domain.enums import Language
supplement_score, from abogen.application.chapter_selection import build_chapter_payload
should_preselect_chapter,
ensure_at_least_one_chapter_enabled,
)
from abogen.webui.service import PendingJob, JobStatus from abogen.webui.service import PendingJob, JobStatus
from abogen.webui.routes.utils.service import get_service from abogen.webui.routes.utils.service import get_service
from abogen.tts_plugin.utils import is_plugin_registered from abogen.tts_plugin.utils import is_plugin_registered
@@ -24,17 +22,21 @@ from abogen.webui.routes.utils.settings import (
audiobookshelf_manual_available, audiobookshelf_manual_available,
) )
from abogen.webui.routes.utils.voice import ( from abogen.webui.routes.utils.voice import (
inject_recommended_voices,
parse_voice_formula, parse_voice_formula,
template_options,
)
from abogen.domain.speaker_metadata import prepare_speaker_metadata
from abogen.domain.metadata_helpers import expand_metadata_aliases
from abogen.domain.voice_resolution import (
formula_from_profile, formula_from_profile,
resolve_voice_setting, resolve_voice_setting,
resolve_voice_choice, resolve_voice_choice,
prepare_speaker_metadata,
template_options,
) )
from abogen.webui.routes.utils.entity import sync_pronunciation_overrides from abogen.webui.routes.utils.entity import sync_pronunciation_overrides
from abogen.webui.routes.utils.epub import job_download_flags from abogen.webui.routes.utils.epub import job_download_flags
from abogen.webui.routes.utils.common import split_profile_spec, extract_checkbox from abogen.webui.routes.utils.common import split_profile_spec, extract_checkbox
from abogen.utils import calculate_text_length from abogen.domain.text_utils import calculate_text_length
from abogen.voice_profiles import serialize_profiles, normalize_profile_entry from abogen.voice_profiles import serialize_profiles, normalize_profile_entry
from abogen.chunking import ChunkLevel, build_chunks_for_chapters from abogen.chunking import ChunkLevel, build_chunks_for_chapters
from abogen.tts_plugin.utils import get_default_voice from abogen.tts_plugin.utils import get_default_voice
@@ -346,7 +348,10 @@ def apply_book_step_form(
language_fallback = pending.language or settings.get("language", "en") language_fallback = pending.language or settings.get("language", "en")
raw_language = (form.get("language") or language_fallback or "en").strip() raw_language = (form.get("language") or language_fallback or "en").strip()
if raw_language: if raw_language:
pending.language = raw_language try:
pending.language = Language.from_str(raw_language)
except (ValueError, AttributeError):
pending.language = Language.EN_US
subtitle_mode = (form.get("subtitle_mode") or pending.subtitle_mode or "Disabled").strip() subtitle_mode = (form.get("subtitle_mode") or pending.subtitle_mode or "Disabled").strip()
if subtitle_mode: if subtitle_mode:
@@ -513,7 +518,10 @@ def apply_book_step_form(
) )
if resolved_language: if resolved_language:
pending.language = resolved_language try:
pending.language = Language.from_str(str(resolved_language))
except (ValueError, AttributeError):
pass # keep existing language
if profile_selection == "__formula" and custom_formula_raw: if profile_selection == "__formula" and custom_formula_raw:
pending.voice = custom_formula_raw pending.voice = custom_formula_raw
@@ -535,35 +543,27 @@ def apply_book_step_form(
if "meta_subtitle" in form: if "meta_subtitle" in form:
pending.metadata_tags["subtitle"] = str(form.get("meta_subtitle", "")).strip() pending.metadata_tags["subtitle"] = str(form.get("meta_subtitle", "")).strip()
# Collect user-editable metadata fields that have concept aliases
user_metadata: Dict[str, str] = {}
if "meta_author" in form: if "meta_author" in form:
authors = str(form.get("meta_author", "")).strip() user_metadata["author"] = str(form.get("meta_author", "")).strip()
pending.metadata_tags["authors"] = authors
pending.metadata_tags["author"] = authors
if "meta_series" in form: if "meta_series" in form:
series = str(form.get("meta_series", "")).strip() user_metadata["series"] = str(form.get("meta_series", "")).strip()
pending.metadata_tags["series"] = series
pending.metadata_tags["series_name"] = series
pending.metadata_tags["seriesname"] = series
pending.metadata_tags["series_title"] = series
pending.metadata_tags["seriestitle"] = series
# If user manually edits series, update opds_series too so it persists
if "opds_series" in pending.metadata_tags:
pending.metadata_tags["opds_series"] = series
if "meta_series_index" in form: if "meta_series_index" in form:
idx = str(form.get("meta_series_index", "")).strip() user_metadata["series_index"] = str(form.get("meta_series_index", "")).strip()
pending.metadata_tags["series_index"] = idx if "meta_description" in form:
pending.metadata_tags["series_sequence"] = idx user_metadata["description"] = str(form.get("meta_description", "")).strip()
if user_metadata:
expanded = expand_metadata_aliases(user_metadata)
pending.metadata_tags.update(expanded)
# If user manually edits series, update opds_series too so it persists
if "meta_series" in form and "opds_series" in pending.metadata_tags:
pending.metadata_tags["opds_series"] = expanded.get("series", "")
if "meta_publisher" in form: if "meta_publisher" in form:
pending.metadata_tags["publisher"] = str(form.get("meta_publisher", "")).strip() pending.metadata_tags["publisher"] = str(form.get("meta_publisher", "")).strip()
if "meta_description" in form:
desc = str(form.get("meta_description", "")).strip()
pending.metadata_tags["description"] = desc
pending.metadata_tags["summary"] = desc
if coerce_bool(form.get("remove_cover"), False): if coerce_bool(form.get("remove_cover"), False):
pending.cover_image_path = None pending.cover_image_path = None
pending.cover_image_mime = None pending.cover_image_mime = None
@@ -637,36 +637,13 @@ def build_pending_job_from_extraction(
getattr(extraction, "combined_text", "") getattr(extraction, "combined_text", "")
) )
chapters_source = getattr(extraction, "chapters", []) or [] chapters_source = getattr(extraction, "chapters", []) or []
total_chapter_count = len(chapters_source) chapters_payload = build_chapter_payload(chapters_source, source_name=original_name)
chapters_payload: List[Dict[str, Any]] = []
for index, chapter in enumerate(chapters_source):
enabled = should_preselect_chapter(chapter.title, chapter.text, index, total_chapter_count)
chapters_payload.append(
{
"id": f"{index:04d}",
"index": index,
"title": chapter.title,
"text": chapter.text,
"characters": calculate_text_length(chapter.text),
"enabled": enabled,
}
)
if not chapters_payload: raw_language = str(form.get("language") or "a").strip() or "a"
chapters_payload.append( try:
{ language = Language.from_str(raw_language)
"id": "0000", except (ValueError, AttributeError):
"index": 0, language = Language.EN_US
"title": original_name,
"text": "",
"characters": 0,
"enabled": True,
}
)
ensure_at_least_one_chapter_enabled(chapters_payload)
language = str(form.get("language") or "a").strip() or "a"
profiles_map = dict(profiles) if isinstance(profiles, Mapping) else dict(profiles or {}) profiles_map = dict(profiles) if isinstance(profiles, Mapping) else dict(profiles or {})
default_voice_setting = settings.get("default_voice") or "" default_voice_setting = settings.get("default_voice") or ""
resolved_default_voice, inferred_profile, inferred_language = resolve_voice_setting( resolved_default_voice, inferred_profile, inferred_language = resolve_voice_setting(
@@ -768,6 +745,7 @@ def build_pending_job_from_extraction(
run_analysis=initial_analysis, run_analysis=initial_analysis,
speaker_config=speaker_config_payload, speaker_config=speaker_config_payload,
apply_config=bool(speaker_config_payload), apply_config=bool(speaker_config_payload),
inject_recommended=inject_recommended_voices,
) )
normalization_overrides = {} normalization_overrides = {}
@@ -783,6 +761,11 @@ def build_pending_job_from_extraction(
else: else:
normalization_overrides[key] = default_val normalization_overrides[key] = default_val
logging.info(
"[form] Creating PendingJob: language=%s voice=%s speed=%.2f provider=%s",
language, voice, speed, settings.get("tts_provider", "kokoro"),
)
pending = PendingJob( pending = PendingJob(
id=uuid.uuid4().hex, id=uuid.uuid4().hex,
original_filename=original_name, original_filename=original_name,
+4 -39
View File
@@ -2,7 +2,6 @@ import os
from typing import Any, Dict, Mapping, Optional from typing import Any, Dict, Mapping, Optional
from abogen.integrations.calibre_opds import CalibreOPDSClient from abogen.integrations.calibre_opds import CalibreOPDSClient
from abogen.integrations.audiobookshelf import AudiobookshelfConfig
from abogen.utils import load_config, save_config from abogen.utils import load_config, save_config
from abogen.domain.settings_core import ( from abogen.domain.settings_core import (
CHUNK_LEVEL_OPTIONS, CHUNK_LEVEL_OPTIONS,
@@ -11,6 +10,7 @@ from abogen.domain.settings_core import (
SAVE_MODE_LABELS, SAVE_MODE_LABELS,
_NORMALIZATION_BOOLEAN_KEYS, _NORMALIZATION_BOOLEAN_KEYS,
_NORMALIZATION_STRING_KEYS, _NORMALIZATION_STRING_KEYS,
build_audiobookshelf_config,
coerce_bool, coerce_bool,
coerce_float, coerce_float,
coerce_int, coerce_int,
@@ -18,6 +18,7 @@ from abogen.domain.settings_core import (
load_settings, load_settings,
llm_ready, llm_ready,
settings_defaults, settings_defaults,
stored_integration_config,
) )
_NORMALIZATION_GROUPS = [ _NORMALIZATION_GROUPS = [
@@ -124,20 +125,8 @@ def load_integration_settings() -> Dict[str, Dict[str, Any]]:
return integrations return integrations
def stored_integration_config(name: str) -> Dict[str, Any]: # stored_integration_config and build_audiobookshelf_config are imported from
cfg = load_config() or {} # abogen.domain.settings_core — single source of truth for integration config.
# Check under "integrations" first (new structure)
integrations = cfg.get("integrations")
if isinstance(integrations, Mapping):
entry = integrations.get(name)
if isinstance(entry, Mapping):
return dict(entry)
# Fallback to top-level (legacy structure)
entry = cfg.get(name)
if isinstance(entry, Mapping):
return dict(entry)
return {}
def calibre_settings_from_payload(payload: Mapping[str, Any]) -> Dict[str, Any]: def calibre_settings_from_payload(payload: Mapping[str, Any]) -> Dict[str, Any]:
@@ -305,30 +294,6 @@ def audiobookshelf_settings_from_payload(payload: Mapping[str, Any]) -> Dict[str
} }
def build_audiobookshelf_config(settings: Mapping[str, Any]) -> Optional[AudiobookshelfConfig]:
base_url = str(settings.get("base_url") or "").strip()
api_token = str(settings.get("api_token") or "").strip()
library_id = str(settings.get("library_id") or "").strip()
if not (base_url and api_token and library_id):
return None
try:
timeout = float(settings.get("timeout", 3600.0))
except (TypeError, ValueError):
timeout = 3600.0
return AudiobookshelfConfig(
base_url=base_url,
api_token=api_token,
library_id=library_id,
collection_id=(str(settings.get("collection_id") or "").strip() or None),
folder_id=(str(settings.get("folder_id") or "").strip() or None),
verify_ssl=coerce_bool(settings.get("verify_ssl"), True),
send_cover=coerce_bool(settings.get("send_cover"), True),
send_chapters=coerce_bool(settings.get("send_chapters"), True),
send_subtitles=coerce_bool(settings.get("send_subtitles"), False),
timeout=timeout,
)
def calibre_integration_enabled( def calibre_integration_enabled(
integrations: Optional[Mapping[str, Any]] = None, integrations: Optional[Mapping[str, Any]] = None,
) -> bool: ) -> bool:
+25 -36
View File
@@ -1,4 +1,5 @@
import io import io
import logging
import threading import threading
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple
import numpy as np import numpy as np
@@ -6,22 +7,15 @@ import soundfile as sf
from flask import current_app, send_file from flask import current_app, send_file
from flask.typing import ResponseReturnValue from flask.typing import ResponseReturnValue
from abogen.domain.audio_helpers import to_float32
from abogen.domain.device import select_device as _select_device from abogen.domain.device import select_device as _select_device
from abogen.domain.enums import Language from abogen.domain.enums import Language
from abogen.domain.split_pattern import get_split_pattern from abogen.domain.split_pattern import get_split_pattern
from abogen.domain.pronunciation import (
# Kokoro-specific language mapping (engine's responsibility) merge_pronunciation_overrides,
_KOKORO_LANG_MAP = { compile_pronunciation_rules,
Language.EN_US: "a", apply_pronunciation_rules,
Language.EN_GB: "b", )
Language.ES: "e",
Language.FR: "f",
Language.HI: "h",
Language.IT: "i",
Language.JA: "j",
Language.PT_BR: "p",
Language.ZH: "z",
}
SAMPLE_RATE = 24000 SAMPLE_RATE = 24000
@@ -41,7 +35,7 @@ def clear_preview_pipelines() -> None:
_preview_pipelines.clear() _preview_pipelines.clear()
def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]: def _resolve_pipeline(language: Language, use_gpu: bool) -> Tuple[Any, bool]:
devices: List[str] = ["cpu"] devices: List[str] = ["cpu"]
if use_gpu: if use_gpu:
preferred = _select_device() preferred = _select_device()
@@ -51,36 +45,33 @@ def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]:
last_error: Optional[Exception] = None last_error: Optional[Exception] = None
for device in devices: for device in devices:
try: try:
logging.info("[preview] Trying device=%s for language=%s", device, language)
return get_preview_pipeline(language, device), device != "cpu" return get_preview_pipeline(language, device), device != "cpu"
except Exception as exc: except Exception as exc:
last_error = exc last_error = exc
logging.warning("[preview] Device %s failed: %s", device, exc)
raise RuntimeError("Preview pipeline is unavailable") from last_error raise RuntimeError("Preview pipeline is unavailable") from last_error
def get_preview_pipeline(language: str, device: str) -> Any: def get_preview_pipeline(language: Language, device: str) -> Any:
# Convert Language enum to Kokoro single-letter code key = (language, device)
try:
lang = Language.from_str(language) if not isinstance(language, Language) else language
except ValueError:
lang = Language.EN_US
kokoro_code = _KOKORO_LANG_MAP.get(lang, "a")
key = (kokoro_code, device)
with _preview_pipeline_lock: with _preview_pipeline_lock:
pipeline = _preview_pipelines.get(key) pipeline = _preview_pipelines.get(key)
if pipeline is not None: if pipeline is not None:
logging.info("[preview] Using cached pipeline for %s/%s", language, device)
return pipeline return pipeline
from abogen.tts_plugin.utils import create_pipeline from abogen.tts_plugin.utils import create_pipeline
pipeline = create_pipeline("kokoro", lang_code=kokoro_code, device=device) logging.info("[preview] Creating pipeline: provider=kokoro language=%s device=%s", language, device)
pipeline = create_pipeline("kokoro", language=language, device=device)
_preview_pipelines[key] = pipeline _preview_pipelines[key] = pipeline
return pipeline return pipeline
def generate_preview_audio( def generate_preview_audio(
text: str, text: str,
voice_spec: str, voice_spec: str,
language: str, language: Language,
speed: float, speed: float,
use_gpu: bool, use_gpu: bool,
tts_provider: str = "kokoro", tts_provider: str = "kokoro",
@@ -100,8 +91,6 @@ def generate_preview_audio(
source_text = text source_text = text
if pronunciation_overrides or manual_overrides or speakers: if pronunciation_overrides or manual_overrides or speakers:
try: try:
from abogen.webui import conversion_runner as runner
class _PreviewJob: class _PreviewJob:
def __init__(self): def __init__(self):
self.language = language self.language = language
@@ -111,9 +100,9 @@ def generate_preview_audio(
self.pronunciation_overrides = list(pronunciation_overrides or []) self.pronunciation_overrides = list(pronunciation_overrides or [])
job = _PreviewJob() job = _PreviewJob()
merged = runner._merge_pronunciation_overrides(job) merged = merge_pronunciation_overrides(job)
rules = runner._compile_pronunciation_rules(merged) rules = compile_pronunciation_rules(merged)
source_text = runner._apply_pronunciation_rules(source_text, rules) source_text = apply_pronunciation_rules(source_text, rules)
except Exception: except Exception:
current_app.logger.exception("Preview override application failed; using raw text") current_app.logger.exception("Preview override application failed; using raw text")
source_text = text source_text = text
@@ -128,12 +117,12 @@ def generate_preview_audio(
current_app.logger.exception("Preview normalization failed; using raw text") current_app.logger.exception("Preview normalization failed; using raw text")
normalized_text = source_text normalized_text = source_text
preview_split = get_split_pattern(str(language or "a"), "Disabled") preview_split = get_split_pattern(language, "Disabled")
if provider == "supertonic": if provider == "supertonic":
from abogen.tts_plugin.utils import create_pipeline from abogen.tts_plugin.utils import create_pipeline
pipeline = create_pipeline("supertonic") pipeline = create_pipeline("supertonic", language=language)
segments = pipeline( segments = pipeline(
normalized_text, normalized_text,
voice=voice_spec, voice=voice_spec,
@@ -148,9 +137,9 @@ def generate_preview_audio(
voice_choice: Any = voice_spec voice_choice: Any = voice_spec
if voice_spec and "*" in voice_spec: if voice_spec and "*" in voice_spec:
from abogen.voice_formulas import get_new_voice from abogen.domain.voice_loader import resolve_voice
voice_choice = get_new_voice(pipeline, voice_spec, pipeline_uses_gpu) voice_choice = resolve_voice(voice_spec, pipeline, pipeline_uses_gpu)
segments = pipeline( segments = pipeline(
normalized_text, normalized_text,
@@ -167,7 +156,7 @@ def generate_preview_audio(
graphemes = getattr(segment, "graphemes", "").strip() graphemes = getattr(segment, "graphemes", "").strip()
if not graphemes: if not graphemes:
continue continue
audio = _to_float32(getattr(segment, "audio", None)) audio = to_float32(getattr(segment, "audio", None))
if audio.size == 0: if audio.size == 0:
continue continue
remaining = max_samples - accumulated remaining = max_samples - accumulated
@@ -191,7 +180,7 @@ def generate_preview_audio(
def synthesize_preview( def synthesize_preview(
text: str, text: str,
voice_spec: str, voice_spec: str,
language: str, language: Language,
speed: float, speed: float,
use_gpu: bool, use_gpu: bool,
tts_provider: str = "kokoro", tts_provider: str = "kokoro",
+3 -527
View File
@@ -1,9 +1,7 @@
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
from abogen.speaker_configs import slugify_label from abogen.speaker_configs import slugify_label
from abogen.speaker_analysis import analyze_speakers
from abogen.webui.routes.utils.settings import load_settings, settings_defaults, _DEFAULT_ANALYSIS_THRESHOLD, _CHUNK_LEVEL_OPTIONS, _APOSTROPHE_MODE_OPTIONS, _NORMALIZATION_GROUPS from abogen.webui.routes.utils.settings import load_settings, settings_defaults, _DEFAULT_ANALYSIS_THRESHOLD, _CHUNK_LEVEL_OPTIONS, _APOSTROPHE_MODE_OPTIONS, _NORMALIZATION_GROUPS
from abogen.webui.routes.utils.common import split_profile_spec
from abogen.voice_profiles import ( from abogen.voice_profiles import (
load_profiles, load_profiles,
serialize_profiles, serialize_profiles,
@@ -18,282 +16,8 @@ from abogen.constants import (
) )
from abogen.tts_plugin.utils import get_voices from abogen.tts_plugin.utils import get_voices
from abogen.speaker_configs import list_configs from abogen.speaker_configs import list_configs
from abogen.domain.voice_resolution import formula_from_profile
from abogen.domain.voice_catalog import build_voice_catalog, filter_voice_catalog
def build_narrator_roster(
voice: str,
voice_profile: Optional[str],
existing: Optional[Mapping[str, Any]] = None,
) -> Dict[str, Any]:
roster: Dict[str, Any] = {
"narrator": {
"id": "narrator",
"label": "Narrator",
"voice": voice,
}
}
if voice_profile:
roster["narrator"]["voice_profile"] = voice_profile
existing_entry: Optional[Mapping[str, Any]] = None
if existing is not None:
existing_entry = existing.get("narrator") if isinstance(existing, Mapping) else None
if isinstance(existing_entry, Mapping):
roster_entry = roster["narrator"]
for key in ("label", "voice", "voice_profile", "voice_formula", "pronunciation"):
value = existing_entry.get(key)
if value is not None and value != "":
roster_entry[key] = value
return roster
def build_speaker_roster(
analysis: Dict[str, Any],
base_voice: str,
voice_profile: Optional[str],
existing: Optional[Mapping[str, Any]] = None,
order: Optional[Iterable[str]] = None,
) -> Dict[str, Any]:
roster = build_narrator_roster(base_voice, voice_profile, existing)
existing_map: Dict[str, Any] = dict(existing) if isinstance(existing, Mapping) else {}
speakers = analysis.get("speakers", {}) if isinstance(analysis, dict) else {}
ordered_ids: Iterable[str]
if order is not None:
ordered_ids = [sid for sid in order if sid in speakers]
else:
ordered_ids = speakers.keys()
for speaker_id in ordered_ids:
payload = speakers.get(speaker_id, {})
if speaker_id == "narrator":
continue
if isinstance(payload, Mapping) and payload.get("suppressed"):
continue
previous = existing_map.get(speaker_id)
roster[speaker_id] = {
"id": speaker_id,
"label": payload.get("label") or speaker_id.replace("_", " ").title(),
"analysis_confidence": payload.get("confidence"),
"analysis_count": payload.get("count"),
"gender": payload.get("gender", "unknown"),
}
detected_gender = payload.get("detected_gender")
if detected_gender:
roster[speaker_id]["detected_gender"] = detected_gender
samples = payload.get("sample_quotes")
if isinstance(samples, list):
roster[speaker_id]["sample_quotes"] = samples
if isinstance(previous, Mapping):
for key in ("voice", "voice_profile", "voice_formula", "resolved_voice", "pronunciation"):
value = previous.get(key)
if value is not None and value != "":
roster[speaker_id][key] = value
if "sample_quotes" not in roster[speaker_id]:
prev_samples = previous.get("sample_quotes")
if isinstance(prev_samples, list):
roster[speaker_id]["sample_quotes"] = prev_samples
if "detected_gender" not in roster[speaker_id]:
prev_detected = previous.get("detected_gender")
if isinstance(prev_detected, str) and prev_detected:
roster[speaker_id]["detected_gender"] = prev_detected
return roster
def match_configured_speaker(
config_speakers: Mapping[str, Any],
roster_id: str,
roster_label: str,
) -> Optional[Mapping[str, Any]]:
if not config_speakers:
return None
entry = config_speakers.get(roster_id)
if entry:
return cast(Mapping[str, Any], entry)
slug = slugify_label(roster_label)
if slug != roster_id and slug in config_speakers:
return cast(Mapping[str, Any], config_speakers[slug])
lower_label = roster_label.strip().lower()
for record in config_speakers.values():
if not isinstance(record, Mapping):
continue
if str(record.get("label", "")).strip().lower() == lower_label:
return record
return None
def apply_speaker_config_to_roster(
roster: Mapping[str, Any],
config: Optional[Mapping[str, Any]],
*,
persist_changes: bool = False,
fallback_languages: Optional[Iterable[str]] = None,
) -> Tuple[Dict[str, Any], List[str], Optional[Dict[str, Any]]]:
if not isinstance(roster, Mapping):
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
return {}, effective_languages, None
updated_roster: Dict[str, Any] = {key: dict(value) for key, value in roster.items() if isinstance(value, Mapping)}
if not config:
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
return updated_roster, effective_languages, None
speakers_map = config.get("speakers")
if not isinstance(speakers_map, Mapping):
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
return updated_roster, effective_languages, None
config_languages = config.get("languages")
if isinstance(config_languages, list):
allowed_languages = [code for code in config_languages if isinstance(code, str) and code]
else:
allowed_languages = []
if not allowed_languages and fallback_languages:
allowed_languages = [code for code in fallback_languages if isinstance(code, str) and code]
default_voice = config.get("default_voice") if isinstance(config.get("default_voice"), str) else ""
used_voices = {entry.get("resolved_voice") or entry.get("voice") for entry in updated_roster.values()} - {None}
narrator_voice = ""
narrator_entry = updated_roster.get("narrator") if isinstance(updated_roster, Mapping) else None
if isinstance(narrator_entry, Mapping):
narrator_voice = str(
narrator_entry.get("resolved_voice")
or narrator_entry.get("default_voice")
or ""
).strip()
if narrator_voice:
used_voices.add(narrator_voice)
config_changed = False
new_config_payload: Dict[str, Any] = {
"language": config.get("language", "a"),
"languages": allowed_languages,
"default_voice": default_voice,
"speakers": dict(speakers_map),
"version": config.get("version", 1),
"notes": config.get("notes", ""),
}
speakers_payload = new_config_payload["speakers"]
for speaker_id, roster_entry in updated_roster.items():
if speaker_id == "narrator":
continue
label = str(roster_entry.get("label") or speaker_id)
config_entry = match_configured_speaker(speakers_map, speaker_id, label)
if config_entry is None:
continue
voice_id = str(config_entry.get("voice") or "").strip()
voice_profile = str(config_entry.get("voice_profile") or "").strip()
voice_formula = str(config_entry.get("voice_formula") or "").strip()
resolved_voice = str(config_entry.get("resolved_voice") or "").strip()
languages = config_entry.get("languages") if isinstance(config_entry.get("languages"), list) else []
chosen_voice = resolved_voice or voice_formula or voice_id or roster_entry.get("voice")
usable_languages = languages or allowed_languages
if chosen_voice:
roster_entry["resolved_voice"] = chosen_voice
roster_entry["voice"] = chosen_voice if not voice_profile and not voice_formula else roster_entry.get("voice", chosen_voice)
if voice_profile:
roster_entry["voice_profile"] = voice_profile
if voice_formula:
roster_entry["voice_formula"] = voice_formula
roster_entry["resolved_voice"] = voice_formula
if not voice_formula and not voice_profile and resolved_voice:
roster_entry["resolved_voice"] = resolved_voice
roster_entry["config_languages"] = usable_languages or []
if chosen_voice:
used_voices.add(chosen_voice)
# persist updates back to config payload if required
if persist_changes:
slug = config_entry.get("id") or slugify_label(label)
speakers_payload[slug] = {
"id": slug,
"label": label,
"gender": config_entry.get("gender", "unknown"),
"voice": voice_id,
"voice_profile": voice_profile,
"voice_formula": voice_formula,
"resolved_voice": roster_entry.get("resolved_voice", resolved_voice or voice_id),
"languages": usable_languages,
}
new_config = new_config_payload if (persist_changes and config_changed) else None
return updated_roster, allowed_languages, new_config
def filter_voice_catalog(
catalog: Iterable[Mapping[str, Any]],
*,
gender: str,
allowed_languages: Optional[Iterable[str]] = None,
) -> List[str]:
allowed_set = {code.lower() for code in (allowed_languages or []) if isinstance(code, str) and code}
gender_normalized = (gender or "unknown").lower()
gender_code = ""
if gender_normalized == "male":
gender_code = "m"
elif gender_normalized == "female":
gender_code = "f"
matches: List[str] = []
seen: set[str] = set()
def _consider(entry: Mapping[str, Any]) -> None:
voice_id = entry.get("id")
if not isinstance(voice_id, str) or not voice_id:
return
if voice_id in seen:
return
seen.add(voice_id)
matches.append(voice_id)
primary: List[Mapping[str, Any]] = []
fallback: List[Mapping[str, Any]] = []
for entry in catalog:
if not isinstance(entry, Mapping):
continue
voice_lang = str(entry.get("language", "")).lower()
voice_gender_code = str(entry.get("gender_code", "")).lower()
if allowed_set and voice_lang not in allowed_set:
continue
if gender_code and voice_gender_code != gender_code:
fallback.append(entry)
continue
primary.append(entry)
for entry in primary:
_consider(entry)
if not matches:
for entry in fallback:
_consider(entry)
if not matches:
for entry in catalog:
if isinstance(entry, Mapping):
_consider(entry)
return matches
def build_voice_catalog() -> List[Dict[str, str]]:
catalog: List[Dict[str, str]] = []
gender_map = {"f": "Female", "m": "Male"}
for voice_id in get_voices("kokoro"):
prefix, _, rest = voice_id.partition("_")
language_code = prefix[0] if prefix else "a"
gender_code = prefix[1] if len(prefix) > 1 else ""
catalog.append(
{
"id": voice_id,
"language": language_code,
"language_label": LANGUAGE_DESCRIPTIONS.get(language_code, language_code.upper()),
"gender": gender_map.get(gender_code, "Unknown"),
"gender_code": gender_code,
"display_name": rest.replace("_", " ").title() if rest else voice_id,
}
)
return catalog
def inject_recommended_voices( def inject_recommended_voices(
@@ -385,177 +109,6 @@ def extract_speaker_config_form(form: Mapping[str, Any]) -> Tuple[str, Dict[str,
return name, payload, errors return name, payload, errors
def prepare_speaker_metadata(
*,
chapters: List[Dict[str, Any]],
chunks: List[Dict[str, Any]],
analysis_chunks: Optional[List[Dict[str, Any]]] = None,
voice: str,
voice_profile: Optional[str],
threshold: int,
existing_roster: Optional[Mapping[str, Any]] = None,
run_analysis: bool = True,
speaker_config: Optional[Mapping[str, Any]] = None,
apply_config: bool = False,
persist_config: bool = False,
) -> tuple[List[Dict[str, Any]], Dict[str, Any], Dict[str, Any], List[str], Optional[Dict[str, Any]]]:
chunk_list = [dict(chunk) for chunk in chunks]
analysis_source = [dict(chunk) for chunk in (analysis_chunks or chunks)]
threshold_value = max(1, int(threshold))
analysis_enabled = run_analysis
settings_state = load_settings()
global_random_languages = [
code
for code in settings_state.get("speaker_random_languages", [])
if isinstance(code, str) and code
]
if not analysis_enabled:
for chunk in chunk_list:
chunk["speaker_id"] = "narrator"
chunk["speaker_label"] = "Narrator"
analysis_payload = {
"version": "1.0",
"narrator": "narrator",
"assignments": {str(chunk.get("id")): "narrator" for chunk in chunk_list},
"speakers": {
"narrator": {
"id": "narrator",
"label": "Narrator",
"count": len(chunk_list),
"confidence": "low",
"sample_quotes": [],
"suppressed": False,
}
},
"suppressed": [],
"stats": {
"total_chunks": len(chunk_list),
"explicit_chunks": 0,
"active_speakers": 0,
"unique_speakers": 1,
"suppressed": 0,
},
}
roster = build_narrator_roster(voice, voice_profile, existing_roster)
narrator_pron = roster["narrator"].get("pronunciation")
if narrator_pron:
analysis_payload["speakers"]["narrator"]["pronunciation"] = narrator_pron
return chunk_list, roster, analysis_payload, [], None
analysis_result = analyze_speakers(
chapters,
analysis_source,
threshold=threshold_value,
max_speakers=0,
)
analysis_payload = analysis_result.to_dict()
speakers_payload = analysis_payload.get("speakers", {})
ordered_ids = [
sid
for sid, meta in sorted(
(
(sid, meta)
for sid, meta in speakers_payload.items()
if sid != "narrator" and isinstance(meta, Mapping) and not meta.get("suppressed")
),
key=lambda item: item[1].get("count", 0),
reverse=True,
)
]
analysis_payload["ordered_speakers"] = ordered_ids
assignments = analysis_payload.get("assignments", {})
suppressed_ids = analysis_payload.get("suppressed", [])
suppressed_details: List[Dict[str, Any]] = []
speakers_payload = analysis_payload.get("speakers", {})
if isinstance(suppressed_ids, Iterable):
for suppressed_id in suppressed_ids:
speaker_meta = speakers_payload.get(suppressed_id) if isinstance(speakers_payload, dict) else None
if isinstance(speaker_meta, dict):
suppressed_details.append(
{
"id": suppressed_id,
"label": speaker_meta.get("label")
or str(suppressed_id).replace("_", " ").title(),
"pronunciation": speaker_meta.get("pronunciation"),
}
)
else:
suppressed_details.append(
{
"id": suppressed_id,
"label": str(suppressed_id).replace("_", " ").title(),
"pronunciation": None,
}
)
analysis_payload["suppressed_details"] = suppressed_details
roster = build_speaker_roster(
analysis_payload,
voice,
voice_profile,
existing=existing_roster,
order=analysis_payload.get("ordered_speakers"),
)
applied_languages: List[str] = []
updated_config: Optional[Dict[str, Any]] = None
if apply_config and speaker_config:
roster, applied_languages, updated_config = apply_speaker_config_to_roster(
roster,
speaker_config,
persist_changes=persist_config,
fallback_languages=global_random_languages,
)
speakers_payload = analysis_payload.get("speakers")
if isinstance(speakers_payload, dict):
for roster_id, roster_payload in roster.items():
speaker_meta = speakers_payload.get(roster_id)
if isinstance(speaker_meta, dict):
for key in ("voice", "voice_profile", "voice_formula", "resolved_voice"):
value = roster_payload.get(key)
if value:
speaker_meta[key] = value
effective_languages: List[str] = []
if applied_languages:
effective_languages = applied_languages
elif isinstance(analysis_payload.get("config_languages"), list):
effective_languages = [
code for code in analysis_payload.get("config_languages", []) if isinstance(code, str) and code
]
elif global_random_languages:
effective_languages = list(global_random_languages)
if effective_languages:
analysis_payload["config_languages"] = effective_languages
speakers_payload = analysis_payload.get("speakers")
if isinstance(speakers_payload, dict):
for roster_id, roster_payload in roster.items():
if roster_id in speakers_payload and isinstance(roster_payload, dict):
pronunciation_value = roster_payload.get("pronunciation")
if pronunciation_value:
speakers_payload[roster_id]["pronunciation"] = pronunciation_value
fallback_languages = effective_languages or []
inject_recommended_voices(roster, fallback_languages=fallback_languages)
for chunk in chunk_list:
chunk_id = str(chunk.get("id"))
speaker_id = assignments.get(chunk_id, "narrator")
chunk["speaker_id"] = speaker_id
speaker_meta = roster.get(speaker_id)
chunk["speaker_label"] = speaker_meta.get("label") if isinstance(speaker_meta, dict) else speaker_id
return chunk_list, roster, analysis_payload, applied_languages, updated_config
def formula_from_profile(entry: Dict[str, Any]) -> Optional[str]:
from abogen.voice_formulas import pairs_to_formula
voices = entry.get("voices") or []
if not voices:
return None
return pairs_to_formula(voices)
def template_options() -> Dict[str, Any]: def template_options() -> Dict[str, Any]:
current_settings = load_settings() current_settings = load_settings()
profiles = serialize_profiles() profiles = serialize_profiles()
@@ -576,7 +129,7 @@ def template_options() -> Dict[str, Any]:
) )
voice_catalog = build_voice_catalog() voice_catalog = build_voice_catalog()
return { return {
"languages": LANGUAGE_DESCRIPTIONS, "languages": {lang.value: label for lang, label in LANGUAGE_DESCRIPTIONS.items()},
"voices": get_voices("kokoro"), "voices": get_voices("kokoro"),
"subtitle_formats": SUBTITLE_FORMATS, "subtitle_formats": SUBTITLE_FORMATS,
"supported_langs_for_subs": SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION, "supported_langs_for_subs": SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
@@ -601,83 +154,6 @@ def template_options() -> Dict[str, Any]:
} }
def resolve_profile_voice(
profile_name: Optional[str],
*,
profiles: Optional[Mapping[str, Any]] = None,
) -> tuple[str, Optional[str]]:
if not profile_name:
return "", None
source = profiles if isinstance(profiles, Mapping) else None
if source is None:
source = load_profiles()
entry = source.get(profile_name) if isinstance(source, Mapping) else None
if not isinstance(entry, Mapping):
return "", None
formula = formula_from_profile(dict(entry)) or ""
language = entry.get("language") if isinstance(entry.get("language"), str) else None
if isinstance(language, str):
language = language.strip().lower() or None
return formula, language
def resolve_voice_setting(
value: Any,
*,
profiles: Optional[Mapping[str, Any]] = None,
) -> tuple[str, Optional[str], Optional[str]]:
base_spec, profile_name = split_profile_spec(value)
if profile_name:
formula, language = resolve_profile_voice(profile_name, profiles=profiles)
return formula or "", profile_name, language
return base_spec, None, None
def resolve_voice_choice(
language: str,
base_voice: str,
profile_name: str,
custom_formula: str,
profiles: Dict[str, Any],
) -> tuple[str, str, Optional[str]]:
resolved_voice = base_voice
resolved_language = language
selected_profile = None
if profile_name:
from abogen.voice_profiles import normalize_profile_entry
entry_raw = profiles.get(profile_name)
entry = normalize_profile_entry(entry_raw)
provider = str((entry or {}).get("provider") or "").strip().lower()
# Provider-aware behavior:
# - Kokoro profiles typically represent mixes (formula strings).
# - SuperTonic profiles represent a discrete voice id + settings.
# In that case, we return a speaker reference so downstream can
# resolve provider per-speaker and allow mixed-provider casting.
if provider == "supertonic":
resolved_voice = f"speaker:{profile_name}"
selected_profile = profile_name
profile_language = (entry or {}).get("language")
if profile_language:
resolved_language = str(profile_language)
else:
formula = formula_from_profile(entry or {}) if entry else None
if formula:
resolved_voice = formula
selected_profile = profile_name
profile_language = (entry or {}).get("language")
if profile_language:
resolved_language = profile_language
if custom_formula:
resolved_voice = custom_formula
selected_profile = None
return resolved_voice, resolved_language, selected_profile
def parse_voice_formula(formula: str) -> List[tuple[str, float]]: def parse_voice_formula(formula: str) -> List[tuple[str, float]]:
voices = parse_formula_terms(formula) voices = parse_formula_terms(formula)
total = sum(weight for _, weight in voices) total = sum(weight for _, weight in voices)
+5 -2
View File
@@ -2,11 +2,14 @@ from typing import Any, Dict, List, Optional
from flask import Blueprint, render_template, request, jsonify, abort, flash, redirect, url_for from flask import Blueprint, render_template, request, jsonify, abort, flash, redirect, url_for
from flask.typing import ResponseReturnValue from flask.typing import ResponseReturnValue
from abogen.domain.enums import Language
from abogen.webui.routes.utils.voice import ( from abogen.webui.routes.utils.voice import (
template_options, template_options,
parse_voice_formula,
)
from abogen.domain.voice_resolution import (
resolve_voice_setting, resolve_voice_setting,
resolve_voice_choice, resolve_voice_choice,
parse_voice_formula,
) )
from abogen.webui.routes.utils.settings import load_settings, coerce_bool from abogen.webui.routes.utils.settings import load_settings, coerce_bool
from abogen.webui.routes.utils.synthesize import synthesize_preview from abogen.webui.routes.utils.synthesize import synthesize_preview
@@ -39,7 +42,7 @@ def test_voice() -> ResponseReturnValue:
return synthesize_preview( return synthesize_preview(
text=text, text=text,
voice_spec=voice, voice_spec=voice,
language="a", # Default language language=Language.EN_US,
speed=speed, speed=speed,
use_gpu=use_gpu, use_gpu=use_gpu,
) )
+20 -174
View File
@@ -14,24 +14,11 @@ from enum import Enum
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Dict, Iterable, List, Optional, Mapping from typing import Any, Callable, Dict, Iterable, List, Optional, Mapping
from abogen.utils import get_internal_cache_path, get_user_settings_dir, load_config from abogen.domain.metadata_helpers import normalize_metadata_map
from abogen.voice_cache import bootstrap_voice_cache
from abogen.integrations.audiobookshelf import ( from abogen.domain.enums import Language
AudiobookshelfClient, from abogen.utils import get_internal_cache_path, get_user_settings_dir
AudiobookshelfConfig,
AudiobookshelfUploadError,
)
from abogen.domain.metadata_helpers import (
normalize_metadata_casefold as _normalize_metadata_casefold,
split_people_field as _split_people_field,
split_simple_list as _split_simple_list,
first_nonempty as _first_nonempty,
extract_year as _extract_year,
normalize_series_sequence as _normalize_series_sequence,
build_audiobookshelf_metadata as _build_abs_metadata,
load_audiobookshelf_chapters as _load_abs_chapters,
_SERIES_SEQUENCE_TAG_KEYS,
)
def _create_set_event() -> threading.Event: def _create_set_event() -> threading.Event:
@@ -105,7 +92,7 @@ class Job:
id: str id: str
original_filename: str original_filename: str
stored_path: Path stored_path: Path
language: str language: Language
voice: str voice: str
speed: float speed: float
use_gpu: bool use_gpu: bool
@@ -265,23 +252,6 @@ class Job:
} }
def build_audiobookshelf_metadata(job: Job) -> Dict[str, Any]:
filename = Path(job.original_filename or "").stem or job.original_filename or "Audiobook"
return _build_abs_metadata(
job.metadata_tags,
language=job.language or "",
filename=filename,
)
def load_audiobookshelf_chapters(job: Job) -> Optional[List[Dict[str, Any]]]:
metadata_ref = job.result.artifacts.get("metadata")
if not metadata_ref:
return None
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
return _load_abs_chapters(metadata_path)
def _existing_paths(paths: Iterable[Any]) -> List[Path]: def _existing_paths(paths: Iterable[Any]) -> List[Path]:
resolved: List[Path] = [] resolved: List[Path] = []
for item in paths: for item in paths:
@@ -296,7 +266,7 @@ class PendingJob:
id: str id: str
original_filename: str original_filename: str
stored_path: Path stored_path: Path
language: str language: Language
voice: str voice: str
speed: float speed: float
use_gpu: bool use_gpu: bool
@@ -367,7 +337,6 @@ class ConversionService:
self._pending_jobs: Dict[str, PendingJob] = {} self._pending_jobs: Dict[str, PendingJob] = {}
self._state_path = self._determine_state_path() self._state_path = self._determine_state_path()
self._ensure_directories() self._ensure_directories()
self._bootstrap_voice_cache()
self._load_state() self._load_state()
# Public API --------------------------------------------------------- # Public API ---------------------------------------------------------
@@ -384,7 +353,7 @@ class ConversionService:
*, *,
original_filename: str, original_filename: str,
stored_path: Path, stored_path: Path,
language: str, language: Language,
voice: str, voice: str,
speed: float, speed: float,
tts_provider: str = "kokoro", tts_provider: str = "kokoro",
@@ -428,7 +397,7 @@ class ConversionService:
normalization_overrides: Optional[Mapping[str, Any]] = None, normalization_overrides: Optional[Mapping[str, Any]] = None,
) -> Job: ) -> Job:
job_id = uuid.uuid4().hex job_id = uuid.uuid4().hex
normalized_metadata = self._normalize_metadata_tags(metadata_tags) normalized_metadata = normalize_metadata_map(metadata_tags)
normalized_chapters = self._normalize_chapters(chapters) normalized_chapters = self._normalize_chapters(chapters)
normalized_chunks = self._normalize_chunks(chunks) normalized_chunks = self._normalize_chunks(chunks)
if total_characters <= 0 and normalized_chapters: if total_characters <= 0 and normalized_chapters:
@@ -697,23 +666,6 @@ class ConversionService:
self._uploads_root.mkdir(parents=True, exist_ok=True) self._uploads_root.mkdir(parents=True, exist_ok=True)
self._state_path.parent.mkdir(parents=True, exist_ok=True) self._state_path.parent.mkdir(parents=True, exist_ok=True)
def _bootstrap_voice_cache(self) -> None:
try:
downloaded, errors = bootstrap_voice_cache(
on_progress=lambda msg: _JOB_LOGGER.debug("[voice cache] %s", msg)
)
except RuntimeError as exc:
_JOB_LOGGER.warning("Voice cache bootstrap skipped: %s", exc)
return
if downloaded:
count = len(downloaded)
suffix = "s" if count != 1 else ""
_JOB_LOGGER.info("Voice cache ready: downloaded %d new asset%s.", count, suffix)
if errors:
for voice_id, message in errors.items():
_JOB_LOGGER.warning("Voice cache failed for %s: %s", voice_id, message)
def _ensure_worker(self) -> None: def _ensure_worker(self) -> None:
with self._lock: with self._lock:
if self._worker_thread and self._worker_thread.is_alive(): if self._worker_thread and self._worker_thread.is_alive():
@@ -780,7 +732,6 @@ class ConversionService:
elif job.status != JobStatus.FAILED: elif job.status != JobStatus.FAILED:
job.status = JobStatus.COMPLETED job.status = JobStatus.COMPLETED
job.add_log("Job completed", level="success") job.add_log("Job completed", level="success")
self._post_completion_hooks(job)
job.finished_at = time.time() job.finished_at = time.time()
finally: finally:
job.pause_event.set() job.pause_event.set()
@@ -801,105 +752,6 @@ class ConversionService:
self._queue.remove(job_id) self._queue.remove(job_id)
self._update_queue_positions_locked() self._update_queue_positions_locked()
def _post_completion_hooks(self, job: Job) -> None:
try:
self._maybe_send_to_audiobookshelf(job)
except AudiobookshelfUploadError as exc:
job.add_log(f"Audiobookshelf upload failed: {exc}", level="error")
except Exception as exc: # pragma: no cover - defensive guard
job.add_log(f"Audiobookshelf integration error: {exc}", level="error")
def _maybe_send_to_audiobookshelf(self, job: Job) -> None:
cfg = load_config() or {}
integration_cfg = cfg.get("audiobookshelf")
if not isinstance(integration_cfg, Mapping):
return
enabled = self._coerce_bool(integration_cfg.get("enabled"), False)
auto_send = self._coerce_bool(integration_cfg.get("auto_send"), False)
if not (enabled and auto_send):
return
base_url = str(integration_cfg.get("base_url") or "").strip()
api_token = str(integration_cfg.get("api_token") or "").strip()
library_id = str(integration_cfg.get("library_id") or "").strip()
folder_id = str(integration_cfg.get("folder_id") or "").strip()
if not base_url or not api_token or not library_id:
job.add_log(
"Audiobookshelf upload skipped: configure base URL, API token, and library ID first.",
level="warning",
)
return
if not folder_id:
job.add_log(
"Audiobookshelf upload skipped: enter the folder name or ID in the Audiobookshelf settings.",
level="warning",
)
return
audio_ref = job.result.audio_path
audio_path = audio_ref if isinstance(audio_ref, Path) else Path(str(audio_ref)) if audio_ref else None
if not audio_path or not audio_path.exists():
job.add_log("Audiobookshelf upload skipped: audio output not found.", level="warning")
return
timeout_raw = integration_cfg.get("timeout", 3600.0)
try:
timeout_value = float(timeout_raw)
except (TypeError, ValueError):
timeout_value = 3600.0
config = AudiobookshelfConfig(
base_url=base_url,
api_token=api_token,
library_id=library_id,
collection_id=(str(integration_cfg.get("collection_id") or "").strip() or None),
folder_id=folder_id,
verify_ssl=self._coerce_bool(integration_cfg.get("verify_ssl"), True),
send_cover=self._coerce_bool(integration_cfg.get("send_cover"), True),
send_chapters=self._coerce_bool(integration_cfg.get("send_chapters"), True),
send_subtitles=self._coerce_bool(integration_cfg.get("send_subtitles"), False),
timeout=timeout_value,
)
cover_ref = job.cover_image_path
cover_path = None
if config.send_cover and cover_ref:
cover_candidate = cover_ref if isinstance(cover_ref, Path) else Path(str(cover_ref))
if cover_candidate.exists():
cover_path = cover_candidate
subtitles = _existing_paths(job.result.subtitle_paths) if config.send_subtitles else None
chapters = load_audiobookshelf_chapters(job) if config.send_chapters else None
metadata = build_audiobookshelf_metadata(job)
client = AudiobookshelfClient(config)
display_title = metadata.get("title") or audio_path.stem
try:
existing_items = client.find_existing_items(display_title, folder_id=config.folder_id)
except AudiobookshelfUploadError as exc:
job.add_log(f"Audiobookshelf lookup failed: {exc}", level="error")
return
if existing_items:
job.add_log(
f"Removing existing Audiobookshelf item(s) for '{display_title}' before upload.",
level="info",
)
try:
client.delete_items(existing_items)
except Exception as exc:
job.add_log(f"Failed to remove existing item(s): {exc}", level="warning")
client.upload_audiobook(
audio_path,
metadata=metadata,
cover_path=cover_path,
chapters=chapters,
subtitles=subtitles,
)
job.add_log("Audiobookshelf upload queued.", level="info")
# Persistence ------------------------------------------------------ # Persistence ------------------------------------------------------
def _serialize_job(self, job: Job) -> Dict[str, Any]: def _serialize_job(self, job: Job) -> Dict[str, Any]:
result_audio = str(job.result.audio_path) if job.result.audio_path else None result_audio = str(job.result.audio_path) if job.result.audio_path else None
@@ -910,7 +762,7 @@ class ConversionService:
"id": job.id, "id": job.id,
"original_filename": job.original_filename, "original_filename": job.original_filename,
"stored_path": str(job.stored_path), "stored_path": str(job.stored_path),
"language": job.language, "language": job.language.value if isinstance(job.language, Language) else str(job.language),
"tts_provider": getattr(job, "tts_provider", "kokoro"), "tts_provider": getattr(job, "tts_provider", "kokoro"),
"voice": job.voice, "voice": job.voice,
"speed": job.speed, "speed": job.speed,
@@ -1026,11 +878,19 @@ class ConversionService:
stored_path = Path(payload["stored_path"]) stored_path = Path(payload["stored_path"])
output_folder_raw = payload.get("output_folder") output_folder_raw = payload.get("output_folder")
output_folder = Path(output_folder_raw) if output_folder_raw else None output_folder = Path(output_folder_raw) if output_folder_raw else None
raw_lang = payload.get("language", "")
if isinstance(raw_lang, Language):
language = raw_lang
else:
try:
language = Language.from_str(str(raw_lang or "").strip())
except (ValueError, AttributeError):
language = Language.EN_US
job = Job( job = Job(
id=payload["id"], id=payload["id"],
original_filename=payload["original_filename"], original_filename=payload["original_filename"],
stored_path=stored_path, stored_path=stored_path,
language=payload.get("language", "a"), language=language,
tts_provider=str(payload.get("tts_provider") or "kokoro"), tts_provider=str(payload.get("tts_provider") or "kokoro"),
voice=payload.get("voice", ""), voice=payload.get("voice", ""),
speed=float(payload.get("speed", 1.0)), speed=float(payload.get("speed", 1.0)),
@@ -1177,20 +1037,6 @@ class ConversionService:
except (TypeError, ValueError): except (TypeError, ValueError):
return None return None
@staticmethod
def _normalize_metadata_tags(values: Optional[Mapping[str, Any]]) -> Dict[str, str]:
if not values:
return {}
normalized: Dict[str, str] = {}
for key, raw_value in values.items():
if raw_value is None:
continue
key_str = str(key).strip()
if not key_str:
continue
normalized[key_str] = str(raw_value)
return normalized
@classmethod @classmethod
def _normalize_chapters(cls, chapters: Optional[Iterable[Any]]) -> List[Dict[str, Any]]: def _normalize_chapters(cls, chapters: Optional[Iterable[Any]]) -> List[Dict[str, Any]]:
if not chapters: if not chapters:
@@ -1267,7 +1113,7 @@ class ConversionService:
entry["enabled"] = enabled entry["enabled"] = enabled
metadata_payload = raw_dict.get("metadata") or raw_dict.get("metadata_tags") metadata_payload = raw_dict.get("metadata") or raw_dict.get("metadata_tags")
normalized_metadata = cls._normalize_metadata_tags(metadata_payload) normalized_metadata = normalize_metadata_map(metadata_payload)
if normalized_metadata: if normalized_metadata:
entry["metadata"] = normalized_metadata entry["metadata"] = normalized_metadata
+1 -1
View File
@@ -29,8 +29,8 @@ def apply_form_to_settings(current: dict, form: Mapping[str, Any]) -> dict:
DEFAULT_ANALYSIS_THRESHOLD, DEFAULT_ANALYSIS_THRESHOLD,
_NORMALIZATION_BOOLEAN_KEYS, _NORMALIZATION_BOOLEAN_KEYS,
_NORMALIZATION_STRING_KEYS, _NORMALIZATION_STRING_KEYS,
stored_integration_config,
) )
from abogen.webui.routes.utils.settings import stored_integration_config
from abogen.webui.routes.utils.common import extract_checkbox from abogen.webui.routes.utils.common import extract_checkbox
from abogen.utils import load_config from abogen.utils import load_config
# General settings # General settings
+3
View File
@@ -452,6 +452,9 @@ const initDashboard = () => {
return; return;
} }
openUploadModal(dropzone); openUploadModal(dropzone);
if (sourceFileInput) {
sourceFileInput.click();
}
}); });
dropzone.addEventListener("keydown", (event) => { dropzone.addEventListener("keydown", (event) => {
+3 -1
View File
@@ -165,6 +165,7 @@ def create_engine(
""" """
try: try:
KPipeline = _load_kpipeline() KPipeline = _load_kpipeline()
from plugins.kokoro.engine import engine_language
# Determine repo_id from model_path or use default # Determine repo_id from model_path or use default
repo_id = "hexgrad/Kokoro-82M" repo_id = "hexgrad/Kokoro-82M"
@@ -172,8 +173,9 @@ def create_engine(
# If a specific model path is provided, use it as repo_id # If a specific model path is provided, use it as repo_id
repo_id = str(model_path) repo_id = str(model_path)
kokoro_code = engine_language(config.language)
pipeline = KPipeline( pipeline = KPipeline(
lang_code=config.lang_code, lang_code=kokoro_code,
repo_id=repo_id, repo_id=repo_id,
device=config.device, device=config.device,
) )
+58
View File
@@ -2,6 +2,10 @@
This module adapts the existing Kokoro backend to the new Engine/EngineSession This module adapts the existing Kokoro backend to the new Engine/EngineSession
protocol. It wraps the KokoroBackend without modifying it. protocol. It wraps the KokoroBackend without modifying it.
Language mapping: this is the engine's responsibility. The engine knows
which languages it supports and converts Language enum internal format.
Callers outside this module never see engine-specific codes.
""" """
from __future__ import annotations from __future__ import annotations
@@ -11,6 +15,7 @@ from typing import Any
import numpy as np import numpy as np
from abogen.domain.enums import Language
from abogen.tts_plugin.capabilities import VoiceLister from abogen.tts_plugin.capabilities import VoiceLister
from abogen.tts_plugin.engine import Engine, EngineSession from abogen.tts_plugin.engine import Engine, EngineSession
from abogen.tts_plugin.errors import EngineError from abogen.tts_plugin.errors import EngineError
@@ -27,6 +32,59 @@ logger = logging.getLogger(__name__)
# Sample rate for Kokoro audio # Sample rate for Kokoro audio
_KOKORO_SAMPLE_RATE = 24000 _KOKORO_SAMPLE_RATE = 24000
# Engine-internal language mapping: Language enum → kokoro code.
# ONLY visible inside this module — callers never see kokoro codes.
_KOKORO_LANG_MAP: dict[Language, str] = {
Language.EN_US: "a",
Language.EN_GB: "b",
Language.ES: "e",
Language.FR: "f",
Language.HI: "h",
Language.IT: "i",
Language.JA: "j",
Language.PT_BR: "p",
Language.ZH: "z",
}
# Reverse mapping: engine-internal code → Language enum.
# Used by voice catalog and other places that need to convert
# engine codes back to Language enum (e.g. voice ID prefix extraction).
_CODE_TO_LANGUAGE: dict[str, Language] = {v: k for k, v in _KOKORO_LANG_MAP.items()}
def supported_languages() -> list[Language]:
"""Return the list of Language enum values this engine supports.
This is the engine's responsibility — the engine knows which
languages it supports and exposes them as Language enum values.
UI layers query this to populate language selectors.
"""
return list(_KOKORO_LANG_MAP.keys())
def engine_language(lang: Language) -> str:
"""Map a Language enum to the engine's internal code.
This is the engine's responsibility — the engine owns the mapping
between Language enum and its internal format. Callers pass Language
enum; the engine converts internally. The returned string is ONLY
used inside the engine implementation.
"""
return _KOKORO_LANG_MAP.get(lang, "a")
def language_for_voice_id(voice_id: str) -> Language:
"""Determine which Language a voice belongs to from its voice ID.
Kokoro voice IDs encode language as a prefix (e.g. "af_heart" "a" EN_US).
This is kokoro-specific knowledge that stays inside the engine.
Callers pass a voice ID string; the engine returns a Language enum.
"""
prefix = str(voice_id or "").strip()[:1].lower()
if prefix in _CODE_TO_LANGUAGE:
return _CODE_TO_LANGUAGE[prefix]
return Language.EN_US
class KokoroSession: class KokoroSession:
"""EngineSession implementation for Kokoro. """EngineSession implementation for Kokoro.
+3 -2
View File
@@ -32,11 +32,12 @@ from abogen.tts_plugin.types import EngineConfig
from .engine import SuperTonicEngine from .engine import SuperTonicEngine
def _load_supertonic_pipeline() -> Any: def _load_supertonic_pipeline(language: Any = None) -> Any:
"""Lazy-load SuperTonic dependencies and create pipeline.""" """Lazy-load SuperTonic dependencies and create pipeline."""
from plugins.supertonic.pipeline import SupertonicPipeline from plugins.supertonic.pipeline import SupertonicPipeline
return SupertonicPipeline( return SupertonicPipeline(
language=language,
sample_rate=24000, sample_rate=24000,
auto_download=True, auto_download=True,
total_steps=5, total_steps=5,
@@ -128,7 +129,7 @@ def create_engine(
EngineError: On failure. Cleans up partially created resources. EngineError: On failure. Cleans up partially created resources.
""" """
try: try:
pipeline = _load_supertonic_pipeline() pipeline = _load_supertonic_pipeline(language=config.language)
engine = SuperTonicEngine(pipeline) engine = SuperTonicEngine(pipeline)
return engine return engine
except Exception as e: except Exception as e:
+56
View File
@@ -12,6 +12,7 @@ from typing import Any
import numpy as np import numpy as np
from abogen.domain.enums import Language
from abogen.tts_plugin.capabilities import VoiceLister from abogen.tts_plugin.capabilities import VoiceLister
from abogen.tts_plugin.engine import Engine, EngineSession from abogen.tts_plugin.engine import Engine, EngineSession
from abogen.tts_plugin.errors import EngineError from abogen.tts_plugin.errors import EngineError
@@ -28,6 +29,61 @@ logger = logging.getLogger(__name__)
# Sample rate for SuperTonic audio # Sample rate for SuperTonic audio
_SUPERTONIC_SAMPLE_RATE = 24000 _SUPERTONIC_SAMPLE_RATE = 24000
# Engine-internal language mapping: Language enum → Supertonic ISO 639-1 code.
_SUPERTONIC_LANG_MAP: dict[Language, str] = {
Language.EN_US: "en",
Language.EN_GB: "en",
Language.AR: "ar",
Language.BG: "bg",
Language.CS: "cs",
Language.DA: "da",
Language.DE: "de",
Language.EL: "el",
Language.ES: "es",
Language.ET: "et",
Language.FI: "fi",
Language.FR: "fr",
Language.HI: "hi",
Language.HR: "hr",
Language.HU: "hu",
Language.ID: "id",
Language.IT: "it",
Language.JA: "ja",
Language.KO: "ko",
Language.LT: "lt",
Language.LV: "lv",
Language.NL: "nl",
Language.PL: "pl",
Language.PT_BR: "pt",
Language.RO: "ro",
Language.RU: "ru",
Language.SK: "sk",
Language.SL: "sl",
Language.SV: "sv",
Language.TR: "tr",
Language.UK: "uk",
Language.VI: "vi",
}
def supported_languages() -> list[Language]:
"""Return the list of Language enum values this engine supports."""
return list(_SUPERTONIC_LANG_MAP.keys())
def engine_language(lang: Language) -> str:
"""Map a Language enum to the engine's internal ISO 639-1 code.
Raises ValueError for unsupported languages.
"""
result = _SUPERTONIC_LANG_MAP.get(lang)
if result is None:
raise ValueError(
f"Supertonic does not support language: {lang!r}. "
f"Supported: {supported_languages()}"
)
return result
class SuperTonicSession: class SuperTonicSession:
"""EngineSession implementation for SuperTonic. """EngineSession implementation for SuperTonic.
+9
View File
@@ -158,6 +158,7 @@ class SupertonicPipeline:
def __init__( def __init__(
self, self,
*, *,
language: Any = None,
sample_rate: int, sample_rate: int,
auto_download: bool = True, auto_download: bool = True,
total_steps: int = 5, total_steps: int = 5,
@@ -167,6 +168,13 @@ class SupertonicPipeline:
self.total_steps = int(total_steps) self.total_steps = int(total_steps)
self.max_chunk_length = int(max_chunk_length) self.max_chunk_length = int(max_chunk_length)
# Resolve language to ISO 639-1 code for Supertonic
if language is not None:
from plugins.supertonic.engine import engine_language
self._lang = engine_language(language)
else:
self._lang = "en"
_configure_supertonic_gpu() _configure_supertonic_gpu()
try: try:
@@ -212,6 +220,7 @@ class SupertonicPipeline:
max_chunk_length=self.max_chunk_length, max_chunk_length=self.max_chunk_length,
silence_duration=0.0, silence_duration=0.0,
verbose=False, verbose=False,
lang=self._lang,
) )
break break
except ValueError as exc: except ValueError as exc:
+3 -1
View File
@@ -14,6 +14,8 @@ from unittest.mock import MagicMock, patch
import numpy as np import numpy as np
from abogen.domain.enums import Language
from abogen.tts_plugin.engine import Engine, EngineSession from abogen.tts_plugin.engine import Engine, EngineSession
from abogen.tts_plugin.errors import EngineError from abogen.tts_plugin.errors import EngineError
from abogen.tts_plugin.plugin_manager import PluginManager, get_plugin_manager, reset_plugin_manager from abogen.tts_plugin.plugin_manager import PluginManager, get_plugin_manager, reset_plugin_manager
@@ -328,7 +330,7 @@ class TestRegression:
manager._loaded = True manager._loaded = True
with patch("abogen.tts_plugin.utils.get_plugin_manager", return_value=manager): with patch("abogen.tts_plugin.utils.get_plugin_manager", return_value=manager):
backend = create_pipeline("mock_tts", lang_code="a", device="cpu") backend = create_pipeline("mock_tts", language=Language.EN_US, device="cpu")
# Old interface: pipeline(text, voice=..., speed=..., split_pattern=...) # Old interface: pipeline(text, voice=..., speed=..., split_pattern=...)
segments = list(backend( segments = list(backend(
@@ -3,6 +3,7 @@
import pytest import pytest
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
from abogen.domain.enums import Language
from abogen.tts_plugin.plugin_manager import PluginManager, get_plugin_manager, reset_plugin_manager from abogen.tts_plugin.plugin_manager import PluginManager, get_plugin_manager, reset_plugin_manager
from abogen.tts_plugin.utils import Pipeline, create_pipeline from abogen.tts_plugin.utils import Pipeline, create_pipeline
from abogen.tts_plugin.engine import Engine, EngineSession from abogen.tts_plugin.engine import Engine, EngineSession
@@ -175,7 +176,7 @@ class TestCreatePipelineCompat:
mock_engine = FakeEngine() mock_engine = FakeEngine()
mock_manager.create_engine.return_value = mock_engine mock_manager.create_engine.return_value = mock_engine
backend = create_pipeline("kokoro", lang_code="a", device="cpu") backend = create_pipeline("kokoro", language=Language.EN_US, device="cpu")
assert callable(backend) assert callable(backend)
mock_manager.create_engine.assert_called_once() mock_manager.create_engine.assert_called_once()
@@ -185,7 +186,7 @@ class TestCreatePipelineCompat:
assert call_args.kwargs["model_path"] is None assert call_args.kwargs["model_path"] is None
assert isinstance(call_args.kwargs["config"], EngineConfig) assert isinstance(call_args.kwargs["config"], EngineConfig)
assert call_args.kwargs["config"].device == "cpu" assert call_args.kwargs["config"].device == "cpu"
assert call_args.kwargs["config"].lang_code == "a" assert call_args.kwargs["config"].language == Language.EN_US
def test_create_pipeline_raises_for_unknown_plugin(self): def test_create_pipeline_raises_for_unknown_plugin(self):
"""create_pipeline raises KeyError for unknown plugins.""" """create_pipeline raises KeyError for unknown plugins."""
+13 -12
View File
@@ -8,6 +8,7 @@ These tests verify that value objects satisfy the architectural requirements:
import pytest import pytest
from abogen.domain.enums import Language
from abogen.tts_plugin.types import ( from abogen.tts_plugin.types import (
AudioFormat, AudioFormat,
Duration, Duration,
@@ -192,23 +193,23 @@ class TestEngineConfigContract:
config = EngineConfig(device="cuda:0") config = EngineConfig(device="cuda:0")
assert config.device == "cuda:0" assert config.device == "cuda:0"
def test_default_lang_code(self) -> None: def test_default_language(self) -> None:
config = EngineConfig() config = EngineConfig()
assert config.lang_code == "a" assert config.language == Language.EN_US
def test_custom_lang_code(self) -> None: def test_custom_language(self) -> None:
config = EngineConfig(lang_code="j") config = EngineConfig(language=Language.JA)
assert config.lang_code == "j" assert config.language == Language.JA
def test_immutability(self) -> None: def test_immutability(self) -> None:
config = EngineConfig() config = EngineConfig()
with pytest.raises(AttributeError): with pytest.raises(AttributeError):
config.device = "cuda:0" # type: ignore[misc] config.device = "cuda:0" # type: ignore[misc]
def test_immutability_lang_code(self) -> None: def test_immutability_language(self) -> None:
config = EngineConfig() config = EngineConfig()
with pytest.raises(AttributeError): with pytest.raises(AttributeError):
config.lang_code = "j" # type: ignore[misc] config.language = Language.JA # type: ignore[misc]
def test_unknown_keys_ignored_per_spec(self) -> None: def test_unknown_keys_ignored_per_spec(self) -> None:
"""Architecture spec: Unknown keys are ignored (no error). """Architecture spec: Unknown keys are ignored (no error).
@@ -225,11 +226,11 @@ class TestEngineConfigContract:
EngineConfig may contain fields that are not relevant to every plugin. EngineConfig may contain fields that are not relevant to every plugin.
Plugins MUST ignore fields they do not need, not raise on them. Plugins MUST ignore fields they do not need, not raise on them.
""" """
config = EngineConfig(device="cuda:0", lang_code="j") config = EngineConfig(device="cuda:0", language=Language.JA)
assert config.device == "cuda:0" assert config.device == "cuda:0"
assert config.lang_code == "j" assert config.language == Language.JA
# A plugin that only needs device simply reads config.device # A plugin that only needs device simply reads config.device
# and ignores config.lang_code — this must not raise. # and ignores config.language — this must not raise.
def test_engine_config_contains_engine_instance_configuration(self) -> None: def test_engine_config_contains_engine_instance_configuration(self) -> None:
"""Architecture Amendment #1: EngineConfig definition. """Architecture Amendment #1: EngineConfig definition.
@@ -238,7 +239,7 @@ class TestEngineConfigContract:
Engine instance is created and that remain constant throughout Engine instance is created and that remain constant throughout
the lifetime of that Engine. the lifetime of that Engine.
""" """
config = EngineConfig(device="cpu", lang_code="a") config = EngineConfig(device="cpu", language=Language.EN_US)
# Both fields are init-time, immutable, engine-scoped. # Both fields are init-time, immutable, engine-scoped.
assert config.device == "cpu" assert config.device == "cpu"
assert config.lang_code == "a" assert config.language == Language.EN_US
+102
View File
@@ -0,0 +1,102 @@
"""Tests for application/chapter_selection.py."""
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from dataclasses import dataclass
from abogen.application.chapter_selection import build_chapter_payload
@dataclass
class FakeChapter:
title: str
text: str
class TestBuildChapterPayload:
def test_empty_chapters(self):
result = build_chapter_payload([], source_name="book.txt")
assert len(result) == 1
assert result[0]["id"] == "0000"
assert result[0]["title"] == "book.txt"
assert result[0]["text"] == ""
assert result[0]["characters"] == 0
assert result[0]["enabled"] is True
def test_single_chapter_always_enabled(self):
chapters = [FakeChapter("Chapter 1", "Once upon a time.")]
result = build_chapter_payload(chapters)
assert len(result) == 1
assert result[0]["title"] == "Chapter 1"
assert result[0]["enabled"] is True
assert result[0]["index"] == 0
assert result[0]["id"] == "0000"
def test_content_chapters_preselected(self):
chapters = [
FakeChapter("Chapter 1", "The story begins with a long enough text to pass the threshold."),
FakeChapter("Chapter 2", "The story continues with another substantial chunk of text."),
]
result = build_chapter_payload(chapters)
assert all(ch["enabled"] for ch in result)
def test_supplement_not_preselected(self):
chapters = [
FakeChapter("Chapter 1", "The story begins with a long enough text to pass the threshold."),
FakeChapter("Title Page", ""),
FakeChapter("Copyright", "All rights reserved."),
FakeChapter("Table of Contents", ""),
FakeChapter("Chapter 2", "The story continues with another substantial chunk of text."),
]
result = build_chapter_payload(chapters)
titles_enabled = {ch["title"]: ch["enabled"] for ch in result}
assert titles_enabled["Chapter 1"] is True
assert titles_enabled["Chapter 2"] is True
assert titles_enabled["Title Page"] is False
assert titles_enabled["Copyright"] is False
assert titles_enabled["Table of Contents"] is False
def test_at_least_one_enabled(self):
chapters = [
FakeChapter("Title Page", ""),
FakeChapter("Copyright", "All rights reserved."),
]
result = build_chapter_payload(chapters)
assert any(ch["enabled"] for ch in result)
def test_characters_calculated(self):
chapters = [FakeChapter("Ch1", "Hello world")]
result = build_chapter_payload(chapters)
assert result[0]["characters"] == 11
def test_ids_are_zero_padded(self):
chapters = [FakeChapter(f"Ch{i}", f"text {i}") for i in range(5)]
result = build_chapter_payload(chapters)
ids = [ch["id"] for ch in result]
assert ids == ["0000", "0001", "0002", "0003", "0004"]
def test_indices_are_sequential(self):
chapters = [FakeChapter(f"Ch{i}", f"text {i}") for i in range(3)]
result = build_chapter_payload(chapters)
indices = [ch["index"] for ch in result]
assert indices == [0, 1, 2]
def test_source_name_used_for_empty(self):
result = build_chapter_payload([], source_name="mybook.epub")
assert result[0]["title"] == "mybook.epub"
def test_default_source_name(self):
result = build_chapter_payload([])
assert result[0]["title"] == ""
def test_none_title_and_text(self):
class BadChapter:
def __init__(self):
self.title = None
self.text = None
result = build_chapter_payload([BadChapter()])
assert result[0]["title"] == ""
assert result[0]["text"] == ""
assert result[0]["enabled"] is True # single chapter always enabled
+130 -61
View File
@@ -11,6 +11,12 @@ from unittest.mock import MagicMock, patch
import numpy as np import numpy as np
import pytest import pytest
from abogen.application.conversion_config import (
CoverConfig,
PronunciationConfig,
SaveConfig,
SubtitleConfig,
)
from abogen.application.conversion_request import ConversionRequest from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_models import ( from abogen.application.conversion_models import (
ChapterPlan, ChapterPlan,
@@ -48,7 +54,7 @@ class FakeBackend:
def __init__(self): def __init__(self):
self.synthesized: List[str] = [] self.synthesized: List[str] = []
def __call__(self, text: str, *, voice: Any, speed: float = 1.0, split_pattern: str = "") -> List: def __call__(self, text: str, *, voice: Any, speed: float = 1.0, split_pattern: str = "", **kwargs: Any) -> List:
self.synthesized.append(text) self.synthesized.append(text)
class FakeSegment: class FakeSegment:
@@ -106,6 +112,23 @@ class FakeVoiceResolver:
) )
@pytest.fixture(autouse=True)
def _mock_pool_and_resolver():
"""Mock PipelinePool and _create_voice_resolver for all service tests."""
fake_pool = FakePipelineProvider()
fake_resolver = FakeVoiceResolver()
with patch(
"abogen.domain.pipeline_factory.PipelinePool",
return_value=fake_pool,
), patch(
"abogen.domain.voice_loader.VoiceCache",
), patch(
"abogen.application.conversion_service._create_voice_resolver",
return_value=fake_resolver,
):
yield
# ─── Tests for conversion_service.py ─────────────────────────────── # ─── Tests for conversion_service.py ───────────────────────────────
@@ -120,14 +143,11 @@ class TestConversionService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello world", direct_text="Hello world",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
events = FakeEvents() events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
result = run_conversion(req, events, pipeline, resolver) result = run_conversion(req, events)
assert result is not None assert result is not None
assert result.audio_path is not None assert result.audio_path is not None
@@ -141,14 +161,11 @@ class TestConversionService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
events = FakeEvents() events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
result = run_conversion(req, events, pipeline, resolver) result = run_conversion(req, events)
log_messages = [msg for msg, _ in events.logs] log_messages = [msg for msg, _ in events.logs]
assert any("Preparing conversion pipeline" in msg for msg in log_messages) assert any("Preparing conversion pipeline" in msg for msg in log_messages)
@@ -164,16 +181,13 @@ class TestConversionService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
events = FakeEvents() events = FakeEvents()
events.cancelled = True events.cancelled = True
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
with pytest.raises(RuntimeError, match="Conversion cancelled"): with pytest.raises(RuntimeError, match="Conversion cancelled"):
run_conversion(req, events, pipeline, resolver) run_conversion(req, events)
def test_service_handles_empty_text(self): def test_service_handles_empty_text(self):
"""Service raises ValueError for empty text.""" """Service raises ValueError for empty text."""
@@ -181,11 +195,9 @@ class TestConversionService:
req = ConversionRequest(direct_text="", voice="M1") req = ConversionRequest(direct_text="", voice="M1")
events = FakeEvents() events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
with pytest.raises(ValueError, match="No text content"): with pytest.raises(ValueError, match="No text content"):
run_conversion(req, events, pipeline, resolver) run_conversion(req, events)
def test_service_multi_chapter(self): def test_service_multi_chapter(self):
"""Service handles multi-chapter conversion.""" """Service handles multi-chapter conversion."""
@@ -195,14 +207,11 @@ class TestConversionService:
req = ConversionRequest( req = ConversionRequest(
direct_text="<<CHAPTER_MARKER:Ch1>>\nText A\n<<CHAPTER_MARKER:Ch2>>\nText B", direct_text="<<CHAPTER_MARKER:Ch1>>\nText A\n<<CHAPTER_MARKER:Ch2>>\nText B",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
events = FakeEvents() events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
result = run_conversion(req, events, pipeline, resolver) result = run_conversion(req, events)
assert result.total_chapters == 2 assert result.total_chapters == 2
@@ -214,17 +223,14 @@ class TestConversionService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Body text", direct_text="Body text",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
read_title_intro=True, read_title_intro=True,
read_closing_outro=True, read_closing_outro=True,
metadata_tags={"title": "Test Book", "author": "Author"}, metadata_tags={"title": "Test Book", "author": "Author"},
) )
events = FakeEvents() events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
result = run_conversion(req, events, pipeline, resolver) result = run_conversion(req, events)
assert result is not None assert result is not None
@@ -234,17 +240,67 @@ class TestConversionService:
req = ConversionRequest(direct_text="Hello", voice="M1") req = ConversionRequest(direct_text="Hello", voice="M1")
events = FakeEvents() events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
# Mock build_conversion_plan to raise an error # Mock build_conversion_plan to raise an error
with patch("abogen.application.conversion_service.build_conversion_plan", side_effect=RuntimeError("Test error")): with patch("abogen.application.conversion_service.build_conversion_plan", side_effect=RuntimeError("Test error")):
with pytest.raises(RuntimeError, match="Test error"): with pytest.raises(RuntimeError, match="Test error"):
run_conversion(req, events, pipeline, resolver) run_conversion(req, events)
log_messages = [msg for msg, _ in events.logs] log_messages = [msg for msg, _ in events.logs]
assert any("Conversion failed" in msg for msg in log_messages) assert any("Conversion failed" in msg for msg in log_messages)
def test_tts_context_applies_normalization_overrides(self):
"""Service applies normalization_overrides from request to apostrophe config."""
from abogen.application.conversion_service import run_conversion
with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest(
direct_text="Hello",
voice="M1",
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
pronunciation=PronunciationConfig(
normalization_overrides={"normalization_numbers": False},
),
)
events = FakeEvents()
result = run_conversion(req, events)
assert result is not None
def test_tts_context_rejects_unconfigured_llm_mode(self):
"""Service raises RuntimeError if LLM apostrophe mode is selected but unconfigured."""
from abogen.application.conversion_service import run_conversion
with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest(
direct_text="Hello",
voice="M1",
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
pronunciation=PronunciationConfig(
normalization_overrides={"normalization_apostrophe_mode": "llm"},
),
)
events = FakeEvents()
with pytest.raises(RuntimeError, match="LLM.*apostrophe"):
run_conversion(req, events)
def test_usage_counter_populated_in_result(self):
"""usage_counter is created and accessible in result."""
from abogen.application.conversion_service import run_conversion
with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest(
direct_text="Hello",
voice="M1",
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
)
events = FakeEvents()
result = run_conversion(req, events)
assert hasattr(result, "usage_counter")
assert isinstance(result.usage_counter, dict)
# ─── Tests for output_layout_service.py ───────────────────────────── # ─── Tests for output_layout_service.py ─────────────────────────────
@@ -260,8 +316,7 @@ class TestOutputLayoutService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
layout = resolve_output_layout(req) layout = resolve_output_layout(req)
@@ -278,7 +333,7 @@ class TestOutputLayoutService:
req = ConversionRequest( req = ConversionRequest(
source_path=source, source_path=source,
voice="M1", voice="M1",
save_mode="save_next_to_input", save=SaveConfig(mode="save_next_to_input"),
) )
layout = resolve_output_layout(req) layout = resolve_output_layout(req)
@@ -292,9 +347,11 @@ class TestOutputLayoutService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(
output_folder=Path(tmpdir), mode="custom_folder",
save_as_project=True, output_folder=Path(tmpdir),
save_as_project=True,
),
original_filename="test.wav", original_filename="test.wav",
) )
layout = resolve_output_layout(req) layout = resolve_output_layout(req)
@@ -334,7 +391,7 @@ class TestOutputLayoutService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
separate_chapters_format="wav", save=SaveConfig(separate_chapters_format="wav"),
) )
path = resolve_chapter_path(layout, req, "Chapter 1", 1) path = resolve_chapter_path(layout, req, "Chapter 1", 1)
@@ -353,7 +410,7 @@ class TestOutputLayoutService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
separate_chapters_format="wav", save=SaveConfig(separate_chapters_format="wav"),
) )
path = resolve_chapter_path(layout, req, "", 3) path = resolve_chapter_path(layout, req, "", 3)
@@ -367,7 +424,7 @@ class TestOutputLayoutService:
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
output_format="m4b", output_format="m4b",
merge_chapters_at_end=False, save=SaveConfig(merge_chapters_at_end=False),
) )
assert should_merge_output(req) is True assert should_merge_output(req) is True
@@ -378,7 +435,7 @@ class TestOutputLayoutService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_chapters_separately=False, save=SaveConfig(save_chapters_separately=False),
) )
assert should_merge_output(req) is True assert should_merge_output(req) is True
@@ -389,8 +446,10 @@ class TestOutputLayoutService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_chapters_separately=True, save=SaveConfig(
merge_chapters_at_end=True, save_chapters_separately=True,
merge_chapters_at_end=True,
),
) )
assert should_merge_output(req) is True assert should_merge_output(req) is True
@@ -401,8 +460,10 @@ class TestOutputLayoutService:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_chapters_separately=True, save=SaveConfig(
merge_chapters_at_end=False, save_chapters_separately=True,
merge_chapters_at_end=False,
),
) )
assert should_merge_output(req) is False assert should_merge_output(req) is False
@@ -432,19 +493,28 @@ class TestExecutorGaps:
with pytest.raises(ValueError, match="output_layout"): with pytest.raises(ValueError, match="output_layout"):
execute_conversion(plan, events, pipeline, resolver, tts_context) execute_conversion(plan, events, pipeline, resolver, tts_context)
def test_executor_m4b_forces_merge(self): @patch("subprocess.Popen")
def test_executor_m4b_forces_merge(self, mock_popen):
"""Executor forces merge for m4b format.""" """Executor forces merge for m4b format."""
from abogen.application.conversion_executor import execute_conversion from abogen.application.conversion_executor import execute_conversion
mock_proc = mock_popen.return_value
mock_proc.returncode = 0
mock_proc.wait.return_value = 0
mock_proc.stdin = MagicMock()
mock_proc.communicate.return_value = (b"", b"")
with tempfile.TemporaryDirectory() as tmpdir: with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(
output_folder=Path(tmpdir), mode="custom_folder",
output_folder=Path(tmpdir),
save_chapters_separately=True,
merge_chapters_at_end=False,
),
output_format="m4b", output_format="m4b",
save_chapters_separately=True,
merge_chapters_at_end=False,
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -485,10 +555,12 @@ class TestExecutorGaps:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(
output_folder=Path(tmpdir), mode="custom_folder",
save_chapters_separately=True, output_folder=Path(tmpdir),
merge_chapters_at_end=True, save_chapters_separately=True,
merge_chapters_at_end=True,
),
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -538,8 +610,7 @@ class TestExecutorGaps:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -596,8 +667,7 @@ class TestExecutorGaps:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -637,8 +707,7 @@ class TestExecutorGaps:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
silence_between_chapters=1.0, silence_between_chapters=1.0,
) )
plan = ConversionPlan( plan = ConversionPlan(
+2 -1
View File
@@ -928,10 +928,11 @@ class TestValueObjectsBehavioral:
def test_engine_config_defaults(self) -> None: def test_engine_config_defaults(self) -> None:
from abogen.tts_plugin.types import EngineConfig from abogen.tts_plugin.types import EngineConfig
from abogen.domain.enums import Language
config = EngineConfig() config = EngineConfig()
assert config.device == "cpu" assert config.device == "cpu"
assert config.lang_code == "a" assert config.language == Language.EN_US
def test_parameter_values_defaults(self) -> None: def test_parameter_values_defaults(self) -> None:
pv = ParameterValues() pv = ParameterValues()
-195
View File
@@ -1,195 +0,0 @@
from __future__ import annotations
import sys
import types
def _install_dependency_stubs() -> None:
if "ebooklib" not in sys.modules:
ebooklib_stub = types.ModuleType("ebooklib")
epub_stub = types.ModuleType("ebooklib.epub")
setattr(ebooklib_stub, "epub", epub_stub)
sys.modules["ebooklib"] = ebooklib_stub
sys.modules["ebooklib.epub"] = epub_stub
if "dotenv" not in sys.modules:
dotenv_stub = types.ModuleType("dotenv")
def _noop(*_, **__):
return None
setattr(dotenv_stub, "load_dotenv", _noop)
setattr(dotenv_stub, "find_dotenv", lambda *_, **__: "")
sys.modules["dotenv"] = dotenv_stub
if "numpy" not in sys.modules:
numpy_stub = types.ModuleType("numpy")
class _DummyArray(list):
pass
def _zeros(shape, dtype=None):
size = 1
if isinstance(shape, int):
size = shape
elif shape:
size = 1
for dimension in shape:
size *= int(dimension)
return [0.0] * size
setattr(numpy_stub, "ndarray", _DummyArray)
setattr(numpy_stub, "zeros", _zeros)
setattr(numpy_stub, "float32", "float32")
setattr(numpy_stub, "array", lambda data, dtype=None: data)
setattr(numpy_stub, "asarray", lambda data, dtype=None: data)
setattr(
numpy_stub,
"concatenate",
lambda seq, axis=0: sum((list(item) for item in seq), []),
)
sys.modules["numpy"] = numpy_stub
if "soundfile" not in sys.modules:
soundfile_stub = types.ModuleType("soundfile")
class _DummySoundFile:
def __init__(self, *_, **__):
pass
def write(self, *_args, **_kwargs):
return None
def close(self):
return None
setattr(soundfile_stub, "SoundFile", _DummySoundFile)
setattr(soundfile_stub, "write", lambda *_args, **_kwargs: None)
sys.modules["soundfile"] = soundfile_stub
if "fitz" not in sys.modules:
sys.modules["fitz"] = types.ModuleType("fitz")
if "markdown" not in sys.modules:
markdown_stub = types.ModuleType("markdown")
class _DummyMarkdown:
def __init__(self, *_, **__):
pass
def convert(self, text: str) -> str:
return text
setattr(markdown_stub, "Markdown", _DummyMarkdown)
sys.modules["markdown"] = markdown_stub
if "bs4" not in sys.modules:
bs4_stub = types.ModuleType("bs4")
class _DummySoup:
def __init__(self, *_, **__):
pass
def select(self, *_, **__):
return []
def find_all(self, *_, **__):
return []
setattr(bs4_stub, "BeautifulSoup", _DummySoup)
setattr(bs4_stub, "NavigableString", str)
sys.modules["bs4"] = bs4_stub
_install_dependency_stubs()
from abogen.text_extractor import ExtractedChapter
from abogen.webui.conversion_runner import _apply_chapter_overrides, _merge_metadata
def _sample_chapters() -> list[ExtractedChapter]:
return [
ExtractedChapter(title="Chapter 1", text="Original one"),
ExtractedChapter(title="Chapter 2", text="Original two"),
ExtractedChapter(title="Chapter 3", text="Original three"),
]
def test_apply_chapter_overrides_with_custom_text() -> None:
overrides = [
{"index": 0, "enabled": True, "title": "Intro", "text": "Hello world"},
{"index": 1, "enabled": False},
]
selected, metadata, diagnostics = _apply_chapter_overrides(
_sample_chapters(), overrides
)
assert len(selected) == 1
assert selected[0].title == "Intro"
assert selected[0].text == "Hello world"
assert overrides[0]["characters"] == len("Hello world")
assert metadata == {}
assert diagnostics == []
def test_apply_chapter_overrides_uses_original_content_when_text_missing() -> None:
overrides = [
{"index": 1, "enabled": True},
]
selected, metadata, diagnostics = _apply_chapter_overrides(
_sample_chapters(), overrides
)
assert len(selected) == 1
assert selected[0].title == "Chapter 2"
assert selected[0].text == "Original two"
assert overrides[0]["text"] == "Original two"
assert overrides[0]["characters"] == len("Original two")
assert metadata == {}
assert diagnostics == []
def test_apply_chapter_overrides_collects_metadata_updates() -> None:
overrides = [
{
"index": 2,
"enabled": True,
"metadata": {"artist": "Test Author", "year": 2024},
}
]
selected, metadata, diagnostics = _apply_chapter_overrides(
_sample_chapters(), overrides
)
assert len(selected) == 1
assert metadata == {"artist": "Test Author", "year": "2024"}
assert diagnostics == []
def test_apply_chapter_overrides_reports_diagnostics_for_invalid_payload() -> None:
overrides = [
{"enabled": True, "title": "Missing"},
]
selected, metadata, diagnostics = _apply_chapter_overrides(
_sample_chapters(), overrides
)
assert selected == []
assert metadata == {}
assert diagnostics and "Skipped chapter override" in diagnostics[0]
def test_merge_metadata_prefers_overrides_and_drops_none_values() -> None:
extracted = {"title": "Original", "artist": "Someone"}
overrides = {"artist": "Another", "genre": "Fiction", "year": None}
merged = _merge_metadata(extracted, overrides)
assert merged["title"] == "Original"
assert merged["artist"] == "Another"
assert merged["genre"] == "Fiction"
assert "year" not in merged
-500
View File
@@ -1,500 +0,0 @@
"""Tests for conversion adapters (WebUI + PyQt).
Covers field mapping, event bridging, voice resolution, and cancellation behavior.
"""
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_ports import ResolvedVoice
# ─── WebUI adapter tests ───────────────────────────────────────────
class TestWebUIAdapter:
"""Test WebUI conversion adapter field mapping."""
def _make_job(self, **overrides):
"""Create a mock WebUI Job with default values."""
defaults = dict(
stored_path="/tmp/test.epub",
original_filename="test.epub",
language="a",
tts_provider="kokoro",
voice="M1",
voice_profile=None,
speed=1.0,
use_gpu=False,
supertonic_total_steps=5,
output_format="wav",
subtitle_mode="Disabled",
subtitle_format="srt",
max_subtitle_words=50,
save_mode="save_next_to_input",
output_folder=None,
save_chapters_separately=False,
merge_chapters_at_end=True,
separate_chapters_format="wav",
save_as_project=False,
silence_between_chapters=2.0,
chapter_intro_delay=0.0,
replace_single_newlines=False,
read_title_intro=False,
read_closing_outro=False,
auto_prefix_chapter_titles=True,
normalize_chapter_opening_caps=False,
pronunciation_overrides=[],
manual_overrides=[],
heteronym_overrides=[],
normalization_overrides={},
chapters=[],
chunks=[],
chunk_level="paragraph",
speaker_mode="single",
speakers={},
metadata_tags={},
cover_image_path=None,
cover_image_mime=None,
generate_epub3=False,
cancel_requested=False,
)
defaults.update(overrides)
return SimpleNamespace(**defaults)
def test_basic_field_mapping(self):
from abogen.webui.conversion_adapter import build_conversion_request_from_job
job = self._make_job()
req = build_conversion_request_from_job(job)
assert isinstance(req, ConversionRequest)
assert req.source_path == Path("/tmp/test.epub")
assert req.original_filename == "test.epub"
assert req.language == "a"
assert req.voice == "M1"
assert req.speed == 1.0
assert req.output_format == "wav"
def test_optional_fields_mapped(self):
from abogen.webui.conversion_adapter import build_conversion_request_from_job
job = self._make_job(
voice_profile="custom_profile",
output_folder="/output",
cover_image_path="/cover.jpg",
cover_image_mime="image/jpeg",
metadata_tags={"title": "Test"},
)
req = build_conversion_request_from_job(job)
assert req.voice_profile == "custom_profile"
assert req.output_folder == Path("/output")
assert req.cover_image_path == Path("/cover.jpg")
assert req.cover_image_mime == "image/jpeg"
assert req.metadata_tags == {"title": "Test"}
def test_none_paths_result_in_none(self):
from abogen.webui.conversion_adapter import build_conversion_request_from_job
job = self._make_job(
stored_path=None,
output_folder=None,
cover_image_path=None,
)
req = build_conversion_request_from_job(job)
assert req.source_path is None
assert req.output_folder is None
assert req.cover_image_path is None
def test_chapter_overrides_mapped(self):
from abogen.webui.conversion_adapter import build_conversion_request_from_job
chapters = [{"title": "Ch1", "voice": "F1"}]
job = self._make_job(chapters=chapters)
req = build_conversion_request_from_job(job)
assert req.chapter_overrides == chapters
def test_chunks_mapped(self):
from abogen.webui.conversion_adapter import build_conversion_request_from_job
chunks = [{"text": "Hello", "speaker": "A"}]
job = self._make_job(chunks=chunks)
req = build_conversion_request_from_job(job)
assert req.chunks == chunks
def test_pronunciation_overrides_mapped(self):
from abogen.webui.conversion_adapter import build_conversion_request_from_job
job = self._make_job(
pronunciation_overrides=["word=pron"],
manual_overrides=["manual=override"],
heteronym_overrides=["read=reed"],
)
req = build_conversion_request_from_job(job)
assert req.pronunciation_overrides == ["word=pron"]
assert req.manual_overrides == ["manual=override"]
assert req.heteronym_overrides == ["read=reed"]
def test_none_defaults_handled(self):
from abogen.webui.conversion_adapter import build_conversion_request_from_job
from abogen.domain.enums import Language, OutputFormat, SubtitleMode, SaveMode
job = self._make_job(
language=None,
voice=None,
speed=None,
output_format=None,
subtitle_mode=None,
save_mode=None,
silence_between_chapters=None,
chapter_intro_delay=None,
supertonic_total_steps=None,
max_subtitle_words=None,
)
req = build_conversion_request_from_job(job)
# None values pass through adapter; ConversionRequest.__post_init__
# applies defaults and clamping for numeric fields.
assert req.language == Language.EN_US
assert req.speed == 1.0
assert req.output_format == OutputFormat.WAV
assert req.subtitle_mode == SubtitleMode.DISABLED
assert req.save_mode == SaveMode.SAVE_NEXT_TO_INPUT
assert req.silence_between_chapters == 2.0
assert req.chapter_intro_delay == 0.0
assert req.supertonic_total_steps == 5
assert req.max_subtitle_words == 50
class TestWebUIEvents:
"""Test WebUI ConversionEvents implementation."""
def test_log_calls_add_log(self):
from abogen.webui.conversion_adapter import WebJobEvents
job = SimpleNamespace(add_log=MagicMock())
events = WebJobEvents(job)
events.log("test message", level="info")
job.add_log.assert_called_once_with("test message", level="info")
def test_progress_updates_job(self):
from abogen.webui.conversion_adapter import WebJobEvents
job = SimpleNamespace(progress=0.0, etr_str="")
events = WebJobEvents(job)
events.progress(50, "2m 30s")
assert job.progress == 0.5
assert job.etr_str == "2m 30s"
def test_check_cancelled_raises(self):
from abogen.webui.conversion_adapter import ConversionCancelled, WebJobEvents
job = SimpleNamespace(cancel_requested=True)
events = WebJobEvents(job)
with pytest.raises(ConversionCancelled):
events.check_cancelled()
def test_check_not_cancelled_passes(self):
from abogen.webui.conversion_adapter import WebJobEvents
job = SimpleNamespace(cancel_requested=False)
events = WebJobEvents(job)
events.check_cancelled() # Should not raise
class TestWebUIPipelineProvider:
"""Test WebUI PipelineProvider implementation."""
def test_get_returns_backend(self):
from abogen.webui.conversion_adapter import WebPipelineProvider
backend = MagicMock()
pool = SimpleNamespace(get=MagicMock(return_value=backend))
provider = WebPipelineProvider(pool)
result = provider.get("kokoro", "a", False)
assert result is backend
pool.get.assert_called_once_with("kokoro", "a", False)
class TestWebUIVoiceResolver:
"""Test WebUI VoiceResolver implementation."""
def test_resolve_returns_resolved_voice(self):
from abogen.webui.conversion_adapter import WebVoiceResolver
def resolve_fn(spec):
return ("kokoro", spec, "M1", 1.0, 5)
resolver = WebVoiceResolver(resolve_fn)
result = resolver.resolve("M1")
assert isinstance(result, ResolvedVoice)
assert result.provider == "kokoro"
assert result.voice == "M1"
assert result.speed == 1.0
assert result.supertonic_steps == 5
def test_resolve_none_speed_defaults(self):
from abogen.webui.conversion_adapter import WebVoiceResolver
def resolve_fn(spec):
return ("kokoro", spec, "M1", None, None)
resolver = WebVoiceResolver(resolve_fn)
result = resolver.resolve("M1")
assert result.speed == 1.0
assert result.supertonic_steps == 5
# ─── PyQt adapter tests ────────────────────────────────────────────
class TestPyQtAdapter:
"""Test PyQt conversion adapter field mapping."""
def _make_thread(self, **overrides):
"""Create a mock ConversionThread with default values."""
defaults = dict(
file_name="/tmp/test.epub",
lang_code="a",
voice="M1",
voice_profile=None,
speed=1.0,
use_gpu=False,
supertonic_total_steps=5,
output_format="wav",
subtitle_mode="Disabled",
subtitle_format="srt",
max_subtitle_words=50,
save_option="save_next_to_input",
output_folder=None,
save_chapters_separately=False,
merge_chapters_at_end=True,
separate_chapters_format="wav",
save_as_project=False,
silence_duration=2.0,
chapter_intro_delay=0.0,
replace_single_newlines=False,
read_title_intro=False,
read_closing_outro=True,
auto_prefix_chapter_titles=True,
normalize_chapter_opening_caps=False,
pronunciation_overrides=[],
manual_overrides=[],
heteronym_overrides=[],
normalization_overrides=None,
metadata_tags={},
cover_image_path=None,
cover_image_mime=None,
generate_epub3=False,
is_direct_text=False,
from_queue=False,
display_path=None,
save_base_path=None,
cancel_requested=False,
)
defaults.update(overrides)
return SimpleNamespace(**defaults)
def test_basic_field_mapping(self):
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
thread = self._make_thread()
req = build_conversion_request_from_thread(thread)
assert isinstance(req, ConversionRequest)
assert req.source_path == Path("/tmp/test.epub")
assert req.language == "a"
assert req.voice == "M1"
assert req.speed == 1.0
assert req.output_format == "wav"
def test_direct_text_mode(self):
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
thread = self._make_thread(
is_direct_text=True,
file_name="Hello world",
)
req = build_conversion_request_from_thread(thread)
assert req.source_path is None
assert req.direct_text == "Hello world"
def test_from_queue_uses_save_base_path(self):
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
thread = self._make_thread(
from_queue=True,
save_base_path="/queue/book.epub",
display_path="/display/book.epub",
)
req = build_conversion_request_from_thread(thread)
assert req.original_filename == "book.epub"
def test_display_path_used_when_not_from_queue(self):
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
thread = self._make_thread(
from_queue=False,
display_path="/display/book.epub",
save_base_path="/queue/book.epub",
)
req = build_conversion_request_from_thread(thread)
assert req.original_filename == "book.epub"
def test_output_folder_mapped(self):
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
thread = self._make_thread(output_folder="/output")
req = build_conversion_request_from_thread(thread)
assert req.output_folder == Path("/output")
def test_none_defaults_handled(self):
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
from abogen.domain.enums import Language, OutputFormat, SubtitleMode, SaveMode
thread = self._make_thread(
lang_code=None,
voice=None,
speed=None,
output_format=None,
subtitle_mode=None,
save_option=None,
silence_duration=None,
chapter_intro_delay=None,
supertonic_total_steps=None,
max_subtitle_words=None,
)
req = build_conversion_request_from_thread(thread)
# None values pass through adapter; ConversionRequest.__post_init__
# applies defaults and clamping for numeric fields.
assert req.language == Language.EN_US
assert req.speed == 1.0
assert req.output_format == OutputFormat.WAV
assert req.subtitle_mode == SubtitleMode.DISABLED
assert req.save_mode == SaveMode.SAVE_NEXT_TO_INPUT
assert req.silence_between_chapters == 2.0
assert req.chapter_intro_delay == 0.0
assert req.supertonic_total_steps == 5
assert req.max_subtitle_words == 50
def test_chapter_chunks_not_mapped(self):
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
thread = self._make_thread()
req = build_conversion_request_from_thread(thread)
assert req.chapter_overrides == []
assert req.chunks == []
assert req.chunk_level == "paragraph"
assert req.speaker_mode == "single"
assert req.speakers == {}
class TestPyQtEvents:
"""Test PyQt ConversionEvents implementation."""
def test_log_emits_signal(self):
from abogen.pyqt.conversion_adapter import PyQtEvents
thread = SimpleNamespace(
log_updated=MagicMock(),
)
events = PyQtEvents(thread)
events.log("test message", level="info")
thread.log_updated.emit.assert_called_once()
def test_progress_emits_signal(self):
from abogen.pyqt.conversion_adapter import PyQtEvents
thread = SimpleNamespace(
progress_updated=MagicMock(),
)
events = PyQtEvents(thread)
events.progress(50, "2m 30s")
thread.progress_updated.emit.assert_called_once_with(50, "2m 30s")
def test_check_cancelled_raises(self):
from abogen.pyqt.conversion_adapter import ConversionCancelled, PyQtEvents
thread = SimpleNamespace(cancel_requested=True)
events = PyQtEvents(thread)
with pytest.raises(ConversionCancelled):
events.check_cancelled()
def test_check_not_cancelled_passes(self):
from abogen.pyqt.conversion_adapter import PyQtEvents
thread = SimpleNamespace(cancel_requested=False)
events = PyQtEvents(thread)
events.check_cancelled() # Should not raise
class TestPyQtPipelineProvider:
"""Test PyQt PipelineProvider implementation."""
def test_get_returns_backend(self):
from abogen.pyqt.conversion_adapter import PyQtPipelineProvider
backend = MagicMock()
provider = PyQtPipelineProvider(backend)
result = provider.get("kokoro", "a", False)
assert result is backend
def test_dispose_all_noop(self):
from abogen.pyqt.conversion_adapter import PyQtPipelineProvider
backend = MagicMock()
provider = PyQtPipelineProvider(backend)
provider.dispose_all() # Should not raise
class TestPyQtVoiceResolver:
"""Test PyQt VoiceResolver implementation."""
def test_resolve_returns_resolved_voice(self):
from abogen.pyqt.conversion_adapter import PyQtVoiceResolver
loaded_voice = MagicMock()
thread = SimpleNamespace(
load_voice_cached=MagicMock(return_value=loaded_voice),
backend=MagicMock(),
speed=1.0,
supertonic_total_steps=5,
)
resolver = PyQtVoiceResolver(thread)
result = resolver.resolve("M1")
assert isinstance(result, ResolvedVoice)
assert result.provider == "kokoro"
assert result.voice is loaded_voice
assert result.speed == 1.0
assert result.supertonic_steps == 5
-240
View File
@@ -1,240 +0,0 @@
import sys
import types
if "soundfile" not in sys.modules:
soundfile_stub = types.ModuleType("soundfile")
class _SoundFileStub: # pragma: no cover - placeholder to satisfy imports
def __init__(self, *args: object, **kwargs: object) -> None:
raise RuntimeError("soundfile is not installed in the test environment")
soundfile_stub.SoundFile = _SoundFileStub # type: ignore[attr-defined]
sys.modules["soundfile"] = soundfile_stub
if "static_ffmpeg" not in sys.modules:
sys.modules["static_ffmpeg"] = types.ModuleType("static_ffmpeg")
if "ebooklib" not in sys.modules:
ebooklib_stub = types.ModuleType("ebooklib")
ebooklib_epub_stub = types.ModuleType("ebooklib.epub")
ebooklib_stub.epub = ebooklib_epub_stub # type: ignore[attr-defined]
sys.modules["ebooklib"] = ebooklib_stub
sys.modules["ebooklib.epub"] = ebooklib_epub_stub
if "fitz" not in sys.modules:
sys.modules["fitz"] = types.ModuleType("fitz")
if "markdown" not in sys.modules:
markdown_stub = types.ModuleType("markdown")
class _MarkdownStub:
def __init__(self, *args: object, **kwargs: object) -> None:
self.toc_tokens = []
def convert(self, text: str) -> str:
return text
markdown_stub.Markdown = _MarkdownStub # type: ignore[attr-defined]
sys.modules["markdown"] = markdown_stub
if "bs4" not in sys.modules:
bs4_stub = types.ModuleType("bs4")
class _BeautifulSoupStub:
def __init__(self, *args: object, **kwargs: object) -> None:
self._text = ""
def find(self, *args: object, **kwargs: object) -> None:
return None
def get_text(self) -> str:
return self._text
def decompose(self) -> None: # pragma: no cover - compatibility shim
return None
class _NavigableStringStub(str):
pass
bs4_stub.BeautifulSoup = _BeautifulSoupStub # type: ignore[attr-defined]
bs4_stub.NavigableString = _NavigableStringStub # type: ignore[attr-defined]
sys.modules["bs4"] = bs4_stub
from abogen.webui.conversion_runner import (
_format_spoken_chapter_title,
_headings_equivalent,
_normalize_chapter_opening_caps,
_strip_duplicate_heading_line,
)
def test_format_spoken_chapter_title_adds_prefix() -> None:
assert _format_spoken_chapter_title("1: A Tale", 1, True) == "Chapter 1. A Tale"
def test_format_spoken_chapter_title_respects_existing_prefix() -> None:
assert (
_format_spoken_chapter_title("Chapter 2: Story", 2, True) == "Chapter 2: Story"
)
def test_format_spoken_chapter_title_handles_empty_title() -> None:
assert _format_spoken_chapter_title("", 4, True) == "Chapter 4"
def test_format_spoken_chapter_title_trims_delimiters() -> None:
assert (
_format_spoken_chapter_title("7 - Into the Wild", 7, True)
== "Chapter 7. Into the Wild"
)
def test_headings_equivalent_ignores_case_and_prefix() -> None:
assert _headings_equivalent("1: The House", "Chapter 1: The House")
def test_strip_duplicate_heading_line_removes_first_match() -> None:
text, removed = _strip_duplicate_heading_line(
"Chapter 3: Intro\nBody text", "Chapter 3: Intro"
)
assert removed is True
assert text.strip() == "Body text"
def test_normalize_chapter_opening_caps_basic_title() -> None:
normalized, changed = _normalize_chapter_opening_caps("ALL CAPS TITLE")
assert normalized == "All Caps Title"
assert changed is True
def test_normalize_chapter_opening_caps_respects_acronyms() -> None:
normalized, changed = _normalize_chapter_opening_caps("NASA MISSION LOG")
assert normalized == "NASA Mission Log"
assert changed is True
def test_normalize_chapter_opening_caps_handles_roman_numerals() -> None:
normalized, changed = _normalize_chapter_opening_caps("IV. THE RETURN")
assert normalized == "IV. The Return"
assert changed is True
def test_normalize_chapter_opening_caps_keeps_mixed_case() -> None:
normalized, changed = _normalize_chapter_opening_caps("Already Mixed Case")
assert normalized == "Already Mixed Case"
assert changed is False
class TestApplyChapterTextTransforms:
"""Tests for the combined heading-strip + opening-caps helper."""
def test_both_enabled_heading_matches(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"Chapter 1: The Beginning\nBody text here",
heading_text="Chapter 1: The Beginning",
raw_title="Chapter 1: The Beginning",
strip_heading=True,
normalize_caps=True,
)
assert heading_removed is True
assert "Body text here" in text
assert "Chapter 1" not in text
def test_heading_fallback_to_number(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"1. The Beginning\nBody text",
heading_text="Chapter 1: The Beginning",
raw_title="1: The Beginning",
strip_heading=True,
normalize_caps=False,
)
assert heading_removed is True
assert "Body text" in text
def test_only_heading_strip(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"Chapter 1: Title\nBody text",
heading_text="Chapter 1: Title",
raw_title="",
strip_heading=True,
normalize_caps=False,
)
assert heading_removed is True
assert caps_changed is False
def test_only_opening_caps(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"ALL CAPS START OF CHAPTER",
heading_text="Chapter 1",
raw_title="",
strip_heading=False,
normalize_caps=True,
)
assert heading_removed is False
assert caps_changed is True
assert text == "All Caps Start Of Chapter"
def test_both_disabled_no_change(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
original = "Some text here"
text, heading_removed, caps_changed = apply_chapter_text_transforms(
original,
heading_text="Chapter 1",
raw_title="",
strip_heading=False,
normalize_caps=False,
)
assert text == original
assert heading_removed is False
assert caps_changed is False
def test_heading_not_matching(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"Completely different text",
heading_text="Chapter 1: Title",
raw_title="",
strip_heading=True,
normalize_caps=False,
)
assert heading_removed is False
assert text == "Completely different text"
def test_empty_text(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"",
heading_text="Chapter 1",
raw_title="",
strip_heading=True,
normalize_caps=True,
)
assert text == ""
assert heading_removed is False
assert caps_changed is False
def test_both_enabled_text_only_has_caps(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"NASA MISSION LOG",
heading_text="Chapter 1",
raw_title="",
strip_heading=True,
normalize_caps=True,
)
assert heading_removed is False
assert caps_changed is True
assert text == "NASA Mission Log"
+14 -17
View File
@@ -14,6 +14,8 @@ from unittest.mock import MagicMock, patch
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any from typing import Any
from abogen.domain.config_types import SubtitleConfig
from abogen.domain.enums import Language, SubtitleMode
from abogen.domain.conversion_engine import ( from abogen.domain.conversion_engine import (
synthesize_text, synthesize_text,
SynthParams, SynthParams,
@@ -56,7 +58,7 @@ class FakeBackend:
self.segment_duration = segment_duration self.segment_duration = segment_duration
self.call_count = 0 self.call_count = 0
def __call__(self, text: str, voice: Any, speed: float = 1.0, split_pattern: str = ""): def __call__(self, text: str, voice: Any, speed: float = 1.0, split_pattern: str = "", **kwargs: Any):
self.call_count += 1 self.call_count += 1
# Return fake segment objects with required attributes # Return fake segment objects with required attributes
@dataclass @dataclass
@@ -252,9 +254,8 @@ class TestProcessAndWriteSubtitles:
process_and_write_subtitles( process_and_write_subtitles(
[], [],
writer, writer,
subtitle_mode="Sentence", subtitle=SubtitleConfig(mode=SubtitleMode.SENTENCE, max_words=5),
max_subtitle_words=5, language=Language.EN_US,
lang_code="a",
use_spacy_segmentation=False, use_spacy_segmentation=False,
fallback_end_time=10.0, fallback_end_time=10.0,
) )
@@ -269,9 +270,8 @@ class TestProcessAndWriteSubtitles:
process_and_write_subtitles( process_and_write_subtitles(
tokens, tokens,
writer, writer,
subtitle_mode="Sentence", subtitle=SubtitleConfig(mode=SubtitleMode.SENTENCE, max_words=5),
max_subtitle_words=5, language=Language.EN_US,
lang_code="a",
use_spacy_segmentation=False, use_spacy_segmentation=False,
fallback_end_time=2.0, fallback_end_time=2.0,
) )
@@ -291,9 +291,8 @@ class TestProcessAndWriteSubtitles:
process_and_write_subtitles( process_and_write_subtitles(
tokens, tokens,
writer, writer,
subtitle_mode="Line", subtitle=SubtitleConfig(mode=SubtitleMode.LINE, max_words=5),
max_subtitle_words=5, language=Language.EN_US,
lang_code="a",
use_spacy_segmentation=False, use_spacy_segmentation=False,
fallback_end_time=3.0, fallback_end_time=3.0,
) )
@@ -310,9 +309,8 @@ class TestProcessAndWriteSubtitles:
process_and_write_subtitles( process_and_write_subtitles(
tokens, tokens,
writer, writer,
subtitle_mode="Disabled", subtitle=SubtitleConfig(mode=SubtitleMode.DISABLED, max_words=5),
max_subtitle_words=5, language=Language.EN_US,
lang_code="a",
use_spacy_segmentation=False, use_spacy_segmentation=False,
fallback_end_time=2.0, fallback_end_time=2.0,
) )
@@ -348,7 +346,7 @@ class TestFullPipeline:
audio_sink=merged_sink, audio_sink=merged_sink,
subtitle_mode="Sentence", subtitle_mode="Sentence",
max_subtitle_words=5, max_subtitle_words=5,
lang_code="a", language=Language.EN_US,
use_spacy_segmentation=False, use_spacy_segmentation=False,
) )
@@ -365,9 +363,8 @@ class TestFullPipeline:
process_and_write_subtitles( process_and_write_subtitles(
tokens, tokens,
subtitle_writer, subtitle_writer,
subtitle_mode="Sentence", subtitle=SubtitleConfig(mode=SubtitleMode.SENTENCE, max_words=5),
max_subtitle_words=5, language=Language.EN_US,
lang_code="a",
use_spacy_segmentation=False, use_spacy_segmentation=False,
fallback_end_time=stats.current_time, fallback_end_time=stats.current_time,
) )
+393 -17
View File
@@ -12,6 +12,7 @@ from unittest.mock import MagicMock
import numpy as np import numpy as np
import pytest import pytest
from abogen.application.conversion_config import SaveConfig
from abogen.application.conversion_executor import execute_conversion from abogen.application.conversion_executor import execute_conversion
from abogen.application.conversion_models import ( from abogen.application.conversion_models import (
ChapterPlan, ChapterPlan,
@@ -72,7 +73,7 @@ class FakeBackend:
def __init__(self): def __init__(self):
self.synthesized: List[str] = [] self.synthesized: List[str] = []
def __call__(self, text: str, *, voice: Any, speed: float = 1.0, split_pattern: str = "") -> List: def __call__(self, text: str, *, voice: Any, speed: float = 1.0, split_pattern: str = "", **kwargs: Any) -> List:
"""Return fake TTS segments.""" """Return fake TTS segments."""
self.synthesized.append(text) self.synthesized.append(text)
@@ -150,8 +151,7 @@ class TestExecuteConversion:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello world", direct_text="Hello world",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -198,10 +198,7 @@ class TestExecuteConversion:
req = ConversionRequest( req = ConversionRequest(
direct_text="Text", direct_text="Text",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir), save_chapters_separately=True, merge_chapters_at_end=True),
output_folder=Path(tmpdir),
save_chapters_separately=True,
merge_chapters_at_end=True,
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -262,8 +259,7 @@ class TestExecuteConversion:
req = ConversionRequest( req = ConversionRequest(
direct_text="Text", direct_text="Text",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -315,8 +311,7 @@ class TestExecuteConversion:
req = ConversionRequest( req = ConversionRequest(
direct_text="Text", direct_text="Text",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -377,8 +372,7 @@ class TestExecuteConversion:
req = ConversionRequest( req = ConversionRequest(
direct_text="Text", direct_text="Text",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -424,8 +418,7 @@ class TestExecuteConversion:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello world", direct_text="Hello world",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -471,8 +464,7 @@ class TestExecuteConversion:
req = ConversionRequest( req = ConversionRequest(
direct_text="Text", direct_text="Text",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
plan = ConversionPlan( plan = ConversionPlan(
request=req, request=req,
@@ -511,3 +503,387 @@ class TestExecuteConversion:
assert result.metadata["title"] == "Test Book" assert result.metadata["title"] == "Test Book"
assert result.metadata["author"] == "Author" assert result.metadata["author"] == "Author"
class TestHeadingDedup:
"""Tests for heading dedup in executor."""
def test_heading_dedup_strips_matching_first_line(self):
"""When first segment matches heading, it should be stripped."""
with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest(
direct_text="Hello",
voice="M1",
auto_prefix_chapter_titles=True,
)
# Simulate: heading = "Chapter 1", first segment = "Chapter 1: The Beginning"
# headings_equivalent should match these
plan = ConversionPlan(
request=req,
metadata={},
chapters=[
ChapterPlan(
index=1,
title="Chapter 1",
original_title="Chapter 1",
body_text="Chapter 1: The Beginning\nBody text here",
segments=[
SegmentPlan(
text="Chapter 1: The Beginning",
voice_spec="M1",
kind="body",
source="chapter",
),
SegmentPlan(
text="Body text here",
voice_spec="M1",
kind="body",
source="chapter",
),
],
voice_spec="M1",
)
],
output_layout=OutputLayout(
parent_dir=Path(tmpdir),
audio_dir=Path(tmpdir),
),
)
events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
tts_context = TTSContext()
result = execute_conversion(
plan, events, pipeline, resolver, tts_context
)
# The executor should have logged the heading
log_messages = [m for m, _ in events.logs if "Title:" in m]
assert len(log_messages) >= 1
def test_heading_dedup_no_match_preserves_all(self):
"""When first segment doesn't match heading, nothing is stripped."""
with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest(
direct_text="Hello",
voice="M1",
auto_prefix_chapter_titles=True,
)
plan = ConversionPlan(
request=req,
metadata={},
chapters=[
ChapterPlan(
index=1,
title="Chapter 1",
original_title="Chapter 1",
body_text="Completely different text\nMore text",
segments=[
SegmentPlan(
text="Completely different text",
voice_spec="M1",
kind="body",
source="chapter",
),
SegmentPlan(
text="More text",
voice_spec="M1",
kind="body",
source="chapter",
),
],
voice_spec="M1",
)
],
output_layout=OutputLayout(
parent_dir=Path(tmpdir),
audio_dir=Path(tmpdir),
),
)
events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
tts_context = TTSContext()
result = execute_conversion(
plan, events, pipeline, resolver, tts_context
)
# Both segments should be synthesized (heading + 2 body segments)
assert result.total_segments >= 2
class TestMarkerCollector:
"""Tests for MarkerCollector."""
def test_chapter_marker_has_voices_list(self):
"""Chapter markers should have 'voices' as list of dicts."""
with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest(direct_text="Hello", voice="M1")
plan = ConversionPlan(
request=req,
metadata={},
chapters=[
ChapterPlan(
index=1,
title="Chapter 1",
original_title="Chapter 1",
body_text="Body text",
segments=[
SegmentPlan(
text="Body text",
voice_spec="M1",
kind="body",
source="chapter",
),
],
voice_spec="M1",
)
],
output_layout=OutputLayout(
parent_dir=Path(tmpdir),
audio_dir=Path(tmpdir),
),
)
events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
tts_context = TTSContext()
result = execute_conversion(plan, events, pipeline, resolver, tts_context)
assert len(result.chapter_markers) == 1
marker = result.chapter_markers[0]
assert "voices" in marker
assert isinstance(marker["voices"], list)
assert len(marker["voices"]) == 1
assert marker["voices"][0]["provider"] == "kokoro"
assert marker["voices"][0]["voice"] == "M1"
def test_outro_marker_recorded(self):
"""Outro should be recorded as a chapter marker."""
from abogen.application.conversion_models import IntroOutroSpec
with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest(direct_text="Hello", voice="M1")
plan = ConversionPlan(
request=req,
metadata={},
chapters=[
ChapterPlan(
index=1,
title="Chapter 1",
original_title="Chapter 1",
body_text="Body text",
segments=[
SegmentPlan(
text="Body text",
voice_spec="M1",
kind="body",
source="chapter",
),
],
voice_spec="M1",
)
],
outro=IntroOutroSpec(
enabled=True,
text="Thanks for listening",
voice_spec="M1",
kind="outro",
),
output_layout=OutputLayout(
parent_dir=Path(tmpdir),
audio_dir=Path(tmpdir),
),
)
events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
tts_context = TTSContext()
result = execute_conversion(plan, events, pipeline, resolver, tts_context)
# Should have chapter marker + outro marker
assert len(result.chapter_markers) == 2
outro_marker = result.chapter_markers[1]
assert outro_marker["title"] == "Outro"
assert "start" in outro_marker
assert "end" in outro_marker
assert outro_marker["end"] > outro_marker["start"]
def test_chunk_marker_voice_is_dict(self):
"""Chunk markers should have 'voice' as dict with provider."""
with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest(direct_text="Hello", voice="M1")
plan = ConversionPlan(
request=req,
metadata={},
chapters=[
ChapterPlan(
index=1,
title="Chapter 1",
original_title="Chapter 1",
body_text="Body text",
segments=[
SegmentPlan(
text="Body text",
voice_spec="M1",
kind="body",
source="chunk",
chunk_id="chunk_001",
chunk_index=0,
speaker_id="narrator",
level="paragraph",
),
],
voice_spec="M1",
)
],
output_layout=OutputLayout(
parent_dir=Path(tmpdir),
audio_dir=Path(tmpdir),
),
)
events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
tts_context = TTSContext()
result = execute_conversion(plan, events, pipeline, resolver, tts_context)
assert len(result.chunk_markers) == 1
chunk = result.chunk_markers[0]
assert isinstance(chunk["voice"], dict)
assert chunk["voice"]["provider"] == "kokoro"
assert chunk["voice"]["voice"] == "M1"
def test_multi_speaker_collects_unique_voices(self):
"""Multi-speaker chapters should collect all unique voices."""
with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest(direct_text="Hello", voice="M1")
plan = ConversionPlan(
request=req,
metadata={},
chapters=[
ChapterPlan(
index=1,
title="Chapter 1",
original_title="Chapter 1",
body_text="Body text",
segments=[
SegmentPlan(
text="Narrator speaks",
voice_spec="M1",
kind="body",
source="chapter",
),
SegmentPlan(
text="Character speaks",
voice_spec="F1",
kind="body",
source="chapter",
),
],
voice_spec="M1",
)
],
output_layout=OutputLayout(
parent_dir=Path(tmpdir),
audio_dir=Path(tmpdir),
),
)
events = FakeEvents()
pipeline = FakePipelineProvider()
resolver = FakeVoiceResolver()
tts_context = TTSContext()
result = execute_conversion(plan, events, pipeline, resolver, tts_context)
marker = result.chapter_markers[0]
assert len(marker["voices"]) == 2
voice_specs = {v["voice"] for v in marker["voices"]}
assert "M1" in voice_specs
assert "F1" in voice_specs
class TestFfmetadataVoiceFormat:
"""Tests for ffmetadata rendering with new voice format."""
def test_render_ffmetadata_with_voices_list(self):
"""ffmetadata should render voices list as comma-separated string."""
from abogen.infrastructure.exporters import ExportService
svc = ExportService()
chapters = [
{
"title": "Chapter 1",
"start": 0.0,
"end": 60.0,
"voices": [
{"provider": "kokoro", "voice": "M1"},
{"provider": "kokoro", "voice": "F1"},
],
}
]
content = svc.render_ffmetadata({}, chapters)
assert "voice=M1@kokoro, F1@kokoro" in content
def test_render_ffmetadata_with_empty_voices(self):
"""ffmetadata should handle empty voices list."""
from abogen.infrastructure.exporters import ExportService
svc = ExportService()
chapters = [
{
"title": "Chapter 1",
"start": 0.0,
"end": 60.0,
"voices": [],
}
]
content = svc.render_ffmetadata({}, chapters)
assert "voice=" not in content
class TestEpub3VoiceFormat:
"""Tests for EPUB3 voice handling with new format."""
def test_chunk_overlay_voice_is_dict(self):
"""ChunkOverlay should accept voice as dict."""
from abogen.epub3.exporter import ChunkOverlay
overlay = ChunkOverlay(
id="test",
text="hello",
original_text=None,
start=0.0,
end=1.0,
speaker_id="narrator",
voice={"provider": "kokoro", "voice": "M1"},
)
assert isinstance(overlay.voice, dict)
assert overlay.voice["provider"] == "kokoro"
def test_render_chunk_inline_with_voice_dict(self):
"""_render_chunk_inline should render voice dict as data-voice attribute."""
from abogen.epub3.exporter import ChunkOverlay, _render_chunk_inline
overlay = ChunkOverlay(
id="chunk_001",
text="Hello world",
original_text=None,
start=0.0,
end=1.0,
speaker_id="narrator",
voice={"provider": "kokoro", "voice": "M1"},
)
html = _render_chunk_inline(overlay)
assert 'data-voice="M1@kokoro"' in html
+122 -11
View File
@@ -25,6 +25,7 @@ from abogen.application.conversion_models import (
OutputLayout, OutputLayout,
SegmentPlan, SegmentPlan,
) )
from abogen.application.conversion_config import ChapterChunkConfig, WordSubstitutionConfig
from abogen.application.conversion_planner import build_conversion_plan from abogen.application.conversion_planner import build_conversion_plan
from abogen.application.conversion_request import ConversionRequest from abogen.application.conversion_request import ConversionRequest
@@ -74,10 +75,12 @@ class TestBuildConversionPlan:
req = ConversionRequest( req = ConversionRequest(
direct_text="Some text", direct_text="Some text",
voice="M1", voice="M1",
chunks=[ chapter_chunk=ChapterChunkConfig(
{"text": "Chunk 1", "speaker_id": "narrator"}, chunks=[
{"text": "Chunk 2", "speaker_id": "narrator"}, {"text": "Chunk 1", "speaker_id": "narrator"},
], {"text": "Chunk 2", "speaker_id": "narrator"},
],
),
) )
plan = build_conversion_plan(req) plan = build_conversion_plan(req)
@@ -92,11 +95,13 @@ class TestBuildConversionPlan:
req = ConversionRequest( req = ConversionRequest(
direct_text="Text", direct_text="Text",
voice="M1", voice="M1",
chunks=[ chapter_chunk=ChapterChunkConfig(
{"text": "Narrator speaks", "speaker_id": "narrator"}, chunks=[
{"text": "Character speaks", "speaker_id": "alice", "voice": "F1"}, {"text": "Narrator speaks", "speaker_id": "narrator"},
], {"text": "Character speaks", "speaker_id": "alice", "voice": "F1"},
speakers={"alice": {"voice": "F1"}}, ],
speakers={"alice": {"voice": "F1"}},
),
) )
plan = build_conversion_plan(req) plan = build_conversion_plan(req)
@@ -120,12 +125,12 @@ class TestBuildConversionPlan:
def test_output_layout(self): def test_output_layout(self):
"""Output layout is resolved from request.""" """Output layout is resolved from request."""
from abogen.application.conversion_config import SaveConfig
with tempfile.TemporaryDirectory() as tmpdir: with tempfile.TemporaryDirectory() as tmpdir:
req = ConversionRequest( req = ConversionRequest(
direct_text="Hello", direct_text="Hello",
voice="M1", voice="M1",
save_mode="custom_folder", save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
output_folder=Path(tmpdir),
) )
plan = build_conversion_plan(req) plan = build_conversion_plan(req)
@@ -198,6 +203,84 @@ class TestBuildConversionPlan:
assert plan.chapters[0].segments[0].kind == "body" assert plan.chapters[0].segments[0].kind == "body"
class TestWordSubstitution:
"""Tests for word substitution in the planner."""
def test_basic_substitution(self):
"""Single word substitution is applied."""
req = ConversionRequest(
direct_text="The quick brown fox",
voice="M1",
word_substitution=WordSubstitutionConfig(
substitutions_list="fox|cat",
),
)
plan = build_conversion_plan(req)
assert "cat" in plan.chapters[0].body_text
assert "fox" not in plan.chapters[0].body_text
def test_multiple_substitutions(self):
"""Multiple word substitutions are applied."""
req = ConversionRequest(
direct_text="The quick brown fox jumps",
voice="M1",
word_substitution=WordSubstitutionConfig(
substitutions_list="fox|cat\nquick|slow",
),
)
plan = build_conversion_plan(req)
text = plan.chapters[0].body_text
assert "cat" in text
assert "slow" in text
def test_substitution_preserves_chapter_markers(self):
"""Chapter markers are preserved during substitution."""
req = ConversionRequest(
direct_text="<<CHAPTER_MARKER:Ch1>>\nThe quick brown fox",
voice="M1",
word_substitution=WordSubstitutionConfig(
substitutions_list="fox|cat",
),
)
plan = build_conversion_plan(req)
assert len(plan.chapters) >= 1
assert "cat" in plan.chapters[0].body_text
def test_chunks_assigned_to_correct_chapter(self):
"""Chunks are grouped by chapter_index and only assigned to matching chapters."""
req = ConversionRequest(
direct_text="<<CHAPTER_MARKER:Ch1>>\nText A\n<<CHAPTER_MARKER:Ch2>>\nText B",
voice="M1",
chapter_chunk=ChapterChunkConfig(
chunks=[
{"text": "Ch1 chunk", "chapter_index": 0},
{"text": "Ch2 chunk", "chapter_index": 1},
],
),
)
plan = build_conversion_plan(req)
assert len(plan.chapters) == 2
# Ch1 should have only its chunk
ch1_texts = [s.text for s in plan.chapters[0].segments]
assert "Ch1 chunk" in ch1_texts
assert "Ch2 chunk" not in ch1_texts
# Ch2 should have only its chunk
ch2_texts = [s.text for s in plan.chapters[1].segments]
assert "Ch2 chunk" in ch2_texts
assert "Ch1 chunk" not in ch2_texts
def test_no_substitution_when_disabled(self):
"""No substitution when word_substitution is None."""
req = ConversionRequest(
direct_text="The quick brown fox",
voice="M1",
word_substitution=None,
)
plan = build_conversion_plan(req)
assert "fox" in plan.chapters[0].body_text
class TestPlannerWithFileSource: class TestPlannerWithFileSource:
"""Tests using actual file sources (not direct_text).""" """Tests using actual file sources (not direct_text)."""
@@ -558,3 +641,31 @@ class TestFeatureParity:
if output_format.lower() == "m4b": if output_format.lower() == "m4b":
merge_chapters_at_end = True merge_chapters_at_end = True
assert merge_chapters_at_end is True assert merge_chapters_at_end is True
class TestCapsNormalization:
"""Tests for caps normalization in planner."""
def test_caps_normalization_applied_when_enabled(self):
"""When normalize_chapter_opening_caps=True, body text is normalized."""
req = ConversionRequest(
direct_text="<<CHAPTER_MARKER:Chapter 1>>\nALL CAPS OPENING TEXT here",
voice="M1",
normalize_chapter_opening_caps=True,
)
plan = build_conversion_plan(req)
body = plan.chapters[0].body_text
# ALL CAPS should be normalized to Title Case
assert body != "ALL CAPS OPENING TEXT here"
assert "ALL CAPS" not in body
def test_caps_normalization_skipped_when_disabled(self):
"""When normalize_chapter_opening_caps=False, body text is unchanged."""
req = ConversionRequest(
direct_text="<<CHAPTER_MARKER:Chapter 1>>\nALL CAPS OPENING TEXT here",
voice="M1",
normalize_chapter_opening_caps=False,
)
plan = build_conversion_plan(req)
body = plan.chapters[0].body_text
assert "ALL CAPS OPENING TEXT" in body
+38 -31
View File
@@ -16,6 +16,14 @@ from pathlib import Path
from unittest.mock import MagicMock from unittest.mock import MagicMock
from abogen.application.conversion_request import ConversionRequest, ConversionRequestError from abogen.application.conversion_request import ConversionRequest, ConversionRequestError
from abogen.application.conversion_config import (
ChapterChunkConfig,
CoverConfig,
PronunciationConfig,
SaveConfig,
SubtitleConfig,
WordSubstitutionConfig,
)
from abogen.domain.enums import Language, OutputFormat, SaveMode, SubtitleFormat, SubtitleMode from abogen.domain.enums import Language, OutputFormat, SaveMode, SubtitleFormat, SubtitleMode
from abogen.domain.normalization import TTSContext from abogen.domain.normalization import TTSContext
from abogen.domain.settings_core import settings_defaults from abogen.domain.settings_core import settings_defaults
@@ -36,13 +44,13 @@ class TestConversionRequestBasics:
assert "merge_chapters_at_end" in defaults assert "merge_chapters_at_end" in defaults
def test_split_pattern_computation(self): def test_split_pattern_computation(self):
pattern = get_split_pattern("a", "Disabled") pattern = get_split_pattern(Language.EN_US, "Disabled")
assert isinstance(pattern, str) assert isinstance(pattern, str)
assert len(pattern) > 0 assert len(pattern) > 0
def test_split_pattern_varies_by_subtitle_mode(self): def test_split_pattern_varies_by_subtitle_mode(self):
pattern_disabled = get_split_pattern("a", "Disabled") pattern_disabled = get_split_pattern(Language.EN_US, "Disabled")
pattern_sentence = get_split_pattern("a", "Sentence") pattern_sentence = get_split_pattern(Language.EN_US, "Sentence")
# Different modes should produce different patterns # Different modes should produce different patterns
assert isinstance(pattern_disabled, str) assert isinstance(pattern_disabled, str)
assert isinstance(pattern_sentence, str) assert isinstance(pattern_sentence, str)
@@ -201,23 +209,11 @@ class TestConversionRequestValidation:
def test_defaults_are_valid(self): def test_defaults_are_valid(self):
req = ConversionRequest() req = ConversionRequest()
assert req.max_subtitle_words == 50 assert req.subtitle.max_words == 50
assert req.speed == 1.0 assert req.speed == 1.0
assert req.supertonic_total_steps == 5 assert req.supertonic_total_steps == 5
assert req.output_format == OutputFormat.WAV assert req.output_format == OutputFormat.WAV
assert req.subtitle_mode == SubtitleMode.DISABLED assert req.subtitle.mode == SubtitleMode.DISABLED
def test_max_subtitle_words_clamped_below_min(self):
req = ConversionRequest(max_subtitle_words=0)
assert req.max_subtitle_words == 1
def test_max_subtitle_words_clamped_above_max(self):
req = ConversionRequest(max_subtitle_words=999)
assert req.max_subtitle_words == 500
def test_max_subtitle_words_valid(self):
req = ConversionRequest(max_subtitle_words=100)
assert req.max_subtitle_words == 100
def test_speed_clamped_below_min(self): def test_speed_clamped_below_min(self):
req = ConversionRequest(speed=0.1) req = ConversionRequest(speed=0.1)
@@ -248,16 +244,12 @@ class TestConversionRequestValidation:
assert req.chapter_intro_delay == 0.0 assert req.chapter_intro_delay == 0.0
def test_invalid_chunk_level_raises(self): def test_invalid_chunk_level_raises(self):
with pytest.raises(ConversionRequestError, match="chunk_level"): with pytest.raises(ValueError, match="chunk_level"):
ConversionRequest(chunk_level="invalid") ConversionRequest(chapter_chunk=ChapterChunkConfig(chunk_level="invalid"))
def test_invalid_speaker_mode_raises(self): def test_invalid_speaker_mode_raises(self):
with pytest.raises(ConversionRequestError, match="speaker_mode"): with pytest.raises(ValueError, match="speaker_mode"):
ConversionRequest(speaker_mode="invalid") ConversionRequest(chapter_chunk=ChapterChunkConfig(speaker_mode="invalid"))
def test_invalid_max_subtitle_words_type_raises(self):
with pytest.raises(ConversionRequestError, match="max_subtitle_words"):
ConversionRequest(max_subtitle_words="not_a_number")
def test_invalid_speed_type_raises(self): def test_invalid_speed_type_raises(self):
with pytest.raises(ConversionRequestError, match="speed"): with pytest.raises(ConversionRequestError, match="speed"):
@@ -275,12 +267,27 @@ class TestConversionRequestValidation:
req = ConversionRequest( req = ConversionRequest(
language=Language.FR, language=Language.FR,
output_format=OutputFormat.MP3, output_format=OutputFormat.MP3,
subtitle_mode=SubtitleMode.SENTENCE, subtitle=SubtitleConfig(
subtitle_format=SubtitleFormat.ASS, mode=SubtitleMode.SENTENCE,
save_mode=SaveMode.CUSTOM_FOLDER, format=SubtitleFormat.ASS,
),
save=SaveConfig(mode=SaveMode.CUSTOM_FOLDER),
) )
assert req.language == Language.FR assert req.language == Language.FR
assert req.output_format == OutputFormat.MP3 assert req.output_format == OutputFormat.MP3
assert req.subtitle_mode == SubtitleMode.SENTENCE assert req.subtitle.mode == SubtitleMode.SENTENCE
assert req.subtitle_format == SubtitleFormat.ASS assert req.subtitle.format == SubtitleFormat.ASS
assert req.save_mode == SaveMode.CUSTOM_FOLDER assert req.save.mode == SaveMode.CUSTOM_FOLDER
def test_config_objects_constructed(self):
req = ConversionRequest(
subtitle=SubtitleConfig(max_words=100),
cover=CoverConfig(path=Path("/tmp/cover.jpg"), mime="image/jpeg"),
pronunciation=PronunciationConfig(normalization_overrides={"key": "val"}),
save=SaveConfig(save_as_project=True),
)
assert req.subtitle.max_words == 100
assert req.cover.path == Path("/tmp/cover.jpg")
assert req.cover.mime == "image/jpeg"
assert req.pronunciation.normalization_overrides == {"key": "val"}
assert req.save.save_as_project is True
-120
View File
@@ -1,120 +0,0 @@
import sys
import types
if "soundfile" not in sys.modules:
soundfile_stub = types.ModuleType("soundfile")
class _SoundFileStub: # pragma: no cover - placeholder to satisfy imports
def __init__(self, *args: object, **kwargs: object) -> None:
raise RuntimeError("soundfile is not installed in the test environment")
soundfile_stub.SoundFile = _SoundFileStub # type: ignore[attr-defined]
sys.modules["soundfile"] = soundfile_stub
if "static_ffmpeg" not in sys.modules:
sys.modules["static_ffmpeg"] = types.ModuleType("static_ffmpeg")
if "ebooklib" not in sys.modules:
ebooklib_stub = types.ModuleType("ebooklib")
ebooklib_epub_stub = types.ModuleType("ebooklib.epub")
ebooklib_stub.epub = ebooklib_epub_stub # type: ignore[attr-defined]
sys.modules["ebooklib"] = ebooklib_stub
sys.modules["ebooklib.epub"] = ebooklib_epub_stub
if "fitz" not in sys.modules:
sys.modules["fitz"] = types.ModuleType("fitz")
if "markdown" not in sys.modules:
markdown_stub = types.ModuleType("markdown")
class _MarkdownStub:
def __init__(self, *args: object, **kwargs: object) -> None:
self.toc_tokens = []
def convert(self, text: str) -> str:
return text
markdown_stub.Markdown = _MarkdownStub # type: ignore[attr-defined]
sys.modules["markdown"] = markdown_stub
if "bs4" not in sys.modules:
bs4_stub = types.ModuleType("bs4")
class _BeautifulSoupStub:
def __init__(self, *args: object, **kwargs: object) -> None:
self._text = ""
def find(self, *args: object, **kwargs: object) -> None:
return None
def get_text(self) -> str:
return self._text
def decompose(self) -> None: # pragma: no cover - compatibility shim
return None
class _NavigableStringStub(str):
pass
bs4_stub.BeautifulSoup = _BeautifulSoupStub # type: ignore[attr-defined]
bs4_stub.NavigableString = _NavigableStringStub # type: ignore[attr-defined]
sys.modules["bs4"] = bs4_stub
from abogen.webui.conversion_runner import _build_outro_text, _build_title_intro_text
def test_title_intro_includes_series_sentence() -> None:
metadata = {
"title": "Galactic Chronicles",
"author": "Jane Doe",
"series": "Chronicles",
"series_index": "2",
}
intro_text = _build_title_intro_text(metadata, "chronicles.mp3")
assert intro_text.startswith("Book 2 of the Chronicles.")
assert "Galactic Chronicles." in intro_text
assert "By Jane Doe." in intro_text
def test_series_sentence_skips_duplicate_article() -> None:
metadata = {
"title": "Iron Council",
"authors": "China Miéville",
"series": "The Bas-Lag",
"series_index": "3",
}
intro_text = _build_title_intro_text(metadata, "iron_council.mp3")
assert "Book 3 of The Bas-Lag." in intro_text
assert "of the The" not in intro_text
def test_outro_appends_series_information() -> None:
metadata = {
"title": "Abaddon's Gate",
"authors": "James S. A. Corey",
"series": "The Expanse",
"series_index": "3",
}
outro_text = _build_outro_text(metadata, "abaddon.mp3")
assert outro_text.startswith("The end of Abaddon's Gate from James S. A. Corey.")
assert outro_text.endswith("Book 3 of The Expanse.")
def test_series_number_preserves_decimal_positions() -> None:
metadata = {
"title": "Interlude",
"author": "Alex Writer",
"series": "Chronicles",
"series_index": "2.5",
}
intro_text = _build_title_intro_text(metadata, "interlude.mp3")
assert "Book 2.5 of the Chronicles." in intro_text
-52
View File
@@ -1,52 +0,0 @@
from types import SimpleNamespace
from typing import cast
from abogen.tts_plugin.utils import get_voices
from abogen.webui.conversion_runner import (
_chapter_voice_spec,
_chunk_voice_spec,
_collect_required_voice_ids,
)
from abogen.webui.service import Job
def _sample_job(formula: str) -> Job:
return cast(
Job,
SimpleNamespace(
voice="__custom_mix",
speakers={
"narrator": {
"resolved_voice": formula,
}
},
chapters=[],
chunks=[{}],
),
)
def test_chapter_voice_spec_uses_resolved_formula():
formula = "af_nova*0.7+am_liam*0.3"
job = _sample_job(formula)
assert _chapter_voice_spec(job, None) == formula
def test_chunk_voice_fallback_uses_resolved_formula():
formula = "af_nova*0.7+am_liam*0.3"
job = _sample_job(formula)
result = _chunk_voice_spec(job, {}, "")
assert result == formula
def test_voice_collection_includes_formula_components():
formula = "af_nova*0.7+am_liam*0.3"
job = _sample_job(formula)
voices = _collect_required_voice_ids(job)
assert {"af_nova", "am_liam"}.issubset(voices)
assert voices.issuperset(get_voices("kokoro"))
+82
View File
@@ -181,3 +181,85 @@ class TestTtsSegments:
)) ))
assert results[0].chunk_start == 10.0 assert results[0].chunk_start == 10.0
assert results[1].chunk_start == 11.0 assert results[1].chunk_start == 11.0
class TestSpacyPreTtsSegmentation:
"""Tests for spacy_pre_tts_segmentation()."""
def test_disabled_when_use_spacy_false(self):
from abogen.domain.conversion_pipeline import spacy_pre_tts_segmentation
from abogen.domain.enums import Language
segments, split = spacy_pre_tts_segmentation(
"Hello world",
Language.EN_US,
"Disabled",
use_spacy_segmentation=False,
)
assert segments == ["Hello world"]
assert isinstance(split, str)
def test_disabled_for_disabled_subtitle_mode(self):
from abogen.domain.conversion_pipeline import spacy_pre_tts_segmentation
from abogen.domain.enums import Language
segments, split = spacy_pre_tts_segmentation(
"Hello world",
Language.FR,
"Disabled",
use_spacy_segmentation=True,
)
assert segments == ["Hello world"]
def test_disabled_for_line_subtitle_mode(self):
from abogen.domain.conversion_pipeline import spacy_pre_tts_segmentation
from abogen.domain.enums import Language
segments, split = spacy_pre_tts_segmentation(
"Hello world",
Language.ES,
"Line",
use_spacy_segmentation=True,
)
assert segments == ["Hello world"]
def test_disabled_for_subtitle_input(self):
from abogen.domain.conversion_pipeline import spacy_pre_tts_segmentation
from abogen.domain.enums import Language
segments, split = spacy_pre_tts_segmentation(
"Hello world",
Language.FR,
"Sentence",
is_subtitle_input=True,
use_spacy_segmentation=True,
)
assert segments == ["Hello world"]
def test_english_excluded_from_pre_tts(self):
"""English uses spaCy only for post-TTS subtitles, not pre-TTS."""
from abogen.domain.conversion_pipeline import spacy_pre_tts_segmentation
from abogen.domain.enums import Language
segments, split = spacy_pre_tts_segmentation(
"Hello world. How are you?",
Language.EN_US,
"Sentence",
use_spacy_segmentation=True,
)
# English should return single segment (no pre-TTS segmentation)
assert len(segments) == 1
def test_returns_at_least_one_segment(self):
from unittest.mock import patch
from abogen.domain.conversion_pipeline import spacy_pre_tts_segmentation
from abogen.domain.enums import Language
with patch("abogen.spacy_utils.segment_sentences", return_value=None):
segments, split = spacy_pre_tts_segmentation(
"",
Language.FR,
"Sentence",
use_spacy_segmentation=True,
)
assert len(segments) >= 1
+160 -2
View File
@@ -1,8 +1,10 @@
"""Tests for domain/normalization.py — prepare_text_for_tts.""" """Tests for domain/normalization.py — prepare_text_for_tts, build_tts_context."""
import pytest import pytest
from unittest.mock import patch, MagicMock from unittest.mock import patch, MagicMock
from abogen.domain.normalization import prepare_text_for_tts, normalize_text_for_pipeline from abogen.domain.config_types import PronunciationConfig, SubtitleConfig
from abogen.domain.enums import Language, SubtitleMode
from abogen.domain.normalization import prepare_text_for_tts, normalize_text_for_pipeline, build_tts_context, TTSContext
class TestPrepareTextForTts: class TestPrepareTextForTts:
@@ -145,3 +147,159 @@ class TestNormalizeTextForPipeline:
normalization_overrides={"normalization_apostrophe_mode": "spacy"}, normalization_overrides={"normalization_apostrophe_mode": "spacy"},
) )
assert isinstance(result, str) assert isinstance(result, str)
class TestBuildTtsContext:
"""Tests for the build_tts_context factory."""
def test_returns_tts_context(self):
ctx = build_tts_context(language=Language.EN_US)
assert isinstance(ctx, TTSContext)
def test_default_split_pattern(self):
ctx = build_tts_context(language=Language.EN_US, subtitle="Disabled")
assert isinstance(ctx.split_pattern, str)
assert len(ctx.split_pattern) > 0
def test_english_uses_newline_split(self):
ctx = build_tts_context(language=Language.EN_US, subtitle="Disabled")
assert ctx.split_pattern == "\n"
def test_cjk_uses_punctuation_split(self):
ctx = build_tts_context(language=Language.JA, subtitle="Disabled")
assert r"\n" in ctx.split_pattern
def test_pronunciation_overrides_compiled(self):
overrides = [
{
"token": "epub",
"pronunciation": "ee-pub",
"normalized": "epub",
}
]
ctx = build_tts_context(
language=Language.EN_US,
pronunciation=PronunciationConfig(pronunciation_overrides=overrides),
)
assert ctx.pronunciation_rules is not None
assert len(ctx.pronunciation_rules) >= 1
def test_manual_overrides_included(self):
overrides = [
{
"token": "gif",
"pronunciation": "jif",
"normalized": "gif",
}
]
ctx = build_tts_context(
language=Language.EN_US,
pronunciation=PronunciationConfig(manual_overrides=overrides),
)
assert ctx.pronunciation_rules is not None
assert len(ctx.pronunciation_rules) >= 1
def test_manual_overrides_win_over_pronunciation(self):
pronunciation = [
{"token": "x", "pronunciation": "WRONG", "normalized": "x"}
]
manual = [
{"token": "x", "pronunciation": "RIGHT", "normalized": "x"}
]
ctx = build_tts_context(
language=Language.EN_US,
pronunciation=PronunciationConfig(
pronunciation_overrides=pronunciation,
manual_overrides=manual,
),
)
found_right = any(
r.get("replacement") == "RIGHT" for r in ctx.pronunciation_rules
)
found_wrong = any(
r.get("replacement") == "WRONG" for r in ctx.pronunciation_rules
)
assert found_right
assert not found_wrong
def test_heteronym_overrides_compiled(self):
overrides = [
{
"token": "read",
"pronunciation": "red",
"context": "past tense",
}
]
ctx = build_tts_context(
language=Language.EN_US,
pronunciation=PronunciationConfig(heteronym_overrides=overrides),
)
assert ctx.heteronym_rules is not None
def test_usage_counter_passed_through(self):
counter = {}
ctx = build_tts_context(language=Language.EN_US, usage_counter=counter)
assert ctx.usage_counter is counter
def test_usage_counter_default_empty(self):
ctx = build_tts_context(language=Language.EN_US)
assert ctx.usage_counter == {}
def test_normalization_overrides_stored(self):
overrides = {"normalization_numbers": False}
ctx = build_tts_context(
language=Language.EN_US,
pronunciation=PronunciationConfig(normalization_overrides=overrides),
)
assert ctx.normalization_overrides is overrides
def test_speakers_used_for_pronunciation(self):
speakers = {
"narrator": {
"token": "route",
"pronunciation": "root",
"resolved_voice": "M1",
}
}
ctx = build_tts_context(language=Language.EN_US, speakers=speakers)
assert ctx.pronunciation_rules is not None
assert len(ctx.pronunciation_rules) >= 1
def test_log_callback_called_on_num2words_missing(self):
logs = []
with patch("abogen.domain.normalization.get_runtime_settings", return_value={
"normalization_apostrophe_mode": "spacy",
"normalization_enabled": True,
"normalization_numbers": True,
}):
with patch("abogen.normalization_settings.build_apostrophe_config") as mock_cfg:
mock_cfg.return_value = MagicMock(convert_numbers=True)
with patch("builtins.__import__", side_effect=ImportError):
try:
build_tts_context(language=Language.EN_US, log_callback=lambda lvl, msg: logs.append((lvl, msg)))
except ImportError:
pass
# If num2words is missing and convert_numbers is True, a warning should be logged
# (depends on mock behavior, so just check no crash)
def test_llm_mode_raises_if_not_configured(self):
with patch("abogen.domain.normalization.get_runtime_settings", return_value={
"normalization_apostrophe_mode": "llm",
}):
with pytest.raises(RuntimeError, match="LLM"):
build_tts_context(language=Language.EN_US)
def test_dict_source_accepted(self):
"""merge_pronunciation_overrides should accept a dict."""
source = {
"pronunciation_overrides": [
{"token": "test", "pronunciation": "test-est", "normalized": "test"}
],
"manual_overrides": [],
"speakers": {},
"language": "a",
}
from abogen.domain.pronunciation import merge_pronunciation_overrides
result = merge_pronunciation_overrides(source)
assert isinstance(result, list)
assert len(result) >= 1
+26 -27
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
from abogen.domain.enums import Language
from abogen.domain.pipeline_factory import ( from abogen.domain.pipeline_factory import (
PipelinePool, PipelinePool,
create_pipeline_for_job, create_pipeline_for_job,
@@ -31,8 +32,8 @@ class TestCreatePipelineForJob:
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True) @patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
def test_supertonic_provider(self, _reg, mock_create): def test_supertonic_provider(self, _reg, mock_create):
mock_create.return_value = MagicMock() mock_create.return_value = MagicMock()
result = create_pipeline_for_job("supertonic", "en", use_gpu=True) result = create_pipeline_for_job("supertonic", Language.EN_US, use_gpu=True)
mock_create.assert_called_once_with("supertonic") mock_create.assert_called_once_with("supertonic", language=Language.EN_US)
assert result is mock_create.return_value assert result is mock_create.return_value
@patch("abogen.domain.pipeline_factory.create_pipeline") @patch("abogen.domain.pipeline_factory.create_pipeline")
@@ -40,43 +41,41 @@ class TestCreatePipelineForJob:
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu") @patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_kokoro_provider(self, _dev, _reg, mock_create): def test_kokoro_provider(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock() mock_create.return_value = MagicMock()
result = create_pipeline_for_job("kokoro", "en", use_gpu=False) result = create_pipeline_for_job("kokoro", Language.EN_US, use_gpu=False)
# "en" → fallback to EN_US → kokoro code "a" mock_create.assert_called_once_with("kokoro", language=Language.EN_US, device="cpu")
mock_create.assert_called_once_with("kokoro", lang_code="a", device="cpu")
assert result is mock_create.return_value assert result is mock_create.return_value
@patch("abogen.domain.pipeline_factory.create_pipeline") @patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True) @patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu") @patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_kokoro_provider_iso_code(self, _dev, _reg, mock_create): def test_kokoro_provider_en_gb(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock() mock_create.return_value = MagicMock()
result = create_pipeline_for_job("kokoro", "en-GB", use_gpu=False) result = create_pipeline_for_job("kokoro", Language.EN_GB, use_gpu=False)
# "en-GB" → EN_GB → kokoro code "b" mock_create.assert_called_once_with("kokoro", language=Language.EN_GB, device="cpu")
mock_create.assert_called_once_with("kokoro", lang_code="b", device="cpu")
@patch("abogen.domain.pipeline_factory.create_pipeline") @patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=False) @patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=False)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu") @patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_unknown_provider_falls_back_to_kokoro(self, _dev, _reg, mock_create): def test_unknown_provider_falls_back_to_kokoro(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock() mock_create.return_value = MagicMock()
result = create_pipeline_for_job("unknown_provider", "en", use_gpu=False) result = create_pipeline_for_job("unknown_provider", Language.EN_US, use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="a", device="cpu") mock_create.assert_called_once_with("kokoro", language=Language.EN_US, device="cpu")
@patch("abogen.domain.pipeline_factory.create_pipeline") @patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True) @patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu") @patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_empty_provider_defaults_to_kokoro(self, _dev, _reg, mock_create): def test_empty_provider_defaults_to_kokoro(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock() mock_create.return_value = MagicMock()
result = create_pipeline_for_job("", "en", use_gpu=False) result = create_pipeline_for_job("", Language.EN_US, use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="a", device="cpu") mock_create.assert_called_once_with("kokoro", language=Language.EN_US, device="cpu")
@patch("abogen.domain.pipeline_factory.create_pipeline") @patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True) @patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu") @patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_none_provider_defaults_to_kokoro(self, _dev, _reg, mock_create): def test_none_provider_defaults_to_kokoro(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock() mock_create.return_value = MagicMock()
result = create_pipeline_for_job(None, "en", use_gpu=False) result = create_pipeline_for_job(None, Language.EN_US, use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="a", device="cpu") mock_create.assert_called_once_with("kokoro", language=Language.EN_US, device="cpu")
class TestDisposePipelines: class TestDisposePipelines:
@@ -110,11 +109,11 @@ class TestPipelinePool:
mock_create.return_value = mock_pipeline mock_create.return_value = mock_pipeline
pool = PipelinePool() pool = PipelinePool()
result = pool.get("kokoro", "en", use_gpu=True) result = pool.get("kokoro", Language.EN_US, use_gpu=True)
assert result is mock_pipeline assert result is mock_pipeline
mock_create.assert_called_once() mock_create.assert_called_once()
result2 = pool.get("kokoro", "en", use_gpu=True) result2 = pool.get("kokoro", Language.EN_US, use_gpu=True)
assert result2 is mock_pipeline assert result2 is mock_pipeline
assert mock_create.call_count == 1 assert mock_create.call_count == 1
@@ -124,11 +123,11 @@ class TestPipelinePool:
mock_create.return_value = MagicMock() mock_create.return_value = MagicMock()
pool = PipelinePool() pool = PipelinePool()
job = MagicMock() request = MagicMock()
pool.get("kokoro", "en", use_gpu=True, job=job) pool.get("kokoro", Language.EN_US, use_gpu=True, request=request)
assert mock_cache.call_count == 1 assert mock_cache.call_count == 1
pool.get("kokoro", "en", use_gpu=True, job=job) pool.get("kokoro", Language.EN_US, use_gpu=True, request=request)
assert mock_cache.call_count == 1 assert mock_cache.call_count == 1
@patch("abogen.domain.pipeline_factory.initialize_voice_cache") @patch("abogen.domain.pipeline_factory.initialize_voice_cache")
@@ -136,7 +135,7 @@ class TestPipelinePool:
def test_get_no_job_skips_voice_cache(self, mock_create, mock_cache): def test_get_no_job_skips_voice_cache(self, mock_create, mock_cache):
mock_create.return_value = MagicMock() mock_create.return_value = MagicMock()
pool = PipelinePool() pool = PipelinePool()
pool.get("kokoro", "en", use_gpu=True) pool.get("kokoro", Language.EN_US, use_gpu=True)
mock_cache.assert_not_called() mock_cache.assert_not_called()
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job") @patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@@ -146,8 +145,8 @@ class TestPipelinePool:
mock_create.side_effect = [p1, p2] mock_create.side_effect = [p1, p2]
pool = PipelinePool() pool = PipelinePool()
r1 = pool.get("kokoro", "en", use_gpu=True) r1 = pool.get("kokoro", Language.EN_US, use_gpu=True)
r2 = pool.get("supertonic", "en", use_gpu=True) r2 = pool.get("supertonic", Language.EN_US, use_gpu=True)
assert r1 is p1 assert r1 is p1
assert r2 is p2 assert r2 is p2
assert mock_create.call_count == 2 assert mock_create.call_count == 2
@@ -160,8 +159,8 @@ class TestPipelinePool:
mock_create.side_effect = [p1, p2] mock_create.side_effect = [p1, p2]
pool = PipelinePool() pool = PipelinePool()
pool.get("kokoro", "en", use_gpu=True) pool.get("kokoro", Language.EN_US, use_gpu=True)
pool.get("supertonic", "en", use_gpu=True) pool.get("supertonic", Language.EN_US, use_gpu=True)
pool.dispose_all() pool.dispose_all()
p1.dispose.assert_called_once() p1.dispose.assert_called_once()
@@ -181,5 +180,5 @@ class TestPipelinePool:
def test_unknown_provider_falls_back(self, _reg, _cache, mock_create): def test_unknown_provider_falls_back(self, _reg, _cache, mock_create):
mock_create.return_value = MagicMock() mock_create.return_value = MagicMock()
pool = PipelinePool() pool = PipelinePool()
pool.get("bogus_provider", "en", use_gpu=True) pool.get("bogus_provider", Language.EN_US, use_gpu=True)
mock_create.assert_called_once_with("kokoro", "en", True) mock_create.assert_called_once_with("kokoro", Language.EN_US, True)
+758
View File
@@ -0,0 +1,758 @@
"""Tests for domain speaker metadata functions.
Tests for build_narrator_roster, build_speaker_roster, match_configured_speaker,
apply_speaker_config_to_roster, and prepare_speaker_metadata.
"""
from __future__ import annotations
from typing import Any, Dict
from unittest.mock import patch, MagicMock
import pytest
# ---------------------------------------------------------------------------
# build_narrator_roster
# ---------------------------------------------------------------------------
class TestBuildNarratorRoster:
"""Tests for build_narrator_roster()."""
def test_basic_roster(self):
from abogen.domain.speaker_metadata import build_narrator_roster
roster = build_narrator_roster("af_heart", None)
assert "narrator" in roster
assert roster["narrator"]["voice"] == "af_heart"
assert roster["narrator"]["label"] == "Narrator"
def test_with_voice_profile(self):
from abogen.domain.speaker_metadata import build_narrator_roster
roster = build_narrator_roster("af_heart", "my_profile")
assert roster["narrator"]["voice_profile"] == "my_profile"
def test_without_voice_profile(self):
from abogen.domain.speaker_metadata import build_narrator_roster
roster = build_narrator_roster("af_heart", None)
assert "voice_profile" not in roster["narrator"]
def test_merges_existing_overrides(self):
from abogen.domain.speaker_metadata import build_narrator_roster
existing = {
"narrator": {
"label": "Custom Narrator",
"voice": "am_echo",
"pronunciation": "NAH-rah-tor",
}
}
roster = build_narrator_roster("af_heart", None, existing=existing)
assert roster["narrator"]["label"] == "Custom Narrator"
assert roster["narrator"]["voice"] == "am_echo"
assert roster["narrator"]["pronunciation"] == "NAH-rah-tor"
def test_existing_none_ignored(self):
from abogen.domain.speaker_metadata import build_narrator_roster
roster = build_narrator_roster("af_heart", None, existing=None)
assert roster["narrator"]["voice"] == "af_heart"
def test_empty_existing_dict(self):
from abogen.domain.speaker_metadata import build_narrator_roster
roster = build_narrator_roster("af_heart", None, existing={})
assert roster["narrator"]["voice"] == "af_heart"
def test_existing_without_narrator_key(self):
from abogen.domain.speaker_metadata import build_narrator_roster
existing = {"other_speaker": {"label": "Other"}}
roster = build_narrator_roster("af_heart", None, existing=existing)
assert roster["narrator"]["voice"] == "af_heart"
def test_empty_string_values_not_overridden(self):
from abogen.domain.speaker_metadata import build_narrator_roster
existing = {"narrator": {"label": "", "voice": ""}}
roster = build_narrator_roster("af_heart", None, existing=existing)
assert roster["narrator"]["label"] == "Narrator"
assert roster["narrator"]["voice"] == "af_heart"
# ---------------------------------------------------------------------------
# build_speaker_roster
# ---------------------------------------------------------------------------
class TestBuildSpeakerRoster:
"""Tests for build_speaker_roster()."""
def test_single_narrator(self):
from abogen.domain.speaker_metadata import build_speaker_roster
analysis = {"speakers": {"narrator": {"label": "Narrator", "count": 10}}}
roster = build_speaker_roster(analysis, "af_heart", None)
assert list(roster.keys()) == ["narrator"]
assert roster["narrator"]["voice"] == "af_heart"
def test_multiple_speakers(self):
from abogen.domain.speaker_metadata import build_speaker_roster
analysis = {
"speakers": {
"narrator": {"label": "Narrator", "count": 10},
"alice": {"label": "Alice", "count": 5, "gender": "female"},
"bob": {"label": "Bob", "count": 3, "gender": "male"},
}
}
roster = build_speaker_roster(analysis, "af_heart", None)
assert "narrator" in roster
assert "alice" in roster
assert "bob" in roster
assert roster["alice"]["label"] == "Alice"
assert roster["alice"]["gender"] == "female"
def test_suppressed_speakers_excluded(self):
from abogen.domain.speaker_metadata import build_speaker_roster
analysis = {
"speakers": {
"narrator": {"label": "Narrator", "count": 10},
"alice": {"label": "Alice", "count": 5},
"bob": {"label": "Bob", "count": 1, "suppressed": True},
}
}
roster = build_speaker_roster(analysis, "af_heart", None)
assert "bob" not in roster
def test_order_respected(self):
from abogen.domain.speaker_metadata import build_speaker_roster
analysis = {
"speakers": {
"narrator": {"label": "Narrator", "count": 10},
"alice": {"label": "Alice", "count": 5},
"bob": {"label": "Bob", "count": 3},
}
}
roster = build_speaker_roster(analysis, "af_heart", None, order=["bob", "alice"])
keys = list(roster.keys())
assert keys.index("bob") < keys.index("alice")
def test_existing_assignments_preserved(self):
from abogen.domain.speaker_metadata import build_speaker_roster
analysis = {
"speakers": {
"narrator": {"label": "Narrator", "count": 10},
"alice": {"label": "Alice", "count": 5, "gender": "female"},
}
}
existing = {
"narrator": {"voice": "am_echo"},
"alice": {"voice": "af_nicole", "pronunciation": "AH-leece"},
}
roster = build_speaker_roster(analysis, "af_heart", None, existing=existing)
assert roster["alice"]["voice"] == "af_nicole"
assert roster["alice"]["pronunciation"] == "AH-leece"
assert roster["narrator"]["voice"] == "am_echo"
def test_empty_analysis(self):
from abogen.domain.speaker_metadata import build_speaker_roster
roster = build_speaker_roster({}, "af_heart", None)
assert "narrator" in roster
assert len(roster) == 1
def test_sample_quotes_preserved(self):
from abogen.domain.speaker_metadata import build_speaker_roster
analysis = {
"speakers": {
"narrator": {"label": "Narrator", "count": 10},
"alice": {
"label": "Alice",
"count": 5,
"sample_quotes": ["Hello!", "Goodbye!"],
},
}
}
roster = build_speaker_roster(analysis, "af_heart", None)
assert roster["alice"]["sample_quotes"] == ["Hello!", "Goodbye!"]
def test_detected_gender_preserved(self):
from abogen.domain.speaker_metadata import build_speaker_roster
analysis = {
"speakers": {
"narrator": {"label": "Narrator", "count": 10},
"alice": {"label": "Alice", "count": 5, "detected_gender": "female"},
}
}
roster = build_speaker_roster(analysis, "af_heart", None)
assert roster["alice"]["detected_gender"] == "female"
def test_default_label_from_id(self):
from abogen.domain.speaker_metadata import build_speaker_roster
analysis = {
"speakers": {
"narrator": {"label": "Narrator", "count": 10},
"my_character": {"count": 3},
}
}
roster = build_speaker_roster(analysis, "af_heart", None)
assert roster["my_character"]["label"] == "My Character"
# ---------------------------------------------------------------------------
# match_configured_speaker
# ---------------------------------------------------------------------------
class TestMatchConfiguredSpeaker:
"""Tests for match_configured_speaker()."""
def test_match_by_id(self):
from abogen.domain.speaker_metadata import match_configured_speaker
config = {"alice": {"id": "alice", "label": "Alice", "voice": "af_heart"}}
result = match_configured_speaker(config, "alice", "Alice")
assert result is not None
assert result["voice"] == "af_heart"
def test_match_by_slug(self):
from abogen.domain.speaker_metadata import match_configured_speaker
config = {"my_character": {"id": "my_character", "label": "My Character"}}
result = match_configured_speaker(config, "my_character", "My Character")
assert result is not None
def test_match_by_label_lowercase(self):
from abogen.domain.speaker_metadata import match_configured_speaker
config = {"custom_id": {"id": "custom_id", "label": "Alice"}}
result = match_configured_speaker(config, "other_id", "Alice")
assert result is not None
assert result["id"] == "custom_id"
def test_no_match(self):
from abogen.domain.speaker_metadata import match_configured_speaker
config = {"alice": {"id": "alice", "label": "Alice"}}
result = match_configured_speaker(config, "bob", "Bob")
assert result is None
def test_empty_config(self):
from abogen.domain.speaker_metadata import match_configured_speaker
result = match_configured_speaker({}, "alice", "Alice")
assert result is None
def test_none_config(self):
from abogen.domain.speaker_metadata import match_configured_speaker
result = match_configured_speaker(None, "alice", "Alice") # type: ignore
assert result is None
# ---------------------------------------------------------------------------
# apply_speaker_config_to_roster
# ---------------------------------------------------------------------------
class TestApplySpeakerConfigToRoster:
"""Tests for apply_speaker_config_to_roster()."""
def test_no_config_returns_roster_unchanged(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {"narrator": {"id": "narrator", "voice": "af_heart"}}
result, languages, config = apply_speaker_config_to_roster(roster, None)
assert result["narrator"]["voice"] == "af_heart"
assert languages == []
assert config is None
def test_empty_config_returns_roster_unchanged(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {"narrator": {"id": "narrator", "voice": "af_heart"}}
result, languages, config = apply_speaker_config_to_roster(roster, {})
assert result["narrator"]["voice"] == "af_heart"
def test_config_without_speakers_map(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {"narrator": {"id": "narrator", "voice": "af_heart"}}
config = {"language": "a"}
result, languages, config = apply_speaker_config_to_roster(roster, config)
assert result["narrator"]["voice"] == "af_heart"
def test_applies_voice_from_config(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {
"narrator": {"id": "narrator", "voice": "af_heart"},
"alice": {"id": "alice", "label": "Alice", "voice": "af_heart"},
}
config = {
"speakers": {
"alice": {"id": "alice", "voice": "af_nicole", "gender": "female"}
},
"languages": ["a"],
}
result, languages, updated_config = apply_speaker_config_to_roster(
roster, config, persist_changes=True
)
assert result["alice"]["voice"] == "af_nicole"
assert result["alice"]["resolved_voice"] == "af_nicole"
def test_applies_voice_profile(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {
"narrator": {"id": "narrator", "voice": "af_heart"},
"alice": {"id": "alice", "label": "Alice"},
}
config = {
"speakers": {
"alice": {"id": "alice", "voice_profile": "my_profile"}
}
}
result, _, _ = apply_speaker_config_to_roster(roster, config)
assert result["alice"]["voice_profile"] == "my_profile"
def test_applies_voice_formula(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {
"narrator": {"id": "narrator", "voice": "af_heart"},
"alice": {"id": "alice", "label": "Alice"},
}
config = {
"speakers": {
"alice": {"id": "alice", "voice_formula": "af_heart(0.6)+af_nicole(0.4)"}
}
}
result, _, _ = apply_speaker_config_to_roster(roster, config)
assert result["alice"]["voice_formula"] == "af_heart(0.6)+af_nicole(0.4)"
assert result["alice"]["resolved_voice"] == "af_heart(0.6)+af_nicole(0.4)"
def test_persist_changes_returns_updated_config(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {
"narrator": {"id": "narrator", "voice": "af_heart"},
"alice": {"id": "alice", "label": "Alice", "voice": "af_heart"},
}
config = {
"language": "a",
"languages": ["a"],
"speakers": {
"alice": {"id": "alice", "voice": "af_nicole", "gender": "female"}
},
"version": 1,
}
_, _, updated_config = apply_speaker_config_to_roster(
roster, config, persist_changes=True
)
# config_changed is False by default, so updated_config should be None
# unless there's actual change logic triggered
# The function has config_changed = False and never sets it to True
# so updated_config should be None even with persist_changes=True
assert updated_config is None
def test_fallback_languages_used(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {"narrator": {"id": "narrator", "voice": "af_heart"}}
result, languages, _ = apply_speaker_config_to_roster(
roster, None, fallback_languages=["a", "b"]
)
assert languages == ["a", "b"]
def test_config_languages_take_precedence(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {"narrator": {"id": "narrator", "voice": "af_heart"}}
config = {"languages": ["a"], "speakers": {}}
_, languages, _ = apply_speaker_config_to_roster(
roster, config, fallback_languages=["a", "b"]
)
assert languages == ["a"]
def test_empty_roster_returns_empty(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
result, languages, config = apply_speaker_config_to_roster({}, None)
assert result == {}
assert languages == []
def test_non_mapping_roster_returns_empty(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
result, languages, config = apply_speaker_config_to_roster("invalid", None) # type: ignore
assert result == {}
assert languages == []
def test_narrator_not_modified(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {
"narrator": {"id": "narrator", "voice": "af_heart"},
"alice": {"id": "alice", "label": "Alice"},
}
config = {
"speakers": {
"narrator": {"id": "narrator", "voice": "am_echo"},
"alice": {"id": "alice", "voice": "af_nicole"},
}
}
result, _, _ = apply_speaker_config_to_roster(roster, config)
assert result["narrator"]["voice"] == "af_heart"
assert result["alice"]["voice"] == "af_nicole"
def test_config_languages_applied_to_roster_entry(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {
"narrator": {"id": "narrator", "voice": "af_heart"},
"alice": {"id": "alice", "label": "Alice"},
}
config = {
"languages": ["a", "b"],
"speakers": {
"alice": {"id": "alice", "voice": "af_nicole"}
},
}
result, _, _ = apply_speaker_config_to_roster(roster, config)
assert result["alice"]["config_languages"] == ["a", "b"]
def test_speaker_specific_languages_override(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {
"narrator": {"id": "narrator", "voice": "af_heart"},
"alice": {"id": "alice", "label": "Alice"},
}
config = {
"languages": ["a"],
"speakers": {
"alice": {"id": "alice", "voice": "af_nicole", "languages": ["a", "b"]}
},
}
result, _, _ = apply_speaker_config_to_roster(roster, config)
assert result["alice"]["config_languages"] == ["a", "b"]
def test_resolved_voice_takes_precedence(self):
from abogen.domain.speaker_metadata import apply_speaker_config_to_roster
roster = {
"narrator": {"id": "narrator", "voice": "af_heart"},
"alice": {"id": "alice", "label": "Alice"},
}
config = {
"speakers": {
"alice": {
"id": "alice",
"voice": "af_heart",
"resolved_voice": "af_nicole",
}
}
}
result, _, _ = apply_speaker_config_to_roster(roster, config)
assert result["alice"]["resolved_voice"] == "af_nicole"
# ---------------------------------------------------------------------------
# prepare_speaker_metadata
# ---------------------------------------------------------------------------
class TestPrepareSpeakerMetadata:
"""Tests for prepare_speaker_metadata()."""
def _make_chunks(self, count=3):
return [{"id": str(i), "text": f"Chunk {i}"} for i in range(count)]
def _make_chapters(self):
return [{"title": "Chapter 1", "chunks": self._make_chunks()}]
@patch("abogen.domain.speaker_metadata.load_settings")
def test_no_analysis(self, mock_settings):
from abogen.domain.speaker_metadata import prepare_speaker_metadata
mock_settings.return_value = {"speaker_random_languages": []}
chunks = self._make_chunks()
result = prepare_speaker_metadata(
chapters=self._make_chapters(),
chunks=chunks,
voice="af_heart",
voice_profile=None,
threshold=3,
run_analysis=False,
)
chunk_list, roster, analysis, languages, config = result
assert all(c["speaker_id"] == "narrator" for c in chunk_list)
assert all(c["speaker_label"] == "Narrator" for c in chunk_list)
assert "narrator" in roster
assert languages == []
assert config is None
@patch("abogen.domain.speaker_metadata.load_settings")
def test_no_analysis_with_existing_roster(self, mock_settings):
from abogen.domain.speaker_metadata import prepare_speaker_metadata
mock_settings.return_value = {"speaker_random_languages": []}
existing = {"narrator": {"voice": "am_echo", "pronunciation": "test"}}
result = prepare_speaker_metadata(
chapters=self._make_chapters(),
chunks=self._make_chunks(),
voice="af_heart",
voice_profile=None,
threshold=3,
run_analysis=False,
existing_roster=existing,
)
_, roster, _, _, _ = result
assert roster["narrator"]["voice"] == "am_echo"
assert roster["narrator"]["pronunciation"] == "test"
@patch("abogen.domain.speaker_metadata.load_settings")
@patch("abogen.domain.speaker_metadata.analyze_speakers")
def test_with_analysis(self, mock_analyze, mock_settings):
from abogen.domain.speaker_metadata import prepare_speaker_metadata
mock_settings.return_value = {"speaker_random_languages": []}
mock_result = MagicMock()
mock_result.to_dict.return_value = {
"version": "1.0",
"narrator": "narrator",
"assignments": {"0": "narrator", "1": "narrator", "2": "narrator"},
"speakers": {
"narrator": {
"label": "Narrator",
"count": 3,
"confidence": "low",
"sample_quotes": [],
"suppressed": False,
}
},
"suppressed": [],
"stats": {
"total_chunks": 3,
"explicit_chunks": 0,
"active_speakers": 0,
"unique_speakers": 1,
"suppressed": 0,
},
}
mock_analyze.return_value = mock_result
result = prepare_speaker_metadata(
chapters=self._make_chapters(),
chunks=self._make_chunks(),
voice="af_heart",
voice_profile=None,
threshold=3,
run_analysis=True,
)
chunk_list, roster, analysis, _, _ = result
assert "narrator" in roster
assert analysis["version"] == "1.0"
@patch("abogen.domain.speaker_metadata.load_settings")
@patch("abogen.domain.speaker_metadata.analyze_speakers")
def test_inject_recommended_callback_called(self, mock_analyze, mock_settings):
from abogen.domain.speaker_metadata import prepare_speaker_metadata
mock_settings.return_value = {"speaker_random_languages": []}
mock_result = MagicMock()
mock_result.to_dict.return_value = {
"version": "1.0",
"narrator": "narrator",
"assignments": {},
"speakers": {
"narrator": {
"label": "Narrator",
"count": 1,
"confidence": "low",
"sample_quotes": [],
"suppressed": False,
}
},
"suppressed": [],
"stats": {
"total_chunks": 1,
"explicit_chunks": 0,
"active_speakers": 0,
"unique_speakers": 1,
"suppressed": 0,
},
}
mock_analyze.return_value = mock_result
injected = []
callback = lambda roster, **kwargs: injected.append(dict(roster))
result = prepare_speaker_metadata(
chapters=self._make_chapters(),
chunks=self._make_chunks(1),
voice="af_heart",
voice_profile=None,
threshold=3,
run_analysis=True,
inject_recommended=callback,
)
assert len(injected) == 1
assert "narrator" in injected[0]
@patch("abogen.domain.speaker_metadata.load_settings")
def test_inject_recommended_not_called_when_none(self, mock_settings):
from abogen.domain.speaker_metadata import prepare_speaker_metadata
mock_settings.return_value = {"speaker_random_languages": []}
result = prepare_speaker_metadata(
chapters=self._make_chapters(),
chunks=self._make_chunks(),
voice="af_heart",
voice_profile=None,
threshold=3,
run_analysis=False,
inject_recommended=None,
)
assert result is not None
@patch("abogen.domain.speaker_metadata.load_settings")
def test_chunks_are_copies(self, mock_settings):
from abogen.domain.speaker_metadata import prepare_speaker_metadata
mock_settings.return_value = {"speaker_random_languages": []}
original_chunks = [{"id": "0", "text": "Hello"}]
result = prepare_speaker_metadata(
chapters=[{"title": "Ch1", "chunks": original_chunks}],
chunks=original_chunks,
voice="af_heart",
voice_profile=None,
threshold=3,
run_analysis=False,
)
chunk_list = result[0]
assert chunk_list is not original_chunks
assert chunk_list[0] is not original_chunks[0]
@patch("abogen.domain.speaker_metadata.load_settings")
def test_analysis_disabled_sets_narrator_on_all_chunks(self, mock_settings):
from abogen.domain.speaker_metadata import prepare_speaker_metadata
mock_settings.return_value = {"speaker_random_languages": []}
chunks = [{"id": "0"}, {"id": "1"}, {"id": "2"}]
result = prepare_speaker_metadata(
chapters=[{"title": "Ch1", "chunks": chunks}],
chunks=chunks,
voice="af_heart",
voice_profile=None,
threshold=3,
run_analysis=False,
)
for chunk in result[0]:
assert chunk["speaker_id"] == "narrator"
assert chunk["speaker_label"] == "Narrator"
@patch("abogen.domain.speaker_metadata.load_settings")
@patch("abogen.domain.speaker_metadata.analyze_speakers")
def test_speaker_random_languages_used(self, mock_analyze, mock_settings):
from abogen.domain.speaker_metadata import prepare_speaker_metadata
mock_settings.return_value = {"speaker_random_languages": ["a", "b"]}
mock_result = MagicMock()
mock_result.to_dict.return_value = {
"version": "1.0",
"narrator": "narrator",
"assignments": {},
"speakers": {
"narrator": {
"label": "Narrator",
"count": 1,
"confidence": "low",
"sample_quotes": [],
"suppressed": False,
}
},
"suppressed": [],
"stats": {
"total_chunks": 1,
"explicit_chunks": 0,
"active_speakers": 0,
"unique_speakers": 1,
"suppressed": 0,
},
}
mock_analyze.return_value = mock_result
result = prepare_speaker_metadata(
chapters=self._make_chapters(),
chunks=self._make_chunks(1),
voice="af_heart",
voice_profile=None,
threshold=3,
run_analysis=True,
)
_, _, analysis, _, _ = result
assert analysis["config_languages"] == ["a", "b"]
@patch("abogen.domain.speaker_metadata.load_settings")
@patch("abogen.domain.speaker_metadata.analyze_speakers")
def test_apply_config_with_speaker_config(self, mock_analyze, mock_settings):
from abogen.domain.speaker_metadata import prepare_speaker_metadata
mock_settings.return_value = {"speaker_random_languages": []}
mock_result = MagicMock()
mock_result.to_dict.return_value = {
"version": "1.0",
"narrator": "narrator",
"assignments": {"0": "narrator"},
"speakers": {
"narrator": {
"label": "Narrator",
"count": 1,
"confidence": "low",
"sample_quotes": [],
"suppressed": False,
}
},
"suppressed": [],
"stats": {
"total_chunks": 1,
"explicit_chunks": 0,
"active_speakers": 0,
"unique_speakers": 1,
"suppressed": 0,
},
}
mock_analyze.return_value = mock_result
speaker_config = {
"languages": ["a"],
"speakers": {
"narrator": {"id": "narrator", "voice": "am_echo"},
},
}
result = prepare_speaker_metadata(
chapters=self._make_chapters(),
chunks=self._make_chunks(1),
voice="af_heart",
voice_profile=None,
threshold=3,
run_analysis=True,
speaker_config=speaker_config,
apply_config=True,
)
_, roster, _, languages, _ = result
assert roster["narrator"]["voice"] == "af_heart"
assert languages == ["a"]
+44
View File
@@ -0,0 +1,44 @@
"""Tests for domain/text_utils.py."""
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from abogen.domain.text_utils import calculate_text_length
class TestCalculateTextUtilsLength:
def test_empty(self):
assert calculate_text_length("") == 0
def test_plain_text(self):
assert calculate_text_length("Hello world") == 11
def test_strips_newlines(self):
assert calculate_text_length("Hello\nworld") == 10
def test_strips_leading_trailing_spaces(self):
assert calculate_text_length(" Hello ") == 5
def test_strips_chapter_markers(self):
assert calculate_text_length("Hello<<CHAPTER_MARKER:intro>>world") == 10
def test_strips_voice_markers(self):
assert calculate_text_length("Hello<<VOICE:M1>>world") == 10
def test_strips_metadata_tags(self):
assert calculate_text_length("Hello<<METADATA_TITLE:My Book>>world") == 10
def test_strips_multiple_markers(self):
text = "<<CHAPTER_MARKER:ch1>>Hello<<VOICE:M1>> <<METADATA_TITLE:Book>>world"
assert calculate_text_length(text) == 11
def test_strips_mixed_content(self):
text = "<<CHAPTER_MARKER:ch1>>\nHello\n<<VOICE:M1>>\nworld\n"
assert calculate_text_length(text) == 10
def test_preserves_internal_spaces(self):
assert calculate_text_length("Hello world") == 11
def test_only_markers(self):
assert calculate_text_length("<<CHAPTER_MARKER:x>><<VOICE:y>>") == 0
+394
View File
@@ -0,0 +1,394 @@
"""Tests for domain voice resolution functions.
Tests for formula_from_profile, resolve_profile_voice, resolve_voice_setting,
resolve_voice_choice, build_voice_catalog, and filter_voice_catalog.
"""
from __future__ import annotations
from typing import Any, Dict
from unittest.mock import patch, MagicMock
import pytest
# ---------------------------------------------------------------------------
# formula_from_profile
# ---------------------------------------------------------------------------
class TestFormulaFromProfile:
"""Tests for formula_from_profile()."""
def test_kokoro_profile_with_voices(self):
from abogen.domain.voice_resolution import formula_from_profile
entry = {"voices": [("af_heart", 0.6), ("am_echo", 0.4)]}
result = formula_from_profile(entry)
assert result is not None
assert "af_heart" in result
assert "am_echo" in result
def test_empty_voices_returns_none(self):
from abogen.domain.voice_resolution import formula_from_profile
entry = {"voices": []}
assert formula_from_profile(entry) is None
def test_no_voices_key_returns_none(self):
from abogen.domain.voice_resolution import formula_from_profile
entry = {"language": "a"}
assert formula_from_profile(entry) is None
def test_none_entry_returns_none(self):
from abogen.domain.voice_resolution import formula_from_profile
assert formula_from_profile(None) is None # type: ignore
def test_non_dict_entry_returns_none(self):
from abogen.domain.voice_resolution import formula_from_profile
assert formula_from_profile("invalid") is None # type: ignore
def test_supertonic_profile_no_voices(self):
from abogen.domain.voice_resolution import formula_from_profile
entry = {"provider": "supertonic", "voice": "M1"}
assert formula_from_profile(entry) is None
# ---------------------------------------------------------------------------
# resolve_profile_voice
# ---------------------------------------------------------------------------
class TestResolveProfileVoice:
"""Tests for resolve_profile_voice()."""
def test_resolves_kokoro_profile(self):
from abogen.domain.voice_resolution import resolve_profile_voice
profiles = {
"MyMix": {
"provider": "kokoro",
"language": "a",
"voices": [("af_heart", 0.5), ("am_echo", 0.5)],
}
}
formula, language = resolve_profile_voice("MyMix", profiles=profiles)
assert "af_heart" in formula
assert "am_echo" in formula
assert language == "a"
def test_empty_profile_name(self):
from abogen.domain.voice_resolution import resolve_profile_voice
formula, language = resolve_profile_voice("", profiles={})
assert formula == ""
assert language is None
def test_none_profile_name(self):
from abogen.domain.voice_resolution import resolve_profile_voice
formula, language = resolve_profile_voice(None, profiles={})
assert formula == ""
assert language is None
def test_nonexistent_profile(self):
from abogen.domain.voice_resolution import resolve_profile_voice
formula, language = resolve_profile_voice("Nonexistent", profiles={})
assert formula == ""
assert language is None
def test_profile_without_language(self):
from abogen.domain.voice_resolution import resolve_profile_voice
profiles = {
"NoLang": {
"provider": "kokoro",
"voices": [("af_heart", 1.0)],
}
}
formula, language = resolve_profile_voice("NoLang", profiles=profiles)
assert "af_heart" in formula
assert language is None
# ---------------------------------------------------------------------------
# resolve_voice_setting
# ---------------------------------------------------------------------------
class TestResolveVoiceSetting:
"""Tests for resolve_voice_setting()."""
def test_plain_voice_spec(self):
from abogen.domain.voice_resolution import resolve_voice_setting
spec, profile, language = resolve_voice_setting("af_heart")
assert spec == "af_heart"
assert profile is None
assert language is None
def test_profile_prefix(self):
from abogen.domain.voice_resolution import resolve_voice_setting
profiles = {
"MyMix": {
"provider": "kokoro",
"language": "a",
"voices": [("af_heart", 0.5), ("am_echo", 0.5)],
}
}
spec, profile, language = resolve_voice_setting("profile:MyMix", profiles=profiles)
assert "af_heart" in spec
assert profile == "MyMix"
assert language == "a"
def test_speaker_prefix(self):
from abogen.domain.voice_resolution import resolve_voice_setting
profiles = {
"MyMix": {
"provider": "kokoro",
"language": "e",
"voices": [("bf_sage", 1.0)],
}
}
spec, profile, language = resolve_voice_setting("speaker:MyMix", profiles=profiles)
assert "bf_sage" in spec
assert profile == "MyMix"
assert language == "e"
def test_empty_value(self):
from abogen.domain.voice_resolution import resolve_voice_setting
spec, profile, language = resolve_voice_setting("")
assert spec == ""
assert profile is None
assert language is None
# ---------------------------------------------------------------------------
# resolve_voice_choice
# ---------------------------------------------------------------------------
class TestResolveVoiceChoice:
"""Tests for resolve_voice_choice()."""
def test_plain_voice(self):
from abogen.domain.voice_resolution import resolve_voice_choice
voice, lang, profile = resolve_voice_choice(
language="a",
base_voice="af_heart",
profile_name="",
custom_formula="",
profiles={},
)
assert voice == "af_heart"
assert lang == "a"
assert profile is None
def test_kokoro_profile(self):
from abogen.domain.voice_resolution import resolve_voice_choice
profiles = {
"MyMix": {
"provider": "kokoro",
"language": "a",
"voices": [("af_heart", 0.5), ("am_echo", 0.5)],
}
}
voice, lang, profile = resolve_voice_choice(
language="a",
base_voice="af_heart",
profile_name="MyMix",
custom_formula="",
profiles=profiles,
)
assert "af_heart" in voice
assert "am_echo" in voice
assert lang == "a"
assert profile == "MyMix"
def test_supertonic_profile(self):
from abogen.domain.voice_resolution import resolve_voice_choice
profiles = {
"MyST": {
"provider": "supertonic",
"language": "a",
"voice": "M1",
}
}
voice, lang, profile = resolve_voice_choice(
language="a",
base_voice="M1",
profile_name="MyST",
custom_formula="",
profiles=profiles,
)
assert voice == "speaker:MyST"
assert lang == "a"
assert profile == "MyST"
def test_custom_formula_overrides_profile(self):
from abogen.domain.voice_resolution import resolve_voice_choice
profiles = {
"MyMix": {
"provider": "kokoro",
"language": "a",
"voices": [("af_heart", 1.0)],
}
}
voice, lang, profile = resolve_voice_choice(
language="a",
base_voice="af_heart",
profile_name="MyMix",
custom_formula="af_heart*0.3+am_echo*0.7",
profiles=profiles,
)
assert voice == "af_heart*0.3+am_echo*0.7"
assert profile is None
def test_profile_language_override(self):
from abogen.domain.voice_resolution import resolve_voice_choice
profiles = {
"GermanMix": {
"provider": "kokoro",
"language": "g",
"voices": [("af_heart", 1.0)],
}
}
voice, lang, profile = resolve_voice_choice(
language="a",
base_voice="af_heart",
profile_name="GermanMix",
custom_formula="",
profiles=profiles,
)
assert lang == "g"
assert profile == "GermanMix"
# ---------------------------------------------------------------------------
# build_voice_catalog
# ---------------------------------------------------------------------------
class TestBuildVoiceCatalog:
"""Tests for build_voice_catalog()."""
@patch("plugins.kokoro.engine.language_for_voice_id")
@patch("abogen.domain.voice_catalog.get_voices")
def test_builds_catalog_with_metadata(self, mock_voices, mock_lang):
from abogen.domain.voice_catalog import build_voice_catalog
mock_voices.return_value = ("af_heart", "am_echo")
mock_lang.side_effect = lambda vid: MagicMock(value="a")
catalog = build_voice_catalog()
assert len(catalog) == 2
assert catalog[0]["id"] == "af_heart"
assert catalog[0]["gender"] == "Female"
assert catalog[0]["gender_code"] == "f"
assert catalog[0]["language"] == "a"
assert "Heart" in catalog[0]["display_name"]
assert catalog[1]["id"] == "am_echo"
assert catalog[1]["gender"] == "Male"
assert catalog[1]["gender_code"] == "m"
@patch("plugins.kokoro.engine.language_for_voice_id")
@patch("abogen.domain.voice_catalog.get_voices")
def test_empty_voices(self, mock_voices, mock_lang):
from abogen.domain.voice_catalog import build_voice_catalog
mock_voices.return_value = ()
catalog = build_voice_catalog()
assert catalog == []
@patch("plugins.kokoro.engine.language_for_voice_id")
@patch("abogen.domain.voice_catalog.get_voices")
def test_display_name_formatting(self, mock_voices, mock_lang):
from abogen.domain.voice_catalog import build_voice_catalog
mock_voices.return_value = ("bf_sage",)
mock_lang.side_effect = lambda vid: MagicMock(value="a")
catalog = build_voice_catalog()
assert catalog[0]["display_name"] == "Sage"
# ---------------------------------------------------------------------------
# filter_voice_catalog
# ---------------------------------------------------------------------------
class TestFilterVoiceCatalog:
"""Tests for filter_voice_catalog()."""
def _catalog(self):
return [
{"id": "af_heart", "language": "a", "gender_code": "f"},
{"id": "am_echo", "language": "a", "gender_code": "m"},
{"id": "bf_sage", "language": "b", "gender_code": "f"},
]
def test_filter_by_female(self):
from abogen.domain.voice_catalog import filter_voice_catalog
result = filter_voice_catalog(self._catalog(), gender="female")
assert "af_heart" in result
assert "bf_sage" in result
assert "am_echo" not in result
def test_filter_by_male(self):
from abogen.domain.voice_catalog import filter_voice_catalog
result = filter_voice_catalog(self._catalog(), gender="male")
assert "am_echo" in result
assert "af_heart" not in result
def test_filter_by_language(self):
from abogen.domain.voice_catalog import filter_voice_catalog
result = filter_voice_catalog(
self._catalog(), gender="female", allowed_languages=["a"]
)
assert "af_heart" in result
assert "bf_sage" not in result
def test_fallback_to_any_gender(self):
from abogen.domain.voice_catalog import filter_voice_catalog
catalog = [
{"id": "af_heart", "language": "a", "gender_code": "f"},
]
result = filter_voice_catalog(catalog, gender="male")
assert "af_heart" in result
def test_fallback_to_any_language(self):
from abogen.domain.voice_catalog import filter_voice_catalog
catalog = [
{"id": "af_heart", "language": "a", "gender_code": "f"},
]
result = filter_voice_catalog(
catalog, gender="female", allowed_languages=["b"]
)
assert "af_heart" in result
def test_empty_catalog(self):
from abogen.domain.voice_catalog import filter_voice_catalog
result = filter_voice_catalog([], gender="female")
assert result == []
+2 -2
View File
@@ -69,9 +69,9 @@ class TestRenderFfmetadata:
assert "title=Ch 1" in result assert "title=Ch 1" in result
def test_renders_voice_in_chapter(self): def test_renders_voice_in_chapter(self):
chapters = [{"start": 0.0, "end": 5.0, "voice": "af_heart"}] chapters = [{"start": 0.0, "end": 5.0, "voices": [{"provider": "kokoro", "voice": "af_heart"}]}]
result = self.svc.render_ffmetadata({}, chapters) result = self.svc.render_ffmetadata({}, chapters)
assert "voice=af_heart" in result assert "voice=af_heart@kokoro" in result
def test_skips_chapters_without_times(self): def test_skips_chapters_without_times(self):
chapters = [{"title": "No times"}] chapters = [{"title": "No times"}]
+2 -2
View File
@@ -14,7 +14,7 @@ def test_render_ffmetadata_includes_chapters(tmp_path):
"publisher": "ACME=Corp", "publisher": "ACME=Corp",
} }
chapters = [ chapters = [
{"start": 0.0, "end": 5.0, "title": "Intro", "voice": "voice_a"}, {"start": 0.0, "end": 5.0, "title": "Intro", "voices": [{"provider": "kokoro", "voice": "voice_a"}]},
{"start": 5.0, "end": 12.345, "title": "Chapter 2"}, {"start": 5.0, "end": 12.345, "title": "Chapter 2"},
] ]
@@ -28,7 +28,7 @@ def test_render_ffmetadata_includes_chapters(tmp_path):
assert rendered.count("[CHAPTER]") == 2 assert rendered.count("[CHAPTER]") == 2
assert "START=0" in rendered assert "START=0" in rendered
assert "END=5000" in rendered assert "END=5000" in rendered
assert "voice=voice_a" in rendered assert "voice=voice_a@kokoro" in rendered
audio_path = tmp_path / "book.m4b" audio_path = tmp_path / "book.m4b"
metadata_path = svc.write_ffmetadata_file(audio_path, metadata, chapters) metadata_path = svc.write_ffmetadata_file(audio_path, metadata, chapters)
-221
View File
@@ -1,221 +0,0 @@
"""Import/layering tests for the conversion flow unification.
Verifies that:
- Application layer does not import from PyQt or WebUI
- Adapters import from application/domain (not the other way around)
- All application models are importable without GUI/Flask side effects
"""
import importlib
import sys
from pathlib import Path
import pytest
# ─── Application layer imports ─────────────────────────────────────
class TestApplicationLayerImports:
"""Verify application layer has no PyQt/WebUI imports."""
APPLICATION_DIR = Path(__file__).parent.parent / "abogen" / "application"
def _get_python_files(self):
"""Get all Python files in the application directory."""
return list(self.APPLICATION_DIR.glob("*.py"))
def test_no_pyqt_imports(self):
"""Application layer must not import from abogen.pyqt."""
import re
forbidden = re.compile(r"from\s+abogen\.pyqt|import\s+abogen\.pyqt")
violations = []
for py_file in self._get_python_files():
content = py_file.read_text(encoding="utf-8")
if forbidden.search(content):
violations.append(py_file.name)
assert not violations, f"Application files import from PyQt: {violations}"
def test_no_webui_imports(self):
"""Application layer must not import from abogen.webui."""
import re
forbidden = re.compile(r"from\s+abogen\.webui|import\s+abogen\.webui")
violations = []
for py_file in self._get_python_files():
content = py_file.read_text(encoding="utf-8")
if forbidden.search(content):
violations.append(py_file.name)
assert not violations, f"Application files import from WebUI: {violations}"
def test_no_flask_imports(self):
"""Application layer must not import Flask."""
import re
forbidden = re.compile(r"from\s+flask|import\s+flask")
violations = []
for py_file in self._get_python_files():
content = py_file.read_text(encoding="utf-8")
if forbidden.search(content):
violations.append(py_file.name)
assert not violations, f"Application files import Flask: {violations}"
def test_no_qthread_imports(self):
"""Application layer must not import QThread."""
import re
forbidden = re.compile(r"from\s+PyQt6|import\s+PyQt6")
violations = []
for py_file in self._get_python_files():
content = py_file.read_text(encoding="utf-8")
if forbidden.search(content):
violations.append(py_file.name)
assert not violations, f"Application files import PyQt6: {violations}"
class TestApplicationModelsImportable:
"""Verify application models are importable without side effects."""
def test_conversion_request_importable(self):
from abogen.application.conversion_request import ConversionRequest
assert ConversionRequest is not None
def test_conversion_models_importable(self):
from abogen.application.conversion_models import (
ChapterPlan,
ConversionPlan,
IntroOutroSpec,
OutputLayout,
SegmentPlan,
)
assert all(
cls is not None
for cls in [ChapterPlan, ConversionPlan, IntroOutroSpec, OutputLayout, SegmentPlan]
)
def test_conversion_result_importable(self):
from abogen.application.conversion_result import ConversionError, ConversionResult
assert ConversionResult is not None
assert ConversionError is not None
def test_conversion_ports_importable(self):
from abogen.application.conversion_ports import (
AudioSink,
ConversionEvents,
PipelineProvider,
ResolvedVoice,
SubtitleWriter,
VoiceResolver,
)
assert all(
cls is not None
for cls in [
AudioSink,
ConversionEvents,
PipelineProvider,
ResolvedVoice,
SubtitleWriter,
VoiceResolver,
]
)
def test_output_layout_service_importable(self):
from abogen.application.output_layout_service import (
resolve_chapter_path,
resolve_merged_path,
resolve_output_layout,
should_merge_output,
)
assert all(
fn is not None
for fn in [resolve_chapter_path, resolve_merged_path, resolve_output_layout, should_merge_output]
)
def test_conversion_planner_importable(self):
from abogen.application.conversion_planner import build_conversion_plan
assert build_conversion_plan is not None
def test_conversion_executor_importable(self):
from abogen.application.conversion_executor import execute_conversion
assert execute_conversion is not None
def test_conversion_service_importable(self):
from abogen.application.conversion_service import run_conversion
assert run_conversion is not None
class TestAdapterImports:
"""Verify adapters import from application/domain correctly."""
def test_webui_adapter_imports_application(self):
from abogen.webui.conversion_adapter import (
WebJobEvents,
WebPipelineProvider,
WebVoiceResolver,
build_conversion_request_from_job,
)
assert all(
cls is not None
for cls in [
WebJobEvents,
WebPipelineProvider,
WebVoiceResolver,
build_conversion_request_from_job,
]
)
def test_pyqt_adapter_imports_application(self):
from abogen.pyqt.conversion_adapter import (
PyQtEvents,
PyQtPipelineProvider,
PyQtVoiceResolver,
build_conversion_request_from_thread,
)
assert all(
cls is not None
for cls in [
PyQtEvents,
PyQtPipelineProvider,
PyQtVoiceResolver,
build_conversion_request_from_thread,
]
)
def test_webui_adapter_does_not_import_pyqt(self):
"""WebUI adapter must not import from PyQt."""
import re
adapter_path = Path(__file__).parent.parent / "abogen" / "webui" / "conversion_adapter.py"
content = adapter_path.read_text(encoding="utf-8")
forbidden = re.compile(r"from\s+abogen\.pyqt|import\s+abogen\.pyqt")
assert not forbidden.search(content), "WebUI adapter imports from PyQt"
def test_pyqt_adapter_does_not_import_webui(self):
"""PyQt adapter must not import from WebUI."""
import re
adapter_path = Path(__file__).parent.parent / "abogen" / "pyqt" / "conversion_adapter.py"
content = adapter_path.read_text(encoding="utf-8")
forbidden = re.compile(r"from\s+abogen\.webui|import\s+abogen\.webui")
assert not forbidden.search(content), "PyQt adapter imports from WebUI"
+525
View File
@@ -0,0 +1,525 @@
"""Tests for application/integration_hooks.py — PostConversionHooks
and domain/settings_core.py build_audiobookshelf_config."""
from __future__ import annotations
from pathlib import Path
from typing import Any, Dict, List
from unittest.mock import MagicMock, patch
import pytest
from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_result import ConversionResult
from abogen.application.integration_hooks import PostConversionHooks
from abogen.domain.enums import Language
from abogen.domain.settings_core import build_audiobookshelf_config
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_request(**overrides: Any) -> ConversionRequest:
defaults = dict(
source_path=Path("/tmp/test.txt"),
original_filename="test.txt",
language=Language.EN_US,
voice="M1",
speed=1.0,
use_gpu=False,
)
defaults.update(overrides)
return ConversionRequest(**defaults)
def _make_result(**overrides: Any) -> ConversionResult:
defaults: Dict[str, Any] = dict(
metadata={"title": "Test Book"},
)
defaults.update(overrides)
return ConversionResult(**defaults)
class _FakeEvents:
def __init__(self) -> None:
self.logs: List[tuple[str, str]] = []
def log(self, message: str, level: str = "info") -> None:
self.logs.append((message, level))
def progress(self, pct: int, etr: str) -> None:
pass
def check_cancelled(self) -> None:
pass
def _abs_settings(**overrides: Any) -> Dict[str, Any]:
"""Build a minimal Audiobookshelf settings dict."""
settings: Dict[str, Any] = {
"enabled": True,
"auto_send": True,
"base_url": "https://example.com",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
}
settings.update(overrides)
return settings
# ---------------------------------------------------------------------------
# build_audiobookshelf_config tests (domain layer)
# ---------------------------------------------------------------------------
class TestBuildAbsConfig:
"""build_audiobookshelf_config from domain.settings_core."""
def test_returns_none_when_base_url_missing(self) -> None:
result = build_audiobookshelf_config({
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
})
assert result is None
def test_returns_none_when_api_token_missing(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"library_id": "lib",
"folder_id": "fld",
})
assert result is None
def test_returns_none_when_library_id_missing(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"api_token": "tok",
"folder_id": "fld",
})
assert result is None
def test_returns_none_when_folder_id_missing(self) -> None:
# folder_id is optional in AudiobookshelfConfig, so this should succeed
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"api_token": "tok",
"library_id": "lib",
})
assert result is not None
def test_returns_config_when_all_required_fields_present(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
})
assert result is not None
assert result.base_url == "https://example.com"
assert result.api_token == "tok"
assert result.library_id == "lib"
assert result.folder_id == "fld"
def test_preserves_trailing_slash_in_base_url(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com/",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
})
assert result is not None
# normalization is done by AudiobookshelfClient, not config
assert result.base_url == "https://example.com/"
def test_preserves_api_suffix_in_base_url(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com/api",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
})
assert result is not None
# normalization is done by AudiobookshelfClient, not config
assert result.base_url == "https://example.com/api"
def test_applies_default_timeout(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
})
assert result is not None
assert result.timeout == 3600.0
def test_applies_custom_timeout(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
"timeout": 7200.0,
})
assert result is not None
assert result.timeout == 7200.0
def test_invalid_timeout_falls_back_to_default(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
"timeout": "invalid",
})
assert result is not None
assert result.timeout == 3600.0
def test_collection_id_is_optional(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
})
assert result is not None
assert result.collection_id is None
def test_collection_id_when_provided(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
"collection_id": "col123",
})
assert result is not None
assert result.collection_id == "col123"
def test_boolean_flags_default(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
})
assert result is not None
assert result.verify_ssl is True
assert result.send_cover is True
assert result.send_chapters is True
assert result.send_subtitles is False
def test_boolean_flags_custom(self) -> None:
result = build_audiobookshelf_config({
"base_url": "https://example.com",
"api_token": "tok",
"library_id": "lib",
"folder_id": "fld",
"verify_ssl": False,
"send_cover": False,
"send_chapters": False,
"send_subtitles": True,
})
assert result is not None
assert result.verify_ssl is False
assert result.send_cover is False
assert result.send_chapters is False
assert result.send_subtitles is True
# ---------------------------------------------------------------------------
# Hook skipping tests
# ---------------------------------------------------------------------------
class TestPostConversionHooks:
"""PostConversionHooks.run() skipping logic."""
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_skip_when_no_audiobookshelf_config(self, mock_stored: MagicMock) -> None:
mock_stored.return_value = {}
hooks = PostConversionHooks()
request = _make_request()
result = _make_result()
events = _FakeEvents()
hooks.run(request, result, events)
assert events.logs == []
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_skip_when_enabled_false(self, mock_stored: MagicMock) -> None:
mock_stored.return_value = _abs_settings(enabled=False)
hooks = PostConversionHooks()
request = _make_request()
result = _make_result()
events = _FakeEvents()
hooks.run(request, result, events)
assert events.logs == []
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_skip_when_auto_send_false(self, mock_stored: MagicMock) -> None:
mock_stored.return_value = _abs_settings(auto_send=False)
hooks = PostConversionHooks()
request = _make_request()
result = _make_result()
events = _FakeEvents()
hooks.run(request, result, events)
assert events.logs == []
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_skip_when_config_incomplete(self, mock_stored: MagicMock) -> None:
mock_stored.return_value = _abs_settings(base_url="")
hooks = PostConversionHooks()
request = _make_request()
result = _make_result()
events = _FakeEvents()
hooks.run(request, result, events)
assert len(events.logs) == 1
assert "configure" in events.logs[0][0].lower()
assert events.logs[0][1] == "warning"
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_skip_when_audio_path_missing(self, mock_stored: MagicMock) -> None:
mock_stored.return_value = _abs_settings()
hooks = PostConversionHooks()
request = _make_request()
result = _make_result(audio_path=None)
events = _FakeEvents()
hooks.run(request, result, events)
assert len(events.logs) == 1
assert "audio output not found" in events.logs[0][0].lower()
assert events.logs[0][1] == "warning"
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_skip_when_audio_file_does_not_exist(self, mock_stored: MagicMock, tmp_path: Path) -> None:
mock_stored.return_value = _abs_settings()
hooks = PostConversionHooks()
request = _make_request()
result = _make_result(audio_path=tmp_path / "nonexistent.mp3")
events = _FakeEvents()
hooks.run(request, result, events)
assert len(events.logs) == 1
assert "audio output not found" in events.logs[0][0].lower()
# ---------------------------------------------------------------------------
# Upload flow tests (mocked client)
# ---------------------------------------------------------------------------
class TestAudiobookshelfUpload:
"""Test the upload flow with mocked AudiobookshelfClient."""
@patch("abogen.application.integration_hooks.AudiobookshelfClient")
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_successful_upload(
self, mock_stored: MagicMock, mock_client_cls: MagicMock, tmp_path: Path,
) -> None:
audio_path = tmp_path / "book.mp3"
audio_path.write_bytes(b"audio-content")
mock_client = MagicMock()
mock_client.find_existing_items.return_value = []
mock_client_cls.return_value = mock_client
mock_stored.return_value = _abs_settings()
hooks = PostConversionHooks()
request = _make_request(original_filename="book.mp3")
result = _make_result(audio_path=audio_path)
events = _FakeEvents()
hooks.run(request, result, events)
mock_client.find_existing_items.assert_called_once()
mock_client.upload_audiobook.assert_called_once()
assert any("upload queued" in msg.lower() for msg, _ in events.logs)
@patch("abogen.application.integration_hooks.AudiobookshelfClient")
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_deletes_existing_items_before_upload(
self, mock_stored: MagicMock, mock_client_cls: MagicMock, tmp_path: Path,
) -> None:
audio_path = tmp_path / "book.mp3"
audio_path.write_bytes(b"audio-content")
mock_client = MagicMock()
mock_client.find_existing_items.return_value = [{"id": "existing-1"}]
mock_client_cls.return_value = mock_client
mock_stored.return_value = _abs_settings()
hooks = PostConversionHooks()
request = _make_request(original_filename="book.mp3")
result = _make_result(audio_path=audio_path)
events = _FakeEvents()
hooks.run(request, result, events)
mock_client.delete_items.assert_called_once_with([{"id": "existing-1"}])
mock_client.upload_audiobook.assert_called_once()
@patch("abogen.application.integration_hooks.AudiobookshelfClient")
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_lookup_error_logged_not_raised(
self, mock_stored: MagicMock, mock_client_cls: MagicMock, tmp_path: Path,
) -> None:
from abogen.integrations.audiobookshelf import AudiobookshelfUploadError
audio_path = tmp_path / "book.mp3"
audio_path.write_bytes(b"audio-content")
mock_client = MagicMock()
mock_client.find_existing_items.side_effect = AudiobookshelfUploadError("connection refused")
mock_client_cls.return_value = mock_client
mock_stored.return_value = _abs_settings()
hooks = PostConversionHooks()
request = _make_request(original_filename="book.mp3")
result = _make_result(audio_path=audio_path)
events = _FakeEvents()
hooks.run(request, result, events)
assert any("lookup failed" in msg.lower() for msg, _ in events.logs)
mock_client.upload_audiobook.assert_not_called()
@patch("abogen.application.integration_hooks.AudiobookshelfClient")
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_upload_error_logged_not_raised(
self, mock_stored: MagicMock, mock_client_cls: MagicMock, tmp_path: Path,
) -> None:
from abogen.integrations.audiobookshelf import AudiobookshelfUploadError
audio_path = tmp_path / "book.mp3"
audio_path.write_bytes(b"audio-content")
mock_client = MagicMock()
mock_client.find_existing_items.return_value = []
mock_client.upload_audiobook.side_effect = AudiobookshelfUploadError("timeout")
mock_client_cls.return_value = mock_client
mock_stored.return_value = _abs_settings()
hooks = PostConversionHooks()
request = _make_request(original_filename="book.mp3")
result = _make_result(audio_path=audio_path)
events = _FakeEvents()
hooks.run(request, result, events)
assert any("upload failed" in msg.lower() for msg, _ in events.logs)
@patch("abogen.application.integration_hooks.AudiobookshelfClient")
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_delete_error_logged_not_raised(
self, mock_stored: MagicMock, mock_client_cls: MagicMock, tmp_path: Path,
) -> None:
audio_path = tmp_path / "book.mp3"
audio_path.write_bytes(b"audio-content")
mock_client = MagicMock()
mock_client.find_existing_items.return_value = [{"id": "old-item"}]
mock_client.delete_items.side_effect = Exception("network error")
mock_client_cls.return_value = mock_client
mock_stored.return_value = _abs_settings()
hooks = PostConversionHooks()
request = _make_request(original_filename="book.mp3")
result = _make_result(audio_path=audio_path)
events = _FakeEvents()
hooks.run(request, result, events)
assert any("failed to remove" in msg.lower() for msg, _ in events.logs)
mock_client.upload_audiobook.assert_called_once()
@patch("abogen.application.integration_hooks.AudiobookshelfClient")
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_cover_included_when_exists(
self, mock_stored: MagicMock, mock_client_cls: MagicMock, tmp_path: Path,
) -> None:
audio_path = tmp_path / "book.mp3"
audio_path.write_bytes(b"audio-content")
cover_path = tmp_path / "cover.jpg"
cover_path.write_bytes(b"jpeg-content")
mock_client = MagicMock()
mock_client.find_existing_items.return_value = []
mock_client_cls.return_value = mock_client
mock_stored.return_value = _abs_settings(send_cover=True)
hooks = PostConversionHooks()
request = _make_request(original_filename="book.mp3")
from abogen.application.conversion_config import CoverConfig
request.cover = CoverConfig(path=cover_path, mime="image/jpeg")
result = _make_result(audio_path=audio_path)
events = _FakeEvents()
hooks.run(request, result, events)
call_kwargs = mock_client.upload_audiobook.call_args
assert call_kwargs[1]["cover_path"] == cover_path
@patch("abogen.application.integration_hooks.AudiobookshelfClient")
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_subtitles_included_when_enabled(
self, mock_stored: MagicMock, mock_client_cls: MagicMock, tmp_path: Path,
) -> None:
audio_path = tmp_path / "book.mp3"
audio_path.write_bytes(b"audio-content")
subtitle_path = tmp_path / "book.srt"
subtitle_path.write_bytes(b"srt-content")
mock_client = MagicMock()
mock_client.find_existing_items.return_value = []
mock_client_cls.return_value = mock_client
mock_stored.return_value = _abs_settings(send_subtitles=True)
hooks = PostConversionHooks()
request = _make_request(original_filename="book.mp3")
result = _make_result(audio_path=audio_path)
result.subtitle_paths = [subtitle_path]
events = _FakeEvents()
hooks.run(request, result, events)
call_kwargs = mock_client.upload_audiobook.call_args
assert call_kwargs[1]["subtitles"] == [subtitle_path]
@patch("abogen.application.integration_hooks.AudiobookshelfClient")
@patch("abogen.application.integration_hooks.stored_integration_config")
def test_subtitles_skipped_when_disabled(
self, mock_stored: MagicMock, mock_client_cls: MagicMock, tmp_path: Path,
) -> None:
audio_path = tmp_path / "book.mp3"
audio_path.write_bytes(b"audio-content")
subtitle_path = tmp_path / "book.srt"
subtitle_path.write_bytes(b"srt-content")
mock_client = MagicMock()
mock_client.find_existing_items.return_value = []
mock_client_cls.return_value = mock_client
mock_stored.return_value = _abs_settings(send_subtitles=False)
hooks = PostConversionHooks()
request = _make_request(original_filename="book.mp3")
result = _make_result(audio_path=audio_path)
result.subtitle_paths = [subtitle_path]
events = _FakeEvents()
hooks.run(request, result, events)
call_kwargs = mock_client.upload_audiobook.call_args
assert call_kwargs[1]["subtitles"] is None

Some files were not shown because too many files have changed in this diff Show More