Compare commits

..
Author SHA1 Message Date
Deniz Şafak 9201f58770 Add k0sm0naft to GitHub funding list 2026-07-23 16:27:54 +03:00
Deniz Şafak 6274a02d5e Fix empty voice list when launched via desktop shortcut
PluginManager.discover() used a relative path 'plugins', which resolved
against the CWD. When launched from a desktop shortcut the CWD is ~, so
the plugins directory was never found and no voices appeared in the list.

Fall back to the project-relative plugins path when the default relative
path doesn't resolve.
2026-07-23 03:55:07 +03:00
Deniz Şafak 342ea0dfac Fix spaCy unknown language error: map Kokoro single-letter codes to Language enum in get_spacy_model 2026-07-23 03:33:06 +03:00
Deniz Şafak 27f88b759d Fix spurious HF HEAD requests: return early on cache hit in tracked_hf_hub_download 2026-07-23 03:26:58 +03:00
Deniz Şafak dbcbb1c8a9 Fix NameError: add missing 'from pathlib import Path' in pyqt/conversion.py 2026-07-23 03:18:51 +03:00
Deniz Şafak bd99ee1ba1 fix: subtitle FakeToken split, missing run_tts_segment_loop import
- subtitle_generation: split multi-sentence FakeToken into separate entries
- conversion.py: add missing run_tts_segment_loop import

No changes to spacy_utils or Language enum.
2026-07-23 03:15:21 +03:00
Deniz Şafak d5cddb9749 fix: pass mock job object to merge_pronunciation_overrides instead of positional args 2026-07-23 02:33:18 +03:00
Deniz Şafak ec55918b04 fix: add load_single_voice to Pipeline wrapper to prevent formula string being used as download filename 2026-07-23 02:28:41 +03:00
Deniz Şafak 14913b45e9 fix: import importlib.util explicitly (not auto-loaded in Python 3.12) — broke plugin loading, causing empty voice lists 2026-07-23 02:17:46 +03:00
Deniz Şafak a0fdabd81f fix: suppress harmless Qt portal registration warning on Linux 2026-07-23 01:42:41 +03:00
Deniz Şafak 473631b84e fix: use theme-aware GREY_BACKGROUND for word substitutions instructions label 2026-07-23 01:41:13 +03:00
Deniz Şafak 0f5003dfdd fix: add missing imports for get_resource_path and load_integration_settings 2026-07-23 01:37:50 +03:00
Artem Akymenko fcec4e9fe5 fix: test_stretch_reduces_duration — remove stale self param, fix mock data size, fix atempo assertion 2026-07-22 15:32:18 +03:00
Artem Akymenko 72d5e3d1db fix: add keys() method to VoiceCache for resolve_intro compatibility 2026-07-22 15:31:43 +03:00
Artem Akymenko d3682e7672 refactor: dynamic ConversionRequest validation, remove 'or default' from adapters
- __post_init__: _apply_none_defaults() iterates dataclasses.fields() dynamically
- _NUMERIC_CONSTRAINTS and _ENUM_CONSTRAINTS dicts replace per-field if chains
- Both adapters pass values as-is (no 'or default' fallbacks)
- 18 validation tests + updated adapter tests for Enum assertions
2026-07-22 11:33:44 +00:00
Artem Akymenko 0805e9fdae refactor: Language Enum with ISO codes
- Language enum: en-US, en-GB, es, fr, hi, it, ja, pt-BR, zh
- Engine-specific mappings (kokoro → single-letter) live in pipeline_factory and synthesize
- spacy_utils uses Language enum keys for model mapping
- split_pattern uses Language enum properties (is_cjk)
- Updated all tests to use ISO codes
2026-07-22 10:54:39 +00:00
Artem Akymenko 4aef73ff85 refactor: remove infrastructure enum duplicates
- SubtitleFormat/SubtitleMode now only in domain/enums.py
- Added VTT to SubtitleFormat
- Renamed SENTENCE_HIGHLIGHTING → SENTENCE_HIGHLIGHT for consistency
- Infrastructure subtitle_writer imports from domain
2026-07-22 09:16:40 +00:00
Artem Akymenko f6a8008f51 refactor: typed Enums for format/mode fields
- SubtitleMode, OutputFormat, SaveMode, SubtitleFormat, InputFormat
- Properties: dot_ext, is_lossless, is_book, is_subtitle
- from_str/from_path class methods with normalization
- Updated domain and application layers to use Enums
- 17 new tests for enum validation and properties
2026-07-22 09:01:50 +00:00
Artem Akymenko dc5257252f refactor: run_tts_segment_loop also accepts SynthParams
- Reduces from 14 params to 5 unique params + SynthParams
- synthesize_text now passes params through cleanly
- PyQt intro/outro direct calls updated
2026-07-22 08:28:19 +00:00
Artem Akymenko c4cebb8822 refactor: SynthParams dataclass for synthesize_text
- Frozen dataclass in domain/conversion_engine.py with common params
- synthesize_text now takes params=SynthParams + unique kwargs
- Executor, PyQt legacy, WebUI legacy, and tests updated
- Adding new common params now only requires changing the dataclass
2026-07-22 08:21:45 +00:00
Artem Akymenko 93f5a46485 refactor: deduplicate synth params in executor
- Compute use_spacy and effective_subtitle_mode once instead of 6x each
- Reduces repeated ternary expressions across synthesize_text calls
2026-07-22 11:06:14 +03:00
Artem Akymenko 5f169a4921 refactor: replace executor _slugify with domain sanitize_filename_for_chapter
- Use existing domain function instead of duplicated local implementation
- Domain version includes OS-specific sanitization
2026-07-22 11:06:14 +03:00
Artem Akymenko df5705779e refactor: extract ConversionCancelled to conversion_ports
- Single definition in application layer
- Both adapters import from ports instead of defining locally
2026-07-22 11:06:14 +03:00
Artem Akymenko d0fe221176 refactor: ConversionPlan.request forward ref
- request type: Any → ConversionRequest via TYPE_CHECKING
2026-07-22 07:31:46 +00:00
Artem Akymenko 17700426fd clean: dead code removal + unused imports
- executor: remove dead subtitle_writer stub (lines 121-125)
- executor: replace getattr with direct field access on ResolvedVoice
- service: remove duplicate split_pattern import
- service: remove unused import time
- planner: remove unused import os
- adapters: remove unused import threading, time
2026-07-22 07:21:23 +00:00
Artem Akymenko 16b3f7d8a8 fix: clean direct_text in planner + remove redundant if/else
- _extract_source_text now applies clean_text() to direct_text (was skipped)
- _parse_chapters simplified: identical branches collapsed to single call
2026-07-22 07:05:50 +00:00
Artem Akymenko 1a3741ec50 fix: max_subtitle_words default 5 → 50
All UI layers use 50 (settings, Job, Thread, adapters). The value 5 was
incorrect and only masked by adapter fallbacks.
2026-07-21 14:58:58 +03:00
Artem Akymenko 7f317ca784 test: adapter field mapping + import layering tests
- 28 adapter tests (WebUI + PyQt): field mapping, events, provider, resolver
- 15 import/layering tests: no PyQt/WebUI in app layer, all models importable
2026-07-21 11:47:46 +00:00
Artem Akymenko d0e42ee691 fix: executor subtitle_writer leak + adapter Path/None-default fixes
- executor: manage subtitle_writer via ExitStack (stack.callback)
- executor: remove manual subtitle_writer.close()
- WebUI adapter: wrap source_path, output_folder, cover_image_path in Path()
- WebUI adapter: remove unused threading/time imports
- PyQt adapter: fix getattr(attr, None) or default for 4 fields
2026-07-21 11:47:36 +00:00
Artem Akymenko b1392084e1 chore: add .coverage to .gitignore 2026-07-21 09:35:36 +00:00
Artem Akymenko 71916aa39f fix: conversion_service import bug + coverage tests
Fix import error in _prepare_tts_context:
- apply_normalization_overrides doesn't exist in domain.normalization
- merge_pronunciation_overrides expects job-like object, not two lists
- Use _MockJob adapter to bridge ConversionRequest to existing API

Add test_application_coverage.py (23 tests):
- ConversionService: simple, logs, cancellation, empty text, multi-chapter, intro/outro, error
- OutputLayoutService: custom folder, source path, project, merged path, chapter path, should_merge
- Executor gaps: no layout, m4b, separate chapters, no intro/outro, voice fallback, silence

Coverage: 80% -> 92%
2026-07-21 09:34:47 +00:00
Artem Akymenko 4c4434c309 fix: sanitize_output_stem signature + audio_sink import
Two pre-existing bugs found during test coverage analysis:

1. sanitize_output_stem() only accepted 1 arg but resolve_project_layout
   passed 2 args (name, index) via sanitize_fn parameter.
   Fix: added optional index parameter to sanitize_output_stem.

2. audio_sink.py imported get_internal_cache_path from
   abogen.infrastructure.cache which doesn't exist.
   Fix: import from abogen.utils where the function lives.
2026-07-21 09:18:51 +00:00
Artem Akymenko 7973de3868 feat: ConversionService
Main orchestrator for the conversion flow. Both UIs call run_conversion().

Functions:
- run_conversion(request, events, pipeline_provider, voice_resolver) -> ConversionResult
- _prepare_tts_context(request, events) -> TTSContext

The service ties together planner, executor, and finalizers.
2026-07-21 11:30:15 +03:00
Artem Akymenko fd659d0f4f feat: PyQt adapter
Converts PyQt ConversionThread to ConversionRequest for the application layer.

Functions:
- build_conversion_request_from_thread(thread) -> ConversionRequest
- PyQtEvents: wraps thread signals for logging, progress, cancellation
- PyQtPipelineProvider: wraps existing backend
- PyQtVoiceResolver: wraps load_voice_cached

Subtitle file/timestamp special paths remain in ConversionThread.run().
2026-07-21 11:29:47 +03:00
Artem Akymenko e53251ef81 feat: WebUI adapter
Converts WebUI Job to ConversionRequest for the application layer.

Functions:
- build_conversion_request_from_job(job) -> ConversionRequest
- WebJobEvents: wraps Job for logging, progress, cancellation
- WebPipelineProvider: wraps PipelinePool for TTS backends
- WebVoiceResolver: wraps voice resolution function

The adapter is the bridge between WebUI layer and application/domain.
Application layer never accesses Job directly.
2026-07-21 11:29:38 +03:00
Artem Akymenko cd3cc9bce7 refactor: extract OutputLayoutService
Extract output path resolution from conversion_planner.py into
application/output_layout_service.py as a standalone service.

Functions:
- resolve_output_layout(request) -> OutputLayout
- resolve_merged_path(layout, request) -> Path
- resolve_chapter_path(layout, request, title, index) -> Path
- should_merge_output(request) -> bool

Planner now imports from output_layout_service instead of inline logic.
2026-07-21 11:17:28 +03:00
Artem Akymenko 75a3ad517a test: executor tests with fake backend/sink/ports
7 tests for the unified conversion executor:
- simple text conversion
- multi-chapter with separate chapter output
- voice markers
- intro/outro
- cancellation behavior
- progress reporting
- metadata preservation

Uses FakeBackend, FakeAudioSink, FakeSubtitleWriter, FakeEvents,
FakePipelineProvider, FakeVoiceResolver to test without real TTS.
2026-07-21 11:16:41 +03:00
Artem Akymenko a6b7ce69aa feat: unified conversion executor (execute_conversion)
Takes ConversionPlan + ports, executes TTS conversion, returns ConversionResult.
- Opens/closes audio sinks and subtitle writers
- Processes intro/outro
- Executes chapter loop with heading + body segments
- Collects chapter_markers and chunk_markers
- Uses domain functions only (no UI imports)
2026-07-21 11:16:41 +03:00
Artem Akymenko 680418fa1d test: planner tests + domain regression tests
51 tests for the unified conversion planner:
- build_conversion_plan: direct text, voice markers, chunks, chapters, intro/outro, output layout
- Domain regression: chapter parsing, voice markers, TTSContext, voice resolution, intro/outro, output paths, subtitles
- All tests use domain functions only (no UI, no TTS, no audio I/O)
2026-07-21 11:16:41 +03:00
Artem Akymenko 53b850ef41 feat: unified conversion planner (build_conversion_plan)
Pure function that takes ConversionRequest -> ConversionPlan.
Handles chapter parsing, voice markers, chunks, intro/outro, output layout.
Replaces duplicated planning logic in both PyQt and WebUI runners.
2026-07-21 11:16:41 +03:00
Artem Akymenko 7ed4eca68c feat: application layer models and ports for conversion unification
- application/conversion_models.py: SegmentPlan, ChapterPlan, ConversionPlan, OutputLayout, IntroOutroSpec
- application/conversion_request.py: ConversionRequest (normalized input)
- application/conversion_result.py: ConversionResult, ConversionError (normalized output)
- application/conversion_ports.py: protocols (ConversionEvents, PipelineProvider, VoiceResolver, SubtitleWriter, AudioSink)

These are pure data models and interfaces. No implementation yet.
2026-07-21 11:16:41 +03:00
Artem Akymenko e1e49e8a0f test: regression tests for conversion flow unification
Three new test files describing expected behavior before refactoring:
- test_conversion_planner.py: chapter parsing, voice markers, TTSContext, intro/outro, output paths, subtitles (30 tests)
- test_conversion_request.py: settings, context building, chapter selection, cancellation/logging protocols (15 tests)
- test_conversion_executor.py: synthesize_text, process_and_write_subtitles, full pipeline with fake backend/sink (12 tests)

All 1310 tests pass (1253 existing + 57 new).
2026-07-21 11:16:40 +03:00
Artem Akymenko 28998e1e5c refactor: delete redundant _prepare_project_layout wrapper
resolve_project_layout() from domain already handles mkdir.
Callers now use the domain function directly.

Tests: 1253 passed
2026-07-20 10:10:50 +00:00
Artem Akymenko 8a220a936c refactor: extract extract_metadata_for_file() domain function, delete PyQt wrapper
domain/metadata_extraction.py gains extract_metadata_for_file() combining
read_text_for_metadata + extract_metadata_from_text. PyQt _extract_metadata_dict
deleted, calls replaced with domain function.

Tests: 1253 passed
2026-07-20 09:49:33 +00:00
Artem Akymenko ccc2cdb166 refactor: replace manual suffix loop with resolve_unique_path (#4)
PyQt output path resolution now uses resolve_unique_path() from domain
instead of a hand-rolled counter loop. Output path logic is now fully
shared via domain functions.

Tests: 1253 passed
2026-07-20 09:27:31 +00:00
Artem Akymenko 79ff7e4682 refactor: delete _process_subtitle_tokens wrapper, use domain function directly (#6)
PyQt now calls process_subtitle_tokens() from domain instead of a thin
wrapper that just forwarded self.subtitle_mode/lang_code/use_spacy.

Tests: 1253 passed
2026-07-20 09:22:31 +00:00
Artem Akymenko 2a54b8fdf1 refactor: extract synthesize_text() domain function (#2)
Combines TTSContext.normalize() + run_tts_segment_loop() into a single
domain function. Both UIs call synthesize_text() instead of inlining
normalize → TTS loop. UI-specific concerns (provider resolution,
progress display, cancellation) stay in the UI layer.

Tests: 1253 passed
2026-07-20 09:12:40 +00:00
Artem Akymenko 68e5adb091 refactor: consolidate voice resolution via resolve_voice_choice (#3)
Chapter and chunk loops now call resolve_voice_choice() instead of
inlining _resolve_voice_target + cache check + resolve_voice.
Reduces 3 duplicated voice resolution blocks to 1 closure.

Tests: 1253 passed
2026-07-20 09:05:26 +00:00
Artem Akymenko c4870eece6 refactor: extract TTSContext dataclass for normalization parameters (#5)
Bundles pronunciation_rules, heteronym_rules, normalization_overrides,
usage_counter, and split_pattern into a single TTSContext dataclass.
Both UIs create it once and use tts_context.normalize() instead of
threading 5 separate parameters through prepare_text_for_tts calls.

Tests: 1253 passed
2026-07-20 09:00:55 +00:00
Artem Akymenko 8144a7a507 refactor: unify intro/outro through domain; extract subtitle writer creation
- domain/intro_outro.py: resolve_intro(), resolve_outro() return IntroOutroSpec
- Both UIs call domain for text building + voice spec resolution
- PyQt uses resolve_intro/resolve_outro instead of direct calls
- infrastructure/subtitle_writer.py: resolve_subtitle_format(), make_subtitle_writer()
- Deleted duplicate _create_subtitle_writer() from WebUI
- Deleted duplicate _subtitle_alignment_from_format() from PyQt
- domain/conversion_engine.py: run_tts_segment_loop() for TTS iteration
- VoiceCache class in domain/voice_loader.py used by both UIs
- Tests: 1253 passed
2026-07-20 08:32:55 +00:00
Artem Akymenko f38700025a unify voice caching: VoiceCache class used by both WebUI and PyQt
- domain/voice_loader.py: VoiceCache class now used by both UIs;
  resolve_voice() and load_voice_cached() accept VoiceCache or plain dict;
  added hasattr(pipeline, 'load_single_voice') safety check from WebUI
- conversion_runner.py: replaced local _resolve_voice() with domain's
  resolve_voice(); voice_cache changed from Dict to VoiceCache instance;
  all cache access uses VoiceCache.get()/set() API
- pyqt/conversion.py: self.voice_cache changed from Dict to VoiceCache
- debug_tts_runner.py: imports resolve_voice from domain instead of
  removed _resolve_voice from conversion_runner
2026-07-20 08:02:34 +00:00
Artem Akymenko 804517f5b2 extract subtitle writer creation: resolve_subtitle_format() + make_subtitle_writer()
- infrastructure/subtitle_writer.py: add resolve_subtitle_format() that
  maps format strings (e.g. 'ass_centered_narrow') to (extension, alignment),
  and make_subtitle_writer() convenience that resolves + creates writer or None
- conversion_runner.py: replace _create_subtitle_writer() with make_subtitle_writer()
- pyqt/conversion.py: replace _subtitle_alignment_from_format() and 3 manual
  create_subtitle_writer() call sites with resolve_subtitle_format()/make_subtitle_writer()
2026-07-20 07:52:15 +00:00
Artem Akymenko 5d30903149 extract conversion_engine: shared TTS segment iteration loop for WebUI and PyQt
- domain/conversion_engine.py: run_tts_segment_loop() with CancelChecker,
  SegmentStats, SegmentInfo protocols; on_segment callback for per-segment
  subtitle processing; process_and_write_subtitles() helper
- conversion_runner.py: emit_text() delegates TTS iteration to engine
- pyqt/conversion.py: inner tts_segments loop replaced with engine call,
  on_segment handles dual merged+chapter subtitle writers
- routes/utils/settings.py: re-exports load_settings, coerce_int/float,
  llm_ready, settings_defaults from domain for backward compat
2026-07-20 07:35:58 +00:00
Artem Akymenko 476063bc3d refactor: move load_settings() to domain, simplify settings.py
- load_settings() now in domain/settings_core.py (shared by all UIs)
- settings.py delegates to domain instead of reimplementing
- settings.py: 456 → 430 lines
2026-07-20 06:56:42 +00:00
Artem Akymenko 079e185108 refactor: simplify normalize_setting_value() via Setting.normalizer
- Added normalizer callable to Setting dataclass
- Moved special-case logic (_norm_save_mode, _norm_voice_spec, etc.)
  into registry entries as normalizers
- normalize_setting_value() reduced from 25 lines to 10 lines
- Single dispatch: normalizer → coerce → fallback
2026-07-20 06:48:06 +00:00
Artem Akymenko 69c398ebf0 refactor(pyqt): replace hardcoded config defaults with SETTINGS_REGISTRY
gui.py now reads defaults from all_settings_defaults() instead of
hardcoding values like 50, True, 'wav', etc. One source of truth
for all settings across Web UI and Desktop GUI.
2026-07-19 15:57:49 +00:00
Artem Akymenko dbe73254a4 refactor: add SETTINGS_REGISTRY contract to domain/settings_core.py
- Setting dataclass: key, type, default, min/max, valid_values, scope
- 72 settings total: 54 shared, 18 PyQt-only
- validate_setting() checks types and ranges
- Setting.coerce() handles type conversion with bounds
- settings_defaults() / all_settings_defaults() derived from registry
- BOOLEAN_SETTINGS, FLOAT_SETTINGS, INT_SETTINGS now auto-derived
- 17 tests validating schema, coercion, and validation
2026-07-19 13:24:52 +00:00
Artem Akymenko 64e8a8f4e6 refactor: extract shared settings_core and Flask route logic to domain/services 2026-07-19 15:52:35 +03:00
Artem Akymenko aec3462f1f refactor: both UIs use shared tts_segments() from domain/conversion_pipeline.py
- domain/conversion_pipeline.py: add tts_segments() for pre-normalized text;
  emit_text_segments() now delegates to tts_segments() internally
- pyqt/conversion.py: inner TTS loop replaced with tts_segments() iterator;
  removed FakeToken import (handled by domain)
- webui/conversion_runner.py: emit_text() inner loop replaced with
  tts_segments() iterator; removed FakeToken import
- Both UIs now share the same TTS emission logic — normalization + backend
  invocation + segment iteration + token extraction
- +3 tests for tts_segments (no-normalization, chunk_start, basic)
- 1188 tests pass
2026-07-19 11:47:14 +00:00
Artem Akymenko 1193185833 refactor: create domain/conversion_pipeline.py with shared TTS emission loop
- domain/conversion_pipeline.py: emit_text_segments() — generator yielding
  SegmentResult for each TTS segment; emit_text_to_sinks() — convenience
  wrapper handling audio writing + token accumulation + subtitle flushing
- Both WebUI and PyQt can call these instead of reimplementing the TTS loop
- Caller provides backend, voice, speed, split_pattern; domain handles
  normalization, TTS invocation, token extraction
- +7 tests (segment yielding, empty audio skip, chunk_start, tokens, fallback)
- 1185 tests pass
2026-07-19 10:31:58 +00:00
Artem Akymenko a99cf58c79 refactor: consolidate voice formula building into voice_formulas.py
- voice_formulas.py: add pairs_to_formula() as canonical implementation
- webui/routes/utils/voice.py: formula_from_profile() and pairs_to_formula()
  now delegate to voice_formulas.pairs_to_formula()
- pyqt/gui.py: get_voice_formula() now uses voice_formulas.pairs_to_formula()
  instead of inline string formatting
- Eliminates 3 duplicate implementations of voice*weight formula building
- +9 tests
- 1178 tests pass
2026-07-19 10:02:14 +00:00
Artem Akymenko fe62b6b44c refactor: extract book metadata logic from PyQt to domain
- domain/metadata_extraction.py: add format_metadata_tags(),
  extract_book_metadata_epub(), extract_book_metadata_pdf(),
  extract_book_metadata_markdown(), _save_cover_to_cache()
- pyqt/book_handler.py: _extract_book_metadata() reduced from ~165 lines
  to ~10 lines by delegating to domain; _format_metadata_tags() reduced
  from ~55 lines to ~15 lines; ebooklib/fitz imports moved to domain
- +16 tests (format_metadata_tags, save_cover, markdown extraction)
- 1169 tests pass
2026-07-19 09:51:50 +00:00
Artem Akymenko 0e216f3786 refactor: extract _process_subtitle_file domain logic to shared modules
- domain/subtitle_processor.py: parse_subtitle_file, format_time_range,
  speed_up_audio, fit_audio_to_duration (moved from audio_buffer),
  process_subtitle_entries (core TTS loop with cancel/log/progress callbacks)
- domain/audio_buffer.py: add fit_audio_to_duration, ffmpeg_time_stretch
- domain/output_paths.py: add resolve_unique_path (collision-safe filename)
- pyqt/conversion.py: _process_subtitle_file reduced from ~350 to ~100 lines
  by delegating to domain functions; removed 4 unused subtitle parser imports
- +33 tests (8 resolve_unique_path, 14 subtitle_processor, 8 audio_buffer,
  3 format_time_range)
- 1153 tests pass
2026-07-19 09:34:17 +00:00
Artem Akymenko 380cdee0cb refactor(pyqt): use create_pipeline_for_job() in LoadPipelineThread
Replace direct create_pipeline() call with domain function for
consistent provider validation and device resolution.
2026-07-19 07:57:09 +00:00
Artem Akymenko c76cf74efc refactor: unify duplicated logic between WebUI and PyQt
domain/output_paths.py:
- Add sanitize_filename_for_chapter() with OS safety + smart truncation
- Keep existing slugify() for backward compatibility

domain/text_chapters.py (NEW):
- parse_chapters_from_text() combines intro preservation (PyQt) + clean_text (WebUI)

PyQt/conversion.py:
- Replace 3x copy-pasted ASS/SRT headers with create_subtitle_writer()
- Replace inline M4B muxing with ExportService.embed_m4b_metadata()
- Replace inline chapter splitting with parse_chapters_from_text()
- Replace inline chapter filename sanitization with sanitize_filename_for_chapter()
- Remove unused _CHAPTER_MARKER_SEARCH_PATTERN import

Tests: 1152 passed (+21 new)
2026-07-19 07:50:02 +00:00
Artem Akymenko a299947bb1 refactor(webui): replace inline get_pipeline/resolve_voice_target closures with domain modules
- Replace get_pipeline() closure with PipelinePool from domain/pipeline_factory
- Replace resolve_voice_target() closure with domain function from voice_utils
- Remove dead _load_pipeline() function and unused is_plugin_registered import
- Add 33 tests for resolve_voice_target and PipelinePool
- Add 10 regression tests verifying domain extraction preserves behavior
- 1131 tests pass (+61 new)
2026-07-18 14:13:17 +03:00
Artem Akymenko 957c6778f6 refactor(pyt): dedup voice formula resolution in PyQt
Replace 2 inline 'if * in voice: get_new_voice(...)' patterns with
resolve_voice() from domain/voice_loader.py. Removes unused import
of get_new_voice.
2026-07-18 14:13:16 +03:00
Artem Akymenko fcdaf2b2a8 refactor(domain): extract FakeToken to domain/tokens.py
Shared token stub used by both WebUI and PyQt for languages
without per-word token support.
2026-07-18 14:12:18 +03:00
Artem Akymenko d8634f812d refactor(domain): extract audio sink abstraction to domain layer
- New domain/audio_sink.py: AudioSink context manager + open_audio_sink() factory
  - Supports WAV/FLAC (soundfile) and MP3/Opus/M4B (ffmpeg pipe)
  - cancel_check, extra_ffmpeg_args, ffmpeg_cmd parameters
  - 17 tests in test_domain_audio_sink.py

- WebUI: replaced local AudioSink + _open_audio_sink with domain module
  - Removed 35 lines, 3 call sites now use open_audio_sink()

- PyQt: replaced 3 inline audio output setups with domain module
  - New _open_merged_sink() helper encapsulates m4b cover art logic
  - ExitStack for automatic cleanup in main conversion
  - Removed ~140 lines of duplicated ffmpeg/soundfile boilerplate
2026-07-18 14:12:01 +03:00
Artem Akymenko 85b5851786 refactor(pyt): replace all inline float32 conversion with to_float32()
PyQt had 7 inline float32 conversion patterns:
  hasattr(x, 'numpy') ? x.numpy().astype('float32') : x.astype('float32')
spread across TTS loop, subtitle processing, and streaming.

All replaced with domain.audio_helpers.to_float32() which handles:
- None → zeros
- PyTorch tensors → .detach().cpu().numpy()
- Plain numpy → asarray(dtype=float32)
- reshape(-1) for consistent 1D output

The old inline code missed .detach() and .cpu() on GPU tensors,
causing potential crashes. Now both UIs use the same robust conversion.

1053 tests pass.
2026-07-18 06:58:21 +00:00
Artem Akymenko e77c8b3372 fix: review cleanup — imports, _FakeToken, use_spacy_segmentation
- Move pronunciation imports from inside run() to top-level imports
- Extract _FakeToken to module level (was redefined every loop iteration)
- use_spacy_segmentation now mirrors PyQt logic: pass the flag,
  let process_subtitle_tokens filter by language internally
2026-07-18 06:52:28 +00:00
Artem Akymenko 294069e53e refactor(webui): token-level subtitle processing via process_subtitle_tokens — P0
Before: WebUI wrote one subtitle entry per TTS segment (no sentence
grouping, no comma splitting, no karaoke highlighting). The subtitle
modes 'Sentence', 'Sentence + Comma', and 'Sentence + Highlighting'
produced broken output.

After: emit_text() accumulates tokens_with_timestamps from each
segment's .tokens attribute, then flushes them through
domain.subtitle_generation.process_subtitle_tokens() at the end.
This gives the WebUI the same subtitle quality as the PyQt desktop GUI:
- Sentence mode: groups tokens into sentences
- Sentence + Comma: splits on commas within sentences
- Sentence + Highlighting: karaoke timing per word
- Word-count mode: groups by N words

Also removed the duplicate _to_float32 function from synthesize.py
(now imports from domain.audio_helpers).

1053 tests pass.
2026-07-18 06:36:19 +00:00
Artem Akymenko 4ff09be664 refactor(pyt): add text normalization via prepare_text_for_tts — P0
PyQt desktop GUI now calls the shared normalization pipeline before
TTS synthesis, matching the Web UI's behavior:

1. Heteronym sentence rules (context-dependent pronunciation)
2. Pronunciation rules (token-level replacements)
3. Pipeline normalization (apostrophe handling, LLM)

Before: PyQt passed raw text to the backend — no normalization at all,
resulting in inferior audio quality compared to the Web UI.

The normalization rules are compiled once at the start of run() from
pronunciation_overrides and heteronym_overrides (currently None since
the PyQt GUI doesn't expose these settings yet — basic apostrophe
normalization still applies).

1053 tests pass.
2026-07-18 09:28:53 +03:00
Artem Akymenko a1d93820b1 refactor(domain): unify ETR calculation — both UIs now use calc_etr_str
Before:
  - PyQt: inline ETR using chars-based formula
  - WebUI: Job.estimated_time_remaining using progress-based formula
  (different formulas → different ETR estimates)

After:
  - Both UIs call domain.progress.calc_etr_str(elapsed, done, total)
  - Same formula, same ETR, single source of truth
  - WebUI now stores etr_str on Job and displays it directly
  - Job.estimated_time_remaining property kept for backward compat

domain/progress.py: ProgressTracker class + calc_etr_str function
1053 tests pass.
2026-07-16 09:31:26 +00:00
Artem Akymenko 0c1a3c1904 refactor(webui): use shared get_split_pattern instead of hardcoded \\n+
All three Web UI consumers now call domain.split_pattern.get_split_pattern()
which selects the correct split pattern based on language and subtitle mode.

Before: WebUI always split on \\n+ regardless of language (CJK missed
punctuation-based splitting that PyQt already had).
After: Both UIs share identical language-aware splitting logic.

1038 tests pass.
2026-07-16 09:12:46 +00:00
Artem Akymenko 2228f37c06 refactor(domain): add prepare_text_for_tts — unified normalization pipeline
New function chains all three normalization stages:
  1. Heteronym sentence rules (context-dependent pronunciation)
  2. Pronunciation rules (token-level replacements)
  3. Pipeline normalization (apostrophe, LLM)

This is the single entry point that both Web UI and PyQt should call
before TTS synthesis. Currently only Web UI uses it; PyQt has NO
normalization — this unlocks that capability.

Updated conversion_runner.emit_text to use the new function.
1038 tests pass.
2026-07-16 08:53:15 +00:00
Artem Akymenko 832e2c5197 refactor(domain): extract chapter classification heuristics from form.py
Moved supplement_score, should_preselect_chapter, and
ensure_at_least_one_chapter_enabled to domain/chapter_classification.py.

Also moved coerce_bool from settings.py to common.py to break circular
import introduced in previous commit.

1025 tests pass.
2026-07-16 08:13:37 +00:00
Artem Akymenko 17229b2390 refactor(webui): deduplicate _extract_checkbox (3 copies → 1)
Extracted extract_checkbox from settings.py, form.py (x2) into
webui/routes/utils/common.py. Moved coerce_bool from settings.py
to common.py to break circular import.

Fixed bug in second form.py copy (was missing __contains__ check).
1006 tests pass.
2026-07-16 07:46:51 +00:00
Artem Akymenko c2c584e741 refactor(domain): deduplicate metadata helpers across 3 layers
Extracted 8 metadata functions from service.py, exporters.py, and
audiobookshelf.py into domain/metadata_helpers.py:
- normalize_metadata_casefold, split_people_field, split_simple_list
- first_nonempty, extract_year, normalize_series_sequence
- build_audiobookshelf_metadata, load_audiobookshelf_chapters

service.py, exporters.py, and audiobookshelf.py now import from domain
instead of maintaining separate copies. Thin wrappers adapt to layer
interfaces (Job objects, etc.).

Net -231 lines. 1006 tests pass.
2026-07-16 07:34:58 +00:00
Artem Akymenko 8ccdc85ccb refactor(webui): rename preview.py to synthesize.py and remove dead code
- Rename abogen/webui/routes/utils/preview.py → synthesize.py
  The file contains the core TTS synthesis pipeline (generate_preview_audio,
  synthesize_preview), not just preview logic. Name now matches responsibility.
- Remove dead code from voice.py: get_preview_pipeline(), synthesize_audio_from_normalized(),
  _preview_pipeline_lock, _preview_pipelines, and unused imports (threading, numpy,
  create_pipeline, get_new_voice, _select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN).
  These were never called — identical logic lives in synthesize.py.
- Update imports in api.py, voices.py, and test_preview_applies_manual_overrides.py
- 7 new tests in test_synthesize_module.py enforce file naming and import rules
- 7 tests in test_domain_imports.py updated for renamed module
2026-07-16 10:19:01 +03:00
Artem Akymenko ef07a8b5b2 fix(tests): mock spacy in plugin tests to fix externally-managed-environment failures 2026-07-16 10:02:01 +03:00
Artem Akymenko 1268a83cff fix(webui): voice.py imports from domain modules instead of conversion_runner
Replace private imports (_select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN)
from abogen.webui.conversion_runner with proper domain imports:
- select_device from abogen.domain.device
- to_float32, SAMPLE_RATE from abogen.domain.audio_helpers
- SPLIT_PATTERN defined locally (r'\n+')

Also verifies preview.py already uses domain imports correctly.

7 new tests in tests/test_domain_imports.py enforce the architecture rule.
2026-07-16 06:57:12 +00:00
Artem Akymenko ef6faff2e8 refactor: extract metadata processing logic to domain layer
- Replace manual metadata extraction with regex in pyqt/conversion.py
  with calls to domain/metadata_extraction.py functions
- Remove duplicate _embed_m4b_metadata and _apply_m4b_chapters_with_mutagen
  functions from webui/conversion_runner.py
- Use ExportService.embed_m4b_metadata for m4b metadata embedding
- Reduce code duplication between PyQt and WebUI interfaces
2026-07-15 20:44:19 +03:00
Artem Akymenko da9d5e7eb9 fix(tests): audio_buffer and subtitle_generation tests
- Fix mix_audio to return target buffer (was not modifying in-place)
- Fix samples_for_duration to return 0 for negative durations
- Fix test assertions for numpy 2.x compatibility (share_memory -> shares_memory)
- Adjust subtitle_generation tests to match actual behavior
2026-07-15 20:26:31 +03:00
Artem Akymenko acb000b9e6 refactor: extract voice loading logic to domain layer
- Add abogen/domain/voice_loader.py with:
  - VoiceCache class: unified cache for loaded voices
  - resolve_voice(): load voice with optional caching
  - load_voice_cached(): compatibility wrapper for PyQt

- Update abogen/pyqt/conversion.py:
  - Replace load_voice_cached method body with call to domain function
  - Maintain backward compatibility with existing interface

- Add tests/test_voice_loader.py with unit tests for VoiceCache and voice loading
2026-07-15 20:20:18 +03:00
Artem Akymenko d6c66dc18a refactor: extract subtitle token processing to domain layer
- Add abogen/domain/subtitle_generation.py with:
  - process_subtitle_tokens(): main function for converting TTS tokens to subtitles
  - Support for all subtitle modes: Line, Sentence, Sentence + Comma, Sentence + Highlighting
  - Support for word-count based grouping (e.g., '5' for 5 words per entry)
  - spaCy integration for English sentence boundary detection
  - Karaoke highlighting tags for Sentence + Highlighting mode
  - Punctuation constants for sentence splitting

- Update abogen/pyqt/conversion.py:
  - Replace _process_subtitle_tokens method body with call to domain function
  - Remove ~260 lines of duplicate logic

- Add tests/test_subtitle_generation.py with comprehensive unit tests
2026-07-15 20:14:02 +03:00
Artem Akymenko 0d46076bf6 refactor: extract audio buffer operations to domain layer
- Add abogen/domain/audio_buffer.py with core audio operations:
  - create_silence(): create silence audio buffer
  - mix_audio(): mix source into target buffer with auto-resize
  - normalize_audio(): normalize to prevent clipping
  - ensure_buffer_size(): extend buffer to minimum size
  - concatenate_audio(): join multiple audio buffers
  - audio_duration(): calculate duration from samples
  - samples_for_duration(): calculate samples from duration
  - SAMPLE_RATE constant (24000)

- Update abogen/pyqt/conversion.py:
  - Import and use create_silence for chapter silence
  - Use mix_audio for subtitle file mixing
  - Use normalize_audio for clipping prevention
  - Use create_silence for padding in subtitle processing

- Update abogen/webui/conversion_runner.py:
  - Import and use create_silence in append_silence
  - Replace np.zeros with domain function

- Add tests/test_audio_buffer.py with comprehensive unit tests
2026-07-15 20:02:33 +03:00
Artem Akymenko 7fef9c1d93 extract normalize_text_for_pipeline to domain/normalization.py 2026-07-15 15:19:01 +00:00
Artem Akymenko 56cfd0810d extract resolve_fallback_voice_spec to domain/voice_resolution.py; fix missing get_default_voice import and __custom_mix reset bug 2026-07-15 15:01:17 +00:00
Artem Akymenko 7bd3177241 extract select_device to domain/device.py; fix bug where conversion_runner didn't check torch availability 2026-07-15 14:45:38 +00:00
Artem Akymenko d5c2a81733 Merge pull request #191 from hydraxman/fix/large-chapter-form-limits
fix(webui): allow large chapter forms
2026-07-15 17:35:52 +03:00
Artem Akymenko 514e29a761 extract apply_chapter_text_transforms to domain/chapter_titles.py 2026-07-15 14:29:29 +00:00
Artem Akymenko 86042a3315 fix bugs, remove dead code and unused imports in conversion_runner.py 2026-07-15 13:50:18 +00:00
Artem Akymenko 50d75eb2fc standardize m4b encoding to VBR -q:a 2; replace remaining ffmpeg blocks in desktop GUI with domain modules 2026-07-15 13:28:21 +00:00
Artem Akymenko ae9ab70421 refactor: extract audio helpers to domain/audio_helpers.py
- Extract build_ffmpeg_command, to_float32, apply_m4b_chapters_with_mutagen
- _apply_m4b_chapters_with_mutagen becomes thin wrapper with error handling
- Add tests/test_audio_helpers.py (12 tests)
- conversion_runner.py: 1410 → 1320 lines
- All tests pass
2026-07-15 12:15:10 +00:00
Artem Akymenko 4364276a5b refactor: extract output path utilities to domain/output_paths.py
- Extract slugify, sanitize_output_stem, output_timestamp_token, build_output_path
- Extract apply_newline_policy, resolve_output_directory, resolve_project_layout
- _prepare_output_dir and _prepare_project_layout become thin wrappers with mkdir
- Add tests/test_output_paths.py (21 tests)
- conversion_runner.py: 1443 → 1410 lines
- All tests pass
2026-07-15 11:56:06 +00:00
Artem Akymenko 914e77de46 refactor: wire up domain/voice_utils.py and remove duplicates
- Import supertonic_voice_from_spec, split_speaker_reference, formula_from_kokoro_entry
- Import infer_provider_from_spec, coerce_truthy from domain/voice_utils.py
- Remove duplicate function bodies from conversion_runner.py
- conversion_runner.py: 1518 → 1443 lines
- All tests pass
2026-07-15 11:06:29 +00:00
Artem Akymenko 1d7a2aeed6 refactor: extract chunk utils to domain/chunk_utils.py
- Extract safe_int, group_chunks_by_chapter, record_override_usage, chunk_text_for_tts
- Add tests/test_chunk_utils.py (15 tests)
- Update test_chunk_helpers.py and test_chunk_text_for_tts_prefers_raw.py imports
- conversion_runner.py: 1574 → 1518 lines
- All tests pass
2026-07-15 11:00:18 +00:00
Artem Akymenko a26e02b017 refactor: wire up domain/chapter_overrides.py and domain/metadata_merge.py
- Update chapter_overrides.py to return tuple matching original signature
- Import apply_chapter_overrides and merge_metadata from domain modules
- Remove old function bodies from conversion_runner.py
- Add tests/test_chapter_merge_normalize.py (19 tests)
- conversion_runner.py: 1677 → 1574 lines
- All tests pass
2026-07-15 10:36:31 +00:00
Artem Akymenko c94347b33b refactor: extract voice resolution to domain/voice_resolution.py
- Extract spec_to_voice_ids, job_voice_fallback, collect_required_voice_ids
- Extract initialize_voice_cache, chapter_voice_spec, chunk_voice_spec
- Add tests/test_voice_resolution.py (29 tests)
- conversion_runner.py: 1822 → 1677 lines
- All tests pass
2026-07-15 10:20:58 +00:00
Artem Akymenko b7a48e3204 refactor: extract pronunciation rules to domain/pronunciation.py
- Extract compile_pronunciation_rules, compile_heteronym_sentence_rules
- Extract apply_pronunciation_rules, apply_heteronym_sentence_rules
- Extract merge_pronunciation_overrides
- Add tests/test_pronunciation.py (31 tests)
- All tests pass
2026-07-14 18:27:22 +00:00
Artem Akymenko feb38a24ec fix: create missing domain/file_type.py from previous incomplete refactoring 2026-07-14 18:27:13 +00:00
Artem Akymenko f63590932d refactor: extract voice utils to domain/voice_utils.py
- Extract infer_provider_from_spec, supertonic_voice_from_spec, split_speaker_reference, formula_from_kokoro_entry, coerce_truthy to domain/voice_utils.py
- Add tests/test_voice_utils.py with 24 tests
- All tests match old behavior
2026-07-14 11:02:34 +00:00
Artem Akymenko 7777e58f1d refactor: extract title/outro builders into domain/title_builder.py
- Extract build_title_intro_text and build_outro_text into domain/title_builder.py
- Uses metadata_helpers for metadata processing
- Remove _build_title_intro_text and _build_outro_text from conversion_runner.py
- Add tests/test_title_builder.py with 12 tests
- All tests match old behavior
2026-07-14 10:27:48 +00:00
Artem Akymenko 364c179bd6 refactor: extract metadata helpers into domain/metadata_helpers.py
- Extract normalize_metadata_map, format_author_sentence, ensure_sentence
- Extract normalize_series_number, extract_series_metadata, format_series_sentence
- Remove _SERIES_NAME_KEYS, _SERIES_NUMBER_KEYS, _SERIES_NUMBER_RE from conversion_runner.py
- Add tests/test_metadata_helpers.py with 32 tests
- All tests match old behavior
2026-07-14 10:26:05 +00:00
Artem Akymenko 60ba01557e refactor: extract chapter title processing into domain/chapter_titles.py
- Extract simplify_heading_text, headings_equivalent, strip_duplicate_heading_line
- Extract normalize_caps_word, normalize_chapter_opening_caps
- Extract format_spoken_chapter_title
- Remove _HEADING_SANITIZE_RE, _HEADING_NUMBER_PREFIX_RE, _ACRONYM_ALLOWLIST, _ROMAN_NUMERAL_CHARS, _CAPS_WORD_RE from conversion_runner.py
- Add tests/test_chapter_titles.py with 31 tests
- All tests match old behavior
2026-07-14 10:17:20 +00:00
Artem Akymenko 39eac9b032 refactor: replace _srt_time/_ass_time with _format_timestamp from infrastructure/subtitle_writer.py
- Remove _srt_time() and _ass_time() methods from ConversionThread
- Use _format_timestamp() from infrastructure/subtitle_writer.py instead
- Supports both SRT (ass=False) and ASS (ass=True) formats
- All existing tests pass
2026-07-14 10:14:58 +00:00
Artem Akymenko 1499a3b426 refactor: extract _get_split_pattern into domain/split_pattern.py
- Extract unified split pattern logic to domain/split_pattern.py
- Add get_split_pattern() function with language and subtitle_mode support
- Remove duplicated logic from pyqt/conversion.py
- Update pyqt/conversion.py to use domain.split_pattern.get_split_pattern
- Add tests/test_split_pattern.py with 20 tests covering English, CJK, Spanish, French, and pattern structure
2026-07-14 10:11:52 +00:00
Artem Akymenko 013c80b92c refactor: migrate FFmpeg metadata functions to infrastructure/exporters.py
- Extract FFmpeg metadata functions to infrastructure/exporters.py as ExportService
- _escape_ffmetadata_value → _escape_ffmetadata_value
- _render_ffmetadata → render_ffmetadata
- _write_ffmetadata_file → write_ffmetadata_file
- _metadata_to_ffmpeg_args → _metadata_to_ffmpeg_args
- _apply_m4b_chapters_with_mutagen → _apply_m4b_chapters_mutagen
- _embed_m4b_metadata → embed_m4b_metadata
- Add tests/test_exporters.py with 28 tests for ExportService
- Update tests/test_ffmetadata.py to use ExportService
- Update conversion_runner.py to use ExportService
- All tests pass with new implementation matching old behavior
2026-07-14 10:09:27 +00:00
Artem Akymenko 62f42a9f79 refactor: migrate SubtitleWriter to infrastructure/subtitle_writer.py
- Extract SubtitleWriter classes (SrtWriter, AssWriter, VttWriter) to infrastructure/subtitle_writer.py
- Add create_subtitle_writer() factory function
- Remove old SubtitleWriter class and _format_timestamp from conversion_runner.py
- Use create_subtitle_writer() factory from infrastructure layer
- Add tests/test_subtitle_writer.py with 28 tests covering SrtWriter, AssWriter, VttWriter
- All tests match old _format_timestamp behavior
2026-07-14 10:06:51 +00:00
Bryan Nathan 2c4d13bf56 fix(webui): allow large chapter forms 2026-07-14 08:51:40 +08:00
Artem Akymenko b7026a666d refactor(shutdown): move shutdown logic in one place 2026-07-12 20:19:33 +03:00
Artem Akymenko c380a58496 tts: fix kokoro AlbertModel import for transformers 5.x (plugin architecture)
- Monkey-patch transformers.AlbertModel in plugins/kokoro/__init__.py before kokoro imports it
- Works with transformers 4.x (no-op) and 5.x (adds moved symbol)
- No pinning, forks, or extra files needed
2026-07-12 20:17:25 +03:00
Artem Akymenko b8386b43f7 Merge pull request #190 from denizsafak/tts-plugin-refactor
refactor(tts)!: replace legacy backend with plugin architecture
2026-07-12 18:29:54 +03:00
Artem Akymenko 26e71cc2ac chore: add .gitattributes for consistent LF line endings
- Add .gitattributes with text=auto eol=lf for all text file types
- Ensures consistent line endings across platforms
- Prevents future CRLF/LF diffs in pull requests
2026-07-12 16:20:44 +03:00
Artem Akymenko d8fcfb1cce chore: normalize line endings to LF, add .gitattributes
- Add .gitattributes with text=auto eol=lf for all text files
- Renormalize all files in index to LF line endings
- Fixes massive whitespace-only diffs between main and feature branch
2026-07-12 16:20:42 +03:00
Artem Akymenko c85ea9d64f refactor(tests): add auto-discovery test system for TTS plugins
- Create tests/plugins/ with auto-discovery fixtures and generic tests
- Add conftest.py with plugin_ids, loaded_plugin, host_context fixtures
- Add test_all_plugins.py with 3 test classes:
  - TestAllPluginsManifest: validates manifest structure
  - TestAllPluginsEngine: validates engine lifecycle contract
  - TestAllPluginsCapabilities: validates capability implementation
- Update docs/testing.md with auto-discovery documentation
- Plugin-specific tests remain in tests/test_*_plugin.py for integration

New plugins in plugins/ are now automatically tested without manual test creation.
2026-07-12 16:20:30 +03:00
Artem Akymenko f151a1ae0d docs: archive historical plans, remove duplicates
- Move migration-roadmap.md, epub3_upgrade_plan.md, entities_step_overhaul_plan.md to docs/archive/
- Remove duplicate tts-plugin-architecture.md
2026-07-12 16:20:30 +03:00
Artem Akymenko 096ea58d74 docs: rewrite developer-guide as architectural reference
- Remove implementation details that will rot (code templates, tutorials)
- Keep only stable contracts: ownership, lifecycle, protocols, error semantics
- 270 lines → reference that only changes when architecture changes
2026-07-12 16:20:28 +03:00
Artem Akymenko 65cb0c75e5 fix(tests): pre-existing SuperTonic plugin test mock
Fix _make_mock_engine() to return 10 voices matching manifest and raise EngineError after dispose.
All 528 tests now pass.
2026-07-12 16:20:20 +03:00
Artem Akymenko 780e9bd780 refactor(cleanup): remove Legacy TTS Architecture
Delete legacy backend infrastructure:
- abogen/tts_backend.py (TTSBackend protocol, TTSBackendMetadata)
- abogen/tts_backend_registry.py (TTSBackendRegistry, global singleton, register_backend)
- abogen/tts_backends/ (kokoro.py, supertonic.py, __init__.py)

Delete legacy tests:
- tests/test_tts_backend.py
- tests/test_kokoro_backend.py
- tests/test_voice_formula_resolution.py
- tests/test_tts_supertonic_unsupported_chars.py

Production code now uses only Plugin Architecture via create_pipeline().
All contract, behavioral, and integration tests pass.
2 pre-existing failures in test_supertonic_plugin.py (mock engine mismatch).
2026-07-12 16:20:20 +03:00
Artem Akymenko c094b94704 feat(tts-plugin): complete Plugin Architecture refactor
- Normalize Pipeline public API: create_pipeline(plugin_id, *, lang_code, device)
- EngineConfig: add lang_code field per Architecture Amendment #1
- Kokoro plugin reads config.lang_code (fixes functional regression)
- Static voice catalog in PluginManifest.voices (None = dynamic/VoiceLister)
- get_voices() reads from manifest without creating Engine
- Remove dead kwargs (sample_rate, auto_download, total_steps) from SuperTonic
- Clean up unused imports and dead code in engine implementations
- Fix test expectations for VoiceLister (mock overrides)
- Add clear_preview_pipelines() for resource management
2026-07-12 16:20:20 +03:00
Artem Akymenko 735098d7cd feat: add static voice catalog to PluginManifest
- Add  to PluginManifest
  - None = not declared (use VoiceLister fallback)
  - () = explicitly no static voices
  - Non-empty = static catalog available without Engine instantiation

- Update get_voices() to check manifest first, fall back to Engine
- Declare 54 Kokoro voices and 10 SuperTonic voices in manifests
- Remove hardcoded voice lists from engine.py files
- Engine.listVoices() now returns [] (manifest is source of truth)

- Clean up dead create_pipeline() kwargs (sample_rate, auto_download, total_steps)
  - SuperTonic plugin uses internal defaults
  - total_steps is per-request parameter via Pipeline.__call__() kwargs

- Add clear_preview_pipelines() for resource cleanup
- Fix test mocks to override listVoices()
- Update Architecture Amendment #1 doc
2026-07-12 16:20:16 +03:00
Artem Akymenko 5d1e7165bb feat: finalize behavioral regression suite
- Add 96 behavioral regression tests parametrized for both Kokoro and SuperTonic
- Remove legacy TTSBackendRegistry tests (13) from behavioral suite
- Remove mock-only capability tests (Preview, Streaming, Cancellation) not implemented by either plugin
- Fix get_voices() to pass required args to create_engine() + error handling
- All 598 tests pass
2026-07-12 16:20:06 +03:00
Artem Akymenko 9150a80459 refactor: eliminate remaining legacy dependencies from production code
Task 1: Replace hardcoded VoiceLister bypass in get_voices()
- Use PluginManager → Engine → VoiceLister instead of direct imports
- No more hardcoded imports of plugins.kokoro.engine / plugins.supertonic.engine

Task 2: Remove SuperTonic Plugin dependency on legacy backend
- Create self-contained plugins/supertonic/pipeline.py
- Plugin no longer imports from abogen.tts_backends

Production code now has zero imports from:
- abogen.tts_backend
- abogen.tts_backend_registry
- abogen.tts_backends
2026-07-12 16:20:06 +03:00
Artem Akymenko a76d338931 refactor: remove compatibility layer, use Plugin Architecture directly
- Delete abogen/tts_plugin/compat.py (CompatBackend, create_backend, get_metadata, etc.)
- Add abogen/tts_plugin/utils.py with direct Plugin Manager functions:
  get_voices, get_default_voice, is_plugin_registered, resolve_voice_to_plugin, create_pipeline
- Update all 16 consumer files to import from utils instead of compat
- Update __init__.py to re-export utils instead of compat
- Update 5 test files and add TestNoCompatLayer regression tests
- All 493 tests pass
2026-07-12 16:20:06 +03:00
Artem Akymenko 985e16f1f8 feat: migrate remaining consumers to new Plugin Architecture
- Add compatibility functions to tts_plugin/compat.py:
  - get_metadata(): returns TTSBackendMetadata with voices
  - is_registered_backend(): checks if plugin is loaded
  - resolve_backend_for_voice(): resolves backend for voice spec
  - get_default_voice(): gets default voice for backend

- Update tts_plugin/__init__.py to export new functions

- Migrate all consumers from old tts_backend_registry:
  - WebUI: conversion_runner, debug_tts_runner, routes/api, routes/utils/*
  - PyQt UI: gui, predownload_gui, voice_formula_gui
  - Voice utilities: voice_cache, voice_formulas, voice_profiles
  - Other: subtitle_utils, utils, predownload_gui (root)

- Update tests to use new plugin architecture

Old architecture remains intact as fallback.
2026-07-12 16:20:06 +03:00
Artem Akymenko 25d45ffd36 refactor: extract EngineContractMixin base class for plugin tests
- Add tests/contracts/engine_contract.py with shared Engine/Session tests
- TestKokoroEngineContract and TestSuperTonicEngineContract inherit from it
- Eliminates protocol test duplication between plugins
- Any new plugin just inherits EngineContractMixin to verify compliance
2026-07-12 16:20:06 +03:00
Artem Akymenko 6284c501ed feat: add SuperTonic TTS plugin
- Add plugins/supertonic/ with Engine and EngineSession implementations
- Reuse existing SupertonicPipeline from abogen.tts_backends.supertonic
- Implement VoiceLister capability (M1-M5, F1-F5 voices)
- Declare no streaming support via capabilities
- Add 28 tests: plugin loading, protocol compliance, lifecycle, voice listing, parameters, errors
- All 237 tests pass
2026-07-12 16:20:06 +03:00
Artem Akymenko 23f1efcc62 refactor: rename integration test file, remove PR reference from docstring 2026-07-12 16:20:05 +03:00
Artem Akymenko a05357bab9 feat: add PluginManager, compat adapter, and consumer migration
- Add PluginManager singleton for plugin discovery and engine caching
- Add CompatBackend adapter wrapping Engine/EngineSession into old create_backend() API
- Update tts_plugin/__init__.py with public exports
- Migrate preview.py and its test to use compat.create_backend
- Add integration and plugin manager contract tests
2026-07-12 16:20:05 +03:00
Artem Akymenko d129b0abe8 feat: add Kokoro plugin vertical slice
Implement first TTS plugin using new Plugin Architecture:
- plugins/kokoro/: Plugin package with manifest and entry point
- plugins/kokoro/engine.py: KokoroEngine and KokoroSession adapters
- Wraps existing KokoroBackend without modifying it
- Implements VoiceLister capability
- Satisfies Engine/EngineSession protocol
- Passes all 163 contract tests

Tests:
- Plugin loading through Plugin Loader
- Manifest validation
- Engine creation and lifecycle
- Session synthesis and dispose
- VoiceLister capability

15 new tests for Kokoro plugin.
2026-07-12 16:20:05 +03:00
Artem Akymenko 6eda8516cc feat: add plugin loader infrastructure
Implement plugin loading and validation:
- loader.py: discover, import, validate plugins
- validate PLUGIN_MANIFEST, MODEL_REQUIREMENTS, create_engine
- validate api_version compatibility (major must match)
- validate capabilities (reject unknown)
- diagnostic messages for all error cases
- no partial registration after error

Test plugins:
- fake_plugin: minimal valid plugin for testing
- missing_manifest: no PLUGIN_MANIFEST
- invalid_api_version: major version mismatch
- invalid_capabilities: unknown capabilities
- missing_create_engine: no create_engine function
- import_error: raises ImportError during import
- missing_model_requirements: no MODEL_REQUIREMENTS

39 new tests covering all loader functionality.
2026-07-12 16:20:05 +03:00
Artem Akymenko 0f568120f4 feat: add contract test suite for Plugin API
Create reusable contract tests for TTS Plugin Architecture:
- conftest.py: shared fixtures and stubs (FakeEngine, FakeSession, etc.)
- test_types_contract.py: value object contracts (frozen, immutability, equality)
- test_errors_contract.py: error hierarchy contracts
- test_manifest_contract.py: manifest type contracts
- test_engine_contract.py: Engine protocol contracts (lifecycle, dispose)
- test_session_contract.py: EngineSession protocol contracts
- test_capabilities_contract.py: capability protocol contracts
- test_host_context_contract.py: HostContext contracts
- test_plugin_contract.py: plugin contract (exports, create_engine)

124 tests covering all public API contracts.
2026-07-12 16:20:05 +03:00
Artem Akymenko 79b3d26f66 feat: add frozen Plugin API skeleton
Create public API structure for TTS Plugin Architecture:
- types.py: immutable value objects (AudioFormat, Duration, VoiceSelection, etc.)
- errors.py: EngineError hierarchy (7 typed exceptions)
- manifest.py: plugin manifest dataclasses (PluginManifest, EngineManifest, etc.)
- engine.py: Engine and EngineSession protocols
- capabilities.py: optional capability interfaces (VoiceLister, PreviewGenerator, etc.)
- host_context.py: HostContext and HttpClient protocol
- plugin.py: plugin contract (create_engine signature)
- __init__.py: public API exports

All interfaces are fully defined but contain no business logic.
API is frozen and ready for implementation in subsequent PRs.
2026-07-12 16:20:05 +03:00
Artem Akymenko f1cc6deae8 Merge pull request #164 from yashupadhyayy1/main
Refactor segment processing with overflow error handling
2026-07-09 19:25:23 +03:00
Artem Akymenko 6f25fc06d0 ci: add UV_LINK_MODE=copy to suppress Windows hardlink warning 2026-07-09 06:58:59 +00:00
Artem Akymenko 32c4d533c9 ci: disable uv cache pruning to preserve wheel files 2026-07-09 06:06:10 +00:00
Artem Akymenko 146000886d ci: add uv cache prune to optimize cache size 2026-07-08 21:14:15 +00:00
Artem Akymenko 31f95137dd ci: replace pip with uv for faster dependency installation 2026-07-08 18:34:35 +00:00
Artem Akymenko 6f02fda41c fix(ci): set QT_QPA_PLATFORM=offscreen for headless PyQt6 tests 2026-07-08 17:36:59 +00:00
Artem Akymenko a3c3462348 fix(ci): install libegl1 on Ubuntu and normalize line endings in epub test
- Add system dependency step for libegl1 to fix PyQt6 import on headless CI
- Normalize CRLF to LF in epub exporter whitespace test for Windows CI
2026-07-08 17:16:33 +00:00
Artem Akymenko 79332204d3 Merge pull request #189 from denizsafak/feat/registry-voice-resolution
refactor: Move backend resolution by voice spec into registry
2026-07-08 20:03:19 +03:00
Artem Akymenko 6deec3b9b6 refactor: move backend resolution by voice spec into registry
- Add resolve_backend_for_voice() to TTSBackendRegistry
- Add module-level wrapper resolve_backend_for_voice()
- Simplify _infer_provider_from_spec() to use registry API
- Simplify _supertonic_voice_from_spec() to only normalize
- Add 11 test cases for the new method

Resolution rules:
1. Empty spec -> fallback
2. Kokoro formula (* or +) -> kokoro
3. Exact voice ID match -> backend id
4. Unknown voice -> fallback
2026-07-08 17:02:33 +00:00
Artem Akymenko c4d14112d4 refactor: replace hardcoded backend ID sets with registry checks
Add TTSBackendRegistry.is_registered() and module-level
is_registered_backend() to validate backend IDs dynamically.
Replace all Category A hardcoded sets (validation-only) in
voice_profiles, api routes, conversion_runner, and form utils.
2026-07-08 16:33:16 +00:00
Artem Akymenko f4cb2c2329 ci: add pytest, use actions/cache@v6 2026-07-08 19:26:42 +03:00
Artem Akymenko 783738882f Merge pull request #188 from denizsafak/refactor/move-kokoro-voices-into-backend
refactor: move VOICES_INTERNAL into KokoroBackend module
2026-07-08 19:23:33 +03:00
Artem Akymenko e94ba5257e refactor: move VOICES_INTERNAL into KokoroBackend module
Make the Kokoro voice list an internal implementation detail of the
backend instead of a shared constant. The rest of the project already
accesses voices via get_metadata('kokoro').voices.

- Move VOICES_INTERNAL from constants.py to kokoro.py as _VOICES_INTERNAL
- Update tests to use get_metadata('kokoro').voices instead of importing
  the constant directly
2026-07-08 16:19:34 +00:00
Artem Akymenko 49d66839dc Merge pull request #186 from denizsafak/refactor/migrate-remaining-voice-metadata-consumers
refactor: migrate remaining consumers to get_metadata API
2026-07-08 19:01:20 +03:00
Artem Akymenko d0e316ea7b Merge pull request #187 from denizsafak/refactor/migrate-pyqt-to-backend-metadata
refactor(pyqt): migrate from VOICES_INTERNAL to get_metadata API
2026-07-08 19:01:02 +03:00
Artem Akymenko bb96ae502c refactor: migrate remaining consumers to get_metadata API
Replace direct VOICES_INTERNAL imports with get_metadata('kokoro').voices:
- abogen/predownload_gui.py
- abogen/subtitle_utils.py
2026-07-08 15:58:51 +00:00
Artem Akymenko a4d25accc1 refactor(pyqt): migrate from VOICES_INTERNAL to get_metadata API
Replace direct VOICES_INTERNAL imports with get_metadata('kokoro').voices
from tts_backend_registry in all PyQt modules:
- abogen/pyqt/gui.py
- abogen/pyqt/predownload_gui.py
- abogen/pyqt/voice_formula_gui.py
2026-07-08 15:57:18 +00:00
Artem Akymenko 66964bfd0b Merge pull request #185 from denizsafak/refactor/use-backend-metadata-in-webui
refactor(webui): replace direct VOICES_INTERNAL/DEFAULT_SUPERTONIC_VOICES with get_metadata API
2026-07-08 18:49:21 +03:00
Yash da68f38b9b Refactor segment processing with overflow error handling
fix: catch OverflowError in emit_text for very large numbers

Fixes #145

When text contains a very large number (e.g. a long decimal from a binary
hash), misaki's pipeline calls num2words() which raises OverflowError and
crashes the entire job. Wrap the segment iterator in try/except so the
chunk is skipped gracefully with a warning log instead of terminating.
2026-05-22 10:07:38 +05:30
201 changed files with 27706 additions and 6358 deletions
+15
View File
@@ -0,0 +1,15 @@
*.py text eol=lf
*.md text eol=lf
*.yml text eol=lf
*.yaml text eol=lf
*.toml text eol=lf
*.json text eol=lf
*.txt text eol=lf
*.html text eol=lf
*.css text eol=lf
*.js text eol=lf
*.sh text eol=lf
*.cfg text eol=lf
*.ini text eol=lf
*.svg text eol=lf
*.j2 text eol=lf
+1 -1
View File
@@ -1,6 +1,6 @@
# These are supported funding model platforms # These are supported funding model platforms
github: [jborza, jeremiahsb, mohangk] github: [jborza, jeremiahsb, mohangk, k0sm0naft]
patreon: # Replace with a single Patreon username patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username ko_fi: # Replace with a single Ko-fi username
+27 -8
View File
@@ -1,5 +1,6 @@
name: pip install name: CI
run-name: pip install run-name: CI
on: on:
push: push:
branches: [main] branches: [main]
@@ -12,8 +13,9 @@ on:
- 'pyproject.toml' - 'pyproject.toml'
- '.github/workflows/**' - '.github/workflows/**'
workflow_dispatch: workflow_dispatch:
jobs: jobs:
install-and-run: test:
strategy: strategy:
matrix: matrix:
os: [ubuntu-latest, macos-14, windows-latest] os: [ubuntu-latest, macos-14, windows-latest]
@@ -23,12 +25,29 @@ jobs:
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v7 uses: actions/checkout@v7
- name: Set up Python - name: Set up Python
uses: actions/setup-python@v6 uses: actions/setup-python@v6
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
cache: pip
- name: Install from repository - name: Install uv
run: python -m pip install . uses: astral-sh/setup-uv@v8.3.1
#- name: Run abogen with:
# run: abogen enable-cache: true
prune-cache: false
cache-dependency-glob: pyproject.toml
- name: Install system dependencies (Ubuntu)
if: runner.os == 'Linux'
run: sudo apt-get update && sudo apt-get install -y libegl1
- name: Install dependencies
run: uv pip install --system .[dev]
env:
UV_LINK_MODE: copy
- name: Run tests
env:
QT_QPA_PLATFORM: offscreen
run: pytest tests/ -v --tb=short
+63 -63
View File
@@ -1,63 +1,63 @@
name: Build multi-arch Docker Image name: Build multi-arch Docker Image
on: on:
# Build and push # Build and push
#release: #release:
# types: [published] # types: [published]
# Build only # Build only
#push: it #push: it
# branches: [main] # branches: [main]
# TODO - enable build on pull requests if build times can be reduced # TODO - enable build on pull requests if build times can be reduced
# pull_request: # pull_request:
workflow_dispatch: workflow_dispatch:
env: env:
IMAGE_REPOSITORY: ghcr.io/denizsafak/abogen IMAGE_REPOSITORY: ghcr.io/denizsafak/abogen
jobs: jobs:
build: build:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v7
- name: Login to Github Container Registry - name: Login to Github Container Registry
# Only if we need to push an image # Only if we need to push an image
# if: ${{ github.event_name == 'release' && github.event.action == 'published' }} # if: ${{ github.event_name == 'release' && github.event.action == 'published' }}
uses: docker/login-action@v3 uses: docker/login-action@v3
with: with:
registry: ghcr.io registry: ghcr.io
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
# Setup for buildx # Setup for buildx
- name: Set up QEMU - name: Set up QEMU
uses: docker/setup-qemu-action@v3 uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx - name: Set up Docker Buildx
id: buildx id: buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
# Debugging information # Debugging information
- name: Docker info - name: Docker info
run: docker info run: docker info
- name: Buildx inspect - name: Buildx inspect
run: docker buildx inspect run: docker buildx inspect
# Build and (optionally) push the image # Build and (optionally) push the image
- name: Build image - name: Build image
uses: docker/build-push-action@v6 uses: docker/build-push-action@v6
with: with:
context: ./abogen context: ./abogen
file: ./abogen/Dockerfile file: ./abogen/Dockerfile
# platforms: linux/amd64,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x # platforms: linux/amd64,linux/arm/v7,linux/arm64,linux/ppc64le,linux/s390x
# platforms: linux/amd64,linux/arm64 # platforms: linux/amd64,linux/arm64
platforms: linux/amd64 # using the solution mentioned in https://github.com/denizsafak/abogen/issues/46 platforms: linux/amd64 # using the solution mentioned in https://github.com/denizsafak/abogen/issues/46
# Only push if we are publishing a release # Only push if we are publishing a release
# push: ${{ github.event_name == 'release' && github.event.action == 'published' }} # push: ${{ github.event_name == 'release' && github.event.action == 'published' }}
push: true push: true
# Use a 'temp' tag, that won't be pushed, for non-release builds # Use a 'temp' tag, that won't be pushed, for non-release builds
tags: ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name || 'latest' }} tags: ${{ env.IMAGE_REPOSITORY }}:${{ github.event.release.tag_name || 'latest' }}
# Use a cache to reduce build times # Use a cache to reduce build times
cache-to: type=gha,mode=max cache-to: type=gha,mode=max
cache-from: type=gha cache-from: type=gha
+1
View File
@@ -39,3 +39,4 @@ dist/
test_assets/ test_assets/
dev_notes/ dev_notes/
.claude/ .claude/
.coverage
+8
View File
@@ -0,0 +1,8 @@
"""Application layer for conversion flow unification.
This package contains the application-level orchestration logic
that bridges UI adapters (PyQt, WebUI) with domain functions.
The main entry point is ConversionService.run() which coordinates
planning, execution, and finalization of a conversion job.
"""
+434
View File
@@ -0,0 +1,434 @@
"""Unified conversion executor.
Takes a ConversionPlan and ports, executes the TTS conversion,
and returns a ConversionResult. No UI imports allowed.
This is Stage 6 of the conversion flow unification plan.
"""
from __future__ import annotations
import time
from contextlib import ExitStack
from typing import Any, Callable, Dict, List, Optional, Tuple
from abogen.application.conversion_models import (
ChapterPlan,
ConversionPlan,
IntroOutroSpec,
SegmentPlan,
)
from abogen.application.conversion_ports import (
AudioSink,
ConversionEvents,
PipelineProvider,
SubtitleWriter,
VoiceResolver,
)
from abogen.application.conversion_result import ConversionResult
from abogen.domain.audio_sink import open_audio_sink
from abogen.domain.conversion_engine import (
SegmentStats,
SynthParams,
process_and_write_subtitles,
synthesize_text,
)
from abogen.domain.enums import OutputFormat, SubtitleMode
from abogen.domain.normalization import TTSContext
from abogen.domain.output_paths import sanitize_filename_for_chapter
from abogen.infrastructure.subtitle_writer import make_subtitle_writer
def execute_conversion(
plan: ConversionPlan,
events: ConversionEvents,
pipeline_provider: PipelineProvider,
voice_resolver: VoiceResolver,
tts_context: TTSContext,
*,
check_cancelled: Optional[Callable[[], None]] = None,
) -> ConversionResult:
"""Execute a conversion plan and return the result.
Args:
plan: The conversion plan from build_conversion_plan()
events: UI-specific callbacks (log, progress, check_cancelled)
pipeline_provider: Provides TTS backends
voice_resolver: Resolves voice specs into loaded voices
tts_context: Normalization context for text processing
check_cancelled: Optional cancellation checker (overrides events.check_cancelled)
Returns:
ConversionResult with paths and markers
Raises:
ConversionCancelled: If conversion is cancelled
"""
request = plan.request
result = ConversionResult(metadata=plan.metadata)
# Determine cancellation checker
if check_cancelled is None:
check_cancelled = lambda: events.check_cancelled()
# Stats for progress tracking
total_characters = sum(
len(ch.body_text) for ch in plan.chapters
)
if plan.intro and plan.intro.enabled:
total_characters += len(plan.intro.text)
if plan.outro and plan.outro.enabled:
total_characters += len(plan.outro.text)
stats = SegmentStats(
processed_chars=0,
current_time=0.0,
etr_start_time=time.time(),
total_characters=total_characters,
)
# Compute subtitle flag once (used in every synthesize_text call)
use_spacy = request.subtitle_mode not in (SubtitleMode.DISABLED, SubtitleMode.LINE)
# Output paths
output_layout = plan.output_layout
if not output_layout:
raise ValueError("ConversionPlan must have an output_layout")
# Determine if merged output is needed
merge_chapters = request.merge_chapters_at_end or not request.save_chapters_separately
if request.output_format == OutputFormat.M4B:
merge_chapters = True
# Resolve voices
base_voice_spec = request.voice or "M1"
base_provider, base_voice_choice, base_speed, base_steps = _resolve_voice(
voice_resolver, base_voice_spec, request
)
# Use ExitStack for resource management
with ExitStack() as stack:
# Open merged audio sink
audio_sink: Optional[AudioSink] = None
audio_path = None
if merge_chapters:
audio_path = output_layout.audio_dir / f"{_base_name(request)}.{request.output_format}"
meta = plan.metadata if plan.metadata else None
audio_sink = stack.enter_context(
open_audio_sink(
audio_path,
request.output_format,
metadata=meta,
cancel_check=check_cancelled,
)
)
result.audio_path = audio_path
# Open subtitle writer if needed
subtitle_writer: Optional[SubtitleWriter] = None
if request.subtitle_mode != SubtitleMode.DISABLED and audio_sink:
subtitle_writer = make_subtitle_writer(
audio_path,
request.subtitle_format,
request.subtitle_mode,
max_words=request.max_subtitle_words,
)
if subtitle_writer:
subtitle_writer.open()
stack.callback(subtitle_writer.close)
result.subtitle_paths.append(subtitle_writer.path)
effective_subtitle_mode = request.subtitle_mode if subtitle_writer else SubtitleMode.DISABLED
synth = SynthParams(
tts_context=tts_context,
stats=stats,
check_cancel=check_cancelled,
on_progress=lambda pct, etr: events.progress(pct, etr),
audio_sink=audio_sink,
subtitle_mode=effective_subtitle_mode,
max_subtitle_words=request.max_subtitle_words,
lang_code=request.language,
use_spacy_segmentation=use_spacy,
)
# Chapter directory
chapter_dir = None
if request.save_chapters_separately and len(plan.chapters) > 1:
chapter_dir = output_layout.audio_dir / "chapters"
chapter_dir.mkdir(parents=True, exist_ok=True)
# Process intro
intro_emitted = False
if plan.intro and plan.intro.enabled and merge_chapters:
events.log(f"Title intro: {plan.intro.text[:80]}")
intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice(
voice_resolver, plan.intro.voice_spec, request
)
intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu)
synthesize_text(
text=plan.intro.text,
params=synth,
backend=intro_backend,
voice=intro_voice,
speed=intro_speed or request.speed,
chapter_sink=None,
preview_callback=lambda text: events.log(f" {text[:80]}"),
)
intro_emitted = True
events.log("Intro synthesized.")
# Chapter loop
for chapter_idx, chapter in enumerate(plan.chapters, 1):
check_cancelled()
chapter_display = f"Chapter {chapter_idx}/{len(plan.chapters)}: {chapter.title}"
events.log(f"Processing {chapter_display}")
# Resolve chapter voice
chapter_provider, chapter_voice, chapter_speed, chapter_steps = _resolve_voice(
voice_resolver, chapter.voice_spec, request
)
chapter_backend = pipeline_provider.get(chapter_provider, request.language, request.use_gpu)
# Per-chapter sink
chapter_sink: Optional[AudioSink] = None
chapter_path = None
if chapter_dir:
chapter_filename = sanitize_filename_for_chapter(chapter.title, chapter_idx)
chapter_path = chapter_dir / f"{chapter_filename}.{request.separate_chapters_format}"
chapter_sink = stack.enter_context(
open_audio_sink(
chapter_path,
request.separate_chapters_format,
cancel_check=check_cancelled,
)
)
result.chapter_paths.append(chapter_path)
# Intro delay before first chapter
if not intro_emitted and plan.intro and plan.intro.enabled:
# Intro will be emitted with first chapter
intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice(
voice_resolver, plan.intro.voice_spec, request
)
intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu)
synthesize_text(
text=plan.intro.text,
params=synth,
backend=intro_backend,
voice=intro_voice,
speed=intro_speed or request.speed,
chapter_sink=chapter_sink,
preview_callback=lambda text: events.log(f" Intro: {text[:80]}"),
)
intro_emitted = True
if request.chapter_intro_delay > 0:
_append_silence(
request.chapter_intro_delay,
chapter_sink=chapter_sink,
audio_sink=audio_sink,
stats=stats,
)
# Process heading
if chapter.title:
heading_text = _format_heading(chapter.title, chapter_idx, request)
if heading_text:
synthesize_text(
text=heading_text,
params=synth,
backend=chapter_backend,
voice=chapter_voice,
speed=chapter_speed or request.speed,
chapter_sink=chapter_sink,
preview_callback=lambda text: events.log(f" Title: {text[:80]}"),
)
if request.chapter_intro_delay > 0:
_append_silence(
request.chapter_intro_delay,
chapter_sink=chapter_sink,
audio_sink=audio_sink,
stats=stats,
)
# Process body segments
chapter_chunk_markers: List[Dict[str, Any]] = []
for seg_idx, segment in enumerate(chapter.segments):
check_cancelled()
# Resolve segment voice (may differ from chapter voice)
if segment.voice_spec != chapter.voice_spec:
seg_provider, seg_voice, seg_speed, seg_steps = _resolve_voice(
voice_resolver, segment.voice_spec, request
)
seg_backend = pipeline_provider.get(seg_provider, request.language, request.use_gpu)
else:
seg_provider = chapter_provider
seg_voice = chapter_voice
seg_speed = chapter_speed
seg_backend = chapter_backend
seg_start_time = stats.current_time
local_segments, accumulated_tokens = synthesize_text(
text=segment.text,
params=synth,
backend=seg_backend,
voice=seg_voice,
speed=seg_speed or request.speed,
chapter_sink=chapter_sink,
preview_callback=lambda text: events.log(f" {text[:80]}"),
)
# Process subtitles
if subtitle_writer and audio_sink and accumulated_tokens:
process_and_write_subtitles(
accumulated_tokens,
subtitle_writer,
subtitle_mode=request.subtitle_mode,
max_subtitle_words=request.max_subtitle_words,
lang_code=request.language,
use_spacy_segmentation=use_spacy,
fallback_end_time=stats.current_time,
)
# Record chunk marker
if segment.source in ("chunk", "voice_marker"):
chapter_chunk_markers.append({
"id": segment.chunk_id,
"chapter_index": chapter_idx - 1,
"chunk_index": segment.chunk_index or seg_idx,
"start": seg_start_time,
"end": stats.current_time,
"speaker_id": segment.speaker_id,
"voice": segment.voice_spec,
"level": segment.level or request.chunk_level,
"characters": len(segment.text),
})
# Silence between chapters
if chapter_idx < len(plan.chapters) and request.silence_between_chapters > 0:
_append_silence(
request.silence_between_chapters,
chapter_sink=chapter_sink,
audio_sink=audio_sink,
stats=stats,
)
# Close chapter sink
if chapter_sink:
chapter_sink.close()
# Add chapter marker
result.chapter_markers.append({
"chapter_index": chapter_idx - 1,
"title": chapter.title,
"start": stats.current_time - (stats.current_time - seg_start_time) if chapter.segments else stats.current_time,
"end": stats.current_time,
})
result.chunk_markers.extend(chapter_chunk_markers)
# Process outro
if plan.outro and plan.outro.enabled and merge_chapters:
events.log(f"Closing outro: {plan.outro.text[:80]}")
outro_provider, outro_voice, outro_speed, outro_steps = _resolve_voice(
voice_resolver, plan.outro.voice_spec, request
)
outro_backend = pipeline_provider.get(outro_provider, request.language, request.use_gpu)
# Silence before outro
if request.silence_between_chapters > 0:
_append_silence(
request.silence_between_chapters,
chapter_sink=None,
audio_sink=audio_sink,
stats=stats,
)
synthesize_text(
text=plan.outro.text,
params=synth,
backend=outro_backend,
voice=outro_voice,
speed=outro_speed or request.speed,
chapter_sink=None,
preview_callback=lambda text: events.log(f" {text[:80]}"),
)
events.log("Outro synthesized.")
# Set result metadata
result.total_chapters = len(plan.chapters)
result.total_segments = sum(len(ch.segments) for ch in plan.chapters)
result.total_characters = total_characters
if output_layout.project_root:
result.project_root = output_layout.project_root
return result
# ─── Helpers ────────────────────────────────────────────────────────
def _resolve_voice(
resolver: VoiceResolver,
voice_spec: str,
request: Any,
) -> Tuple[str, Any, Optional[float], Optional[int]]:
"""Resolve a voice spec and return (provider, voice, speed, steps)."""
try:
resolved = resolver.resolve(voice_spec)
return (
resolved.provider,
resolved.voice,
resolved.speed,
resolved.supertonic_steps,
)
except Exception:
# Fallback to base voice
resolved = resolver.resolve(request.voice or "M1")
return (
resolved.provider,
resolved.voice,
resolved.speed,
resolved.supertonic_steps,
)
def _base_name(request: Any) -> str:
"""Get base name for output file."""
from abogen.domain.output_paths import sanitize_output_stem
if request.original_filename:
return sanitize_output_stem(request.original_filename)
return "output"
def _format_heading(title: str, index: int, request: Any) -> str:
"""Format chapter heading for TTS."""
from abogen.domain.chapter_titles import format_spoken_chapter_title
if request.auto_prefix_chapter_titles:
return format_spoken_chapter_title(title, index, apply_prefix=True)
return title
def _append_silence(
duration: float,
*,
chapter_sink: Optional[AudioSink],
audio_sink: Optional[AudioSink],
stats: SegmentStats,
) -> None:
"""Append silence to sinks."""
from abogen.domain.audio_buffer import create_silence
silence = create_silence(duration)
if silence.size == 0:
return
if chapter_sink:
chapter_sink.write(silence)
if audio_sink:
audio_sink.write(silence)
stats.current_time += duration
+95
View File
@@ -0,0 +1,95 @@
"""Core models for conversion planning.
These dataclasses represent the structured plan for a conversion job.
They are UI-agnostic and describe WHAT to convert, not HOW to do it.
The planning flow:
ConversionRequest -> ConversionPlan -> ConversionResult
ConversionPlan contains:
- ChapterPlan[]: chapters with their segments
- SegmentPlan[]: individual text segments with voice specs
- OutputLayout: where to write outputs
- IntroOutroSpec: optional intro/outro
"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, Any, Dict, List, Optional
if TYPE_CHECKING:
from abogen.application.conversion_request import ConversionRequest
@dataclass
class SegmentPlan:
"""A single text segment with its voice specification.
This is the unified model for:
- Regular chapter body text
- PyQt voice markers (<<VOICE:F1>>)
- WebUI chunks with per-chunk voice/speaker
- Intro/outro text
- Chapter headings
"""
text: str
voice_spec: str
kind: str = "body" # intro, heading, body, outro
speaker_id: str = "narrator"
chunk_id: Optional[str] = None
chunk_index: Optional[int] = None
level: Optional[str] = None # chunk level (paragraph, sentence, etc.)
source: str = "chapter" # chapter, voice_marker, chunk
@dataclass
class ChapterPlan:
"""A chapter with its metadata and segments."""
index: int
title: str
original_title: str
body_text: str
segments: List[SegmentPlan]
voice_spec: str # default voice for this chapter
@dataclass
class OutputLayout:
"""Resolved output paths for a conversion job."""
parent_dir: Path
merged_path: Optional[Path] = None
chapter_dir: Optional[Path] = None
project_root: Optional[Path] = None
audio_dir: Optional[Path] = None
subtitle_dir: Optional[Path] = None
metadata_dir: Optional[Path] = None
@dataclass
class IntroOutroSpec:
"""Intro/outro specification with resolved text and voice."""
enabled: bool = False
text: str = ""
voice_spec: str = ""
kind: str = "intro" # intro or outro
@dataclass
class ConversionPlan:
"""Complete plan for a conversion job.
This is the output of the planning phase and input to the executor.
"""
request: ConversionRequest
metadata: Dict[str, Any]
chapters: List[ChapterPlan]
intro: Optional[IntroOutroSpec] = None
outro: Optional[IntroOutroSpec] = None
output_layout: Optional[OutputLayout] = None
+348
View File
@@ -0,0 +1,348 @@
"""Unified conversion planner.
Pure functions that take a ConversionRequest and produce a ConversionPlan.
No side effects, no I/O — all complexity from both UIs in one place.
This is Stage 2 of the conversion flow unification plan.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
from abogen.application.conversion_models import (
ChapterPlan,
ConversionPlan,
IntroOutroSpec,
OutputLayout,
SegmentPlan,
)
from abogen.application.conversion_request import ConversionRequest
from abogen.application.output_layout_service import resolve_output_layout
from abogen.domain.chapter_overrides import apply_chapter_overrides
from abogen.domain.file_type import auto_select_relevant_chapters
from abogen.domain.intro_outro import resolve_intro, resolve_outro
from abogen.domain.metadata_extraction import extract_metadata_for_file
from abogen.domain.metadata_merge import merge_metadata
from abogen.subtitle_utils import split_text_by_voice_markers
def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
"""Build a complete conversion plan from a request.
This is the single entry point that both UIs will call.
It handles all the planning logic that was previously duplicated
in both PyQt and WebUI conversion runners.
Args:
request: Normalized conversion request
Returns:
ConversionPlan with all chapters, segments, and output layout
Raises:
ValueError: If request is invalid (no source, no chapters, etc.)
"""
# 1. Extract and validate source
source_text = _extract_source_text(request)
if not source_text or not source_text.strip():
raise ValueError("No text content to convert")
# 2. Extract metadata
metadata = _extract_metadata(request)
# 3. Parse chapters
raw_chapters = _parse_chapters(source_text, request)
# 4. Apply chapter selection/overrides
selected_chapters = _apply_selection(raw_chapters, request)
# 5. Build segments for each chapter
chapters = _build_chapters(selected_chapters, request)
# 6. Build intro/outro
intro, outro = _build_intro_outro(metadata, request)
# 7. Resolve output layout
output_layout = resolve_output_layout(request)
return ConversionPlan(
request=request,
metadata=metadata,
chapters=chapters,
intro=intro,
outro=outro,
output_layout=output_layout,
)
def _extract_source_text(request: ConversionRequest) -> Optional[str]:
"""Extract text from request source."""
from abogen.subtitle_utils import clean_text
if request.direct_text:
return clean_text(request.direct_text)
if request.source_path and request.source_path.exists():
encoding = "utf-8"
try:
with open(request.source_path, "r", encoding=encoding, errors="replace") as f:
text = f.read()
except Exception:
return None
return clean_text(text)
return None
def _extract_metadata(request: ConversionRequest) -> Dict[str, Any]:
"""Extract metadata from source file."""
if request.direct_text:
return dict(request.metadata_tags)
if request.source_path and request.source_path.exists():
try:
extraction = extract_metadata_for_file(
str(request.source_path), is_direct_text=False
)
metadata = dict(extraction.metadata) if extraction.metadata else {}
except Exception:
metadata = {}
metadata = merge_metadata(metadata, request.metadata_tags)
return metadata
return dict(request.metadata_tags)
def _parse_chapters(
source_text: str, request: ConversionRequest
) -> List[Tuple[str, str, str]]:
"""Parse source text into raw chapters.
Returns list of (title, body_text, default_voice) tuples.
"""
from abogen.domain.text_chapters import parse_chapters_from_text
# Text is already cleaned in _extract_source_text, so clean=False here
chapters = parse_chapters_from_text(source_text, default_title="text", clean=False)
# Default voice from request
default_voice = request.voice or "M1"
return [(title, text, default_voice) for title, text in chapters]
def _apply_selection(
raw_chapters: List[Tuple[str, str, str]], request: ConversionRequest
) -> List[Tuple[str, str, str]]:
"""Apply chapter selection and overrides."""
from abogen.text_extractor import ExtractedChapter
# Convert to ExtractedChapter objects for auto_select_relevant_chapters
extracted = [
ExtractedChapter(title=title, text=text)
for title, text, _ in raw_chapters
]
# If user specified chapters, apply overrides
if request.chapter_overrides:
selected, _, diagnostics = apply_chapter_overrides(extracted, request.chapter_overrides)
if selected:
# Map back to (title, text, voice) tuples
result = []
for ch in selected:
# Find matching original chapter to get voice
voice = request.voice or "M1"
for orig_title, orig_text, orig_voice in raw_chapters:
if orig_title == ch.title:
voice = orig_voice
break
result.append((ch.title, ch.text or "", voice))
return result
# If no chapters selected, fall through to auto-selection
# Auto-select relevant chapters
from abogen.domain.file_type import infer_file_type
file_type = infer_file_type(request.source_path) if request.source_path else "text"
result = auto_select_relevant_chapters(extracted, file_type)
filtered = result.kept
if filtered:
# Map back to (title, text, voice) tuples
result = []
for ch in filtered:
voice = request.voice or "M1"
for orig_title, orig_text, orig_voice in raw_chapters:
if orig_title == ch.title:
voice = orig_voice
break
result.append((ch.title, ch.text or "", voice))
return result
# Fall back to all chapters
return raw_chapters
def _build_chapters(
selected_chapters: List[Tuple[str, str, str]], request: ConversionRequest
) -> List[ChapterPlan]:
"""Build ChapterPlan with SegmentPlan for each chapter."""
chapters = []
for idx, (title, body_text, default_voice) in enumerate(selected_chapters, 1):
# Build segments for this chapter
segments = _build_segments(body_text, default_voice, request)
chapter = ChapterPlan(
index=idx,
title=title,
original_title=title,
body_text=body_text,
segments=segments,
voice_spec=default_voice,
)
chapters.append(chapter)
return chapters
def _build_segments(
body_text: str, default_voice: str, request: ConversionRequest
) -> List[SegmentPlan]:
"""Build SegmentPlan list for a chapter's body text.
Handles voice markers (PyQt) and chunks (WebUI).
"""
segments = []
# Check for chunks (WebUI style)
if request.chunks:
# Group chunks by chapter (simplified — assume chunks are for current chapter)
for chunk_idx, chunk in enumerate(request.chunks):
chunk_text = chunk.get("normalized_text") or chunk.get("text", "")
if not chunk_text or not chunk_text.strip():
continue
chunk_voice = _resolve_chunk_voice(chunk, default_voice, request)
speaker_id = chunk.get("speaker_id", "narrator")
segments.append(
SegmentPlan(
text=chunk_text.strip(),
voice_spec=chunk_voice,
kind="body",
speaker_id=speaker_id,
chunk_id=chunk.get("id"),
chunk_index=chunk.get("chunk_index", chunk_idx),
level=chunk.get("level", request.chunk_level),
source="chunk",
)
)
return segments
# Check for voice markers (PyQt style)
# Detect markers even if validation fails (voice names may not be loaded yet)
from abogen.subtitle_utils import _VOICE_MARKER_SEARCH_PATTERN
has_voice_markers = bool(_VOICE_MARKER_SEARCH_PATTERN.search(body_text))
voice_segments, last_voice, valid_count, invalid_count = split_text_by_voice_markers(
body_text, default_voice
)
if has_voice_markers or (len(voice_segments) > 1):
# Voice markers were used
for voice_name, segment_text in voice_segments:
if not segment_text or not segment_text.strip():
continue
segments.append(
SegmentPlan(
text=segment_text.strip(),
voice_spec=voice_name,
kind="body",
source="voice_marker",
)
)
return segments
# No voice markers — single segment for entire body
if body_text and body_text.strip():
segments.append(
SegmentPlan(
text=body_text.strip(),
voice_spec=default_voice,
kind="body",
source="chapter",
)
)
return segments
def _resolve_chunk_voice(
chunk: Dict[str, Any], default_voice: str, request: ConversionRequest
) -> str:
"""Resolve voice for a chunk."""
# Check for speaker-based voice
speaker_id = chunk.get("speaker_id", "narrator")
if speaker_id and speaker_id != "narrator" and request.speakers:
speaker_config = request.speakers.get(speaker_id, {})
if isinstance(speaker_config, dict):
voice = speaker_config.get("voice")
if voice:
return voice
# Check for direct voice field
voice = chunk.get("voice")
if voice:
return voice
return default_voice
def _build_intro_outro(
metadata: Dict[str, Any], request: ConversionRequest
) -> Tuple[Optional[IntroOutroSpec], Optional[IntroOutroSpec]]:
"""Build intro and outro specs."""
intro_spec = None
outro_spec = None
# Intro
if request.read_title_intro:
resolved = resolve_intro(
metadata,
request.original_filename,
True,
request.voice or "M1",
request.voice or "M1",
[],
)
if resolved.enabled:
intro_spec = IntroOutroSpec(
enabled=True,
text=resolved.text,
voice_spec=resolved.voice_spec,
kind="intro",
)
# Outro
if request.read_closing_outro:
resolved = resolve_outro(
metadata,
request.original_filename,
True,
request.voice or "M1",
request.voice or "M1",
[],
)
if resolved.enabled:
outro_spec = IntroOutroSpec(
enabled=True,
text=resolved.text,
voice_spec=resolved.voice_spec,
kind="outro",
)
return intro_spec, outro_spec
# Output layout resolution is now in application/output_layout_service.py
+112
View File
@@ -0,0 +1,112 @@
"""Ports / interfaces for the conversion service.
These protocols define how the conversion service communicates with
the outside world (UI, TTS backends, voice resolvers).
The service ONLY depends on these interfaces, never on concrete
implementations (PyQt signals, Flask Job, etc.).
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, List, Optional, Protocol, runtime_checkable
class ConversionCancelled(Exception):
"""Raised when conversion is cancelled by user."""
pass
class ConversionEvents(Protocol):
"""UI-specific actions the conversion service delegates back to the caller.
Implementations:
- PyQt: emits signals (log_updated, progress_updated, etc.)
- WebUI: updates Job attributes (job.add_log, job.progress, etc.)
"""
def log(self, message: str, level: str = "info") -> None:
"""Log a message to the UI."""
...
def progress(self, processed: int, total: int, etr: str) -> None:
"""Update progress display."""
...
def check_cancelled(self) -> None:
"""Check if conversion was cancelled.
Should raise ConversionCancelled (or UI-specific exception)
if cancellation is requested. Normal return means "continue".
"""
...
class PipelineProvider(Protocol):
"""Provides access to TTS backends (Kokoro, SuperTonic, etc.).
Implementations:
- PyQt: wraps self.backend (single pipeline)
- WebUI: wraps PipelinePool (multi-provider)
"""
def get(self, provider: str, language: str, use_gpu: bool) -> Any:
"""Get a TTS backend instance."""
...
def dispose_all(self) -> None:
"""Dispose all backend resources."""
...
@dataclass
class ResolvedVoice:
"""A resolved voice ready for TTS synthesis."""
provider: str
resolved_spec: str
voice: Any # loaded voice tensor or name
speed: float
supertonic_steps: int
class VoiceResolver(Protocol):
"""Resolves voice specs into loaded voice objects.
Implementations:
- PyQt: wraps load_voice_cached + VoiceCache
- WebUI: wraps resolve_voice_choice + PipelinePool + VoiceCache
"""
def resolve(self, voice_spec: str) -> ResolvedVoice:
"""Resolve a voice spec into a loaded voice."""
...
class SubtitleWriter(Protocol):
"""Writes subtitle entries to a file."""
def open(self) -> None:
"""Open the subtitle file for writing."""
...
def write_entry(self, start: float, end: float, text: str) -> None:
"""Write a single subtitle entry."""
...
def close(self) -> None:
"""Close the subtitle file."""
...
class AudioSink(Protocol):
"""Writes audio data to a file."""
def write(self, audio: Any) -> None:
"""Write audio samples to the sink."""
...
def close(self) -> None:
"""Close the audio file."""
...
+156
View File
@@ -0,0 +1,156 @@
"""ConversionRequest — normalized input for a conversion job.
This is NOT a WebUI Job and NOT a PyQt ConversionThread state.
It describes the TASK, not the UI.
UI adapters are responsible for converting their respective state
into a ConversionRequest before calling ConversionService.run().
"""
from __future__ import annotations
import dataclasses
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional
from abogen.domain.enums import Language, OutputFormat, SaveMode, SubtitleFormat, SubtitleMode
class ConversionRequestError(ValueError):
"""Raised when ConversionRequest has invalid field values."""
# Numeric field constraints: attr -> (min, max)
_NUMERIC_CONSTRAINTS: dict[str, tuple[float, float | None]] = {
"max_subtitle_words": (1, 500),
"speed": (0.5, 3.0),
"supertonic_total_steps": (2, 15),
"silence_between_chapters": (0.0, None),
"chapter_intro_delay": (0.0, None),
}
# Enum-like fields that must be in allowed set
_ENUM_CONSTRAINTS: dict[str, tuple[str, ...]] = {
"chunk_level": ("paragraph", "sentence"),
"speaker_mode": ("single", "multi"),
}
@dataclass
class ConversionRequest:
"""Normalized request for a conversion job.
Only contains fields that describe the conversion task itself.
UI-only fields (display, logging, user prompts) stay in adapters.
Validation runs on creation via __post_init__:
- None values → replaced with field default (from declaration)
- Numeric fields → clamped to valid range
- String enums → validated against allowed set
"""
# --- Source ---
source_path: Optional[Path] = None
direct_text: Optional[str] = None
original_filename: str = ""
# --- TTS Settings ---
language: Language = Language.EN_US
tts_provider: str = "kokoro"
voice: str = "M1"
voice_profile: Optional[str] = None
speed: float = 1.0
use_gpu: bool = True
supertonic_total_steps: int = 5
# --- Output Format ---
output_format: OutputFormat = OutputFormat.WAV
subtitle_mode: SubtitleMode = SubtitleMode.DISABLED
subtitle_format: SubtitleFormat = SubtitleFormat.SRT
max_subtitle_words: int = 50
# --- Save Options ---
save_mode: SaveMode = SaveMode.SAVE_NEXT_TO_INPUT
output_folder: Optional[Path] = None
save_chapters_separately: bool = False
merge_chapters_at_end: bool = True
separate_chapters_format: OutputFormat = OutputFormat.WAV
save_as_project: bool = False
# --- Timing ---
silence_between_chapters: float = 2.0
chapter_intro_delay: float = 0.0
# --- Content Processing ---
replace_single_newlines: bool = False
read_title_intro: bool = False
read_closing_outro: bool = True
auto_prefix_chapter_titles: bool = True
normalize_chapter_opening_caps: bool = False
# --- Pronunciation / Normalization ---
pronunciation_overrides: List[Dict[str, Any]] = field(default_factory=list)
manual_overrides: List[Dict[str, Any]] = field(default_factory=list)
heteronym_overrides: List[Dict[str, Any]] = field(default_factory=list)
normalization_overrides: Optional[Dict[str, Any]] = None
# --- Chapter/Chunk Configuration ---
chapter_overrides: List[Dict[str, Any]] = field(default_factory=list)
chunks: List[Dict[str, Any]] = field(default_factory=list)
chunk_level: str = "paragraph"
speaker_mode: str = "single"
speakers: Dict[str, Any] = field(default_factory=dict)
# --- Metadata ---
metadata_tags: Dict[str, Any] = field(default_factory=dict)
# --- Artifacts ---
cover_image_path: Optional[Path] = None
cover_image_mime: Optional[str] = None
generate_epub3: bool = False
def __post_init__(self) -> None:
"""Resolve None → default, then validate and clamp."""
_apply_none_defaults(self)
if not self.tts_provider:
self.tts_provider = "kokoro"
_clamp_numerics(self)
_validate_enums(self)
def _apply_none_defaults(obj: ConversionRequest) -> None:
"""Replace None values with field defaults from dataclass declaration."""
for f in dataclasses.fields(obj):
if getattr(obj, f.name) is not None:
continue
if f.default is not dataclasses.MISSING:
setattr(obj, f.name, f.default)
elif f.default_factory is not dataclasses.MISSING:
setattr(obj, f.name, f.default_factory())
def _clamp_numerics(obj: ConversionRequest) -> None:
"""Clamp numeric fields to valid ranges."""
for attr, (min_v, max_v) in _NUMERIC_CONSTRAINTS.items():
val = getattr(obj, attr)
if val is None:
continue
if not isinstance(val, (int, float)):
raise ConversionRequestError(
f"{attr} must be a number, got {type(val).__name__}"
)
clamped = max(min_v, float(val))
if max_v is not None:
clamped = min(max_v, clamped)
setattr(obj, attr, clamped)
def _validate_enums(obj: ConversionRequest) -> None:
"""Validate string enum fields against allowed values."""
for attr, allowed in _ENUM_CONSTRAINTS.items():
val = getattr(obj, attr)
if val not in allowed:
raise ConversionRequestError(
f"{attr} must be one of {allowed}, got {val!r}"
)
+47
View File
@@ -0,0 +1,47 @@
"""ConversionResult — output of a successful conversion.
Returned by ConversionService.run() after all synthesis and finalization.
UI adapters consume this to update their respective state (Job, signals, etc.).
"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional
@dataclass
class ConversionResult:
"""Output of a successful conversion job."""
# --- Primary outputs ---
audio_path: Optional[Path] = None
subtitle_paths: List[Path] = field(default_factory=list)
chapter_paths: List[Path] = field(default_factory=list)
# --- Markers (for metadata/audiobookshelf) ---
chapter_markers: List[Dict[str, Any]] = field(default_factory=list)
chunk_markers: List[Dict[str, Any]] = field(default_factory=list)
# --- Metadata ---
metadata: Dict[str, Any] = field(default_factory=dict)
# --- Artifacts ---
artifacts: Dict[str, Path] = field(default_factory=dict)
project_root: Optional[Path] = None
epub_path: Optional[Path] = None
# --- Stats ---
total_chapters: int = 0
total_segments: int = 0
total_characters: int = 0
@dataclass
class ConversionError:
"""Error information when conversion fails."""
message: str
details: Optional[str] = None
is_cancelled: bool = False
+172
View File
@@ -0,0 +1,172 @@
"""ConversionService — main orchestrator for the conversion flow.
Ties together planner, executor, and finalizers into a single entry point.
Both UIs (PyQt, WebUI) call ConversionService.run() to execute a conversion.
Responsibilities:
- Prepare TTSContext (normalization settings, pronunciation rules)
- Build ConversionPlan via planner
- Execute conversion via executor
- Handle lifecycle (cleanup, error handling)
- Return ConversionResult
The service NEVER imports from PyQt or WebUI.
"""
from __future__ import annotations
from typing import Any, Callable, Dict, Optional
from abogen.application.conversion_executor import execute_conversion
from abogen.application.conversion_models import ConversionPlan
from abogen.application.conversion_planner import build_conversion_plan
from abogen.application.conversion_ports import (
ConversionEvents,
PipelineProvider,
VoiceResolver,
)
from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_result import ConversionResult
from abogen.domain.enums import SubtitleMode
from abogen.domain.normalization import TTSContext
from abogen.domain.split_pattern import get_split_pattern
def run_conversion(
request: ConversionRequest,
events: ConversionEvents,
pipeline_provider: PipelineProvider,
voice_resolver: VoiceResolver,
) -> ConversionResult:
"""Execute a conversion request and return the result.
This is the single entry point for both UIs. It orchestrates:
1. TTS context preparation
2. Conversion planning
3. Conversion execution
4. Resource cleanup
Args:
request: Normalized conversion request
events: UI-specific callbacks (log, progress, check_cancelled)
pipeline_provider: Provides TTS backends
voice_resolver: Resolves voice specs into loaded voices
Returns:
ConversionResult with paths and markers
Raises:
ConversionCancelled: If conversion was cancelled
ValueError: If request is invalid
Exception: On TTS or I/O errors
"""
try:
# Stage 1: Prepare TTS context
events.log("Preparing conversion pipeline")
tts_context = _prepare_tts_context(request, events)
# Stage 2: Build conversion plan
events.log("Building conversion plan")
plan = build_conversion_plan(request)
# Stage 3: Execute conversion
events.log("Starting conversion")
result = execute_conversion(
plan=plan,
events=events,
pipeline_provider=pipeline_provider,
voice_resolver=voice_resolver,
tts_context=tts_context,
)
# Stage 4: Finalize
events.log("Conversion complete")
return result
except Exception as e:
events.log(f"Conversion failed: {e}", level="error")
raise
def _prepare_tts_context(
request: ConversionRequest,
events: ConversionEvents,
) -> TTSContext:
"""Prepare TTSContext with normalization settings.
This compiles pronunciation/heteronym rules and creates the
normalization context used during conversion.
Args:
request: Conversion request with override settings
events: For logging warnings about missing features
Returns:
TTSContext ready for text normalization
"""
from abogen.domain.normalization import (
build_apostrophe_config,
get_runtime_settings,
)
from abogen.domain.pronunciation import (
compile_heteronym_sentence_rules,
compile_pronunciation_rules,
merge_pronunciation_overrides,
)
# Get runtime normalization settings
normalization_settings = get_runtime_settings()
# Build apostrophe config
apostrophe_config = build_apostrophe_config(
settings=normalization_settings,
)
# Check for num2words availability
if apostrophe_config.convert_numbers:
try:
import num2words # noqa: F401
except ImportError:
events.log(
"Number normalization is enabled but 'num2words' library is not available. "
"Numbers will NOT be converted to words.",
level="warning",
)
# Compute split pattern
split_pattern = get_split_pattern(
request.language or Language.EN_US,
request.subtitle_mode or SubtitleMode.DISABLED,
)
# Merge pronunciation overrides (manual + pronunciation)
# Create a mock job-like object for merge_pronunciation_overrides
class _MockJob:
def __init__(self, req):
self.pronunciation_overrides = req.pronunciation_overrides
self.manual_overrides = req.manual_overrides
self.heteronym_overrides = req.heteronym_overrides
merged_overrides = merge_pronunciation_overrides(_MockJob(request))
# Compile rules
pronunciation_rules = compile_pronunciation_rules(merged_overrides)
heteronym_rules = compile_heteronym_sentence_rules(request.heteronym_overrides)
if heteronym_rules:
events.log(
f"Applying {len(heteronym_rules)} heteronym override(s) during conversion.",
level="debug",
)
if pronunciation_rules:
events.log(
f"Applying {len(pronunciation_rules)} pronunciation override(s) during conversion.",
level="debug",
)
return TTSContext(
split_pattern=split_pattern,
pronunciation_rules=pronunciation_rules,
heteronym_rules=heteronym_rules,
normalization_overrides=request.normalization_overrides,
)
+150
View File
@@ -0,0 +1,150 @@
"""Output layout resolution service.
Determines where conversion outputs (audio, subtitles, metadata) should be written.
Extracted from conversion_planner.py as a standalone service per plan Stage 5.
Responsibilities:
- Resolve base output directory from save_mode and source_path
- Determine base filename from original_filename
- Find unique output path to avoid overwrites
- Resolve project layout (audio_dir, subtitle_dir, metadata_dir)
- Force merged output for m4b format
- Return OutputLayout dataclass
"""
from __future__ import annotations
from pathlib import Path
from typing import Optional
from abogen.application.conversion_models import OutputLayout
from abogen.application.conversion_request import ConversionRequest
from abogen.domain.enums import OutputFormat, SaveMode, SubtitleFormat
from abogen.domain.output_paths import (
resolve_project_layout,
resolve_unique_path,
sanitize_output_stem,
)
def resolve_output_layout(request: ConversionRequest) -> OutputLayout:
"""Resolve output paths for a conversion request.
This is the single entry point for output path resolution,
used by both UIs and the conversion service.
Args:
request: Normalized conversion request
Returns:
OutputLayout with resolved paths
"""
# Determine base output directory
if request.save_mode == SaveMode.CUSTOM_FOLDER and request.output_folder:
parent_dir = Path(request.output_folder)
elif request.source_path:
parent_dir = request.source_path.parent
else:
parent_dir = Path.cwd()
# Determine base name
if request.original_filename:
base_name = sanitize_output_stem(request.original_filename)
elif request.source_path:
base_name = sanitize_output_stem(request.source_path.stem)
else:
base_name = "output"
# Find unique output path
allowed_exts = {request.output_format, SubtitleFormat.SRT, SubtitleFormat.ASS, "vtt", "mp4", OutputFormat.M4B}
unique_base = resolve_unique_path(
parent_dir, base_name, "", allowed_extensions=allowed_exts
)
# Resolve project layout
project_root = None
audio_dir = parent_dir
subtitle_dir = None
metadata_dir = None
if request.save_as_project:
project_root, audio_dir, subtitle_dir, metadata_dir = resolve_project_layout(
original_filename=request.original_filename,
save_as_project=True,
base_dir=parent_dir,
)
return OutputLayout(
parent_dir=parent_dir,
project_root=project_root,
audio_dir=audio_dir,
subtitle_dir=subtitle_dir,
metadata_dir=metadata_dir,
)
def resolve_merged_path(
layout: OutputLayout,
request: ConversionRequest,
) -> Path:
"""Resolve the merged output audio file path.
Args:
layout: Resolved output layout
request: Conversion request
Returns:
Path to the merged output file
"""
base_name = sanitize_output_stem(
request.original_filename or "output"
)
return layout.audio_dir / f"{base_name}.{request.output_format}"
def resolve_chapter_path(
layout: OutputLayout,
request: ConversionRequest,
chapter_title: str,
chapter_index: int,
) -> Path:
"""Resolve the output path for a separate chapter file.
Args:
layout: Resolved output layout
request: Conversion request
chapter_title: Chapter title for filename
chapter_index: Chapter number (1-based)
Returns:
Path to the chapter output file
"""
import re
slug = re.sub(r'[^\w\s-]', '', chapter_title.lower())
slug = re.sub(r'[\s_]+', '_', slug).strip('_')
if not slug:
slug = f"chapter_{chapter_index}"
filename = f"{chapter_index:02d}_{slug}.{request.separate_chapters_format}"
return layout.audio_dir / "chapters" / filename
def should_merge_output(request: ConversionRequest) -> bool:
"""Determine if merged output is required.
Rules:
- m4b format always forces merged output
- If save_chapters_separately is False, merged is required
- Otherwise, use merge_chapters_at_end setting
Args:
request: Conversion request
Returns:
True if merged output should be created
"""
if request.output_format == OutputFormat.M4B:
return True
if not request.save_chapters_separately:
return True
return request.merge_chapters_at_end
+30 -30
View File
@@ -1,31 +1,31 @@
<?xml version="1.0" encoding="utf-8"?> <?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools --> <!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
<svg height="800px" width="800px" version="1.1" id="_x32_" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" <svg height="800px" width="800px" version="1.1" id="_x32_" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"
viewBox="0 0 512 512" xml:space="preserve"> viewBox="0 0 512 512" xml:space="preserve">
<style type="text/css"> <style type="text/css">
.st0{fill:#808080;} .st0{fill:#808080;}
</style> </style>
<g> <g>
<path class="st0" d="M502.325,307.303l-39.006-30.805c-6.215-4.908-9.665-12.429-9.668-20.348c0-0.084,0-0.168,0-0.252 <path class="st0" d="M502.325,307.303l-39.006-30.805c-6.215-4.908-9.665-12.429-9.668-20.348c0-0.084,0-0.168,0-0.252
c-0.014-7.936,3.44-15.478,9.667-20.396l39.007-30.806c8.933-7.055,12.093-19.185,7.737-29.701l-17.134-41.366 c-0.014-7.936,3.44-15.478,9.667-20.396l39.007-30.806c8.933-7.055,12.093-19.185,7.737-29.701l-17.134-41.366
c-4.356-10.516-15.167-16.86-26.472-15.532l-49.366,5.8c-7.881,0.926-15.656-1.966-21.258-7.586 c-4.356-10.516-15.167-16.86-26.472-15.532l-49.366,5.8c-7.881,0.926-15.656-1.966-21.258-7.586
c-0.059-0.06-0.118-0.119-0.177-0.178c-5.597-5.602-8.476-13.36-7.552-21.225l5.799-49.363 c-0.059-0.06-0.118-0.119-0.177-0.178c-5.597-5.602-8.476-13.36-7.552-21.225l5.799-49.363
c1.328-11.305-5.015-22.116-15.531-26.472L337.004,1.939c-10.516-4.356-22.646-1.196-29.701,7.736l-30.805,39.005 c1.328-11.305-5.015-22.116-15.531-26.472L337.004,1.939c-10.516-4.356-22.646-1.196-29.701,7.736l-30.805,39.005
c-4.908,6.215-12.43,9.665-20.349,9.668c-0.084,0-0.168,0-0.252,0c-7.935,0.014-15.477-3.44-20.395-9.667L204.697,9.675 c-4.908,6.215-12.43,9.665-20.349,9.668c-0.084,0-0.168,0-0.252,0c-7.935,0.014-15.477-3.44-20.395-9.667L204.697,9.675
c-7.055-8.933-19.185-12.092-29.702-7.736L133.63,19.072c-10.516,4.356-16.86,15.167-15.532,26.473l5.799,49.366 c-7.055-8.933-19.185-12.092-29.702-7.736L133.63,19.072c-10.516,4.356-16.86,15.167-15.532,26.473l5.799,49.366
c0.926,7.881-1.964,15.656-7.585,21.257c-0.059,0.059-0.118,0.118-0.178,0.178c-5.602,5.598-13.36,8.477-21.226,7.552 c0.926,7.881-1.964,15.656-7.585,21.257c-0.059,0.059-0.118,0.118-0.178,0.178c-5.602,5.598-13.36,8.477-21.226,7.552
l-49.363-5.799c-11.305-1.328-22.116,5.015-26.472,15.531L1.939,174.996c-4.356,10.516-1.196,22.646,7.736,29.701l39.006,30.805 l-49.363-5.799c-11.305-1.328-22.116,5.015-26.472,15.531L1.939,174.996c-4.356,10.516-1.196,22.646,7.736,29.701l39.006,30.805
c6.215,4.908,9.665,12.429,9.668,20.348c0,0.084,0,0.167,0,0.251c0.014,7.935-3.44,15.477-9.667,20.395L9.675,307.303 c6.215,4.908,9.665,12.429,9.668,20.348c0,0.084,0,0.167,0,0.251c0.014,7.935-3.44,15.477-9.667,20.395L9.675,307.303
c-8.933,7.055-12.092,19.185-7.736,29.701l17.134,41.365c4.356,10.516,15.168,16.86,26.472,15.532l49.366-5.799 c-8.933,7.055-12.092,19.185-7.736,29.701l17.134,41.365c4.356,10.516,15.168,16.86,26.472,15.532l49.366-5.799
c7.882-0.926,15.656,1.965,21.258,7.586c0.059,0.059,0.118,0.119,0.178,0.178c5.597,5.603,8.476,13.36,7.552,21.226l-5.799,49.364 c7.882-0.926,15.656,1.965,21.258,7.586c0.059,0.059,0.118,0.119,0.178,0.178c5.597,5.603,8.476,13.36,7.552,21.226l-5.799,49.364
c-1.328,11.305,5.015,22.116,15.532,26.472l41.366,17.134c10.516,4.356,22.646,1.196,29.701-7.736l30.804-39.005 c-1.328,11.305,5.015,22.116,15.532,26.472l41.366,17.134c10.516,4.356,22.646,1.196,29.701-7.736l30.804-39.005
c4.908-6.215,12.43-9.665,20.348-9.669c0.084,0,0.168,0,0.251,0c7.936-0.014,15.478,3.44,20.396,9.667l30.806,39.007 c4.908-6.215,12.43-9.665,20.348-9.669c0.084,0,0.168,0,0.251,0c7.936-0.014,15.478,3.44,20.396,9.667l30.806,39.007
c7.055,8.933,19.185,12.093,29.701,7.736l41.366-17.134c10.516-4.356,16.86-15.168,15.532-26.472l-5.8-49.366 c7.055,8.933,19.185,12.093,29.701,7.736l41.366-17.134c10.516-4.356,16.86-15.168,15.532-26.472l-5.8-49.366
c-0.926-7.881,1.965-15.656,7.586-21.257c0.059-0.059,0.119-0.119,0.178-0.178c5.602-5.597,13.36-8.476,21.225-7.552l49.364,5.799 c-0.926-7.881,1.965-15.656,7.586-21.257c0.059-0.059,0.119-0.119,0.178-0.178c5.602-5.597,13.36-8.476,21.225-7.552l49.364,5.799
c11.305,1.328,22.117-5.015,26.472-15.531l17.134-41.365C514.418,326.488,511.258,314.358,502.325,307.303z M281.292,329.698 c11.305,1.328,22.117-5.015,26.472-15.531l17.134-41.365C514.418,326.488,511.258,314.358,502.325,307.303z M281.292,329.698
c-39.68,16.436-85.172-2.407-101.607-42.087c-16.436-39.68,2.407-85.171,42.087-101.608c39.68-16.436,85.172,2.407,101.608,42.088 c-39.68,16.436-85.172-2.407-101.607-42.087c-16.436-39.68,2.407-85.171,42.087-101.608c39.68-16.436,85.172,2.407,101.608,42.088
C339.815,267.771,320.972,313.262,281.292,329.698z"/> C339.815,267.771,320.972,313.262,281.292,329.698z"/>
</g> </g>
</svg> </svg>

Before

Width:  |  Height:  |  Size: 2.6 KiB

After

Width:  |  Height:  |  Size: 2.5 KiB

-58
View File
@@ -63,64 +63,6 @@ SUPPORTED_INPUT_FORMATS = [
# 384 if self.lang_code in 'ab': # 384 if self.lang_code in 'ab':
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION = list(LANGUAGE_DESCRIPTIONS.keys()) SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION = list(LANGUAGE_DESCRIPTIONS.keys())
# Voice and sample text constants
VOICES_INTERNAL = [
"af_alloy",
"af_aoede",
"af_bella",
"af_heart",
"af_jessica",
"af_kore",
"af_nicole",
"af_nova",
"af_river",
"af_sarah",
"af_sky",
"am_adam",
"am_echo",
"am_eric",
"am_fenrir",
"am_liam",
"am_michael",
"am_onyx",
"am_puck",
"am_santa",
"bf_alice",
"bf_emma",
"bf_isabella",
"bf_lily",
"bm_daniel",
"bm_fable",
"bm_george",
"bm_lewis",
"ef_dora",
"em_alex",
"em_santa",
"ff_siwis",
"hf_alpha",
"hf_beta",
"hm_omega",
"hm_psi",
"if_sara",
"im_nicola",
"jf_alpha",
"jf_gongitsune",
"jf_nezumi",
"jf_tebukuro",
"jm_kumo",
"pf_dora",
"pm_alex",
"pm_santa",
"zf_xiaobei",
"zf_xiaoni",
"zf_xiaoxiao",
"zf_xiaoyi",
"zm_yunjian",
"zm_yunxi",
"zm_yunxia",
"zm_yunyang",
]
# Voice and sample text mapping # Voice and sample text mapping
SAMPLE_VOICE_TEXTS = { SAMPLE_VOICE_TEXTS = {
"a": "This is a sample of the selected voice.", "a": "This is a sample of the selected voice.",
+239
View File
@@ -0,0 +1,239 @@
"""Audio buffer operations for audiobook generation.
This module provides core audio buffer manipulation functions including:
- Silence generation
- Audio mixing
- Audio normalization
- Audio buffer resizing
"""
from __future__ import annotations
from typing import Optional
import numpy as np
# Standard sample rate used throughout the application
SAMPLE_RATE = 24000
def create_silence(duration_seconds: float) -> np.ndarray:
"""Create a silence audio buffer.
Args:
duration_seconds: Duration of silence in seconds.
Returns:
Numpy array of float32 zeros with length = duration_seconds * SAMPLE_RATE.
Returns empty array if duration is <= 0.
"""
if duration_seconds <= 0:
return np.array([], dtype="float32")
samples = int(round(duration_seconds * SAMPLE_RATE))
if samples <= 0:
return np.array([], dtype="float32")
return np.zeros(samples, dtype="float32")
def mix_audio(
target: np.ndarray,
source: np.ndarray,
start_sample: int,
end_sample: Optional[int] = None,
) -> np.ndarray:
"""Mix source audio into target buffer at specified position.
This performs additive mixing (target += source). The target buffer
is extended if necessary to accommodate the source audio.
Args:
target: The target audio buffer to mix into.
source: The source audio buffer to mix.
start_sample: Starting sample index in target buffer.
end_sample: Optional end sample index. If None, calculated from source length.
Returns:
The target buffer (possibly extended). If target was extended, returns new array.
"""
if source.size == 0:
return target
if end_sample is None:
end_sample = start_sample + len(source)
# Extend target buffer if needed
if end_sample > len(target):
new_length = end_sample
new_target = np.concatenate([
target,
np.zeros(new_length - len(target), dtype="float32")
])
target = new_target
# Perform the mix (additive)
target[start_sample:end_sample] += source
return target
def normalize_audio(
audio: np.ndarray,
target_peak: float = 1.0,
) -> np.ndarray:
"""Normalize audio buffer to prevent clipping.
If the audio exceeds the target peak (default 1.0), it is scaled down
proportionally to prevent distortion.
Args:
audio: Input audio buffer.
target_peak: Target maximum amplitude (default 1.0).
Returns:
Normalized audio buffer (new array, original is not modified).
"""
if audio.size == 0:
return audio.copy()
max_amplitude = float(np.abs(audio).max())
if max_amplitude <= target_peak:
return audio.copy()
# Scale down to prevent clipping
scale_factor = target_peak / max_amplitude
return (audio * scale_factor).astype("float32")
def ensure_buffer_size(
buffer: np.ndarray,
min_samples: int,
) -> np.ndarray:
"""Ensure audio buffer is at least min_samples long.
If buffer is shorter, it is extended with zeros.
Args:
buffer: Input audio buffer.
min_samples: Minimum required length in samples.
Returns:
Buffer of at least min_samples length (new array if extended).
"""
if len(buffer) >= min_samples:
return buffer
new_buffer = np.zeros(min_samples, dtype="float32")
new_buffer[:len(buffer)] = buffer
return new_buffer
def concatenate_audio(*buffers: np.ndarray) -> np.ndarray:
"""Concatenate multiple audio buffers.
Args:
*buffers: Audio buffers to concatenate.
Returns:
Single concatenated audio buffer.
"""
non_empty = [b for b in buffers if b.size > 0]
if not non_empty:
return np.array([], dtype="float32")
return np.concatenate(non_empty)
def audio_duration(audio: np.ndarray, sample_rate: int = SAMPLE_RATE) -> float:
"""Calculate duration of audio buffer in seconds.
Args:
audio: Audio buffer.
sample_rate: Sample rate in Hz (default SAMPLE_RATE).
Returns:
Duration in seconds.
"""
return len(audio) / sample_rate
def samples_for_duration(duration_seconds: float, sample_rate: int = SAMPLE_RATE) -> int:
"""Calculate number of samples for a given duration.
Args:
duration_seconds: Duration in seconds.
sample_rate: Sample rate in Hz (default SAMPLE_RATE).
Returns:
Number of samples (rounded to nearest integer), or 0 if duration is <= 0.
"""
if duration_seconds <= 0:
return 0
return int(round(duration_seconds * sample_rate))
def fit_audio_to_duration(
audio: np.ndarray,
target_duration: float,
sample_rate: int = SAMPLE_RATE,
) -> np.ndarray:
"""Pad or trim audio to match target duration.
Args:
audio: Input audio buffer.
target_duration: Desired duration in seconds.
sample_rate: Sample rate in Hz.
Returns:
Audio buffer of exact length target_duration * sample_rate.
"""
target_samples = int(target_duration * sample_rate)
if len(audio) < target_samples:
padding = np.zeros(target_samples - len(audio), dtype="float32")
return np.concatenate([audio, padding])
return audio[:target_samples]
def ffmpeg_time_stretch(
audio: np.ndarray,
speed_factor: float,
sample_rate: int = SAMPLE_RATE,
) -> np.ndarray:
"""Time-stretch audio using FFmpeg's atempo filter.
Args:
audio: Input audio buffer (float32).
speed_factor: Speed multiplier (>1.0 = faster).
sample_rate: Sample rate in Hz.
Returns:
Time-stretched audio buffer.
"""
import math
import subprocess
import static_ffmpeg
if speed_factor <= 1.0 or audio.size == 0:
return audio
static_ffmpeg.add_paths()
num_stages = max(1, int(math.ceil(math.log(speed_factor) / math.log(2.0))))
tempo = speed_factor ** (1.0 / num_stages)
filter_str = ",".join([f"atempo={tempo:.6f}"] * num_stages)
proc = subprocess.Popen(
[
"ffmpeg", "-y",
"-f", "f32le", "-ar", str(sample_rate), "-ac", "1",
"-i", "pipe:0",
"-filter:a", filter_str,
"-f", "f32le", "-ar", str(sample_rate), "-ac", "1",
"pipe:1",
],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
out, _ = proc.communicate(input=audio.tobytes())
return np.frombuffer(out, dtype="float32")
+118
View File
@@ -0,0 +1,118 @@
"""Audio helper utilities.
Functions for building ffmpeg commands, converting audio formats,
and applying chapter metadata to MP4 files.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Dict, List, Optional
import numpy as np
SAMPLE_RATE = 24000
def build_ffmpeg_command(path: Path, fmt: str, metadata: Optional[Dict[str, str]] = None) -> list[str]:
from abogen.infrastructure.exporters import ExportService
base = [
"ffmpeg",
"-y",
"-f",
"f32le",
"-ar",
str(SAMPLE_RATE),
"-ac",
"1",
"-i",
"pipe:0",
]
if fmt == "mp3":
base += ["-c:a", "libmp3lame", "-qscale:a", "2"]
elif fmt == "opus":
base += ["-c:a", "libopus", "-b:a", "24000"]
elif fmt == "m4b":
base += ["-c:a", "aac", "-q:a", "2", "-movflags", "+faststart+use_metadata_tags"]
else:
base += ["-c:a", "copy"]
if metadata:
svc = ExportService()
base.extend(svc._metadata_to_ffmpeg_args(metadata))
base.append(str(path))
return base
def to_float32(audio_segment) -> np.ndarray:
if audio_segment is None:
return np.zeros(0, dtype="float32")
tensor = audio_segment
if hasattr(tensor, "detach"):
tensor = tensor.detach()
if hasattr(tensor, "cpu"):
try:
tensor = tensor.cpu()
except Exception:
pass
if hasattr(tensor, "numpy"):
return np.asarray(tensor.numpy(), dtype="float32").reshape(-1)
return np.asarray(tensor, dtype="float32").reshape(-1)
def apply_m4b_chapters_with_mutagen(
audio_path: Path,
chapters: List[Dict[str, Any]],
) -> bool:
"""Apply chapter atoms to an MP4/M4B file using mutagen.
Returns True if chapters were written, False otherwise.
Raises ImportError if mutagen is not installed.
"""
if not chapters:
return False
from fractions import Fraction
from mutagen.mp4 import MP4, MP4Chapter # type: ignore[import]
mp4 = MP4(str(audio_path))
chapter_objects: List[MP4Chapter] = []
for index, entry in enumerate(sorted(chapters, key=lambda item: float(item.get("start") or 0.0))):
start_raw = entry.get("start")
if start_raw is None:
continue
try:
start_seconds = max(0.0, float(start_raw))
except (TypeError, ValueError):
continue
title_value = entry.get("title")
title_text = str(title_value) if title_value else f"Chapter {index + 1}"
start_fraction = Fraction(int(round(start_seconds * 1000)), 1000)
chapter_atom = MP4Chapter(start_fraction, title_text)
end_raw = entry.get("end")
if end_raw is not None:
try:
end_seconds = float(end_raw)
except (TypeError, ValueError):
end_seconds = None
if end_seconds is not None and end_seconds > start_seconds:
chapter_atom.end = Fraction(int(round(end_seconds * 1000)), 1000)
chapter_objects.append(chapter_atom)
if not chapter_objects:
return False
from typing import cast
mp4.chapters = cast(Any, chapter_objects)
mp4.save()
return True
+131
View File
@@ -0,0 +1,131 @@
"""Audio sink abstraction for unified audio output.
Provides a context-manager-based abstraction for writing audio data
to various output formats (WAV, FLAC via soundfile; compressed via ffmpeg).
Usage:
with open_audio_sink(path, "wav") as sink:
sink.write(audio_data)
"""
from __future__ import annotations
import os
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Optional
import numpy as np
from abogen.domain.audio_buffer import SAMPLE_RATE
from abogen.domain.audio_helpers import build_ffmpeg_command
@dataclass(frozen=True)
class AudioSink:
"""Represents an open audio output target."""
write: Callable[[np.ndarray], None]
close: Callable[[], None]
def __enter__(self) -> AudioSink:
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.close()
def _ensure_ffmpeg() -> None:
"""Ensure static ffmpeg binaries are on PATH."""
import static_ffmpeg # type: ignore
ffmpeg_cache_root = _get_ffmpeg_cache_root()
platform_cache = os.path.join(ffmpeg_cache_root, sys.platform)
os.makedirs(platform_cache, exist_ok=True)
try:
import static_ffmpeg.run as static_ffmpeg_run # type: ignore
static_ffmpeg_run.LOCK_FILE = os.path.join(ffmpeg_cache_root, "lock.file")
except Exception:
pass
static_ffmpeg.add_paths(weak=True, download_dir=platform_cache)
def _get_ffmpeg_cache_root() -> str:
from abogen.utils import get_internal_cache_path
return get_internal_cache_path("ffmpeg")
def open_audio_sink(
path: Path,
fmt: str,
*,
metadata: Optional[dict[str, str]] = None,
cancel_check: Optional[Callable[[], bool]] = None,
extra_ffmpeg_args: Optional[list[str]] = None,
ffmpeg_cmd: Optional[list[str]] = None,
) -> AudioSink:
"""Open an audio output sink for writing raw float32 PCM samples.
Args:
path: Output file path.
fmt: Output format ("wav", "flac", "mp3", "opus", "m4b").
metadata: Optional metadata dict (ignored when ffmpeg_cmd is provided).
cancel_check: Optional callable; if it returns True, writes are silently skipped.
extra_ffmpeg_args: Optional extra args inserted after ffmpeg header (ignored when ffmpeg_cmd is provided).
ffmpeg_cmd: Optional pre-built ffmpeg command list (for m4b with cover art etc.).
Returns:
AudioSink with write() and close() methods.
"""
fmt = fmt.lower()
if fmt in {"wav", "flac"}:
import soundfile as sf
soundfile_obj = sf.SoundFile(
path,
mode="w",
samplerate=SAMPLE_RATE,
channels=1,
format=fmt.upper(),
)
def _write_wav(data: np.ndarray) -> None:
if cancel_check and cancel_check():
return
soundfile_obj.write(data)
def _close_wav() -> None:
soundfile_obj.close()
return AudioSink(write=_write_wav, close=_close_wav)
# Compressed formats: pipe through ffmpeg
_ensure_ffmpeg()
if ffmpeg_cmd is not None:
cmd = list(ffmpeg_cmd)
else:
cmd = build_ffmpeg_command(path, fmt, metadata=metadata)
if extra_ffmpeg_args:
cmd[2:2] = extra_ffmpeg_args
process = subprocess.Popen(
cmd, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
)
def _write_compressed(data: np.ndarray) -> None:
if (cancel_check and cancel_check()) or process.stdin is None or process.stdin.closed:
return
process.stdin.write(data.tobytes())
def _close_compressed() -> None:
if process.stdin and not process.stdin.closed:
process.stdin.close()
process.wait()
return AudioSink(write=_write_compressed, close=_close_compressed)
+131
View File
@@ -0,0 +1,131 @@
"""Heuristics for classifying chapters as content vs. supplements.
A 'supplement' is any non-story material that a listener would typically
skip: title page, copyright, table of contents, acknowledgements, etc.
The scoring functions return a float; higher ⇒ more likely to be a
supplement. ``should_preselect_chapter`` turns that score into a
boolean suitable for a web form default.
"""
from __future__ import annotations
import re
from typing import Any, Dict, List, Tuple
# Compiled once at module load these are immutable.
_SUPPLEMENT_TITLE_PATTERNS: List[Tuple[re.Pattern[str], float]] = [
(re.compile(r"\btitle\s+page\b"), 3.0),
(re.compile(r"\bcopyright\b"), 2.4),
(re.compile(r"\btable\s+of\s+contents\b"), 2.8),
(re.compile(r"\bcontents\b"), 2.0),
(re.compile(r"\backnowledg(e)?ments?\b"), 2.0),
(re.compile(r"\bdedication\b"), 2.0),
(re.compile(r"\babout\s+the\s+author(s)?\b"), 2.4),
(re.compile(r"\balso\s+by\b"), 2.0),
(re.compile(r"\bpraise\s+for\b"), 2.0),
(re.compile(r"\bcolophon\b"), 2.2),
(re.compile(r"\bpublication\s+data\b"), 2.2),
(re.compile(r"\btranscriber'?s?\s+note\b"), 2.2),
(re.compile(r"\bglossary\b"), 2.2),
(re.compile(r"\bindex\b"), 2.0),
(re.compile(r"\bbibliograph(y|ies)\b"), 2.0),
(re.compile(r"\breferences\b"), 1.8),
(re.compile(r"\bappendix\b"), 1.9),
]
_CONTENT_TITLE_PATTERNS: List[re.Pattern[str]] = [
re.compile(r"\bchapter\b"),
re.compile(r"\bbook\b"),
re.compile(r"\bpart\b"),
re.compile(r"\bsection\b"),
re.compile(r"\bscene\b"),
re.compile(r"\bprologue\b"),
re.compile(r"\bepilogue\b"),
re.compile(r"\bintroduction\b"),
re.compile(r"\bstory\b"),
]
_SUPPLEMENT_TEXT_KEYWORDS: List[Tuple[str, float]] = [
("copyright", 1.2),
("all rights reserved", 1.1),
("isbn", 0.9),
("library of congress", 1.0),
("table of contents", 1.0),
("dedicated to", 0.8),
("acknowledg", 0.8),
("printed in", 0.6),
("permission", 0.6),
("publisher", 0.5),
("praise for", 0.9),
("also by", 0.9),
("glossary", 0.8),
("index", 0.8),
("newsletter", 3.2),
("mailing list", 2.6),
("sign-up", 2.2),
]
def supplement_score(title: str, text: str, index: int) -> float:
"""Return a score indicating how likely *title*/*text* is a supplement.
Higher values ⇒ more likely to be non-story material (title page,
copyright, acknowledgements, etc.).
"""
normalized_title = (title or "").lower()
score = 0.0
for pattern, weight in _SUPPLEMENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score += weight
for pattern in _CONTENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score -= 2.0
stripped_text = (text or "").strip()
length = len(stripped_text)
if length <= 150:
score += 0.9
elif length <= 400:
score += 0.6
elif length <= 800:
score += 0.35
lowercase_text = stripped_text.lower()
for keyword, weight in _SUPPLEMENT_TEXT_KEYWORDS:
if keyword in lowercase_text:
score += weight
if index == 0 and score > 0:
score += 0.25
return score
def should_preselect_chapter(
title: str,
text: str,
index: int,
total_count: int,
) -> bool:
"""Return True if the chapter should be *enabled* by default in the form.
A single chapter is always preselected. For multi-chapter books, the
chapter is preselected when its supplement score is below 1.9.
"""
if total_count <= 1:
return True
score = supplement_score(title, text, index)
return score < 1.9
def ensure_at_least_one_chapter_enabled(chapters: List[Dict[str, Any]]) -> None:
"""Mutate *chapters* in-place so that at least one has ``enabled=True``."""
if not chapters:
return
if any(chapter.get("enabled") for chapter in chapters):
return
best_index = max(range(len(chapters)), key=lambda idx: chapters[idx].get("characters", 0))
chapters[best_index]["enabled"] = True
+92
View File
@@ -0,0 +1,92 @@
from __future__ import annotations
from typing import Any, Dict, List, Optional, Tuple
from abogen.text_extractor import ExtractedChapter
from abogen.domain.voice_utils import coerce_truthy
def apply_chapter_overrides(
extracted: List[ExtractedChapter],
overrides: List[Dict[str, Any]],
) -> Tuple[List[ExtractedChapter], Dict[str, str], List[str]]:
if not overrides:
return [], {}, []
selected: List[ExtractedChapter] = []
metadata_updates: Dict[str, str] = {}
diagnostics: List[str] = []
for position, payload in enumerate(overrides):
if not isinstance(payload, dict):
diagnostics.append(
f"Skipped chapter override at position {position + 1}: unsupported payload type {type(payload).__name__}."
)
continue
enabled = coerce_truthy(payload.get("enabled", True))
payload["enabled"] = enabled
if not enabled:
continue
metadata_payload = payload.get("metadata") or {}
if isinstance(metadata_payload, dict):
for key, value in metadata_payload.items():
if value is None:
continue
metadata_updates[str(key)] = str(value)
base: Optional[ExtractedChapter] = None
idx_candidate = payload.get("index")
idx_normalized: Optional[int] = None
if isinstance(idx_candidate, int):
idx_normalized = idx_candidate
elif isinstance(idx_candidate, str):
try:
idx_normalized = int(idx_candidate)
except ValueError:
idx_normalized = None
if idx_normalized is not None and 0 <= idx_normalized < len(extracted):
base = extracted[idx_normalized]
payload["index"] = idx_normalized
if base is None:
source_title = payload.get("source_title")
if isinstance(source_title, str):
base = next((chapter for chapter in extracted if chapter.title == source_title), None)
if base is None:
candidate_title = payload.get("title")
if isinstance(candidate_title, str):
base = next((chapter for chapter in extracted if chapter.title == candidate_title), None)
text_override = payload.get("text")
if text_override is not None:
text_value = str(text_override)
elif base is not None:
text_value = base.text
else:
diagnostics.append(
f"Skipped chapter override at position {position + 1}: no text provided and no matching source chapter found."
)
continue
title_override = payload.get("title")
if title_override is not None:
title_value = str(title_override)
elif base is not None:
title_value = base.title
else:
title_value = f"Chapter {position + 1}"
if base and not payload.get("source_title"):
payload["source_title"] = base.title
payload["title"] = title_value
payload["text"] = text_value
payload["characters"] = len(text_value)
payload.setdefault("order", payload.get("order", position))
selected.append(ExtractedChapter(title=title_value, text=text_value))
return selected, metadata_updates, diagnostics
+204
View File
@@ -0,0 +1,204 @@
from __future__ import annotations
import re
from typing import List, Tuple
_HEADING_SANITIZE_RE = re.compile(r"[^a-z0-9]+")
_HEADING_NUMBER_PREFIX_RE = re.compile(
r"^\s*(?P<number>(?:\d+|[ivxlcdm]+))(?P<suffix>(?:[\s.:;-].*)?)$",
re.IGNORECASE,
)
_ACRONYM_ALLOWLIST = {
"AI", "API", "CPU", "DIY", "GPU", "HTML", "HTTP", "HTTPS", "ID",
"JSON", "MP3", "MP4", "M4B", "NASA", "OCR", "PDF", "SQL", "TV",
"TTS", "UK", "UN", "UFO", "OK", "URL", "USA", "US", "VR",
}
_ROMAN_NUMERAL_CHARS = frozenset("IVXLCDM")
_CAPS_WORD_RE = re.compile(r"[A-Z][A-Z0-9'\u2019-]*")
def simplify_heading_text(text: str) -> str:
raw = str(text or "").strip().lower()
if not raw:
return ""
simplified = _HEADING_SANITIZE_RE.sub("", raw)
if simplified.startswith("chapter"):
simplified = simplified[7:]
return simplified
def headings_equivalent(left: str, right: str) -> bool:
simple_left = simplify_heading_text(left)
simple_right = simplify_heading_text(right)
if not simple_left or not simple_right:
return False
if simple_left == simple_right:
return True
if simple_right.startswith(simple_left):
return True
if simple_left.startswith(simple_right):
return True
if len(simple_left) > 5 and simple_left in simple_right:
return True
return False
def strip_duplicate_heading_line(text: str, heading: str) -> Tuple[str, bool]:
source_text = str(text or "")
if not source_text:
return source_text, False
normalized_heading = simplify_heading_text(heading)
if not normalized_heading:
return source_text, False
lines = source_text.splitlines()
new_lines: List[str] = []
removed = False
for line in lines:
stripped = line.strip()
if not removed and stripped:
if headings_equivalent(stripped, heading):
removed = True
continue
new_lines.append(line)
if not removed:
return source_text, False
while new_lines and not new_lines[0].strip():
new_lines.pop(0)
return "\n".join(new_lines), True
def normalize_caps_word(word: str) -> str:
upper = word.upper()
letters = [char for char in upper if char.isalpha()]
if not letters:
return word
if upper in _ACRONYM_ALLOWLIST:
return word
if len(letters) <= 1:
return word
if all(char in _ROMAN_NUMERAL_CHARS for char in letters) and len(letters) <= 7:
return word
parts = re.split(r"(['\-\u2019])", word)
normalized_parts: List[str] = []
for part in parts:
if part in {"'", "-", "\u2019"}:
normalized_parts.append(part)
continue
if not part:
continue
normalized_parts.append(part[0].upper() + part[1:].lower())
return "".join(normalized_parts) or word
def normalize_chapter_opening_caps(text: str) -> Tuple[str, bool]:
if not text:
return text, False
leading_len = len(text) - len(text.lstrip())
leading = text[:leading_len]
working = text[leading_len:]
if not working:
return text, False
builder: List[str] = []
pos = 0
changed = False
while pos < len(working):
char = working[pos]
if char in "\r\n":
builder.append(working[pos:])
pos = len(working)
break
if char.isspace():
builder.append(char)
pos += 1
continue
if char.islower():
builder.append(working[pos:])
pos = len(working)
break
if not char.isalpha():
builder.append(char)
pos += 1
continue
match = _CAPS_WORD_RE.match(working, pos)
if not match:
builder.append(char)
pos += 1
continue
word = match.group(0)
if any(ch.islower() for ch in word):
builder.append(working[pos:])
pos = len(working)
break
normalized = normalize_caps_word(word)
if normalized != word:
changed = True
builder.append(normalized)
pos = match.end()
if pos < len(working):
builder.append(working[pos:])
if not changed:
return text, False
return leading + "".join(builder), True
def format_spoken_chapter_title(title: str, index: int, apply_prefix: bool) -> str:
base = str(title or "").strip()
if not base:
return f"Chapter {index}" if apply_prefix else ""
if not apply_prefix:
return base
lowered = base.lower()
if lowered.startswith("chapter") and (len(lowered) == 7 or not lowered[7].isalpha()):
return base
match = _HEADING_NUMBER_PREFIX_RE.match(base)
if match:
number = match.group("number") or ""
suffix = match.group("suffix") or ""
cleaned_suffix = suffix.lstrip(" .,:;-_ \t\u2013\u2014\u00b7\u2022")
if cleaned_suffix:
return f"Chapter {number}. {cleaned_suffix}"
return f"Chapter {number}"
return base
def apply_chapter_text_transforms(
text: str,
*,
heading_text: str,
raw_title: str,
strip_heading: bool,
normalize_caps: bool,
) -> Tuple[str, bool, bool]:
"""Strip duplicate heading and normalize opening caps.
Returns ``(text, heading_removed, caps_changed)``.
The caller is responsible for state updates (pending flags, logging,
dict mutation, ``continue``).
"""
heading_removed = False
caps_changed = False
if strip_heading and heading_text:
text, heading_removed = strip_duplicate_heading_line(text, heading_text)
if not heading_removed and raw_title:
match = _HEADING_NUMBER_PREFIX_RE.match(raw_title)
if match:
number = match.group("number")
if number:
text, heading_removed = strip_duplicate_heading_line(text, number)
if normalize_caps and text:
text, caps_changed = normalize_chapter_opening_caps(text)
return text, heading_removed, caps_changed
+75
View File
@@ -0,0 +1,75 @@
"""Chunk processing utilities.
Functions for grouping chunks, recording override usage, and selecting
text for TTS synthesis.
"""
from __future__ import annotations
from collections import defaultdict
from typing import Any, Dict, Iterable, Mapping, Optional
from abogen.pronunciation_store import increment_usage
def safe_int(value: Any, default: int = 0) -> int:
try:
return int(value)
except (TypeError, ValueError):
return default
def group_chunks_by_chapter(chunks: Iterable[Dict[str, Any]]) -> Dict[int, List[Dict[str, Any]]]:
grouped: Dict[int, List[Dict[str, Any]]] = defaultdict(list)
for entry in chunks or []:
if not isinstance(entry, dict):
continue
try:
chapter_index = int(entry.get("chapter_index", 0))
except (TypeError, ValueError):
chapter_index = 0
grouped[chapter_index].append(dict(entry))
for chapter_index, items in grouped.items():
items.sort(key=lambda payload: safe_int(payload.get("chunk_index")))
return grouped
def record_override_usage(
job: Any,
usage_counter: Mapping[str, int],
token_map: Mapping[str, str],
) -> None:
if not usage_counter:
return
language = getattr(job, "language", "") or "a"
for normalized, amount in usage_counter.items():
if amount <= 0:
continue
token_value = token_map.get(normalized, normalized)
try:
increment_usage(language=language, token=token_value, amount=int(amount))
except Exception: # pragma: no cover - defensive logging
job.add_log(f"Failed to record usage for override {token_value}", level="warning")
def chunk_text_for_tts(entry: Mapping[str, Any]) -> str:
"""Choose the best source text for synthesis.
We must prefer the raw chunk text (``text`` / ``original_text``) so
manual/pronunciation overrides can match against the original tokens
(e.g. censored words like ``Unfu*k``). ``normalized_text`` may have
already been run through ``normalize_for_pipeline``, which can remove
punctuation and prevent overrides from triggering.
"""
if not isinstance(entry, Mapping):
return ""
return str(
entry.get("text")
or entry.get("original_text")
or entry.get("normalized_text")
or ""
).strip()
+226
View File
@@ -0,0 +1,226 @@
"""Shared TTS iteration loop used by both WebUI and PyQt conversion runners.
The core pattern is identical across both UIs:
for seg in tts_segments(text, backend, voice, speed, split_pattern, current_time):
check_cancel()
update_progress(seg)
write_audio(seg, sink)
accumulate_subtitles(seg)
After the loop, the caller processes accumulated subtitle tokens.
This module provides ``run_tts_segment_loop`` which encapsulates that
iteration, and ``synthesize_text`` which adds normalization on top —
the single entry point both UIs should call for text-to-speech.
"""
from __future__ import annotations
import time
from dataclasses import dataclass, field
from typing import Any, Callable, List, Optional, Protocol
from abogen.domain.audio_sink import AudioSink
from abogen.domain.conversion_pipeline import tts_segments
from abogen.domain.enums import SubtitleMode
from abogen.domain.normalization import TTSContext
from abogen.domain.progress import calc_etr_str
from abogen.domain.subtitle_generation import process_subtitle_tokens
class CancelChecker(Protocol):
"""Returns True if conversion has been cancelled."""
def __call__(self) -> bool: ...
@dataclass
class SegmentStats:
"""Running statistics updated per TTS segment."""
processed_chars: int = 0
current_time: float = 0.0
etr_start_time: float = field(default_factory=time.time)
total_characters: int = 0
@dataclass
class SegmentInfo:
"""Read-only info about a TTS segment, passed to on_segment callback."""
graphemes: str
audio: Any
tokens: list
duration: float
chunk_start: float
def run_tts_segment_loop(
*,
text: str,
params: SynthParams,
backend: Any,
voice: Any,
speed: float,
split_pattern: str,
chapter_sink: Optional[AudioSink] = None,
preview_callback: Optional[Callable[[str], None]] = None,
on_segment: Optional[Callable[[SegmentInfo], None]] = None,
) -> tuple[int, list]:
"""Run the core TTS segment iteration loop.
Args:
text: Normalized text to synthesize.
params: Common synthesis parameters (stats, callbacks, sinks, etc.).
backend: TTS pipeline instance (Kokoro or Supertonic).
voice: Voice name/id for the backend.
speed: Speech speed multiplier.
split_pattern: Regex pattern used by the TTS engine for sentence splitting.
preview_callback: Called with a short preview string per segment.
on_segment: Called with a SegmentInfo for each segment *before*
audio is written. Useful for callers that need per-segment
subtitle processing (e.g. PyQt dual-writer pattern).
When provided, the default subtitle accumulation is skipped.
Returns:
Tuple of (segment_count, accumulated_subtitle_tokens).
The caller is responsible for processing subtitle tokens via
``process_subtitle_tokens`` and writing entries to subtitle writers.
"""
local_segments = 0
accumulated_tokens: list[dict] = []
for seg in tts_segments(
text,
backend=backend,
voice=voice,
speed=speed,
split_pattern=split_pattern,
current_time=params.stats.current_time,
):
if params.check_cancel():
break
local_segments += 1
params.stats.processed_chars += len(seg.graphemes)
# Progress
if params.stats.total_characters:
percent = min(int(params.stats.processed_chars / params.stats.total_characters * 100), 99)
else:
percent = 0 if params.stats.processed_chars == 0 else 99
etr_str = calc_etr_str(
time.time() - params.stats.etr_start_time,
params.stats.processed_chars,
params.stats.total_characters,
)
params.on_progress(percent, etr_str)
# Preview / log
if preview_callback:
preview_callback(seg.graphemes or "[silence]")
# Per-segment callback (for callers needing segment-level access)
if on_segment:
info = SegmentInfo(
graphemes=seg.graphemes,
audio=seg.audio,
tokens=list(seg.tokens) if seg.tokens else [],
duration=seg.duration,
chunk_start=getattr(seg, "chunk_start", params.stats.current_time),
)
on_segment(info)
# Write audio
if chapter_sink:
chapter_sink.write(seg.audio)
if params.audio_sink:
params.audio_sink.write(seg.audio)
# Accumulate subtitle tokens (default path; skipped if on_segment handles it)
if not on_segment and params.subtitle_mode != SubtitleMode.DISABLED and seg.tokens:
accumulated_tokens.extend(seg.tokens)
# Update timing
if params.audio_sink:
params.stats.current_time += seg.duration
return local_segments, accumulated_tokens
def process_and_write_subtitles(
accumulated_tokens: list[dict],
subtitle_writer: Any,
*,
subtitle_mode: str,
max_subtitle_words: int,
lang_code: str,
use_spacy_segmentation: bool,
fallback_end_time: float,
) -> None:
"""Process accumulated subtitle tokens and write entries to a subtitle writer.
This is the standard subtitle post-processing step shared by both UIs.
"""
if not accumulated_tokens or not subtitle_writer:
return
new_entries: list[tuple] = []
process_subtitle_tokens(
accumulated_tokens,
new_entries,
max_subtitle_words,
subtitle_mode,
lang_code,
use_spacy_segmentation=use_spacy_segmentation,
fallback_end_time=fallback_end_time,
)
for start, end, text in new_entries:
subtitle_writer.write_entry(start=start, end=end, text=text)
@dataclass(frozen=True)
class SynthParams:
"""Common parameters for synthesize_text calls.
Packed once by the executor to avoid repeating identical kwargs.
When adding new common params, change only this dataclass.
"""
tts_context: TTSContext
stats: SegmentStats
check_cancel: CancelChecker
on_progress: Callable[[int, str], None]
audio_sink: Optional[AudioSink] = None
subtitle_mode: str = "Disabled"
max_subtitle_words: int = 50
lang_code: str = "a"
use_spacy_segmentation: bool = False
def synthesize_text(
*,
text: str,
params: SynthParams,
backend: Any,
voice: Any,
speed: float,
chapter_sink: Optional[AudioSink] = None,
preview_callback: Optional[Callable[[str], None]] = None,
on_segment: Optional[Callable[[SegmentInfo], None]] = None,
split_pattern_override: Optional[str] = None,
) -> tuple[int, list]:
"""Normalize text and run TTS — the single entry point for both UIs.
Combines TTSContext.normalize() + run_tts_segment_loop() into one call.
UI-specific concerns (provider resolution, progress display) stay in the UI.
"""
normalized = params.tts_context.normalize(text)
return run_tts_segment_loop(
text=normalized,
params=params,
backend=backend,
voice=voice,
speed=speed,
split_pattern=split_pattern_override or params.tts_context.split_pattern,
chapter_sink=chapter_sink,
preview_callback=preview_callback,
on_segment=on_segment,
)
+244
View File
@@ -0,0 +1,244 @@
"""Shared TTS emission pipeline.
Provides the core TTS emission loop used by both WebUI and PyQt conversion runners.
The caller handles audio I/O, progress reporting, and subtitle writing.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass, field
from abogen.domain.enums import SubtitleMode
from typing import Any, Callable, Dict, Iterator, List, Optional
import numpy as np
from abogen.domain.audio_helpers import to_float32
from abogen.domain.normalization import prepare_text_for_tts
from abogen.domain.tokens import FakeToken
from abogen.domain.audio_buffer import SAMPLE_RATE
logger = logging.getLogger(__name__)
@dataclass
class SegmentResult:
"""One TTS segment emitted by the pipeline."""
graphemes: str
audio: np.ndarray
duration: float
chunk_start: float
tokens: List[Dict[str, Any]] = field(default_factory=list)
def tts_segments(
text: str,
*,
backend: Any,
voice: Any,
speed: float,
split_pattern: str,
current_time: float = 0.0,
) -> Iterator[SegmentResult]:
"""Invoke TTS backend on (already normalized) text and yield SegmentResults.
Use this when you've already normalized the text yourself (e.g. after
spaCy sentence segmentation). For raw text, use emit_text_segments() instead.
Args:
text: Already-normalized text to synthesize.
backend: TTS pipeline callable.
voice: Resolved voice.
speed: TTS speed multiplier.
split_pattern: Regex pattern for sentence splitting.
current_time: Current position in the audio timeline (seconds).
Yields:
SegmentResult for each non-empty TTS segment.
"""
segment_iter = backend(
text,
voice=voice,
speed=speed,
split_pattern=split_pattern,
)
chunk_start = current_time
for segment in segment_iter:
graphemes_raw = getattr(segment, "graphemes", "") or ""
graphemes = graphemes_raw.strip()
audio = to_float32(getattr(segment, "audio", None))
if audio.size == 0:
continue
duration = len(audio) / SAMPLE_RATE
tokens_list = getattr(segment, "tokens", [])
if not tokens_list and graphemes:
tokens_list = [FakeToken(graphemes, 0, duration)]
tokens = [
{
"start": chunk_start + (tok.start_ts or 0),
"end": chunk_start + (tok.end_ts or 0),
"text": tok.text,
"whitespace": tok.whitespace,
}
for tok in tokens_list
]
yield SegmentResult(
graphemes=graphemes,
audio=audio,
duration=duration,
chunk_start=chunk_start,
tokens=tokens,
)
chunk_start += duration
def emit_text_segments(
text: str,
*,
backend: Any,
voice: Any,
speed: float,
split_pattern: str,
current_time: float = 0.0,
# normalization
heteronym_rules: Any = None,
pronunciation_rules: Any = None,
normalization_overrides: Any = None,
usage_counter: Optional[Dict[str, int]] = None,
) -> Iterator[SegmentResult]:
"""Normalize text and yield SegmentResults from the TTS backend.
This is the innermost TTS emission loop shared by both UIs. It handles:
1. Text normalization (heteronym + pronunciation rules)
2. TTS backend invocation
3. Segment iteration with token extraction
The caller is responsible for:
- Writing audio to sinks
- Accumulating tokens for subtitle processing
- Progress tracking and cancellation
- Error handling
Args:
text: Raw text to synthesize.
backend: TTS pipeline callable (kokoro or supertonic).
voice: Resolved voice for TTS.
speed: TTS speed multiplier.
split_pattern: Regex pattern for sentence splitting.
current_time: Current position in the audio timeline (seconds).
heteronym_rules: Compiled heteronym rules.
pronunciation_rules: Compiled pronunciation rules.
normalization_overrides: User normalization overrides.
usage_counter: Counter for normalization statistics.
Yields:
SegmentResult for each non-empty TTS segment.
"""
source_text = str(text or "")
normalized = prepare_text_for_tts(
source_text,
heteronym_rules=heteronym_rules,
pronunciation_rules=pronunciation_rules,
normalization_overrides=normalization_overrides,
usage_counter=usage_counter,
)
yield from tts_segments(
normalized,
backend=backend,
voice=voice,
speed=speed,
split_pattern=split_pattern,
current_time=current_time,
)
def emit_text_to_sinks(
text: str,
*,
backend: Any,
voice: Any,
speed: float,
split_pattern: str,
current_time: float = 0.0,
# sinks
audio_sink: Any = None,
chapter_sink: Any = None,
# subtitle
subtitle_writer: Any = None,
subtitle_mode: str = "Disabled",
subtitle_lang: str = "a",
max_subtitle_words: int = 50,
use_spacy_segmentation: bool = True,
# normalization
heteronym_rules: Any = None,
pronunciation_rules: Any = None,
normalization_overrides: Any = None,
usage_counter: Optional[Dict[str, int]] = None,
) -> tuple[int, float, List[Dict[str, Any]]]:
"""Emit TTS audio for text, writing to sinks and collecting subtitle tokens.
Convenience wrapper around emit_text_segments() that handles audio writing
and token accumulation. Returns stats for the caller to update progress.
Returns:
Tuple of (segments_emitted, new_current_time, accumulated_tokens).
"""
from abogen.domain.subtitle_generation import process_subtitle_tokens
segments_emitted = 0
accumulated_tokens: List[Dict[str, Any]] = []
for seg in emit_text_segments(
text,
backend=backend,
voice=voice,
speed=speed,
split_pattern=split_pattern,
current_time=current_time,
heteronym_rules=heteronym_rules,
pronunciation_rules=pronunciation_rules,
normalization_overrides=normalization_overrides,
usage_counter=usage_counter,
):
segments_emitted += 1
# Write audio
if chapter_sink:
chapter_sink.write(seg.audio)
if audio_sink:
audio_sink.write(seg.audio)
# Collect tokens
accumulated_tokens.extend(seg.tokens)
# Flush subtitle tokens
if subtitle_writer and accumulated_tokens:
_use_spacy = subtitle_mode not in (SubtitleMode.DISABLED, SubtitleMode.LINE)
new_entries: List[tuple] = []
process_subtitle_tokens(
accumulated_tokens,
new_entries,
max_subtitle_words,
subtitle_mode,
subtitle_lang,
use_spacy_segmentation=_use_spacy,
fallback_end_time=current_time + sum(t["end"] - t["start"] for t in accumulated_tokens if accumulated_tokens),
)
for start, end, text_entry in new_entries:
subtitle_writer.write_entry(start=start, end=end, text=text_entry)
new_time = current_time
if accumulated_tokens:
new_time = max(t["end"] for t in accumulated_tokens)
return segments_emitted, new_time, accumulated_tokens
+31
View File
@@ -0,0 +1,31 @@
from __future__ import annotations
import platform as _platform
def select_device() -> str:
"""Return the best available compute device (``"mps"``, ``"cuda"``, or ``"cpu"``).
Checks ``torch`` availability at runtime so this can be called from
any context without requiring torch at import time.
"""
try:
import torch # type: ignore[import-not-found]
except Exception:
return "cpu"
system = _platform.system()
if system == "Darwin" and _platform.processor() == "arm":
try:
if torch.backends.mps.is_available(): # type: ignore[union-attr]
return "mps"
except Exception:
pass
return "cpu"
try:
if torch.cuda.is_available(): # type: ignore[union-attr]
return "cuda"
except Exception:
pass
return "cpu"
+179
View File
@@ -0,0 +1,179 @@
"""Domain enums — typed constants for values tied to business logic.
Using Enum instead of bare strings ensures:
- Invalid values are caught at construction time
- IDE autocomplete and type checking work
- Adding new values is explicit (must update Enum)
"""
from __future__ import annotations
from enum import Enum
from pathlib import Path
class SubtitleMode(str, Enum):
"""Subtitle generation mode."""
DISABLED = "Disabled"
LINE = "Line"
SENTENCE = "Sentence"
SENTENCE_COMMA = "Sentence + Comma"
SENTENCE_HIGHLIGHT = "Sentence + Highlighting"
@classmethod
def from_str(cls, value: str) -> SubtitleMode:
"""Parse from user input: case-insensitive, strips whitespace."""
normalized = value.strip()
for member in cls:
if member.value.lower() == normalized.lower():
return member
raise ValueError(f"Invalid SubtitleMode: {value!r}. Valid: {[m.value for m in cls]}")
class OutputFormat(str, Enum):
"""Audio output format."""
WAV = "wav"
MP3 = "mp3"
FLAC = "flac"
OPUS = "opus"
M4B = "m4b"
@property
def dot_ext(self) -> str:
"""File extension with dot: '.wav', '.mp3', etc."""
return f".{self.value}"
@property
def is_lossless(self) -> bool:
"""True for lossless formats."""
return self in (self.WAV, self.FLAC)
@classmethod
def from_str(cls, value: str) -> OutputFormat:
"""Parse from user input: strips dot prefix, case-insensitive."""
normalized = value.strip().lstrip(".").lower()
for member in cls:
if member.value == normalized:
return member
raise ValueError(f"Invalid OutputFormat: {value!r}. Valid: {[m.value for m in cls]}")
class SaveMode(str, Enum):
"""Where to save the output file."""
SAVE_NEXT_TO_INPUT = "save_next_to_input"
SAVE_TO_DESKTOP = "save_to_desktop"
CHOOSE_OUTPUT_FOLDER = "choose_output_folder"
DEFAULT_OUTPUT = "default_output"
CUSTOM_FOLDER = "custom_folder"
class SubtitleFormat(str, Enum):
"""Subtitle file format."""
SRT = "srt"
ASS = "ass"
VTT = "vtt"
@property
def dot_ext(self) -> str:
"""File extension with dot: '.srt', '.ass'."""
return f".{self.value}"
@classmethod
def from_str(cls, value: str) -> SubtitleFormat:
"""Parse from user input: strips dot prefix, case-insensitive."""
normalized = value.strip().lstrip(".").lower()
for member in cls:
if member.value == normalized:
return member
raise ValueError(f"Invalid SubtitleFormat: {value!r}. Valid: {[m.value for m in cls]}")
class InputFormat(str, Enum):
"""Input file format."""
EPUB = "epub"
PDF = "pdf"
TXT = "txt"
MD = "md"
SRT = "srt"
ASS = "ass"
VTT = "vtt"
@property
def is_book(self) -> bool:
"""True for book/document formats (epub, pdf, txt, md)."""
return self in (self.EPUB, self.PDF, self.TXT, self.MD)
@property
def is_subtitle(self) -> bool:
"""True for subtitle formats (srt, ass, vtt)."""
return self in (self.SRT, self.ASS, self.VTT)
@property
def dot_ext(self) -> str:
"""File extension with dot: '.epub', '.srt', etc."""
return f".{self.value}"
@classmethod
def from_path(cls, path: Path) -> InputFormat:
"""Detect format from file path extension."""
suffix = path.suffix.lower().lstrip(".")
if suffix == "markdown":
return cls.MD
try:
return cls(suffix)
except ValueError:
raise ValueError(f"Unsupported input format: {path.suffix!r}. Supported: {[m.value for m in cls]}")
class Language(str, Enum):
"""TTS language code (ISO 639-1 with region where needed).
Each engine (Kokoro, Supertonic) maps these to its own
internal language identifiers.
"""
EN_US = "en-US"
EN_GB = "en-GB"
ES = "es"
FR = "fr"
HI = "hi"
IT = "it"
JA = "ja"
PT_BR = "pt-BR"
ZH = "zh"
@property
def display_name(self) -> str:
"""Human-readable language name."""
_names = {
"en-US": "American English",
"en-GB": "British English",
"es": "Spanish",
"fr": "French",
"hi": "Hindi",
"it": "Italian",
"ja": "Japanese",
"pt-BR": "Brazilian Portuguese",
"zh": "Mandarin Chinese",
}
return _names[self.value]
@property
def is_cjk(self) -> bool:
"""True for CJK languages (Chinese, Japanese)."""
return self in (self.ZH, self.JA)
@property
def supports_subtitle_tokens(self) -> bool:
"""True if this language generates timestamped tokens for subtitles."""
return self in (self.EN_US, self.EN_GB)
@classmethod
def from_str(cls, value: str) -> Language:
"""Parse from user input: ISO code, case-insensitive."""
if isinstance(value, Language):
return value
normalized = value.strip()
for member in cls:
if member.value.lower() == normalized.lower():
return member
raise ValueError(f"Invalid Language: {value!r}. Valid: {[m.value for m in cls]}")
+136
View File
@@ -0,0 +1,136 @@
from __future__ import annotations
import re
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Tuple
from abogen.text_extractor import ExtractedChapter
_SIGNIFICANT_LENGTH_THRESHOLDS: Dict[str, int] = {"epub": 1000, "markdown": 500}
_MIN_SHORT_CONTENT: Dict[str, int] = {"epub": 240, "markdown": 160}
_STRUCTURAL_KEYWORDS = (
"preface",
"prologue",
"introduction",
"foreword",
"epilogue",
"afterword",
"appendix",
"acknowledgment",
"acknowledgement",
)
_STRUCTURAL_MIN_LENGTH = 120
_MAX_SHORT_CHAPTERS = 2
@dataclass
class ChapterFilterResult:
kept: List[ExtractedChapter]
skipped: List[Tuple[str, int]]
def infer_file_type(path: Path) -> str:
suffix = path.suffix.lower()
if suffix == ".epub":
return "epub"
if suffix in {".md", ".markdown"}:
return "markdown"
if suffix == ".pdf":
return "pdf"
if suffix == ".txt":
return "text"
return suffix.lstrip(".") or "text"
def looks_structural(title: str) -> bool:
lowered = title.strip().lower()
if not lowered:
return False
return any(keyword in lowered for keyword in _STRUCTURAL_KEYWORDS)
def chapter_label(file_type: str) -> str:
return "chapters" if file_type.lower() in {"epub", "markdown"} else "pages"
def auto_select_relevant_chapters(
chapters: List[ExtractedChapter],
file_type: str,
) -> ChapterFilterResult:
if not chapters:
return ChapterFilterResult(kept=[], skipped=[])
normalized = file_type.lower()
threshold = _SIGNIFICANT_LENGTH_THRESHOLDS.get(normalized, 0)
min_short = _MIN_SHORT_CONTENT.get(normalized, 0)
kept: List[ExtractedChapter] = []
skipped: List[Tuple[str, int]] = []
short_kept = 0
for chapter in chapters:
stripped = chapter.text.strip()
length = len(stripped)
if length == 0:
skipped.append((chapter.title, length))
continue
keep = False
if threshold == 0:
keep = True
elif length >= threshold:
keep = True
elif not kept:
keep = True
elif min_short and length >= min_short and short_kept < _MAX_SHORT_CHAPTERS:
keep = True
short_kept += 1
elif looks_structural(chapter.title) and length >= _STRUCTURAL_MIN_LENGTH:
keep = True
if keep:
kept.append(chapter)
else:
skipped.append((chapter.title, length))
if kept:
return ChapterFilterResult(kept=kept, skipped=skipped)
longest_idx = None
longest_length = 0
for idx, chapter in enumerate(chapters):
stripped = chapter.text.strip()
if stripped and len(stripped) > longest_length:
longest_length = len(stripped)
longest_idx = idx
if longest_idx is not None:
longest = chapters[longest_idx]
fallback_skipped = [
(chapter.title, len(chapter.text.strip()))
for idx, chapter in enumerate(chapters)
if idx != longest_idx and chapter.text.strip()
]
return ChapterFilterResult(kept=[longest], skipped=fallback_skipped)
return ChapterFilterResult(kept=[], skipped=skipped)
def update_metadata_for_chapter_count(
metadata: Dict[str, Any], count: int, file_type: str
) -> None:
if not metadata or count <= 0:
return
label = "Chapters" if file_type.lower() in {"epub", "markdown"} else "Pages"
metadata["chapter_count"] = str(count)
pattern = re.compile(r"\(\d+\s+(Chapters?|Pages?)\)")
replacement = f"({count} {label})"
for key in ("album", "ALBUM"):
value = metadata.get(key)
if not isinstance(value, str):
continue
metadata[key] = pattern.sub(replacement, value)
+83
View File
@@ -0,0 +1,83 @@
"""Intro/outro text building and voice resolution for audiobook conversion.
Both UIs (WebUI and Desktop) need to:
1. Build intro/outro text from book metadata
2. Resolve which voice to use for intro/outro synthesis
This module provides the shared domain logic. The actual TTS synthesis
and audio writing remain UI-specific.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Dict, Optional
from abogen.domain.title_builder import build_title_intro_text, build_outro_text
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
@dataclass
class IntroOutroSpec:
"""Resolved intro or outro specification ready for TTS synthesis."""
text: str
voice_spec: str
enabled: bool
def resolve_intro(
metadata: Optional[Dict[str, Any]],
original_filename: str,
read_title_intro: bool,
base_voice_spec: str,
job_voice: str,
voice_cache_keys: list[str],
) -> IntroOutroSpec:
"""Resolve the intro specification from job settings and metadata.
Returns an IntroOutroSpec with text and voice_spec populated,
or enabled=False if intro is disabled or text cannot be built.
"""
if not read_title_intro:
return IntroOutroSpec(text="", voice_spec="", enabled=False)
text = build_title_intro_text(metadata, original_filename)
if not text:
return IntroOutroSpec(text="", voice_spec="", enabled=False)
voice_spec = resolve_fallback_voice_spec(
base_voice_spec, job_voice, voice_cache_keys
)
if not voice_spec:
return IntroOutroSpec(text=text, voice_spec="", enabled=False)
return IntroOutroSpec(text=text, voice_spec=voice_spec, enabled=True)
def resolve_outro(
metadata: Optional[Dict[str, Any]],
original_filename: str,
read_closing_outro: bool,
base_voice_spec: str,
job_voice: str,
voice_cache_keys: list[str],
) -> IntroOutroSpec:
"""Resolve the outro specification from job settings and metadata.
Returns an IntroOutroSpec with text and voice_spec populated,
or enabled=False if outro is disabled or text cannot be built.
"""
if not read_closing_outro:
return IntroOutroSpec(text="", voice_spec="", enabled=False)
text = build_outro_text(metadata, original_filename)
if not text:
return IntroOutroSpec(text="", voice_spec="", enabled=False)
voice_spec = resolve_fallback_voice_spec(
base_voice_spec, job_voice, voice_cache_keys
)
if not voice_spec:
return IntroOutroSpec(text=text, voice_spec="", enabled=False)
return IntroOutroSpec(text=text, voice_spec=voice_spec, enabled=True)
+504
View File
@@ -0,0 +1,504 @@
"""Metadata extraction and processing utilities.
This module provides functions for extracting metadata from text content,
formatting metadata tags for TTS embedding, and generating ffmpeg metadata arguments.
"""
from __future__ import annotations
import datetime
import logging
import os
import re
import uuid
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
logger = logging.getLogger(__name__)
def extract_metadata_from_text(text: str) -> Dict[str, Optional[str]]:
"""Extract metadata tags from text content.
Looks for tags in format: <<METADATA_KEY:value>>
Supported tags:
- TITLE, ARTIST, ALBUM, YEAR
- ALBUM_ARTIST, COMPOSER, GENRE
- COVER_PATH
Args:
text: Text content to search for metadata tags.
Returns:
Dictionary with extracted metadata values (None if not found).
"""
metadata = {}
patterns = {
"title": r"<<METADATA_TITLE:([^>]*)>>",
"artist": r"<<METADATA_ARTIST:([^>]*)>>",
"album": r"<<METADATA_ALBUM:([^>]*)>>",
"year": r"<<METADATA_YEAR:([^>]*)>>",
"album_artist": r"<<METADATA_ALBUM_ARTIST:([^>]*)>>",
"composer": r"<<METADATA_COMPOSER:([^>]*)>>",
"genre": r"<<METADATA_GENRE:([^>]*)>>",
"cover_path": r"<<METADATA_COVER_PATH:([^>]*)>>",
}
for key, pattern in patterns.items():
match = re.search(pattern, text)
if match:
metadata[key] = match.group(1).strip()
else:
metadata[key] = None
return metadata
def get_filename_from_path(
file_path: str,
display_path: Optional[str] = None,
from_queue: bool = False,
) -> str:
"""Extract filename (without extension) from path.
Args:
file_path: The file path to extract from.
display_path: Optional display path (used if from_queue is False).
from_queue: Whether the file is from queue.
Returns:
Filename without extension.
"""
if from_queue:
base_path = file_path
else:
base_path = display_path if display_path else file_path
filename = os.path.splitext(os.path.basename(base_path))[0]
return filename
def build_ffmpeg_metadata_args(
metadata: Dict[str, Optional[str]],
filename: str,
) -> List[str]:
"""Build ffmpeg metadata arguments from metadata dictionary.
Args:
metadata: Dictionary with metadata keys and values.
filename: Fallback filename for title/album if not specified.
Returns:
List of ffmpeg metadata arguments.
"""
args = []
# Default values
defaults = {
"title": filename,
"artist": "Unknown",
"album": filename,
"date": str(datetime.datetime.now().year),
"album_artist": "Unknown",
"composer": "Narrator",
"genre": "Audiobook",
}
# Map of metadata keys to ffmpeg metadata keys
key_mapping = {
"title": "title",
"artist": "artist",
"album": "album",
"year": "date", # year -> date for ffmpeg
"album_artist": "album_artist",
"composer": "composer",
"genre": "genre",
}
for metadata_key, ffmpeg_key in key_mapping.items():
value = metadata.get(metadata_key)
if value is None:
value = defaults.get(metadata_key, "")
if value:
args.extend(["-metadata", f"{ffmpeg_key}={value}"])
return args
def extract_metadata_and_build_args(
text: str,
filename: str,
display_path: Optional[str] = None,
from_queue: bool = False,
) -> Tuple[List[str], Optional[str]]:
"""Extract metadata from text and build ffmpeg arguments.
Convenience function that combines extract_metadata_from_text and
build_ffmpeg_metadata_args.
Args:
text: Text content to search for metadata tags.
filename: Fallback filename for title/album.
display_path: Optional display path.
from_queue: Whether the file is from queue.
Returns:
Tuple of (ffmpeg_metadata_args, cover_path).
"""
metadata = extract_metadata_from_text(text)
cover_path = metadata.get("cover_path")
# Get actual filename from path
actual_filename = get_filename_from_path(
file_path=filename,
display_path=display_path,
from_queue=from_queue,
)
args = build_ffmpeg_metadata_args(metadata, actual_filename)
return args, cover_path
def read_text_for_metadata(
file_path: str,
is_direct_text: bool,
direct_text: Optional[str] = None,
encoding: Optional[str] = None,
) -> str:
"""Read text content for metadata extraction.
Args:
file_path: Path to file (or text if is_direct_text).
is_direct_text: Whether file_path contains direct text.
direct_text: Optional direct text (used if is_direct_text).
encoding: File encoding (detected if not provided).
Returns:
Text content for metadata extraction.
"""
if is_direct_text:
return direct_text or file_path
# Read from file
actual_path = direct_text if direct_text else file_path
try:
if encoding is None:
from abogen.utils import detect_encoding
encoding = detect_encoding(actual_path)
with open(actual_path, "r", encoding=encoding, errors="replace") as f:
return f.read()
except Exception:
return ""
def extract_metadata_for_file(
file_path: str,
is_direct_text: bool = False,
) -> Dict[str, Optional[str]]:
"""Extract metadata dict from a file or direct text.
Convenience function combining read_text_for_metadata + extract_metadata_from_text.
Returns empty dict on any error.
"""
try:
text = read_text_for_metadata(
file_path=file_path,
is_direct_text=is_direct_text,
direct_text=file_path if is_direct_text else None,
)
if text:
return extract_metadata_from_text(text) or {}
except Exception:
pass
return {}
def format_metadata_tags(
metadata: Dict[str, Any],
filename: str,
chapter_count: int,
file_type: str,
cover_bytes: Optional[bytes] = None,
cache_dir: Optional[str] = None,
) -> str:
"""Format metadata tags for insertion into TTS text.
Builds <<METADATA_KEY:value>> tags that are later parsed by
extract_metadata_from_text() and fed to ffmpeg.
Args:
metadata: Dict with keys like 'title', 'authors' (list),
'publication_year', 'description', 'cover_image' (bytes).
filename: Fallback filename (without extension) for title/album.
chapter_count: Number of chapters/pages.
file_type: 'epub', 'pdf', or 'markdown'.
cover_bytes: Optional cover image bytes to save to cache.
cache_dir: Directory for cover cache (uses default if None).
Returns:
Newline-joined string of <<METADATA_KEY:value>> tags.
"""
title = metadata.get("title") or filename
authors = metadata.get("authors") or ["Unknown"]
authors_text = ", ".join(authors) if isinstance(authors, list) else str(authors)
year = metadata.get("publication_year") or str(datetime.datetime.now().year)
chapter_label = "Chapters" if file_type in ("epub", "markdown") else "Pages"
chapter_text = f"{chapter_count} {chapter_label}"
tags = [
f"<<METADATA_TITLE:{title}>>",
f"<<METADATA_ARTIST:{authors_text}>>",
f"<<METADATA_ALBUM:{title} ({chapter_text})>>",
f"<<METADATA_YEAR:{year}>>",
f"<<METADATA_ALBUM_ARTIST:{authors_text}>>",
f"<<METADATA_COMPOSER:Narrator>>",
f"<<METADATA_GENRE:Audiobook>>",
]
cover_path = _save_cover_to_cache(cover_bytes, cache_dir)
if cover_path:
tags.append(f"<<METADATA_COVER_PATH:{cover_path}>>")
return "\n".join(tags)
def _save_cover_to_cache(
cover_bytes: Optional[bytes],
cache_dir: Optional[str] = None,
) -> Optional[str]:
"""Save cover image bytes to cache directory.
Args:
cover_bytes: Raw image bytes (e.g. JPEG/PNG).
cache_dir: Directory to save to. If None, returns None.
Returns:
Normalized path to saved cover file, or None on failure.
"""
if not cover_bytes:
return None
if cache_dir is None:
return None
try:
cover_path = os.path.join(cache_dir, f"cover_{uuid.uuid4()}.jpg")
cover_path = os.path.normpath(cover_path)
with open(cover_path, "wb") as f:
f.write(cover_bytes)
return cover_path
except Exception as e:
logger.warning("Failed to save cover image: %s", e)
return None
def extract_book_metadata_epub(book: Any) -> Dict[str, Any]:
"""Extract metadata from an opened ebooklib EPUB book.
Args:
book: An opened ebooklib EPUB book object.
Returns:
Dict with keys: title, authors, description, publisher,
publication_year, cover_image (bytes or None).
"""
import ebooklib
metadata: Dict[str, Any] = {
"title": None,
"authors": [],
"description": None,
"cover_image": None,
"publisher": None,
"publication_year": None,
}
try:
title_items = book.get_metadata("DC", "title")
if title_items and len(title_items) > 0:
metadata["title"] = title_items[0][0]
except Exception as e:
logger.warning("Error extracting title metadata: %s", e)
try:
author_items = book.get_metadata("DC", "creator")
if author_items:
metadata["authors"] = [
author[0] for author in author_items if len(author) > 0
]
except Exception as e:
logger.warning("Error extracting author metadata: %s", e)
try:
desc_items = book.get_metadata("DC", "description")
if desc_items and len(desc_items) > 0:
metadata["description"] = desc_items[0][0]
except Exception as e:
logger.warning("Error extracting description metadata: %s", e)
try:
publisher_items = book.get_metadata("DC", "publisher")
if publisher_items and len(publisher_items) > 0:
metadata["publisher"] = publisher_items[0][0]
except Exception as e:
logger.warning("Error extracting publisher metadata: %s", e)
try:
date_items = book.get_metadata("DC", "date")
if date_items and len(date_items) > 0:
date_str = date_items[0][0]
year_match = re.search(r"\b(19|20)\d{2}\b", date_str)
if year_match:
metadata["publication_year"] = year_match.group(0)
else:
metadata["publication_year"] = date_str
except Exception as e:
logger.warning("Error extracting publication date metadata: %s", e)
for item in book.get_items_of_type(ebooklib.ITEM_COVER):
metadata["cover_image"] = item.get_content()
break
if not metadata["cover_image"]:
for item in book.get_items_of_type(ebooklib.ITEM_IMAGE):
if "cover" in item.get_name().lower():
metadata["cover_image"] = item.get_content()
break
return metadata
def extract_book_metadata_pdf(pdf_doc: Any) -> Dict[str, Any]:
"""Extract metadata from an opened PyMuPDF document.
Args:
pdf_doc: An opened fitz.Document object.
Returns:
Dict with keys: title, authors, description, publisher,
publication_year, cover_image (bytes or None).
"""
metadata: Dict[str, Any] = {
"title": None,
"authors": [],
"description": None,
"cover_image": None,
"publisher": None,
"publication_year": None,
}
pdf_info = pdf_doc.metadata
if pdf_info:
metadata["title"] = pdf_info.get("title", None)
author = pdf_info.get("author", None)
if author:
metadata["authors"] = [author]
metadata["description"] = pdf_info.get("subject", None)
keywords = pdf_info.get("keywords", None)
if keywords:
if metadata["description"]:
metadata["description"] += f"\n\nKeywords: {keywords}"
else:
metadata["description"] = f"Keywords: {keywords}"
metadata["publisher"] = pdf_info.get("creator", None)
if "creationDate" in pdf_info:
date_str = pdf_info["creationDate"]
year_match = re.search(r"D:(\d{4})", date_str)
if year_match:
metadata["publication_year"] = year_match.group(1)
elif "modDate" in pdf_info:
date_str = pdf_info["modDate"]
year_match = re.search(r"D:(\d{4})", date_str)
if year_match:
metadata["publication_year"] = year_match.group(1)
if len(pdf_doc) > 0:
try:
import fitz
pix = pdf_doc[0].get_pixmap(matrix=fitz.Matrix(2, 2))
metadata["cover_image"] = pix.tobytes("png")
except Exception:
pass
return metadata
def extract_book_metadata_markdown(
markdown_text: str,
markdown_toc: Optional[List[Dict[str, Any]]] = None,
) -> Dict[str, Any]:
"""Extract metadata from markdown frontmatter and first heading.
Args:
markdown_text: Raw markdown text content.
markdown_toc: Optional table of contents list (each item has
'level' and 'name' keys).
Returns:
Dict with keys: title, authors, description, publication_year.
cover_image is always None for markdown.
"""
metadata: Dict[str, Any] = {
"title": None,
"authors": [],
"description": None,
"cover_image": None,
"publisher": None,
"publication_year": None,
}
if not markdown_text:
return metadata
frontmatter_match = re.match(
r"^---\s*\n(.*?)\n---\s*\n", markdown_text, re.DOTALL
)
if frontmatter_match:
try:
frontmatter = frontmatter_match.group(1)
title_match = re.search(
r"^title:\s*(.+)$", frontmatter, re.MULTILINE | re.IGNORECASE
)
if title_match:
metadata["title"] = title_match.group(1).strip().strip("\"'")
author_match = re.search(
r"^author:\s*(.+)$", frontmatter, re.MULTILINE | re.IGNORECASE
)
if author_match:
metadata["authors"] = [
author_match.group(1).strip().strip("\"'")
]
desc_match = re.search(
r"^description:\s*(.+)$", frontmatter, re.MULTILINE | re.IGNORECASE
)
if desc_match:
metadata["description"] = (
desc_match.group(1).strip().strip("\"'")
)
date_match = re.search(
r"^date:\s*(.+)$", frontmatter, re.MULTILINE | re.IGNORECASE
)
if date_match:
date_str = date_match.group(1).strip().strip("\"'")
year_match = re.search(r"\b(19|20)\d{2}\b", date_str)
if year_match:
metadata["publication_year"] = year_match.group(0)
except Exception as e:
logger.warning("Error parsing markdown frontmatter: %s", e)
if not metadata["title"] and markdown_toc:
first_h1 = next(
(h for h in markdown_toc if h.get("level") == 1), None
)
if first_h1:
metadata["title"] = first_h1.get("name")
return metadata
+405
View File
@@ -0,0 +1,405 @@
from __future__ import annotations
import json
import math
import re
from pathlib import Path
from typing import Any, Dict, List, Mapping, Optional, Tuple
_SERIES_NAME_KEYS = (
"series",
"series_name",
"series_title",
)
_SERIES_NUMBER_KEYS = (
"series_index",
"series_position",
"series_sequence",
"book_number",
"series_number",
)
_SERIES_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?")
def normalize_metadata_map(values: Optional[Mapping[str, Any]]) -> Dict[str, str]:
normalized: Dict[str, str] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
text = str(value).strip()
if not text:
continue
normalized[str(key).casefold()] = text
return normalized
def format_author_sentence(raw: Optional[str]) -> str:
if raw is None:
return ""
normalized = str(raw).strip()
if not normalized:
return ""
lowered = normalized.casefold()
if lowered in {"unknown", "various"}:
return ""
working = normalized.replace("&", " and ")
segments = [segment.strip() for segment in working.split(",") if segment.strip()]
tokens: List[str] = []
if segments:
for segment in segments:
parts = [part.strip() for part in re.split(r"\band\b", segment, flags=re.IGNORECASE) if part.strip()]
if parts:
tokens.extend(parts)
else:
tokens.append(segment)
else:
parts = [part.strip() for part in re.split(r"\band\b", working, flags=re.IGNORECASE) if part.strip()]
tokens.extend(parts or [normalized])
cleaned = [token for token in tokens if token and token.casefold() not in {"unknown", "various"}]
if not cleaned:
return ""
if len(cleaned) == 1:
return f"By {cleaned[0]}"
if len(cleaned) == 2:
return f"By {cleaned[0]} and {cleaned[1]}"
return f"By {', '.join(cleaned[:-1])}, and {cleaned[-1]}"
def ensure_sentence(text: str) -> str:
cleaned = text.strip()
if not cleaned:
return ""
if cleaned[-1] in ".!?":
return cleaned
return f"{cleaned}."
def normalize_series_number(value: Any) -> Optional[str]:
text = str(value or "").strip()
if not text:
return None
candidate = text.replace(",", ".")
if candidate.replace(".", "", 1).isdigit():
if "." in candidate:
normalized = candidate.rstrip("0").rstrip(".")
return normalized or "0"
try:
return str(int(candidate))
except ValueError:
pass
match = _SERIES_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
return normalized or "0"
try:
return str(int(normalized))
except ValueError:
return normalized
def extract_series_metadata(values: Mapping[str, str]) -> Tuple[Optional[str], Optional[str]]:
series_name: Optional[str] = None
for key in _SERIES_NAME_KEYS:
raw = values.get(key)
if raw:
cleaned = str(raw).strip()
if cleaned:
series_name = cleaned
break
series_number: Optional[str] = None
for key in _SERIES_NUMBER_KEYS:
raw = values.get(key)
if raw is None:
continue
normalized = normalize_series_number(raw)
if normalized:
series_number = normalized
break
return series_name, series_number
def format_series_sentence(series_name: Optional[str], series_number: Optional[str]) -> str:
if not series_name or not series_number:
return ""
name = series_name.strip()
number = series_number.strip()
if not name or not number:
return ""
article = "the " if not name.lower().startswith("the ") else ""
phrase = f"Book {number} of {article}{name}"
return re.sub(r"\s+", " ", phrase).strip()
_PEOPLE_SPLIT_RE = re.compile(r"[;,/&]|\band\b", re.IGNORECASE)
_LIST_SPLIT_RE = re.compile(r"[;,\n]")
_SERIES_SEQUENCE_TAG_KEYS: Tuple[str, ...] = (
"series_index",
"series_position",
"series_sequence",
"series_number",
"seriesnumber",
"book_number",
"booknumber",
)
def normalize_metadata_casefold(values: Optional[Mapping[str, Any]]) -> Dict[str, Any]:
normalized: Dict[str, Any] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
key_text = str(key).strip().lower()
if not key_text:
continue
if isinstance(value, (list, tuple, set)):
normalized[key_text] = value
else:
text = str(value).strip()
if text:
normalized[key_text] = text
return normalized
def split_people_field(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(split_people_field(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _PEOPLE_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def split_simple_list(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(split_simple_list(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _LIST_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def first_nonempty(*values: Any) -> Optional[str]:
for value in values:
if value is None:
continue
if isinstance(value, (list, tuple, set)):
items = list(value)
if not items:
continue
value = items[0]
text = str(value).strip()
if text:
return text
return None
def extract_year(raw: Optional[str]) -> Optional[int]:
if not raw:
return None
text = str(raw).strip()
if not text:
return None
match = re.search(r"(19|20)\d{2}", text)
if match:
try:
return int(match.group(0))
except ValueError:
return None
try:
parsed = int(text)
except ValueError:
return None
if 0 < parsed < 3000:
return parsed
return None
def normalize_series_sequence(raw: Any) -> Optional[str]:
if raw is None:
return None
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return None
text = str(raw)
else:
text = str(raw).strip()
if not text:
return None
candidate = text.replace(",", ".")
match = _SERIES_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
def build_audiobookshelf_metadata(
tags: Mapping[str, Any],
*,
language: str = "",
filename: str = "",
) -> Dict[str, Any]:
normalized = normalize_metadata_casefold(tags)
title = first_nonempty(
normalized.get("title"),
normalized.get("book_title"),
normalized.get("name"),
normalized.get("album"),
filename,
)
authors = split_people_field(
normalized.get("authors")
or normalized.get("author")
or normalized.get("album_artist")
or normalized.get("artist")
)
narrators = split_people_field(normalized.get("narrators") or normalized.get("narrator"))
description = first_nonempty(
normalized.get("description"), normalized.get("summary"), normalized.get("comment")
)
genres = split_simple_list(normalized.get("genre"))
keywords = split_simple_list(normalized.get("tags") or normalized.get("keywords"))
lang = first_nonempty(normalized.get("language"), normalized.get("lang")) or language or ""
series_name = first_nonempty(
normalized.get("series"),
normalized.get("series_name"),
normalized.get("seriesname"),
normalized.get("series_title"),
normalized.get("seriestitle"),
)
series_sequence = None
for key in _SERIES_SEQUENCE_TAG_KEYS:
raw_value = normalized.get(key)
seq = normalize_series_sequence(raw_value)
if seq:
series_sequence = seq
break
if not series_name:
series_sequence = None
data: Dict[str, Any] = {
"title": title,
"subtitle": normalized.get("subtitle"),
"authors": authors,
"narrators": narrators,
"description": description,
"publisher": normalized.get("publisher"),
"genres": genres,
"tags": keywords,
"language": lang,
"publishedYear": extract_year(
normalized.get("published")
or normalized.get("publication_year")
or normalized.get("date")
or normalized.get("year")
),
"seriesName": series_name,
"seriesSequence": series_sequence,
"isbn": first_nonempty(normalized.get("isbn"), normalized.get("asin")),
}
published_date = first_nonempty(
normalized.get("published"), normalized.get("publication_date"), normalized.get("date")
)
if published_date:
data["publishedDate"] = published_date
rating_text = first_nonempty(normalized.get("rating"), normalized.get("my_rating"))
if rating_text:
try:
data["rating"] = float(str(rating_text).strip())
except ValueError:
pass
rating_max_text = first_nonempty(
normalized.get("rating_max"), normalized.get("rating_scale")
)
if rating_max_text:
try:
data["ratingMax"] = float(str(rating_max_text).strip())
except ValueError:
pass
cleaned: Dict[str, Any] = {}
for key, value in data.items():
if value is None:
continue
if isinstance(value, str) and not value.strip():
continue
if isinstance(value, (list, tuple)) and not value:
continue
cleaned[key] = value
return cleaned
def load_audiobookshelf_chapters(
metadata_path: Path,
) -> Optional[List[Dict[str, Any]]]:
if not metadata_path.exists():
return None
try:
payload = json.loads(metadata_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
chapters = payload.get("chapters")
if not isinstance(chapters, list):
return None
cleaned: List[Dict[str, Any]] = []
for entry in chapters:
if not isinstance(entry, Mapping):
continue
title = first_nonempty(entry.get("title"), entry.get("original_title"))
start = entry.get("start")
end = entry.get("end")
if title and start is not None and end is not None:
cleaned.append({"title": str(title), "start": start, "end": end})
return cleaned or None
+23
View File
@@ -0,0 +1,23 @@
from __future__ import annotations
from typing import Any, Dict, Optional
def merge_metadata(
extracted: Optional[Dict[str, Any]],
overrides: Optional[Dict[str, Any]],
) -> Dict[str, str]:
merged: Dict[str, str] = {}
if extracted:
for key, value in extracted.items():
if value is None:
continue
merged[str(key)] = str(value)
if overrides:
for key, value in overrides.items():
key_str = str(key)
if value is None:
merged.pop(key_str, None)
else:
merged[key_str] = str(value)
return merged
+99
View File
@@ -0,0 +1,99 @@
"""OPDS metadata normalization.
Normalizes metadata keys from various OPDS/Calibre sources into
a canonical set of overrides for the audiobook conversion pipeline.
"""
from __future__ import annotations
from typing import Any, Dict, Mapping
def normalize_opds_metadata(metadata_payload: Mapping[str, Any]) -> Dict[str, Any]:
"""Normalize OPDS/Calibre metadata into canonical override keys.
Takes a metadata payload with various key aliases (e.g. 'series'/'series_name',
'tags'/'keywords', 'authors'/'creator') and returns a dict with canonical
keys set.
Args:
metadata_payload: Raw metadata dict from OPDS/Calibre import.
Returns:
Dict with canonical metadata keys (series, series_index, tags,
description, subtitle, publisher, authors).
"""
metadata_overrides: Dict[str, Any] = {}
def _stringify(value: Any) -> str:
if value is None:
return ""
if isinstance(value, (list, tuple, set)):
parts = [str(item).strip() for item in value if item is not None]
return ", ".join(part for part in parts if part)
return str(value).strip()
raw_series = metadata_payload.get("series") or metadata_payload.get("series_name")
series_name = str(raw_series or "").strip()
if series_name:
metadata_overrides["series"] = series_name
metadata_overrides.setdefault("series_name", series_name)
series_index_value = (
metadata_payload.get("series_index")
or metadata_payload.get("series_position")
or metadata_payload.get("series_sequence")
or metadata_payload.get("book_number")
)
if series_index_value is not None:
series_index_text = str(series_index_value).strip()
if series_index_text:
metadata_overrides.setdefault("series_index", series_index_text)
metadata_overrides.setdefault("series_position", series_index_text)
metadata_overrides.setdefault("series_sequence", series_index_text)
metadata_overrides.setdefault("book_number", series_index_text)
tags_value = metadata_payload.get("tags") or metadata_payload.get("keywords")
if tags_value:
tags_text = _stringify(tags_value)
if tags_text:
metadata_overrides.setdefault("tags", tags_text)
metadata_overrides.setdefault("keywords", tags_text)
metadata_overrides.setdefault("genre", tags_text)
description_value = metadata_payload.get("description") or metadata_payload.get("summary")
if description_value:
description_text = _stringify(description_value)
if description_text:
metadata_overrides.setdefault("description", description_text)
metadata_overrides.setdefault("summary", description_text)
subtitle_value = (
metadata_payload.get("subtitle")
or metadata_payload.get("sub_title")
or metadata_payload.get("calibre_subtitle")
)
if subtitle_value:
subtitle_text = _stringify(subtitle_value)
if subtitle_text:
metadata_overrides.setdefault("subtitle", subtitle_text)
publisher_value = metadata_payload.get("publisher")
if publisher_value:
publisher_text = _stringify(publisher_value)
if publisher_text:
metadata_overrides.setdefault("publisher", publisher_text)
authors_value = (
metadata_payload.get("authors")
or metadata_payload.get("author")
or metadata_payload.get("creator")
or metadata_payload.get("dc_creator")
)
if authors_value:
authors_text = _stringify(authors_value)
if authors_text:
metadata_overrides.setdefault("authors", authors_text)
metadata_overrides.setdefault("author", authors_text)
return metadata_overrides
+125
View File
@@ -0,0 +1,125 @@
"""Text normalization convenience helpers.
Provides both the simple ``normalize_text_for_pipeline`` (apostrophe + LLM only)
and the comprehensive ``prepare_text_for_tts`` that chains all three normalization
stages used during conversion: heteronym rules → pronunciation rules → pipeline
normalization. The latter is the single entry point that both the Web UI and
PyQt Desktop GUI should use.
Also provides ``TTSContext`` — a dataclass bundling all pre-compiled normalization
resources so they can be created once and passed as a single object.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Dict, List, Mapping, Optional
from abogen.kokoro_text_normalization import (
ApostropheConfig,
normalize_for_pipeline as _normalize_for_pipeline,
)
from abogen.normalization_settings import (
build_apostrophe_config,
get_runtime_settings,
apply_overrides as _apply_overrides,
)
_BASE_APOSTROPHE_CONFIG = ApostropheConfig()
@dataclass
class TTSContext:
"""Bundles pre-compiled normalization resources for TTS processing.
Created once per conversion job and passed to ``prepare_text_for_tts``
instead of threading 5 separate parameters.
"""
split_pattern: str = r"(?<=[.!?\-])\s+"
pronunciation_rules: Optional[List[Dict[str, Any]]] = None
heteronym_rules: Optional[List[Dict[str, Any]]] = None
normalization_overrides: Optional[Mapping[str, Any]] = None
usage_counter: Dict[str, int] = field(default_factory=dict)
def normalize(self, text: str) -> str:
"""Shorthand: normalize text using this context's compiled rules."""
return prepare_text_for_tts(
text,
heteronym_rules=self.heteronym_rules,
pronunciation_rules=self.pronunciation_rules,
normalization_overrides=self.normalization_overrides,
usage_counter=self.usage_counter,
)
def normalize_text_for_pipeline(
text: str,
*,
normalization_overrides: Optional[Mapping[str, Any]] = None,
) -> str:
"""Normalize text using runtime settings with optional overrides."""
runtime_settings = get_runtime_settings()
if normalization_overrides:
runtime_settings = _apply_overrides(runtime_settings, normalization_overrides)
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
return _normalize_for_pipeline(text, config=apostrophe_config, settings=runtime_settings)
def prepare_text_for_tts(
text: str,
*,
heteronym_rules: Optional[List[Dict[str, Any]]] = None,
pronunciation_rules: Optional[List[Dict[str, Any]]] = None,
normalization_overrides: Optional[Mapping[str, Any]] = None,
usage_counter: Optional[Dict[str, int]] = None,
) -> str:
"""Apply the full text normalization pipeline before TTS synthesis.
Chains three stages in order:
1. Heteronym sentence rules (context-dependent pronunciation)
2. Pronunciation rules (token-level replacements)
3. Pipeline normalization (apostrophe handling, LLM normalization)
This is the **single entry point** that both the Web UI conversion runner
and the PyQt conversion thread should call before passing text to the TTS
backend.
Parameters
----------
text:
Raw text to normalize.
heteronym_rules:
Compiled heteronym rules from ``compile_heteronym_sentence_rules``.
pronunciation_rules:
Compiled pronunciation rules from ``compile_pronunciation_rules``.
normalization_overrides:
User-level overrides for normalization settings (apostrophe mode, etc.).
usage_counter:
Mutable dict that tracks how many times each pronunciation override was
applied. Passed through to ``apply_pronunciation_rules``.
Returns
-------
str
Fully normalized text ready for TTS.
"""
from abogen.domain.pronunciation import (
apply_heteronym_sentence_rules,
apply_pronunciation_rules,
)
result = str(text or "")
if heteronym_rules:
result = apply_heteronym_sentence_rules(result, heteronym_rules)
if pronunciation_rules:
result = apply_pronunciation_rules(result, pronunciation_rules, usage_counter)
runtime_settings = get_runtime_settings()
if normalization_overrides:
runtime_settings = _apply_overrides(runtime_settings, normalization_overrides)
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
return _normalize_for_pipeline(result, config=apostrophe_config, settings=runtime_settings)
+183
View File
@@ -0,0 +1,183 @@
"""Output path resolution utilities.
Pure functions for resolving output directories, building file paths,
and computing project folder layouts.
"""
from __future__ import annotations
import os
import platform
import re
from datetime import datetime
from pathlib import Path
from typing import Any, Callable, List, Optional, Tuple
from abogen.subtitle_utils import sanitize_name_for_os
from abogen.text_extractor import ExtractedChapter
_OUTPUT_SANITIZE_RE = re.compile(r"[^\w\-_.]+")
# OS-specific illegal characters for filenames
_WINDOWS_ILLEGAL_CHARS_RE = re.compile(r'[<>:"/\\|?*\x00-\x1f]')
_UNIX_CONTROL_CHARS_RE = re.compile(r'[\x00-\x1f]')
_RESERVED_NAMES = frozenset(
{"CON", "PRN", "AUX", "NUL"}
| {f"COM{i}" for i in range(1, 10)}
| {f"LPT{i}" for i in range(1, 10)}
)
def slugify(title: str, index: int) -> str:
sanitized = re.sub(r"[^\w\-]+", "_", title.lower()).strip("_")
if not sanitized:
sanitized = f"chapter_{index:02d}"
return sanitized[:80]
def sanitize_filename_for_chapter(title: str, index: int, max_len: int = 80) -> str:
"""Sanitize a chapter name for use as a filename component.
Combines character sanitization, OS safety, and smart truncation
at word boundaries. Prepends zero-padded index prefix.
Args:
title: Raw chapter title.
index: 1-based chapter number for prefix.
max_len: Maximum length of the sanitized portion (excluding prefix).
Returns:
Sanitized string like "01_the_beginning".
"""
# Remove non-word/non-space/non-hyphen chars, then collapse spaces/hyphens
sanitized = re.sub(r"[^\w\s\-]", "", title)
sanitized = re.sub(r"[\s\-]+", "_", sanitized).strip("_")
if not sanitized:
sanitized = f"chapter_{index:02d}"
# OS-specific sanitization
system = platform.system()
if system == "Windows":
sanitized = _WINDOWS_ILLEGAL_CHARS_RE.sub("_", sanitized)
sanitized = sanitized.rstrip(". ")
base = sanitized.split(".")[0].upper()
if base in _RESERVED_NAMES:
sanitized = f"_{sanitized}"
# Linux: only NUL is truly illegal, but control chars are problematic
sanitized = _UNIX_CONTROL_CHARS_RE.sub("_", sanitized)
# Smart truncation at word boundary
if len(sanitized) > max_len:
pos = sanitized[:max_len].rfind("_")
sanitized = sanitized[: pos if pos > 0 else max_len].rstrip("_")
return f"{index:02d}_{sanitized}"
def sanitize_output_stem(name: str, index: int = 0) -> str:
base = Path(name or "").stem
sanitized = _OUTPUT_SANITIZE_RE.sub("_", base).strip("_")
return sanitized or "output"
def output_timestamp_token() -> str:
return datetime.now().strftime("%Y%m%d-%H%M%S")
def build_output_path(directory: Path, original_name: str, extension: str) -> Path:
sanitized = sanitize_output_stem(original_name)
return directory / f"{sanitized}.{extension}"
def apply_newline_policy(chapters: List[ExtractedChapter], replace_single_newlines: bool) -> None:
if not replace_single_newlines:
return
newline_regex = re.compile(r"(?<!\n)\n(?!\n)")
for chapter in chapters:
chapter.text = newline_regex.sub(" ", chapter.text)
from abogen.domain.enums import SaveMode
def resolve_output_directory(
*,
save_mode: str,
stored_path: Path,
output_folder: Optional[str],
desktop_dir: Optional[Path],
user_output_path: Optional[Path],
user_cache_outputs: Optional[Path],
) -> Path:
if save_mode in (SaveMode.SAVE_TO_DESKTOP, "Save to Desktop") and desktop_dir:
return desktop_dir
if save_mode in (SaveMode.SAVE_NEXT_TO_INPUT, "Save next to input file"):
return stored_path.parent
if save_mode in (SaveMode.CHOOSE_OUTPUT_FOLDER, "Choose output folder") and output_folder:
return Path(output_folder)
if save_mode in (SaveMode.DEFAULT_OUTPUT, "Use default save location") and user_output_path:
return user_output_path
return user_cache_outputs or Path(".")
def resolve_project_layout(
*,
original_filename: str,
save_as_project: bool,
base_dir: Path,
timestamp_fn: Callable[[], str] = output_timestamp_token,
sanitize_fn: Callable[[str, int], str] = sanitize_output_stem,
) -> Tuple[Path, Path, Path, Optional[Path]]:
sanitized = sanitize_fn(original_filename, 0)
folder_name = f"{timestamp_fn()}_{sanitized}"
project_root = base_dir / folder_name
project_root.mkdir(parents=True, exist_ok=True)
if save_as_project:
audio_dir = project_root / "audio"
subtitle_dir = project_root / "subtitles"
metadata_dir = project_root / "metadata"
for directory in (audio_dir, subtitle_dir, metadata_dir):
directory.mkdir(parents=True, exist_ok=True)
return project_root, audio_dir, subtitle_dir, metadata_dir
return project_root, project_root, project_root, None
def resolve_unique_path(
parent_dir: str,
base_name: str,
extension: str,
allowed_extensions: Optional[set] = None,
) -> str:
"""Find a unique file path by appending _2, _3, etc. on collision.
Args:
parent_dir: Directory to check for collisions.
base_name: Base filename (without extension).
extension: File extension (without dot).
allowed_extensions: Set of extensions to check against.
If None, checks any existing file/dir with same name.
Returns:
Full path without extension (e.g. "/path/to/name_2").
"""
sanitized = sanitize_name_for_os(base_name, is_folder=True)
counter = 1
while True:
suffix = f"_{counter}" if counter > 1 else ""
candidate = os.path.join(parent_dir, f"{sanitized}{suffix}")
if allowed_extensions is not None:
file_parts = (os.path.splitext(f) for f in os.listdir(parent_dir))
clash = any(
name == f"{sanitized}{suffix}"
and ext[1:].lower() in allowed_extensions
for name, ext in file_parts
)
else:
clash = os.path.exists(candidate)
if not clash:
return candidate
counter += 1
+128
View File
@@ -0,0 +1,128 @@
"""Pipeline creation, caching and lifecycle management.
Provides a unified interface for creating and managing TTS pipelines
across all UI layers (WebUI, PyQt, CLI).
"""
from __future__ import annotations
from typing import Any, Dict, Optional
from abogen.domain.device import select_device
from abogen.domain.enums import Language
from abogen.domain.voice_resolution import initialize_voice_cache
from abogen.tts_plugin.utils import create_pipeline, is_plugin_registered
# Kokoro-specific language mapping (engine's responsibility)
_KOKORO_LANG_MAP = {
Language.EN_US: "a",
Language.EN_GB: "b",
Language.ES: "e",
Language.FR: "f",
Language.HI: "h",
Language.IT: "i",
Language.JA: "j",
Language.PT_BR: "p",
Language.ZH: "z",
}
def resolve_device(use_gpu: bool) -> str:
"""Determine compute device from job and global config flags."""
from abogen.utils import load_config
cfg = load_config()
if use_gpu and cfg.get("use_gpu", True):
return select_device()
return "cpu"
def create_pipeline_for_job(
provider: str,
language: str,
use_gpu: bool,
) -> Any:
"""Create a TTS pipeline with proper device selection.
Handles provider validation, GPU decision, and plugin checks.
"""
provider = str(provider or "kokoro").strip().lower() or "kokoro"
if not is_plugin_registered(provider):
provider = "kokoro"
# Convert Language enum to Kokoro single-letter code
try:
lang = Language.from_str(language) if not isinstance(language, Language) else language
except ValueError:
lang = Language.EN_US # fallback for unknown languages
kokoro_code = _KOKORO_LANG_MAP.get(lang, "a")
if provider == "supertonic":
return create_pipeline("supertonic")
device = resolve_device(use_gpu)
return create_pipeline("kokoro", lang_code=kokoro_code, device=device)
def dispose_pipelines(pipelines: Dict[str, Any]) -> None:
"""Dispose all pipelines in a dict and clear it."""
for p in pipelines.values():
try:
p.dispose()
except Exception:
pass
pipelines.clear()
class PipelinePool:
"""Cache and manage TTS pipelines by provider.
Usage::
pool = PipelinePool()
backend = pool.get("kokoro", "en", use_gpu=True)
# ... use backend ...
pool.dispose_all()
"""
def __init__(self) -> None:
self._pipelines: Dict[str, Any] = {}
self._voice_cache_initialized = False
def get(
self,
provider: str,
language: str,
use_gpu: bool,
*,
job: Any = None,
) -> Any:
"""Get or create a cached pipeline for the given provider.
Args:
provider: TTS provider name ("kokoro" or "supertonic").
language: Language code (for kokoro).
use_gpu: Whether GPU acceleration is requested.
job: Optional job object for voice cache initialization.
"""
provider = str(provider or "kokoro").strip().lower() or "kokoro"
if not is_plugin_registered(provider):
provider = "kokoro"
existing = self._pipelines.get(provider)
if existing is not None:
return existing
pipeline = create_pipeline_for_job(provider, language, use_gpu)
self._pipelines[provider] = pipeline
if provider == "kokoro" and not self._voice_cache_initialized and job is not None:
initialize_voice_cache(job)
self._voice_cache_initialized = True
return pipeline
def dispose_all(self) -> None:
"""Dispose all cached pipelines."""
dispose_pipelines(self._pipelines)
self._voice_cache_initialized = False
+72
View File
@@ -0,0 +1,72 @@
from __future__ import annotations
"""Progress and ETR (estimated time remaining) calculation.
Shared by Web UI and PyQt desktop GUI. Pure math, no UI dependencies.
"""
import time
from dataclasses import dataclass, field
@dataclass
class ProgressTracker:
"""Tracks character-based progress with ETR calculation.
Usage:
tracker = ProgressTracker(total_chars=50000)
# ... as processing occurs:
tracker.update(chars_done=5000)
print(tracker.etr_str) # "00:04:30"
print(tracker.percent) # 10
"""
total_chars: int
_start_time: float = field(default_factory=time.time, repr=False)
_chars_done: int = field(default=0, repr=False)
def update(self, chars_done: int) -> None:
self._chars_done = chars_done
@property
def percent(self) -> int:
if self.total_chars <= 0:
return 0
return min(int(self._chars_done / self.total_chars * 100), 99)
@property
def etr_str(self) -> str:
elapsed = time.time() - self._start_time
if self._chars_done <= 0 or elapsed <= 0.5:
return "Processing..."
avg_time_per_char = elapsed / self._chars_done
remaining = self.total_chars - self._chars_done
if remaining <= 0:
return "00:00:00"
secs = avg_time_per_char * remaining
h = int(secs // 3600)
m = int((secs % 3600) // 60)
s = int(secs % 60)
return f"{h:02d}:{m:02d}:{s:02d}"
def calc_etr_str(elapsed: float, done: int, total: int) -> str:
"""Standalone ETR string calculation (matches PyQt original logic).
Args:
elapsed: seconds since processing started
done: items/characters processed so far
total: total items/characters to process
Returns:
ETR string like "01:23:45" or "Processing..."
"""
if done <= 0 or elapsed <= 0.5:
return "Processing..."
avg_time_per_item = elapsed / done
remaining = total - done
if remaining <= 0:
return "00:00:00"
secs = avg_time_per_item * remaining
h = int(secs // 3600)
m = int((secs % 3600) // 60)
s = int(secs % 60)
return f"{h:02d}:{m:02d}:{s:02d}"
+261
View File
@@ -0,0 +1,261 @@
"""Pronunciation rule compilation and application.
Pure functions for compiling token-level and sentence-level pronunciation
overrides into regex patterns, applying them to text, and merging multiple
override sources with precedence rules.
"""
from __future__ import annotations
import re
from typing import Any, Dict, Iterable, List, Mapping, Optional
from abogen.entity_analysis import normalize_token as normalize_entity_token
from abogen.entity_analysis import normalize_manual_override_token
def compile_pronunciation_rules(
overrides: Optional[Iterable[Mapping[str, Any]]],
) -> List[Dict[str, Any]]:
if not overrides:
return []
candidates: List[Dict[str, Any]] = []
seen: set[str] = set()
for entry in overrides:
if not isinstance(entry, Mapping):
continue
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not pronunciation_value:
continue
token_values: List[str] = []
token_raw = entry.get("token")
if token_raw:
token_value = str(token_raw).strip()
if token_value:
token_values.append(token_value)
normalized_raw = entry.get("normalized")
if normalized_raw:
normalized_value = str(normalized_raw).strip()
if normalized_value:
token_values.append(normalized_value)
if token_raw and not token_values:
fallback = normalize_entity_token(str(token_raw))
if fallback:
token_values.append(fallback)
if not token_values:
continue
usage_normalized = str(entry.get("normalized") or "").strip()
if not usage_normalized and token_values:
usage_normalized = normalize_entity_token(token_values[0]) or token_values[0]
usage_token = str(entry.get("token") or token_values[0])
for token_value in token_values:
key = token_value.casefold()
if key in seen:
continue
seen.add(key)
candidates.append(
{
"token": token_value,
"normalized": usage_normalized,
"replacement": pronunciation_value,
}
)
if not candidates:
return []
candidates.sort(key=lambda item: len(item["token"]), reverse=True)
compiled: List[Dict[str, Any]] = []
for candidate in candidates:
token_value = candidate["token"]
pronunciation_value = candidate["replacement"]
escaped = re.escape(token_value)
pattern = re.compile(rf"(?i)(?<!\w){escaped}(?P<possessive>'s|\u2019s|\u2019)?(?!\w)")
compiled.append(
{
"pattern": pattern,
"replacement": pronunciation_value,
"normalized": candidate.get("normalized") or token_value,
"token": candidate.get("token") or token_value,
}
)
return compiled
def compile_heteronym_sentence_rules(
overrides: Optional[Iterable[Mapping[str, Any]]],
) -> List[Dict[str, Any]]:
if not overrides:
return []
compiled: List[Dict[str, Any]] = []
seen: set[str] = set()
for entry in overrides:
if not isinstance(entry, Mapping):
continue
sentence = str(entry.get("sentence") or "").strip()
if not sentence:
continue
choice = str(entry.get("choice") or "").strip()
if not choice:
continue
replacement_sentence = ""
options = entry.get("options")
if isinstance(options, list):
for opt in options:
if not isinstance(opt, Mapping):
continue
if str(opt.get("key") or "").strip() == choice:
replacement_sentence = str(opt.get("replacement_sentence") or "").strip()
break
if not replacement_sentence:
continue
rule_key = f"{sentence}\n{choice}".casefold()
if rule_key in seen:
continue
seen.add(rule_key)
parts = [p for p in re.split(r"\s+", sentence) if p]
if not parts:
continue
pattern_text = r"\s+".join(re.escape(p) for p in parts)
pattern = re.compile(pattern_text)
compiled.append({"pattern": pattern, "replacement": replacement_sentence})
compiled.sort(key=lambda item: len(item["pattern"].pattern), reverse=True)
return compiled
def apply_heteronym_sentence_rules(text: str, rules: List[Dict[str, Any]]) -> str:
if not text or not rules:
return text
result = text
for rule in rules:
pattern = rule["pattern"]
replacement = rule["replacement"]
result = pattern.sub(replacement, result)
return result
def apply_pronunciation_rules(
text: str,
rules: List[Dict[str, Any]],
usage_counter: Optional[Dict[str, int]] = None,
) -> str:
if not text or not rules:
return text
result = text
for rule in rules:
pattern = rule["pattern"]
pronunciation_value = rule["replacement"]
usage_key = str(rule.get("normalized") or "").strip()
def _replacement(match: re.Match[str]) -> str:
suffix = match.group("possessive") or ""
if usage_counter is not None and usage_key:
usage_counter[usage_key] = usage_counter.get(usage_key, 0) + 1
return pronunciation_value + suffix
result = pattern.sub(_replacement, result)
return result
def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
"""Return pronunciation override entries, ensuring manual overrides are included.
Pending jobs keep both ``manual_overrides`` and ``pronunciation_overrides``, but the
latter can be stale if the UI didn't resync before enqueue. During conversion,
we must merge manual overrides so they always apply (before TTS).
Precedence: manual overrides win over existing entries for the same normalized key.
"""
collected: Dict[str, Dict[str, Any]] = {}
existing = getattr(job, "pronunciation_overrides", None)
if isinstance(existing, list):
for entry in existing:
if not isinstance(entry, Mapping):
continue
token_value = str(entry.get("token") or "").strip()
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = str(entry.get("normalized") or "").strip() or normalize_entity_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(entry.get("voice") or "").strip() or None,
"notes": str(entry.get("notes") or "").strip() or None,
"context": str(entry.get("context") or "").strip() or None,
"source": str(entry.get("source") or "pronunciation"),
"language": getattr(job, "language", None),
}
speakers = getattr(job, "speakers", None)
if isinstance(speakers, dict):
for payload in speakers.values():
if not isinstance(payload, Mapping):
continue
token_value = str(payload.get("token") or "").strip()
pronunciation_value = str(payload.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = normalize_entity_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(
payload.get("resolved_voice")
or payload.get("voice")
or getattr(job, "voice", "")
).strip()
or None,
"notes": None,
"context": None,
"source": "speaker",
"language": getattr(job, "language", None),
}
manual = getattr(job, "manual_overrides", None)
if isinstance(manual, list):
for entry in manual:
if not isinstance(entry, Mapping):
continue
token_value = str(entry.get("token") or "").strip()
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = str(entry.get("normalized") or "").strip() or normalize_manual_override_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(entry.get("voice") or "").strip() or None,
"notes": str(entry.get("notes") or "").strip() or None,
"context": str(entry.get("context") or "").strip() or None,
"source": str(entry.get("source") or "manual"),
"language": getattr(job, "language", None),
}
return list(collected.values())
+580
View File
@@ -0,0 +1,580 @@
"""Shared settings core.
Defines the SETTINGS_REGISTRY — the single source of truth for all settings.
Every setting has a key, type, default, validation rules, and UI scope.
Both Web UI and Desktop GUI must reference this registry.
"""
from __future__ import annotations
import os
import re
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, Mapping, Optional, Sequence
from abogen.constants import (
LANGUAGE_DESCRIPTIONS,
SUBTITLE_FORMATS,
SUPPORTED_SOUND_FORMATS,
)
from abogen.tts_plugin.utils import get_default_voice
from abogen.normalization_settings import (
DEFAULT_LLM_PROMPT,
environment_llm_defaults,
)
# ── Schema ───────────────────────────────────────────────────────────
@dataclass(frozen=True)
class Setting:
"""Contract for a single setting.
Attributes:
key: Config dict key (e.g. "output_format").
type_: Python type (bool, int, float, str, list).
default: Default value or callable returning one.
min_value: Minimum for numeric types.
max_value: Maximum for numeric types.
valid_values: Allowed values for str types (None = any).
gui_only: True if only used by PyQt Desktop GUI.
web_only: True if only used by Web UI.
normalizer: Optional callable(value, default) -> normalized_value.
description: Human-readable explanation.
"""
key: str
type_: type
default: Any
min_value: float | None = None
max_value: float | None = None
valid_values: tuple[Any, ...] | None = None
gui_only: bool = False
web_only: bool = False
normalizer: Callable | None = None
description: str = ""
def coerce(self, value: Any, fallback: Any | None = None) -> Any:
"""Coerce value to the declared type, returning fallback on failure."""
fb = fallback if fallback is not None else self.default
if self.type_ is bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() in {"true", "1", "yes", "on"}
if value is None:
return fb
return bool(value)
if self.type_ is int:
try:
v = int(value)
except (TypeError, ValueError):
return fb
if self.min_value is not None:
v = max(int(self.min_value), v)
if self.max_value is not None:
v = min(int(self.max_value), v)
return v
if self.type_ is float:
try:
v = float(value)
except (TypeError, ValueError):
return fb
if self.min_value is not None:
v = max(self.min_value, v)
if self.max_value is not None:
v = min(self.max_value, v)
return v
if self.type_ is str:
if isinstance(value, str):
v = value.strip()
if self.valid_values and v not in self.valid_values:
return fb
return v
return fb
if self.type_ is list:
if isinstance(value, (list, tuple, set)):
return list(value)
return fb
return value
# ── Normalizers (used by Setting.normalizer) ─────────────────────────
def _norm_save_mode(value: Any, default: str) -> str:
if isinstance(value, str):
if value in SAVE_MODE_LABELS:
return value
if value in LEGACY_SAVE_MODE_MAP:
return LEGACY_SAVE_MODE_MAP[value]
return default
def _norm_voice_spec(value: Any, default: str) -> str:
if isinstance(value, str):
text = value.strip()
if not text:
return default
spec, profile_name = split_profile_spec(text)
if profile_name:
return f"speaker:{profile_name}"
return spec
return default
def _norm_speaker_spec(value: Any, default: str) -> str:
if isinstance(value, str):
text = value.strip()
if not text:
return ""
spec, profile_name = split_profile_spec(text)
if profile_name:
return f"speaker:{profile_name}"
return spec
return ""
def _norm_language_list(value: Any, default: list) -> list:
if isinstance(value, (list, tuple, set)):
return [code for code in value if isinstance(code, str) and code in LANGUAGE_DESCRIPTIONS]
if isinstance(value, str):
parts = [item.strip().lower() for item in value.split(",") if item.strip()]
return [code for code in parts if code in LANGUAGE_DESCRIPTIONS]
return default
def _norm_stripped_str(value: Any, default: str) -> str:
return str(value or "").strip()
def _norm_prompt(value: Any, default: str) -> str:
candidate = str(value or "").strip()
return candidate if candidate else default
# ── Registry ─────────────────────────────────────────────────────────
def _default_output_format() -> str:
return "wav"
def _default_save_mode() -> str:
return "default_output" if has_output_override() else "save_next_to_input"
def _default_llm(key: str) -> str:
return environment_llm_defaults().get(key, "")
SETTINGS_REGISTRY: list[Setting] = [
# ── Core output ──────────────────────────────────────────────
Setting("output_format", str, "wav",
valid_values=tuple(SUPPORTED_SOUND_FORMATS),
description="Audio output format"),
Setting("subtitle_format", str, "srt",
valid_values=tuple(item[0] for item in SUBTITLE_FORMATS),
description="Subtitle file format"),
Setting("save_mode", str, _default_save_mode,
normalizer=_norm_save_mode,
description="Where to save output files"),
Setting("separate_chapters_format", str, "wav",
valid_values=("wav", "flac", "mp3", "opus"),
description="Format for separately saved chapters"),
Setting("chunk_level", str, "paragraph",
valid_values=("paragraph", "sentence"),
description="Text chunking granularity"),
# ── Voice ────────────────────────────────────────────────────
Setting("default_speaker", str, "",
normalizer=_norm_speaker_spec,
description="Default speaker name"),
Setting("default_voice", str, lambda: get_default_voice("kokoro"),
normalizer=_norm_voice_spec,
description="Default TTS voice"),
Setting("speed", float, 1.0, min_value=0.5, max_value=3.0,
gui_only=True,
description="TTS speed multiplier"),
Setting("supertonic_total_steps", int, 5, min_value=2, max_value=15,
description="SuperTonic processing steps"),
Setting("supertonic_speed", float, 1.0, min_value=0.7, max_value=2.0,
description="SuperTonic speed"),
# ── Chapter handling ─────────────────────────────────────────
Setting("silence_between_chapters", float, 2.0, min_value=0.0,
description="Silence gap between chapters (seconds)"),
Setting("chapter_intro_delay", float, 0.5, min_value=0.0,
description="Delay after chapter heading (seconds)"),
Setting("read_title_intro", bool, False,
description="Read chapter title as intro"),
Setting("read_closing_outro", bool, True,
description="Read closing/outro text"),
Setting("normalize_chapter_opening_caps", bool, True,
description="Normalize chapter opening caps"),
Setting("auto_prefix_chapter_titles", bool, True,
description="Auto-prefix chapter titles"),
Setting("save_chapters_separately", bool, False,
description="Save each chapter as separate file"),
Setting("merge_chapters_at_end", bool, True,
description="Merge chapters into single file"),
Setting("save_as_project", bool, False,
description="Save as editable project"),
Setting("generate_epub3", bool, False,
description="Generate EPUB3 output"),
# ── GPU / performance ────────────────────────────────────────
Setting("use_gpu", bool, True,
description="Use GPU acceleration"),
# ── Text processing ──────────────────────────────────────────
Setting("replace_single_newlines", bool, False,
description="Replace single newlines with spaces"),
Setting("max_subtitle_words", int, 50, min_value=1, max_value=500,
description="Max words per subtitle"),
Setting("enable_entity_recognition", bool, True,
description="Enable entity recognition"),
# ── Speaker analysis ─────────────────────────────────────────
Setting("speaker_analysis_threshold", int, 3, min_value=1, max_value=25,
description="Speaker analysis threshold"),
Setting("speaker_pronunciation_sentence", str, "This is {{name}} speaking.",
description="Template for pronunciation samples"),
Setting("speaker_random_languages", list, [],
normalizer=_norm_language_list,
description="Languages for random speaker assignment"),
# ── LLM ──────────────────────────────────────────────────────
Setting("llm_base_url", str, lambda: _default_llm("llm_base_url"),
normalizer=_norm_stripped_str,
description="LLM API base URL"),
Setting("llm_api_key", str, lambda: _default_llm("llm_api_key"),
normalizer=_norm_stripped_str,
description="LLM API key"),
Setting("llm_model", str, lambda: _default_llm("llm_model"),
normalizer=_norm_stripped_str,
description="LLM model name"),
Setting("llm_timeout", float, lambda: _default_llm("llm_timeout") or 30.0,
min_value=1.0,
description="LLM request timeout"),
Setting("llm_prompt", str, lambda: _default_llm("llm_prompt") or DEFAULT_LLM_PROMPT,
normalizer=_norm_prompt,
description="LLM normalization prompt"),
Setting("llm_context_mode", str, lambda: _default_llm("llm_context_mode") or "sentence",
valid_values=("sentence",),
description="LLM context mode"),
# ── Normalization (booleans) ─────────────────────────────────
Setting("normalization_numbers", bool, True,
description="Convert grouped numbers to words"),
Setting("normalization_currency", bool, True,
description="Convert currency symbols"),
Setting("normalization_footnotes", bool, True,
description="Remove footnote indicators"),
Setting("normalization_titles", bool, True,
description="Expand titles and suffixes"),
Setting("normalization_terminal", bool, True,
description="Ensure terminal punctuation"),
Setting("normalization_phoneme_hints", bool, True,
description="Add phoneme hints for possessives"),
Setting("normalization_caps_quotes", bool, True,
description="Convert ALL CAPS in quotes"),
Setting("normalization_internet_slang", bool, False,
description="Expand internet slang"),
Setting("normalization_apostrophes_contractions", bool, True,
description="Expand contractions"),
Setting("normalization_apostrophes_plural_possessives", bool, True,
description="Collapse plural possessives"),
Setting("normalization_apostrophes_sibilant_possessives", bool, True,
description="Mark sibilant possessives"),
Setting("normalization_apostrophes_decades", bool, True,
description="Expand decades"),
Setting("normalization_apostrophes_leading_elisions", bool, True,
description="Expand leading elisions"),
Setting("normalization_contraction_aux_be", bool, True,
description="Expand auxiliary 'be'"),
Setting("normalization_contraction_aux_have", bool, True,
description="Expand auxiliary 'have'"),
Setting("normalization_contraction_modal_will", bool, True,
description="Expand modal 'will'"),
Setting("normalization_contraction_modal_would", bool, True,
description="Expand modal 'would'"),
Setting("normalization_contraction_negation_not", bool, True,
description="Expand negation 'not'"),
Setting("normalization_contraction_let_us", bool, True,
description="Expand 'let's'"),
# ── Normalization (strings) ──────────────────────────────────
Setting("normalization_apostrophe_mode", str, "spacy",
valid_values=("off", "spacy", "llm"),
description="Apostrophe handling mode"),
Setting("normalization_numbers_year_style", str, "american",
valid_values=("american", "off"),
description="Year style for number normalization"),
# ── PyQt GUI-only ────────────────────────────────────────────
Setting("theme", str, "system",
gui_only=True,
description="UI theme"),
Setting("check_updates", bool, True,
gui_only=True,
description="Check for updates on startup"),
Setting("subtitle_mode", str, "Sentence",
gui_only=True,
description="Subtitle display mode"),
Setting("selected_format", str, "wav",
gui_only=True,
description="Last selected audio format"),
Setting("selected_voice", str, "af_heart",
gui_only=True,
description="Last selected voice"),
Setting("selected_profile_name", str, None,
gui_only=True,
description="Last selected profile name"),
Setting("log_window_max_lines", int, 2000, min_value=100,
gui_only=True,
description="Max lines in log window"),
Setting("use_silent_gaps", bool, True,
gui_only=True,
description="Use silent gaps between chunks"),
Setting("subtitle_speed_method", str, "tts",
gui_only=True,
valid_values=("tts", "ffmpeg"),
description="Speed adjustment method for subtitles"),
Setting("use_spacy_segmentation", bool, True,
gui_only=True,
description="Use spaCy for sentence segmentation"),
Setting("word_substitutions_enabled", bool, False,
gui_only=True,
description="Enable word substitutions"),
Setting("word_substitutions_list", str, "",
gui_only=True,
description="Word substitutions list"),
Setting("case_sensitive_substitutions", bool, False,
gui_only=True,
description="Case-sensitive substitutions"),
Setting("replace_all_caps", bool, False,
gui_only=True,
description="Replace ALL CAPS text"),
Setting("replace_numerals", bool, False,
gui_only=True,
description="Replace numerals with words"),
Setting("fix_nonstandard_punctuation", bool, False,
gui_only=True,
description="Fix nonstandard punctuation"),
Setting("queue_override_settings", bool, False,
gui_only=True,
description="Override settings per queue item"),
Setting("disable_kokoro_internet", bool, False,
description="Disable Kokoro internet access"),
]
# ── Registry helpers ─────────────────────────────────────────────────
_REGISTRY_BY_KEY: dict[str, Setting] = {s.key: s for s in SETTINGS_REGISTRY}
SETTING_KEYS: frozenset[str] = frozenset(_REGISTRY_BY_KEY.keys())
GUI_ONLY_KEYS: frozenset[str] = frozenset(s.key for s in SETTINGS_REGISTRY if s.gui_only)
WEB_ONLY_KEYS: frozenset[str] = frozenset(s.key for s in SETTINGS_REGISTRY if s.web_only)
SHARED_KEYS: frozenset[str] = SETTING_KEYS - GUI_ONLY_KEYS - WEB_ONLY_KEYS
BOOLEAN_SETTINGS: frozenset[str] = frozenset(s.key for s in SETTINGS_REGISTRY if s.type_ is bool)
FLOAT_SETTINGS: frozenset[str] = frozenset(s.key for s in SETTINGS_REGISTRY if s.type_ is float)
INT_SETTINGS: frozenset[str] = frozenset(s.key for s in SETTINGS_REGISTRY if s.type_ is int)
# Backward-compatible aliases (used by existing code)
_NORMALIZATION_BOOLEAN_KEYS: frozenset[str] = frozenset(
s.key for s in SETTINGS_REGISTRY
if s.type_ is bool and s.key.startswith("normalization_")
)
_NORMALIZATION_STRING_KEYS: frozenset[str] = frozenset(
s.key for s in SETTINGS_REGISTRY
if s.type_ is str and s.key.startswith("normalization_")
)
def get_setting(key: str) -> Setting | None:
"""Look up a setting by key."""
return _REGISTRY_BY_KEY.get(key)
def has_output_override() -> bool:
return bool(os.environ.get("ABOGEN_OUTPUT_DIR") or os.environ.get("ABOGEN_OUTPUT_ROOT"))
# ── Defaults ─────────────────────────────────────────────────────────
def settings_defaults() -> Dict[str, Any]:
"""Default values for all shared settings (excludes gui_only)."""
result: Dict[str, Any] = {}
for s in SETTINGS_REGISTRY:
if s.gui_only:
continue
result[s.key] = s.default() if callable(s.default) else s.default
return result
def all_settings_defaults() -> Dict[str, Any]:
"""Default values for ALL settings (including gui_only)."""
result: Dict[str, Any] = {}
for s in SETTINGS_REGISTRY:
result[s.key] = s.default() if callable(s.default) else s.default
return result
def load_settings() -> Dict[str, Any]:
"""Load and normalize settings from config file."""
from abogen.utils import load_config
defaults = settings_defaults()
cfg = load_config() or {}
settings: Dict[str, Any] = {}
for key, default in defaults.items():
raw_value = cfg.get(key, default)
settings[key] = normalize_setting_value(key, raw_value, defaults)
return settings
# ── Normalization (delegates to Setting.coerce) ──────────────────────
def normalize_setting_value(key: str, value: Any, defaults: Dict[str, Any]) -> Any:
"""Normalize a single setting value using the registry schema."""
setting = _REGISTRY_BY_KEY.get(key)
if setting is None:
return value if value is not None else defaults.get(key)
fallback = defaults.get(key, setting.default() if callable(setting.default) else setting.default)
if setting.normalizer is not None:
return setting.normalizer(value, fallback)
return setting.coerce(value, fallback)
def validate_setting(key: str, value: Any) -> tuple[bool, str]:
"""Validate a setting value against its schema. Returns (ok, error_message)."""
setting = _REGISTRY_BY_KEY.get(key)
if setting is None:
return False, f"Unknown setting: {key}"
if setting.type_ is str and setting.valid_values is not None:
v = str(value or "").strip()
if v and v not in setting.valid_values:
return False, f"Invalid value '{v}' for {key}. Allowed: {setting.valid_values}"
if setting.type_ is int:
try:
iv = int(value)
except (TypeError, ValueError):
return False, f"Invalid integer value for {key}: {value!r}"
if setting.min_value is not None and iv < setting.min_value:
return False, f"{key} must be >= {setting.min_value}, got {iv}"
if setting.max_value is not None and iv > setting.max_value:
return False, f"{key} must be <= {setting.max_value}, got {iv}"
if setting.type_ is float:
try:
fv = float(value)
except (TypeError, ValueError):
return False, f"Invalid float value for {key}: {value!r}"
if setting.min_value is not None and fv < setting.min_value:
return False, f"{key} must be >= {setting.min_value}, got {fv}"
if setting.max_value is not None and fv > setting.max_value:
return False, f"{key} must be <= {setting.max_value}, got {fv}"
return True, ""
# ── Constants (backward-compatible) ──────────────────────────────────
SAVE_MODE_LABELS = {
"save_next_to_input": "Save next to input file",
"save_to_desktop": "Save to Desktop",
"choose_output_folder": "Choose output folder",
"default_output": "Use default save location",
}
LEGACY_SAVE_MODE_MAP = {label: key for key, label in SAVE_MODE_LABELS.items()}
CHUNK_LEVEL_OPTIONS = [
{"value": "paragraph", "label": "Paragraphs"},
{"value": "sentence", "label": "Sentences"},
]
CHUNK_LEVEL_VALUES = frozenset(option["value"] for option in CHUNK_LEVEL_OPTIONS)
DEFAULT_ANALYSIS_THRESHOLD = 3
# ── Coercion helpers (backward-compatible, delegate to Setting.coerce) ──
def coerce_bool(value: Any, default: bool) -> bool:
return Setting("_", bool, default).coerce(value, default)
def coerce_float(value: Any, default: float) -> float:
return Setting("_", float, default).coerce(value, default)
def coerce_int(value: Any, default: int, *, minimum: int = 1, maximum: int = 200) -> int:
return Setting("_", int, default, min_value=minimum, max_value=maximum).coerce(value, default)
def split_profile_spec(value: Any) -> tuple[str, str | None]:
"""Split 'speaker:Name' or 'profile:Name' into (raw, name)."""
text = str(value or "").strip()
if not text:
return "", None
lowered = text.lower()
if lowered.startswith("profile:") or lowered.startswith("speaker:"):
_, _, remainder = text.partition(":")
name = remainder.strip()
return "", name or None
return text, None
def normalize_save_mode(value: Any, default: str) -> str:
return _norm_save_mode(value, default)
# ── LLM helpers ──────────────────────────────────────────────────────
_PROMPT_TOKEN_RE = re.compile(r"{{\s*([a-zA-Z0-9_]+)\s*}}")
def llm_ready(settings: Mapping[str, Any]) -> bool:
base_url = str(settings.get("llm_base_url") or "").strip()
return bool(base_url)
def render_prompt_template(template: str, context: Mapping[str, str]) -> str:
if not template:
return ""
def _replace(match: re.Match[str]) -> str:
key = match.group(1)
return context.get(key, "")
return _PROMPT_TOKEN_RE.sub(_replace, template)
# ── Integration defaults ─────────────────────────────────────────────
def integration_defaults() -> Dict[str, Dict[str, Any]]:
"""Default values for integration settings."""
return {
"calibre_opds": {
"enabled": False,
"base_url": "",
"username": "",
"password": "",
"verify_ssl": True,
},
"audiobookshelf": {
"enabled": False,
"base_url": "",
"api_token": "",
"library_id": "",
"collection_id": "",
"folder_id": "",
"verify_ssl": True,
"send_cover": True,
"send_chapters": True,
"send_subtitles": False,
"auto_send": False,
"timeout": 30.0,
},
}
+50
View File
@@ -0,0 +1,50 @@
from __future__ import annotations
"""Unified split pattern logic extracted from 3 copies."""
import re
from abogen.domain.enums import Language, SubtitleMode
PUNCTUATION_SENTENCE = r".!?。!?"
PUNCTUATION_SENTENCE_COMMA = r".!?,。!?、,"
def get_split_pattern(language: str, subtitle_mode: str) -> str:
"""Get the appropriate split pattern based on language and subtitle mode.
Args:
language: Language code (a, b, e, f, etc.)
subtitle_mode: Subtitle mode ("Sentence", "Sentence + Comma", "Line", etc.)
Returns:
Split pattern string
"""
try:
lang = Language.from_str(language) if not isinstance(language, Language) else language
except ValueError:
lang = None # unknown language — treat as non-English, non-CJK
try:
mode = SubtitleMode.from_str(subtitle_mode) if not isinstance(subtitle_mode, SubtitleMode) else subtitle_mode
except ValueError:
mode = SubtitleMode.DISABLED
# For English, always use newline splitting only
if lang in (Language.EN_US, Language.EN_GB):
return "\n"
# Determine spacing pattern based on language
spacing = r"\s*" if lang and lang.is_cjk else r"\s+"
# For CJK languages, when subtitle mode is Disabled or Line, prefer
# punctuation-based splitting instead of plain newline splitting.
if mode in (SubtitleMode.DISABLED, SubtitleMode.LINE) and lang and lang.is_cjk:
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
if mode == SubtitleMode.LINE:
return "\n"
elif mode == SubtitleMode.SENTENCE:
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
elif mode == SubtitleMode.SENTENCE_COMMA:
return rf"(?<=[{PUNCTUATION_SENTENCE_COMMA}]){spacing}|\n+"
else:
return r"\n+"
+372
View File
@@ -0,0 +1,372 @@
"""Subtitle generation utilities for audiobook generation.
This module provides functions for processing TTS tokens into subtitle entries
according to various subtitle modes (Line, Sentence, Sentence + Comma,
Sentence + Highlighting).
"""
from __future__ import annotations
import re
from typing import List, Optional, Tuple
from abogen.domain.enums import Language, SubtitleMode
# Punctuation constants for sentence splitting
PUNCTUATION_SENTENCE = ".!?\u061f\u3002\uff01\uff1f" # .!? .?. ??
PUNCTUATION_SENTENCE_COMMA = ".!?,\u3001\u061f\u3002\uff01\uff0c\uff1f" # .!?, ,. ??
def process_subtitle_tokens(
tokens_with_timestamps: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
lang_code: str,
use_spacy_segmentation: bool = False,
fallback_end_time: Optional[float] = None,
) -> None:
"""Process TTS tokens into subtitle entries according to the subtitle mode.
This function modifies subtitle_entries in-place by appending new entries.
Args:
tokens_with_timestamps: List of token dictionaries with 'start', 'end', 'text',
and 'whitespace' keys.
subtitle_entries: List to append subtitle entries to (modified in-place).
Each entry is a tuple of (start_time, end_time, text).
max_subtitle_words: Maximum number of words per subtitle entry.
subtitle_mode: One of "Disabled", "Line", "Sentence", "Sentence + Comma",
"Sentence + Highlighting", or a string like "5" for word-count mode.
lang_code: Language code for spaCy processing (e.g., "a" for English).
use_spacy_segmentation: Whether to use spaCy for sentence boundary detection.
fallback_end_time: Fallback end time for the last entry if none is available.
"""
if not tokens_with_timestamps:
return
processed_tokens = tokens_with_timestamps
# For English with spaCy enabled and sentence-based modes, use spaCy for sentence boundaries
# spaCy is disabled when subtitle mode is "Disabled" or "Line"
use_spacy_for_english = (
use_spacy_segmentation
and subtitle_mode not in [SubtitleMode.DISABLED, SubtitleMode.LINE]
and lang_code in [Language.EN_US, Language.EN_GB]
and subtitle_mode in [SubtitleMode.SENTENCE, SubtitleMode.SENTENCE_COMMA]
)
if subtitle_mode == SubtitleMode.SENTENCE_HIGHLIGHT:
_process_karaoke_highlighting(
processed_tokens, subtitle_entries, max_subtitle_words, fallback_end_time
)
elif subtitle_mode in [SubtitleMode.SENTENCE, SubtitleMode.SENTENCE_COMMA, SubtitleMode.LINE]:
if use_spacy_for_english and subtitle_mode != SubtitleMode.LINE:
_process_spacy_sentences(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, lang_code, fallback_end_time
)
else:
_process_regex_sentences(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
else:
# Word count-based grouping (e.g., "5" for 5-word groups)
_process_word_count(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
def _process_karaoke_highlighting(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens for Sentence + Highlighting mode (karaoke effect)."""
separator = rf"[{re.escape(PUNCTUATION_SENTENCE)}]"
current_sentence = []
word_count = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
# Split sentences based on separator or word count
if (
re.search(separator, token["text"]) and token.get("whitespace") == " "
) or word_count >= max_subtitle_words:
if current_sentence:
# Create karaoke subtitle entry for this sentence
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Generate karaoke text with timing
karaoke_text = ""
for t in current_sentence:
# Calculate duration in centiseconds
duration = (
t["end"] - t["start"]
if t.get("end") is not None and t.get("start") is not None
else 0.5
)
duration_cs = int(duration * 100)
# Add karaoke effect
karaoke_text += f"{{\\kf{duration_cs}}}{t['text']}{t.get('whitespace', '') or ''}"
subtitle_entries.append(
(start_time, end_time, karaoke_text.strip())
)
current_sentence = []
word_count = 0
# Add any remaining tokens as a sentence
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Generate karaoke text for remaining tokens
karaoke_text = ""
for t in current_sentence:
duration = t["end"] - t["start"] if t.get("end") and t.get("start") else 0.5
duration_cs = int(duration * 100)
karaoke_text += f"{{\\kf{duration_cs}}}{t['text']}{t.get('whitespace', '') or ''}"
subtitle_entries.append((start_time, end_time, karaoke_text.strip()))
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_spacy_sentences(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
lang_code: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens using spaCy for sentence boundary detection."""
try:
from abogen.spacy_utils import get_spacy_model
except ImportError:
# Fall back to regex if spaCy is not available
_process_regex_sentences(
tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
return
nlp = get_spacy_model(lang_code)
if not nlp:
_process_regex_sentences(
tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
return
# Build full text and track character positions to token indices
full_text = ""
for token in tokens:
text_part = token["text"] + (token.get("whitespace") or "")
full_text += text_part
# Get sentence boundaries from spaCy
doc = nlp(full_text)
sentence_boundaries = [sent.end_char for sent in doc.sents]
# For "Sentence + Comma" mode, also split on commas
if subtitle_mode == SubtitleMode.SENTENCE_COMMA:
comma_positions = [
i + 1 for i, c in enumerate(full_text) if c == ","
]
sentence_boundaries = sorted(
set(sentence_boundaries + comma_positions)
)
# Group tokens by sentence boundaries
current_sentence = []
word_count = 0
current_char_pos = 0
boundary_idx = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
text_len = len(token["text"]) + len(token.get("whitespace") or "")
current_char_pos += text_len
# Check if we've hit a sentence boundary or max words
at_boundary = (
boundary_idx < len(sentence_boundaries)
and current_char_pos >= sentence_boundaries[boundary_idx]
)
if at_boundary or word_count >= max_subtitle_words:
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = "".join(
t["text"] + (t.get("whitespace") or "")
for t in current_sentence
)
subtitle_entries.append(
(start_time, end_time, sentence_text.strip())
)
current_sentence = []
word_count = 0
if at_boundary:
boundary_idx += 1
# Add remaining tokens
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = "".join(
t["text"] + (t.get("whitespace") or "")
for t in current_sentence
)
subtitle_entries.append(
(start_time, end_time, sentence_text.strip())
)
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_regex_sentences(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens using regex for sentence boundary detection."""
# Define separator pattern based on mode
if subtitle_mode == SubtitleMode.LINE:
separator = r"\n"
elif subtitle_mode == SubtitleMode.SENTENCE:
# Use punctuation without comma
separator = rf"[{re.escape(PUNCTUATION_SENTENCE)}]"
else: # Sentence + Comma
# Use punctuation with comma
separator = rf"[{re.escape(PUNCTUATION_SENTENCE_COMMA)}]"
current_sentence = []
word_count = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
# Split sentences based on separator or word count
if (
re.search(separator, token["text"]) and token.get("whitespace") == " "
) or word_count >= max_subtitle_words:
if current_sentence:
# Create subtitle entry for this sentence
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Simplified text joining logic
sentence_text = ""
for t in current_sentence:
sentence_text += t["text"] + (t.get("whitespace") or "")
subtitle_entries.append(
(start_time, end_time, sentence_text.strip())
)
current_sentence = []
word_count = 0
# Add any remaining tokens as a sentence (split multi-sentence FakeToken)
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = ""
for t in current_sentence:
sentence_text += t["text"] + (t.get("whitespace") or "")
sentence_text = sentence_text.strip()
if len(current_sentence) == 1:
parts = re.split(rf"(?<={separator})\s+", sentence_text)
if len(parts) > 1:
d = end_time - start_time
for i, p in enumerate(parts):
e = end_time if i == len(parts) - 1 else start_time + d * len(p) / len(sentence_text)
subtitle_entries.append((start_time, e, p.strip()))
start_time = e
current_sentence = []
if current_sentence:
subtitle_entries.append((start_time, end_time, sentence_text))
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_word_count(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens by counting spaces (word count mode)."""
try:
word_count = int(subtitle_mode.split()[0])
word_count = min(word_count, max_subtitle_words)
except (ValueError, IndexError):
word_count = 1
current_group = []
space_count = 0
for token in tokens:
current_group.append(token)
# Count spaces after tokens (in the whitespace field)
if token.get("whitespace", "") == " ":
space_count += 1
# Split after counting N spaces
if space_count >= word_count:
text = "".join(
t["text"] + (t.get("whitespace") or "")
for t in current_group
)
subtitle_entries.append(
(
current_group[0]["start"],
current_group[-1]["end"],
text.strip(),
)
)
current_group = []
space_count = 0
# Add any remaining tokens
if current_group:
text = "".join(
t["text"] + (t.get("whitespace") or "") for t in current_group
)
subtitle_entries.append(
(current_group[0]["start"], current_group[-1]["end"], text.strip())
)
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _apply_fallback_end_time(
subtitle_entries: List[Tuple[float, float, str]],
fallback_end_time: Optional[float],
) -> None:
"""Apply fallback end time to the last entry if needed."""
if subtitle_entries and fallback_end_time is not None:
last_entry = subtitle_entries[-1]
start, end, text = last_entry
if end is None or end <= start or end <= 0:
subtitle_entries[-1] = (start, fallback_end_time, text)
+279
View File
@@ -0,0 +1,279 @@
"""Subtitle-to-audio processing pipeline.
Converts subtitle files (SRT/ASS/VTT/timestamp text) into audio by
generating TTS for each entry and mixing into a buffer.
"""
from __future__ import annotations
import logging
import time
from dataclasses import dataclass
from typing import Any, Callable, List, Optional, Tuple
import numpy as np
from abogen.domain.audio_buffer import (
create_silence,
fit_audio_to_duration,
ffmpeg_time_stretch,
mix_audio,
normalize_audio,
SAMPLE_RATE,
)
from abogen.domain.audio_helpers import to_float32
from abogen.domain.progress import calc_etr_str
from abogen.subtitle_utils import (
parse_ass_file,
parse_srt_file,
parse_vtt_file,
parse_timestamp_text_file,
)
logger = logging.getLogger(__name__)
@dataclass
class SubtitleEntry:
"""A single subtitle entry with timing."""
start: float
end: Optional[float]
text: str
def parse_subtitle_file(
file_path: str,
is_timestamp_text: bool = False,
) -> List[Tuple[float, Optional[float], str]]:
"""Parse a subtitle file into (start, end, text) tuples.
Args:
file_path: Path to subtitle file.
is_timestamp_text: Whether to treat as timestamp text file.
Returns:
List of (start_time, end_time, text) tuples.
"""
if is_timestamp_text:
return parse_timestamp_text_file(file_path)
import os
ext = os.path.splitext(file_path)[1].lower()
if ext == ".srt":
return parse_srt_file(file_path)
elif ext == ".vtt":
return parse_vtt_file(file_path)
else:
return parse_ass_file(file_path)
def format_time_range(
start: float,
end: Optional[float],
is_auto_end: bool = False,
) -> str:
"""Format a time range for display in logs.
Args:
start: Start time in seconds.
end: End time in seconds, or None.
is_auto_end: Whether end time is auto-detected.
Returns:
Formatted string like "00:01:23,456 - 00:01:25,789" or "00:01:23 - AUTO".
"""
def _fmt(seconds: float) -> str:
h = int(seconds // 3600)
m = int(seconds % 3600 // 60)
s = int(seconds % 60)
ms = int((seconds - int(seconds)) * 1000)
result = f"{h:02d}:{m:02d}:{s:02d}"
if ms > 0:
result += f",{ms:03d}"
return result
if is_auto_end or end is None:
return f"{_fmt(start)} - AUTO"
return f"{_fmt(start)} - {_fmt(end)}"
def speed_up_audio(
audio: np.ndarray,
speed_factor: float,
method: str = "tts",
*,
backend: Any = None,
text: str = "",
voice: Any = None,
base_speed: float = 1.0,
sample_rate: int = SAMPLE_RATE,
) -> np.ndarray:
"""Speed up audio to fit a time window.
Args:
audio: Input audio buffer.
speed_factor: Required speed multiplier.
method: "ffmpeg" for time-stretch, "tts" for regeneration.
backend: TTS backend (required if method="tts").
text: Text to regenerate (required if method="tts").
voice: Voice to use for regeneration.
base_speed: Base speed for TTS.
sample_rate: Sample rate.
Returns:
Speed-adjusted audio buffer.
"""
if speed_factor <= 1.0:
return audio
if method == "ffmpeg":
logger.info("FFmpeg time-stretch: %.2fx", speed_factor)
return ffmpeg_time_stretch(audio, speed_factor, sample_rate)
# TTS regeneration
if backend is None:
return audio
new_speed = base_speed * speed_factor
logger.info("Regenerating at %.2fx speed", new_speed)
results = [
r for r in backend(text, voice=voice, speed=new_speed, split_pattern=None)
]
chunks = [r.audio for r in results]
if not chunks:
return audio
return np.concatenate([to_float32(c) for c in chunks])
def process_subtitle_entries(
subtitles: List[Tuple[float, Optional[float], str]],
*,
backend: Any,
voice: Any,
speed: float = 1.0,
cancel_check: Callable[[], bool] = lambda: False,
log_callback: Optional[Callable[[str], None]] = None,
progress_callback: Optional[Callable[[int, str], None]] = None,
replace_newlines: bool = True,
use_gaps: bool = False,
is_timestamp_text: bool = False,
subtitle_speed_method: str = "tts",
sample_rate: int = SAMPLE_RATE,
) -> np.ndarray:
"""Process subtitle entries: generate TTS for each and mix into buffer.
This is the core domain logic for subtitle-to-audio conversion.
UI-specific concerns (signals, widgets) are handled via callbacks.
Args:
subtitles: List of (start, end, text) tuples.
backend: TTS pipeline callable.
voice: Resolved voice for TTS.
speed: TTS speed.
cancel_check: Returns True if processing should stop.
log_callback: Called with log messages.
progress_callback: Called with (percent, etr_string).
replace_newlines: Replace \\n with spaces in text.
use_gaps: Whether to use silent gaps between subtitles.
is_timestamp_text: Whether input is timestamp text.
subtitle_speed_method: "ffmpeg" or "tts" for speed adjustment.
sample_rate: Audio sample rate.
Returns:
Mixed audio buffer (float32).
"""
if not subtitles:
return np.array([], dtype="float32")
max_end = max((end for _, end, _ in subtitles if end is not None), default=0)
buffer_samples = int(max_end * sample_rate) + sample_rate
audio_buffer = np.zeros(buffer_samples, dtype="float32")
etr_start = time.time()
total = len(subtitles)
for idx, (start_time, end_time, text) in enumerate(subtitles, 1):
if cancel_check():
break
processed_text = text.replace("\n", " ") if replace_newlines else text
next_start = (
subtitles[idx][0]
if (use_gaps and idx < total)
else float("inf")
)
subtitle_duration = None if end_time is None else end_time - start_time
is_auto_end = is_timestamp_text or (use_gaps and idx == total) or end_time is None
if log_callback:
log_callback(
f"\n[{idx}/{total}] {format_time_range(start_time, end_time, is_auto_end)}: {processed_text}"
)
# Generate TTS
results = [
r for r in backend(
processed_text, voice=voice, speed=speed, split_pattern=None
)
if not cancel_check()
]
if cancel_check():
break
audio_chunks = [r.audio for r in results]
full_audio = (
np.concatenate([to_float32(a) for a in audio_chunks])
if audio_chunks
else np.zeros(int((subtitle_duration or 0) * sample_rate), dtype="float32")
)
audio_duration = len(full_audio) / sample_rate
# Timing adjustment
if is_timestamp_text:
end_time = start_time + audio_duration
subtitle_duration = audio_duration
elif use_gaps:
end_time = min(start_time + audio_duration, next_start)
subtitle_duration = end_time - start_time
elif subtitle_duration is None:
subtitle_duration = audio_duration
end_time = start_time + audio_duration
# Speed up if needed
speedup_threshold = next_start - start_time if use_gaps else subtitle_duration
if audio_duration > speedup_threshold and speedup_threshold > 0:
speed_factor = audio_duration / speedup_threshold
full_audio = speed_up_audio(
full_audio, speed_factor,
method=subtitle_speed_method,
backend=backend, text=processed_text,
voice=voice, base_speed=speed,
sample_rate=sample_rate,
)
audio_duration = len(full_audio) / sample_rate
# Adjust duration after speed change
if use_gaps:
end_time = min(start_time + audio_duration, next_start)
subtitle_duration = end_time - start_time
elif subtitle_duration is None:
subtitle_duration = audio_duration
end_time = start_time + audio_duration
# Pad or trim to subtitle duration
full_audio = fit_audio_to_duration(full_audio, subtitle_duration, sample_rate)
# Mix into buffer
start_sample = int(start_time * sample_rate)
audio_buffer = mix_audio(audio_buffer, full_audio, start_sample)
# Progress
if progress_callback:
percent = min(int(idx / total * 100), 99)
etr = calc_etr_str(time.time() - etr_start, idx, total)
progress_callback(percent, etr)
# Normalize if needed
if np.abs(audio_buffer).max() > 1.0:
logger.info("Normalizing audio (peak: %.2f)", np.abs(audio_buffer).max())
audio_buffer = normalize_audio(audio_buffer)
return audio_buffer
+59
View File
@@ -0,0 +1,59 @@
"""Chapter parsing from raw text.
Provides a unified function for splitting text by chapter markers,
used by both WebUI and PyQt conversion runners.
"""
from __future__ import annotations
import re
from typing import List, Tuple
from abogen.subtitle_utils import clean_text
_CHAPTER_MARKER_RE = re.compile(r"<<CHAPTER_MARKER:(.*?)>>", re.IGNORECASE)
def parse_chapters_from_text(
text: str,
default_title: str = "text",
clean: bool = True,
) -> List[Tuple[str, str]]:
"""Split raw text into chapters using chapter marker patterns.
Preserves content before the first marker as "Introduction" if present.
Optionally applies clean_text() to each chapter segment.
Args:
text: Raw text possibly containing <<CHAPTER_MARKER:Title>> markers.
default_title: Fallback title when no markers are found.
clean: Whether to apply clean_text() to each segment.
Returns:
List of (title, text) tuples.
"""
matches = list(_CHAPTER_MARKER_RE.finditer(text))
if not matches:
cleaned = clean_text(text) if clean else text
return [(default_title, cleaned)]
chapters: List[Tuple[str, str]] = []
# Preserve content before first marker as "Introduction"
first_start = matches[0].start()
if first_start > 0:
intro_text = text[:first_start].strip()
if intro_text:
chapters.append(("Introduction", clean_text(intro_text) if clean else intro_text))
for idx, match in enumerate(matches):
start = match.end()
end = matches[idx + 1].start() if idx + 1 < len(matches) else len(text)
chapter_name = match.group(1).strip() or default_title
chapter_text = text[start:end].strip()
if clean:
chapter_text = clean_text(chapter_text)
chapters.append((chapter_name, chapter_text))
return chapters
+97
View File
@@ -0,0 +1,97 @@
from __future__ import annotations
from pathlib import Path
from typing import Any, Dict, List, Mapping, Optional
from .metadata_helpers import (
ensure_sentence,
extract_series_metadata,
format_author_sentence,
format_series_sentence,
normalize_metadata_map,
)
def build_title_intro_text(
metadata: Optional[Mapping[str, Any]],
fallback_basename: str,
) -> str:
"""Build the title introduction text from metadata."""
normalized = normalize_metadata_map(metadata)
fallback_title = Path(fallback_basename).stem if fallback_basename else ""
title = (
normalized.get("title")
or normalized.get("book_title")
or normalized.get("album")
or fallback_title
)
if not title:
title = fallback_title
subtitle = normalized.get("subtitle") or normalized.get("sub_title")
if subtitle and title and subtitle.casefold() == title.casefold():
subtitle = ""
author_value = ""
for candidate in ("artist", "album_artist", "author", "authors", "writer", "composer"):
value = normalized.get(candidate)
if value:
author_value = value
break
series_name, series_number = extract_series_metadata(normalized)
series_sentence = format_series_sentence(series_name, series_number)
sentences: List[str] = []
if series_sentence:
sentences.append(ensure_sentence(series_sentence))
if title:
sentences.append(ensure_sentence(title))
if subtitle:
sentences.append(ensure_sentence(subtitle))
author_sentence = format_author_sentence(author_value)
if author_sentence:
sentences.append(ensure_sentence(author_sentence))
return " ".join(sentences).strip()
def build_outro_text(
metadata: Optional[Mapping[str, Any]],
fallback_basename: str,
) -> str:
"""Build the outro/closing text from metadata."""
normalized = normalize_metadata_map(metadata)
fallback_title = Path(fallback_basename).stem if fallback_basename else ""
title = (
normalized.get("title")
or normalized.get("book_title")
or normalized.get("album")
or fallback_title
)
author_value = ""
for candidate in ("authors", "author", "album_artist", "artist", "writer", "composer"):
value = normalized.get(candidate)
if value:
author_value = value
break
author_sentence = format_author_sentence(author_value)
authors_fragment = (
author_sentence[3:].strip() if author_sentence.lower().startswith("by ") else author_sentence.strip()
)
if title and authors_fragment:
closing_line = f"The end of {title} from {authors_fragment}"
elif title:
closing_line = f"The end of {title}"
elif authors_fragment:
closing_line = f"The end from {authors_fragment}"
else:
closing_line = "The end"
series_name, series_number = extract_series_metadata(normalized)
series_sentence = format_series_sentence(series_name, series_number)
sentences: List[str] = [ensure_sentence(closing_line)]
if series_sentence:
sentences.append(ensure_sentence(series_sentence))
return " ".join(sentence for sentence in sentences if sentence).strip()
+13
View File
@@ -0,0 +1,13 @@
"""Shared token stubs for TTS processing."""
from __future__ import annotations
class FakeToken:
"""Minimal token stub for languages without per-word token support."""
def __init__(self, text: str, start: float, end: float):
self.text = text
self.start_ts = start
self.end_ts = end
self.whitespace = ""
+128
View File
@@ -0,0 +1,128 @@
"""Voice loading and caching utilities.
This module provides unified voice loading with caching support for both
PyQt and WebUI interfaces.
"""
from __future__ import annotations
from typing import Any, Dict, Optional, Tuple
from abogen.voice_formulas import get_new_voice
class VoiceCache:
"""Thread-safe voice cache for loaded voice tensors."""
def __init__(self):
self._cache: Dict[str, Any] = {}
def get(self, voice_spec: str) -> Optional[Any]:
"""Get cached voice by spec."""
return self._cache.get(voice_spec)
def set(self, voice_spec: str, voice: Any) -> None:
"""Cache a loaded voice."""
self._cache[voice_spec] = voice
def contains(self, voice_spec: str) -> bool:
"""Check if voice is in cache."""
return voice_spec in self._cache
def clear(self) -> None:
"""Clear all cached voices."""
self._cache.clear()
def keys(self):
"""Return cached voice specs."""
return self._cache.keys()
def __contains__(self, voice_spec: str) -> bool:
return self.contains(voice_spec)
def resolve_voice(
voice_spec: str,
pipeline: Any,
use_gpu: bool,
cache: Optional[VoiceCache] = None,
) -> Any:
"""Resolve voice spec to actual voice tensor or name.
If voice_spec contains '*' (formula), loads the voice using get_new_voice.
Otherwise, returns the voice_spec as-is (it's a voice name).
Uses optional cache to avoid reloading same voice multiple times.
Args:
voice_spec: Voice specification (name or formula string with '*').
pipeline: TTS pipeline instance for loading formula voices.
use_gpu: Whether to use GPU for voice loading.
cache: Optional VoiceCache instance for caching loaded voices.
Returns:
Loaded voice tensor (for formulas) or voice name string.
"""
# Check cache first
if cache and cache.contains(voice_spec):
return cache.get(voice_spec)
# Load voice
if "*" in voice_spec:
if pipeline is None or not hasattr(pipeline, "load_single_voice"):
return voice_spec
loaded_voice = get_new_voice(pipeline, voice_spec, use_gpu)
else:
loaded_voice = voice_spec
# Cache it
if cache:
cache.set(voice_spec, loaded_voice)
return loaded_voice
def load_voice_cached(
voice_name: str,
pipeline: Any,
use_gpu: bool,
cache: Any = None,
) -> Any:
"""Load voice with caching (compatibility wrapper for PyQt).
This function maintains backward compatibility with the PyQt interface
while using the unified voice loading logic.
Args:
voice_name: Voice name or formula string.
pipeline: TTS pipeline instance.
use_gpu: Whether to use GPU.
cache: Optional VoiceCache or dict to use as cache.
Returns:
Loaded voice tensor or voice name string.
"""
# Check cache (supports both VoiceCache and plain dict)
if cache is not None:
if isinstance(cache, VoiceCache):
if cache.contains(voice_name):
return cache.get(voice_name)
elif voice_name in cache:
return cache[voice_name]
# Load voice
if "*" in voice_name:
if pipeline is None or not hasattr(pipeline, "load_single_voice"):
return voice_name
loaded_voice = get_new_voice(pipeline, voice_name, use_gpu)
else:
loaded_voice = voice_name
# Cache it
if cache is not None:
if isinstance(cache, VoiceCache):
cache.set(voice_name, loaded_voice)
else:
cache[voice_name] = loaded_voice
return loaded_voice
+190
View File
@@ -0,0 +1,190 @@
"""Voice resolution helpers.
Functions for resolving voice specifications, collecting required voice IDs,
and determining the voice to use for chapters and chunks.
"""
from __future__ import annotations
from typing import Any, Dict, Optional, Set
from abogen.tts_plugin.utils import get_voices, get_default_voice
from abogen.voice_formulas import extract_voice_ids
from abogen.voice_cache import ensure_voice_assets
def spec_to_voice_ids(spec: Any) -> Set[str]:
text = str(spec or "").strip()
if not text:
return set()
if text == "__custom_mix":
return set()
if "*" in text:
try:
return set(extract_voice_ids(text))
except ValueError:
return set()
if text in get_voices("kokoro"):
return {text}
return set()
def job_voice_fallback(job: Any) -> str:
base = str(getattr(job, "voice", "") or "").strip()
if base and base != "__custom_mix":
return base
speakers = getattr(job, "speakers", None)
if isinstance(speakers, dict):
narrator = speakers.get("narrator")
if isinstance(narrator, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = narrator.get(key)
candidate = str(value or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
for payload in speakers.values() or []:
if not isinstance(payload, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
value = payload.get(key)
candidate = str(value or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
for chapter in getattr(job, "chapters", []) or []:
if not isinstance(chapter, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
candidate = str(chapter.get(key) or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
return ""
def collect_required_voice_ids(job: Any) -> Set[str]:
voices: Set[str] = set()
voices.update(spec_to_voice_ids(job.voice))
voices.update(spec_to_voice_ids(job_voice_fallback(job)))
for chapter in getattr(job, "chapters", []) or []:
if not isinstance(chapter, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(chapter.get(key)))
for chunk in getattr(job, "chunks", []) or []:
if not isinstance(chunk, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(chunk.get(key)))
speakers = getattr(job, "speakers", {})
if isinstance(speakers, dict):
for payload in speakers.values() or []:
if not isinstance(payload, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(payload.get(key)))
voices.update(get_voices("kokoro"))
return voices
def initialize_voice_cache(job: Any) -> None:
try:
targets = collect_required_voice_ids(job)
downloaded, errors = ensure_voice_assets(
targets,
on_progress=lambda message: job.add_log(message, level="debug"),
)
except RuntimeError as exc:
job.add_log(f"Voice cache unavailable: {exc}", level="warning")
return
if downloaded:
job.add_log(
f"Cached {len(downloaded)} voice asset{'s' if len(downloaded) != 1 else ''} locally.",
level="info",
)
for voice_id, error in errors.items():
job.add_log(f"Failed to cache voice '{voice_id}': {error}", level="warning")
def chapter_voice_spec(job: Any, override: Optional[Dict[str, Any]]) -> str:
if not override:
return job_voice_fallback(job)
resolved = str(override.get("resolved_voice", "")).strip()
if resolved:
return resolved
formula = str(override.get("voice_formula", "")).strip()
if formula:
return formula
voice = str(override.get("voice", "")).strip()
if voice:
return voice
return job_voice_fallback(job)
def chunk_voice_spec(job: Any, chunk: Dict[str, Any], fallback: str) -> str:
for key in ("resolved_voice", "voice_formula", "voice"):
value = chunk.get(key)
if value:
return str(value)
speaker_id = chunk.get("speaker_id")
speakers = getattr(job, "speakers", None)
if isinstance(speakers, dict) and speaker_id in speakers:
speaker_entry = speakers.get(speaker_id) or {}
if isinstance(speaker_entry, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = speaker_entry.get(key)
if value:
return str(value)
profile_formula = speaker_entry.get("voice_formula")
if profile_formula:
return str(profile_formula)
profile_name = chunk.get("voice_profile")
if profile_name:
if isinstance(speakers, dict):
speaker_entry = speakers.get(profile_name)
if isinstance(speaker_entry, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = speaker_entry.get(key)
if value:
return str(value)
if fallback:
return fallback
return job_voice_fallback(job)
def resolve_fallback_voice_spec(
base_spec: str,
job_voice: str,
voice_cache_keys: list[str],
provider: str = "kokoro",
) -> str:
"""Resolve the voice spec for intro/outro with a priority fallback chain.
Priority: base_spec → job_voice → first voice_cache key → default voice.
``"__custom_mix"`` is treated as empty (it is not a usable voice spec).
"""
spec = base_spec or job_voice
if spec == "__custom_mix":
spec = job_voice or ""
if not spec:
for key in voice_cache_keys:
if key and key != "__custom_mix":
spec = key.split(":", 1)[-1]
break
if not spec:
spec = get_default_voice(provider)
return spec
+130
View File
@@ -0,0 +1,130 @@
from __future__ import annotations
from typing import Any, Dict, Mapping, Optional, Tuple, Set
from abogen.voice_formulas import extract_voice_ids, get_new_voice
from abogen.tts_plugin.utils import get_voices
def infer_provider_from_spec(value: Any, fallback: str = "kokoro") -> str:
"""Infer TTS provider from voice specification."""
raw = str(value or "").strip()
if not raw:
return fallback
if raw.upper() == raw and raw.replace("_", "").isalnum():
return "supertonic"
if raw == "__custom_mix" or "*" in raw or "+" in raw:
return "kokoro"
if raw in get_voices("kokoro"):
return "kokoro"
return fallback
def supertonic_voice_from_spec(spec: Any, fallback: str) -> str:
"""Normalize a voice specification for Supertonic.
This function only performs Supertonic-specific normalization (uppercase conversion
and fallback handling). Backend resolution is handled by the registry.
"""
raw = str(spec or "").strip()
fallback_raw = str(fallback or "").strip()
# Normalize to uppercase for Supertonic voice IDs
upper = raw.upper() if raw else ""
# If empty or contains formula characters, use fallback
if not upper or "*" in upper or "+" in upper:
upper = fallback_raw.upper() if fallback_raw else ""
# If still empty, use default Supertonic voice
if not upper or "*" in upper or "+" in upper:
upper = "M1"
return upper
def split_speaker_reference(value: Any) -> Tuple[Optional[str], str]:
"""Parse speaker/profile reference from string.
Expected format: "speaker:name" or "profile:name"
Returns (name, original) or (None, original) if not a valid reference.
"""
raw = str(value or "").strip()
if not raw or ":" not in raw:
return None, raw
prefix, remainder = raw.split(":", 1)
prefix = prefix.strip().lower()
if prefix not in {"speaker", "profile"}:
return None, raw
name = remainder.strip()
return (name or None), raw
def formula_from_kokoro_entry(entry: Mapping[str, Any]) -> str:
"""Build voice formula string from kokoro entry."""
voices = entry.get("voices") or []
if not voices:
return ""
total = 0.0
parts: list[tuple[str, float]] = []
for item in voices:
if not isinstance(item, (list, tuple)) or len(item) < 2:
continue
name = str(item[0] or "").strip()
try:
weight = float(item[1])
except (TypeError, ValueError):
continue
if name and weight > 0:
parts.append((name, weight))
total += weight
if not parts:
return ""
normalized = [(name, weight / total) for name, weight in parts]
return " + ".join(f"{name}*{weight:.6f}" for name, weight in normalized)
def coerce_truthy(value: Any, default: bool = True) -> bool:
"""Coerce a value to boolean with default."""
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() not in {"false", "0", "no", "off", ""}
if value is None:
return default
return bool(value)
def resolve_voice_target(
raw_spec: str,
normalized_profiles: Dict[str, Dict[str, Any]],
*,
job_voice: str = "M1",
job_tts_provider: str = "kokoro",
job_supertonic_total_steps: int = 5,
job_speed: float = 1.0,
) -> Tuple[str, str, Optional[float], Optional[int]]:
"""Resolve a raw voice spec into (provider, voice_spec, speed_override, steps_override).
Pure function — all dependencies are passed as parameters.
"""
spec = str(raw_spec or "").strip()
speaker_name, _ = split_speaker_reference(spec)
if speaker_name and speaker_name in normalized_profiles:
entry = normalized_profiles[speaker_name]
provider = str(entry.get("provider") or "kokoro").strip().lower() or "kokoro"
if provider == "supertonic":
voice = str(entry.get("voice") or job_voice or "M1").strip() or "M1"
steps = int(entry.get("total_steps") or job_supertonic_total_steps or 5)
speed = float(entry.get("speed") or job_speed or 1.0)
return "supertonic", supertonic_voice_from_spec(voice, job_voice), speed, steps
formula = formula_from_kokoro_entry(entry)
return "kokoro", formula or spec, None, None
fallback_provider = str(job_tts_provider or "kokoro").strip().lower() or "kokoro"
inferred = infer_provider_from_spec(spec, fallback=fallback_provider)
if inferred == "supertonic":
return "supertonic", supertonic_voice_from_spec(spec, job_voice), None, None
return "kokoro", spec, None, None
+1 -1
View File
@@ -19,7 +19,7 @@ def tracked_hf_hub_download(*args, **kwargs):
try: try:
local_kwargs = dict(kwargs) local_kwargs = dict(kwargs)
local_kwargs["local_files_only"] = True local_kwargs["local_files_only"] = True
hf_hub_download(*args, **local_kwargs) return hf_hub_download(*args, **local_kwargs)
except Exception: except Exception:
repo_id = kwargs.get("repo_id", "<unknown repo>") repo_id = kwargs.get("repo_id", "<unknown repo>")
filename = kwargs.get("filename", "<unknown file>") filename = kwargs.get("filename", "<unknown file>")
+447
View File
@@ -0,0 +1,447 @@
from __future__ import annotations
import json
import logging
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Optional, Mapping, Sequence
import static_ffmpeg
from abogen.domain.metadata_helpers import (
normalize_metadata_casefold,
split_people_field,
split_simple_list,
first_nonempty,
extract_year,
normalize_series_sequence,
build_audiobookshelf_metadata as _build_abs_metadata,
load_audiobookshelf_chapters as _load_abs_chapters,
_SERIES_SEQUENCE_TAG_KEYS,
)
from abogen.epub3.exporter import build_epub3_package
from abogen.integrations.audiobookshelf import (
AudiobookshelfClient,
AudiobookshelfConfig,
AudiobookshelfUploadError,
)
from abogen.utils import create_process
logger = logging.getLogger(__name__)
@dataclass
class ExportConfig:
"""Configuration for export operations."""
ffmpeg_path: str = "ffmpeg"
verify_ssl: bool = True
class ExportService:
"""Unified service for audiobook exports (M4B, FFMETADATA, EPUB3, Audiobookshelf)."""
def __init__(self, config: Optional[ExportConfig] = None):
self.config = config or ExportConfig()
static_ffmpeg.add_paths()
# ----------------------------------------------------------------------
# FFMETADATA
# ----------------------------------------------------------------------
def render_ffmetadata(
self,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
) -> str:
"""Render FFMETADATA content."""
lines = [";FFMETADATA1"]
for key, value in (metadata or {}).items():
if value is None:
continue
key_str = str(key).strip()
if not key_str:
continue
lines.append(f"{key_str}={self._escape_ffmetadata_value(value)}")
for chapter in chapters or []:
start = chapter.get("start")
end = chapter.get("end")
if start is None or end is None:
continue
try:
start_ms = max(0, int(round(float(start) * 1000)))
end_ms = int(round(float(end) * 1000))
except (TypeError, ValueError):
continue
if end_ms <= start_ms:
end_ms = start_ms + 1
lines.append("[CHAPTER]")
lines.append("TIMEBASE=1/1000")
lines.append(f"START={start_ms}")
lines.append(f"END={end_ms}")
title = chapter.get("title")
if title:
lines.append(f"title={self._escape_ffmetadata_value(title)}")
voice = chapter.get("voice")
if voice:
lines.append(f"voice={self._escape_ffmetadata_value(voice)}")
return "\n".join(lines) + "\n"
@staticmethod
def _escape_ffmetadata_value(value: Any) -> str:
escaped = str(value).replace("\\", "\\\\").replace("\n", "\\n")
escaped = escaped.replace("=", "\\=").replace(";", "\\;").replace("#", "\\#")
return escaped
def write_ffmetadata_file(
self,
audio_path: Path,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
) -> Optional[Path]:
"""Write FFMETADATA file to temp location."""
content = self.render_ffmetadata(metadata, chapters)
if content.strip() == ";FFMETADATA1":
return None
directory = audio_path.parent if audio_path.parent.exists() else Path(tempfile.gettempdir())
with tempfile.NamedTemporaryFile(
mode="w",
encoding="utf-8",
suffix=".ffmeta",
delete=False,
dir=str(directory),
) as handle:
handle.write(content)
return Path(handle.name)
# ----------------------------------------------------------------------
# M4B Export
# ----------------------------------------------------------------------
def embed_m4b_metadata(
self,
audio_path: Path,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
cover_path: Optional[Path] = None,
cover_mime: Optional[str] = None,
log_callback: Optional[callable] = None,
) -> None:
"""Embed metadata and chapters into M4B file using FFmpeg + Mutagen."""
ffmetadata_path = self.write_ffmetadata_file(audio_path, metadata, chapters)
metadata_args = self._metadata_to_ffmpeg_args(metadata)
cmd = ["ffmpeg", "-y", "-i", str(audio_path)]
if ffmetadata_path:
cmd.extend(["-f", "ffmetadata", "-i", str(ffmetadata_path)])
if cover_path and cover_path.exists():
cmd.extend(["-i", str(cover_path)])
cmd.extend(["-map", "0:a"])
cmd.extend(["-map", "1:v:0", "-c:v:0", "mjpeg", "-disposition:v:0", "attached_pic"])
if cover_mime:
cmd.extend(["-metadata:s:v:0", f"mimetype={cover_mime}"])
cmd.extend(["-metadata:s:v:0", "title=Cover Art"])
else:
cmd.extend(["-map", "0:a"])
cmd.extend(["-c:a", "copy"])
if ffmetadata_path:
cmd.extend(["-map_metadata", "1", "-map_chapters", "1"])
else:
cmd.extend(["-map_metadata", "0"])
if metadata_args:
cmd.extend(metadata_args)
cmd.extend(["-movflags", "+faststart+use_metadata_tags"])
temp_output = audio_path.with_suffix(audio_path.suffix + ".tmp")
if audio_path.suffix.lower() in {".m4b", ".mp4", ".m4a"}:
cmd.extend(["-f", "mp4"])
cmd.append(str(temp_output))
if log_callback:
log_callback("Embedding metadata into M4B output")
process = create_process(cmd, text=True)
return_code = process.wait()
if ffmetadata_path and ffmetadata_path.exists():
try:
ffmetadata_path.unlink()
except OSError:
pass
if return_code != 0:
if temp_output.exists():
temp_output.unlink(missing_ok=True)
raise RuntimeError(f"ffmpeg failed to embed metadata (exit code {return_code})")
temp_output.replace(audio_path)
if log_callback:
log_callback("Embedded metadata and chapters into M4B output", "info")
# Apply chapters via Mutagen for better compatibility
self._apply_m4b_chapters_mutagen(audio_path, chapters, log_callback)
@staticmethod
def _metadata_to_ffmpeg_args(metadata: Dict[str, Any]) -> List[str]:
args = []
for key, value in (metadata or {}).items():
if value in (None, ""):
continue
key_str = str(key).strip()
if not key_str:
continue
normalized_key = key_str.lower()
if normalized_key == "year":
ffmpeg_key = "date"
else:
ffmpeg_key = key_str
args.extend(["-metadata", f"{ffmpeg_key}={value}"])
return args
def _apply_m4b_chapters_mutagen(
self,
audio_path: Path,
chapters: List[Dict[str, Any]],
log_callback: Optional[callable] = None,
) -> bool:
"""Apply chapter atoms using Mutagen."""
if not chapters:
return False
try:
from fractions import Fraction
from mutagen.mp4 import MP4, MP4Chapter
except ImportError:
if log_callback:
log_callback("Unable to write MP4 chapter atoms because mutagen is not installed.", "warning")
return False
try:
mp4 = MP4(str(audio_path))
except Exception as exc:
if log_callback:
log_callback(f"Failed to open m4b for chapter embedding: {exc}", "warning")
return False
chapter_objects = []
for index, entry in enumerate(sorted(chapters, key=lambda item: float(item.get("start") or 0.0))):
start_raw = entry.get("start")
if start_raw is None:
continue
try:
start_seconds = max(0.0, float(start_raw))
except (TypeError, ValueError):
continue
title_value = entry.get("title")
title_text = str(title_value) if title_value else f"Chapter {index + 1}"
start_fraction = Fraction(int(round(start_seconds * 1000)), 1000)
chapter_atom = MP4Chapter(start_fraction, title_text)
end_raw = entry.get("end")
if end_raw is not None:
try:
end_seconds = float(end_raw)
except (TypeError, ValueError):
end_seconds = None
if end_seconds is not None and end_seconds > start_seconds:
chapter_atom.end = Fraction(int(round(end_seconds * 1000)), 1000)
chapter_objects.append(chapter_atom)
if not chapter_objects:
return False
try:
mp4.chapters = chapter_objects
mp4.save()
except Exception as exc:
if log_callback:
log_callback(f"Failed to persist MP4 chapter atoms: {exc}", "warning")
return False
if log_callback:
log_callback(f"Applied {len(chapter_objects)} chapter markers via mutagen", "info")
return True
# ----------------------------------------------------------------------
# EPUB3 Export
# ----------------------------------------------------------------------
def export_epub3(
self,
output_path: Path,
book_id: str,
extraction: Any, # ExtractionResult
metadata_tags: Dict[str, Any],
chapter_markers: Sequence[Dict[str, Any]],
chunk_markers: Sequence[Dict[str, Any]],
chunks: Iterable[Dict[str, Any]],
audio_path: Path,
speaker_mode: str = "single",
cover_path: Optional[Path] = None,
cover_mime: Optional[str] = None,
) -> Path:
"""Export EPUB3 with media overlays."""
return build_epub3_package(
output_path=output_path,
book_id=book_id,
extraction=extraction,
metadata_tags=metadata_tags,
chapter_markers=chapter_markers,
chunk_markers=chunk_markers,
chunks=chunks,
audio_path=audio_path,
speaker_mode=speaker_mode,
cover_image_path=cover_path,
cover_image_mime=cover_mime,
)
# ----------------------------------------------------------------------
# Audiobookshelf Integration
# ----------------------------------------------------------------------
def build_audiobookshelf_metadata(self, job: Any) -> Dict[str, Any]:
"""Build Audiobookshelf metadata from job."""
filename = Path(getattr(job, "original_filename", "") or "").stem or "Audiobook"
return _build_abs_metadata(
getattr(job, "metadata_tags", {}),
language=getattr(job, "language", "") or "",
filename=filename,
)
def load_audiobookshelf_chapters(self, job: Any) -> Optional[List[Dict[str, Any]]]:
"""Load chapters from job artifacts for Audiobookshelf."""
metadata_ref = job.result.artifacts.get("metadata") if getattr(job, "result", None) else None
if not metadata_ref:
return None
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
return _load_abs_chapters(metadata_path)
def upload_audiobookshelf(
self,
job: Any,
audio_path: Path,
subtitle_paths: List[Path],
chapters: List[Dict[str, Any]],
metadata: Dict[str, Any],
cover_path: Optional[Path] = None,
config: Optional[AudiobookshelfConfig] = None,
log_callback: Optional[callable] = None,
) -> None:
"""Upload to Audiobookshelf."""
if config is None:
cfg = getattr(job, "_abs_config", None)
if cfg is None:
from abogen.utils import load_config
global_cfg = load_config() or {}
abs_cfg = global_cfg.get("audiobookshelf")
if isinstance(abs_cfg, Mapping):
config = AudiobookshelfConfig(
base_url=str(abs_cfg.get("base_url") or "").strip(),
api_token=str(abs_cfg.get("api_token") or "").strip(),
library_id=str(abs_cfg.get("library_id") or "").strip(),
collection_id=(str(abs_cfg.get("collection_id") or "").strip() or None),
folder_id=str(abs_cfg.get("folder_id") or "").strip(),
verify_ssl=self._coerce_bool(abs_cfg.get("verify_ssl"), True),
send_cover=self._coerce_bool(abs_cfg.get("send_cover"), True),
send_chapters=self._coerce_bool(abs_cfg.get("send_chapters"), True),
send_subtitles=self._coerce_bool(abs_cfg.get("send_subtitles"), False),
timeout=float(abs_cfg.get("timeout", 3600.0)),
)
else:
if log_callback:
log_callback("Audiobookshelf upload skipped: not configured", "warning")
return
if not config.base_url or not config.api_token or not config.library_id:
if log_callback:
log_callback("Audiobookshelf upload skipped: configure base URL, API token, and library ID first", "warning")
return
if not config.folder_id:
if log_callback:
log_callback("Audiobookshelf upload skipped: enter folder name or ID in settings", "warning")
return
if not audio_path.exists():
if log_callback:
log_callback("Audiobookshelf upload skipped: audio output not found", "warning")
return
existing_subtitles = [p for p in subtitle_paths if p.exists()] if config.send_subtitles else None
chapters_to_send = chapters if config.send_chapters else None
client = AudiobookshelfClient(config)
display_title = metadata.get("title") or audio_path.stem
try:
existing_items = client.find_existing_items(display_title, folder_id=config.folder_id)
except AudiobookshelfUploadError as exc:
if log_callback:
log_callback(f"Audiobookshelf lookup failed: {exc}", "error")
return
if existing_items:
if log_callback:
log_callback(f"Removing existing Audiobookshelf item(s) for '{display_title}' before upload.", "info")
try:
client.delete_items(existing_items)
except Exception as exc:
if log_callback:
log_callback(f"Failed to remove existing item(s): {exc}", "warning")
cover_to_send = cover_path
if config.send_cover and cover_to_send:
if isinstance(cover_to_send, str):
cover_to_send = Path(cover_to_send)
if not cover_to_send.exists():
cover_to_send = None
client.upload_audiobook(
audio_path,
metadata=metadata,
cover_path=cover_to_send,
chapters=chapters_to_send,
subtitles=existing_subtitles,
)
if log_callback:
log_callback("Audiobookshelf upload queued.", "info")
# ----------------------------------------------------------------------
# Helpers
# ----------------------------------------------------------------------
@staticmethod
def _coerce_bool(value: Any, default: bool = True) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
lowered = value.strip().lower()
if lowered in {"true", "1", "yes", "on"}:
return True
if lowered in {"false", "0", "no", "off"}:
return False
return default
if value is None:
return default
return bool(value)
__all__ = [
"ExportConfig",
"ExportService",
]
+357
View File
@@ -0,0 +1,357 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass
from enum import Enum
from pathlib import Path
from typing import List, Optional, TextIO
from abogen.domain.enums import SubtitleFormat, SubtitleMode
from abogen.subtitle_utils import clean_subtitle_text
class SubtitleAlignment(Enum):
LEFT = "left"
CENTER = "center"
NARROW = "narrow"
CENTER_NARROW = "center_narrow"
@dataclass
class SubtitleConfig:
"""Configuration for subtitle writer."""
format: SubtitleFormat
mode: SubtitleMode
alignment: SubtitleAlignment = SubtitleAlignment.LEFT
max_words: int = 50
highlight_color: str = "&H00FFFF00" # ASS highlight color
class SubtitleWriter(ABC):
"""Abstract base class for subtitle writers."""
def __init__(self, path: Path, config: SubtitleConfig):
self.path = path
self.config = config
self._file: Optional[TextIO] = None
self._index = 0
self._opened = False
def open(self) -> None:
"""Open the subtitle file and write header."""
if self._opened:
return
self._file = open(self.path, "w", encoding="utf-8", errors="replace")
self._write_header()
self._opened = True
@abstractmethod
def _write_header(self) -> None:
pass
def write_entry(
self,
start: float,
end: float,
text: str,
voice: Optional[str] = None,
) -> None:
"""Write a subtitle entry."""
if not self._opened:
self.open()
text = clean_subtitle_text(text)
if not text:
return
self._index += 1
self._write_entry(self._index, start, end, text, voice)
@abstractmethod
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
pass
def close(self) -> None:
"""Close the subtitle file."""
if self._file:
self._file.close()
self._file = None
self._opened = False
def __enter__(self) -> "SubtitleWriter":
self.open()
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.close()
class SrtWriter(SubtitleWriter):
"""SRT subtitle writer."""
def _write_header(self) -> None:
pass # SRT has no header
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
self._file.write(f"{index}\n")
self._file.write(f"{start_str} --> {end_str}\n")
self._file.write(f"{text}\n\n")
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = int(seconds % 60)
millis = int((seconds - int(seconds)) * 1000)
return f"{hours:02d}:{minutes:02d}:{secs:02d},{millis:03d}"
class VttWriter(SubtitleWriter):
"""WebVTT subtitle writer."""
def _write_header(self) -> None:
self._file.write("WEBVTT\n\n")
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
self._file.write(f"{index}\n")
self._file.write(f"{start_str} --> {end_str}\n")
self._file.write(f"{text}\n\n")
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = seconds % 60
return f"{hours:02d}:{minutes:02d}:{secs:06.3f}".replace(".", ".")
class AssWriter(SubtitleWriter):
"""ASS subtitle writer with karaoke highlighting support."""
def __init__(self, path: Path, config: SubtitleConfig):
super().__init__(path, config)
self._is_centered = config.alignment in (SubtitleAlignment.CENTER, SubtitleAlignment.CENTER_NARROW)
self._is_narrow = config.alignment in (SubtitleAlignment.NARROW, SubtitleAlignment.CENTER_NARROW)
def _write_header(self) -> None:
margin = "90" if self._is_narrow else "10"
alignment = "5" if self._is_centered else "2"
self._file.write("[Script Info]\n")
self._file.write("Title: Generated by Abogen\n")
self._file.write("ScriptType: v4.00+\n\n")
# Styles
self._file.write("[V4+ Styles]\n")
self._file.write(
"Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, "
"OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, "
"ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, "
"Alignment, MarginL, MarginR, MarginV, Encoding\n"
)
if self.config.mode == SubtitleMode.SENTENCE_HIGHLIGHT:
# Karaoke style with highlighting
self._file.write(
f"Style: Default,Arial,24,&H00FFFFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n"
)
self._file.write(
f"Style: Highlight,Arial,24,&H0000FFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n\n"
)
else:
self._file.write(
f"Style: Default,Arial,24,&H00FFFFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n\n"
)
self._file.write("[Events]\n")
self._file.write(
"Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text\n"
)
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
style = "Default"
if self.config.mode == SubtitleMode.SENTENCE_HIGHLIGHT:
# Add karaoke tags for highlighting
text = self._add_karaoke_tags(text)
style = "Highlight"
alignment_tag = r"{\an5}" if self._is_centered else ""
self._file.write(
f"Dialogue: 0,{start_str},{end_str},{style},,0,0,0,,{alignment_tag}{text}\n"
)
def _add_karaoke_tags(self, text: str) -> str:
"""Add karaoke highlighting tags to text."""
# Simple word-level karaoke timing
words = text.split()
if not words:
return text
# This is a simplified version - real karaoke needs per-word timing
# For now, just return the text with the highlight color
return r"{\k100}" + r"{\k100}".join(words) + r"{\k0}"
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = seconds % 60
return f"{hours}:{minutes:02d}:{secs:05.2f}"
def create_subtitle_writer(
path: Path,
format: str,
mode: str,
alignment: str = "left",
max_words: int = 50,
) -> SubtitleWriter:
"""Factory function to create subtitle writer."""
fmt = SubtitleFormat(format.lower())
mode = SubtitleMode(mode)
align = SubtitleAlignment(alignment.lower())
config = SubtitleConfig(
format=fmt,
mode=mode,
alignment=align,
max_words=max_words,
)
if fmt == SubtitleFormat.SRT:
return SrtWriter(path, config)
elif fmt == SubtitleFormat.VTT:
return VttWriter(path, config)
elif fmt == SubtitleFormat.ASS:
return AssWriter(path, config)
else:
raise ValueError(f"Unsupported subtitle format: {format}")
def resolve_subtitle_format(
subtitle_format: str | None,
subtitle_mode: str,
) -> tuple[str, str]:
"""Resolve a subtitle_format setting string to (file_extension, alignment).
Handles the PyQt convention where format strings encode alignment
(e.g. ``"ass_centered_narrow"`` → extension ``"ass"``, alignment
``"center_narrow"``).
Also enforces that ``"Sentence + Highlighting"`` mode requires ASS.
Returns:
Tuple of (file_extension, alignment) suitable for
:func:`create_subtitle_writer`.
"""
fmt = (subtitle_format or "srt").lower()
if subtitle_mode == "Sentence + Highlighting" and fmt == "srt":
fmt = "ass"
if "ass" in fmt:
extension = "ass"
if "centered_narrow" in fmt:
alignment = "center_narrow"
elif "centered" in fmt:
alignment = "center"
elif "narrow" in fmt:
alignment = "narrow"
else:
alignment = "left"
else:
extension = fmt if fmt in ("srt", "vtt") else "srt"
alignment = "left"
return extension, alignment
def make_subtitle_writer(
audio_path: Path,
subtitle_format: str | None,
subtitle_mode: str,
max_words: int = 50,
) -> SubtitleWriter | None:
"""Convenience: resolve format and create a writer, or return None if disabled.
Returns ``None`` when ``subtitle_mode`` is ``"Disabled"`` or the
format is unsupported.
"""
if subtitle_mode == "Disabled":
return None
extension, alignment = resolve_subtitle_format(subtitle_format, subtitle_mode)
try:
return create_subtitle_writer(
audio_path.with_suffix(f".{extension}"),
extension,
subtitle_mode,
alignment=alignment,
max_words=max_words,
)
except (ValueError, KeyError):
return None
__all__ = [
"SubtitleFormat",
"SubtitleMode",
"SubtitleAlignment",
"SubtitleConfig",
"SubtitleWriter",
"SrtWriter",
"VttWriter",
"AssWriter",
"create_subtitle_writer",
"resolve_subtitle_format",
"make_subtitle_writer",
]
+3 -36
View File
@@ -2,9 +2,7 @@ from __future__ import annotations
import json import json
import logging import logging
import math
import mimetypes import mimetypes
import re
from contextlib import ExitStack from contextlib import ExitStack
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
@@ -12,6 +10,8 @@ from typing import Any, Dict, Iterable, List, Mapping, Optional, Sequence, Tuple
import httpx import httpx
from abogen.domain.metadata_helpers import normalize_series_sequence
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -641,40 +641,7 @@ class AudiobookshelfClient:
for key in preferred_keys: for key in preferred_keys:
if key not in metadata: if key not in metadata:
continue continue
normalized = AudiobookshelfClient._normalize_series_sequence(metadata.get(key)) normalized = normalize_series_sequence(metadata.get(key))
if normalized: if normalized:
return normalized return normalized
return "" return ""
@staticmethod
def _normalize_series_sequence(raw: Any) -> str:
if raw is None:
return ""
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return ""
text = str(raw)
else:
text = str(raw).strip()
if not text:
return ""
candidate = text.replace(",", ".")
match = re.search(r"\d+(?:\.\d+)?", candidate)
if not match:
return ""
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
+5 -15
View File
@@ -2,13 +2,14 @@
from __future__ import annotations from __future__ import annotations
import atexit
import os import os
import platform import platform
import signal
import sys
from abogen.utils import load_config, prevent_sleep_end # Initialise global shutdown handling (atexit, signals, Qt) as early as possible.
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
from abogen.utils import load_config
from abogen.webui.app import main as _run_web_ui from abogen.webui.app import main as _run_web_ui
# Configure Hugging Face Hub behaviour (mirrors legacy GUI defaults). # Configure Hugging Face Hub behaviour (mirrors legacy GUI defaults).
@@ -27,17 +28,6 @@ os.environ.setdefault("MIOPEN_CONV_PRECISE_ROCM_TUNING", "0")
if platform.system() == "Darwin" and platform.processor() == "arm": if platform.system() == "Darwin" and platform.processor() == "arm":
os.environ.setdefault("PYTORCH_ENABLE_MPS_FALLBACK", "1") os.environ.setdefault("PYTORCH_ENABLE_MPS_FALLBACK", "1")
atexit.register(prevent_sleep_end)
def _cleanup_sleep(signum, _frame):
prevent_sleep_end()
sys.exit(0)
signal.signal(signal.SIGINT, _cleanup_sleep)
signal.signal(signal.SIGTERM, _cleanup_sleep)
def main() -> None: def main() -> None:
"""Launch the Flask-based web UI.""" """Launch the Flask-based web UI."""
+5 -4
View File
@@ -21,7 +21,8 @@ from PyQt6.QtWidgets import (
) )
from PyQt6.QtCore import QThread, pyqtSignal from PyQt6.QtCore import QThread, pyqtSignal
from abogen.constants import COLORS, VOICES_INTERNAL from abogen.constants import COLORS
from abogen.tts_plugin.utils import get_voices
from abogen.spacy_utils import SPACY_MODELS from abogen.spacy_utils import SPACY_MODELS
import abogen.hf_tracker import abogen.hf_tracker
@@ -114,7 +115,7 @@ class PreDownloadWorker(QThread):
self._voices_success = False self._voices_success = False
return return
voice_list = VOICES_INTERNAL voice_list = get_voices("kokoro")
for idx, voice in enumerate(voice_list, start=1): for idx, voice in enumerate(voice_list, start=1):
if self._cancelled: if self._cancelled:
self._voices_success = False self._voices_success = False
@@ -462,14 +463,14 @@ class PreDownloadDialog(QDialog):
try: try:
from huggingface_hub import try_to_load_from_cache from huggingface_hub import try_to_load_from_cache
for voice in VOICES_INTERNAL: for voice in get_voices("kokoro"):
if not try_to_load_from_cache( if not try_to_load_from_cache(
repo_id="hexgrad/Kokoro-82M", filename=f"voices/{voice}.pt" repo_id="hexgrad/Kokoro-82M", filename=f"voices/{voice}.pt"
): ):
missing.append(voice) missing.append(voice)
except Exception: except Exception:
# If HF missing, report all as missing # If HF missing, report all as missing
return False, list(VOICES_INTERNAL) return False, list(get_voices("kokoro"))
return (len(missing) == 0), missing return (len(missing) == 0), missing
def _check_kokoro_model(self) -> bool: def _check_kokoro_model(self) -> bool:
+20 -207
View File
@@ -29,6 +29,12 @@ from abogen.utils import (
get_resource_path, get_resource_path,
) )
from abogen.book_parser import get_book_parser from abogen.book_parser import get_book_parser
from abogen.domain.metadata_extraction import (
extract_book_metadata_epub,
extract_book_metadata_pdf,
extract_book_metadata_markdown,
format_metadata_tags,
)
from abogen.subtitle_utils import ( from abogen.subtitle_utils import (
clean_text, clean_text,
@@ -948,169 +954,14 @@ class HandlerDialog(QDialog):
self.previewEdit.setHtml(html_content) self.previewEdit.setHtml(html_content)
def _extract_book_metadata(self): def _extract_book_metadata(self):
metadata = {
"title": None,
"authors": [],
"description": None,
"cover_image": None,
"publisher": None,
"publication_year": None,
}
if self.parser.file_type == "epub": if self.parser.file_type == "epub":
try: return extract_book_metadata_epub(self.book)
title_items = self.book.get_metadata("DC", "title")
if title_items and len(title_items) > 0:
metadata["title"] = title_items[0][0]
except Exception as e:
logging.warning(f"Error extracting title metadata: {e}")
try:
author_items = self.book.get_metadata("DC", "creator")
if author_items:
metadata["authors"] = [
author[0] for author in author_items if len(author) > 0
]
except Exception as e:
logging.warning(f"Error extracting author metadata: {e}")
try:
desc_items = self.book.get_metadata("DC", "description")
if desc_items and len(desc_items) > 0:
metadata["description"] = desc_items[0][0]
except Exception as e:
logging.warning(f"Error extracting description metadata: {e}")
try:
publisher_items = self.book.get_metadata("DC", "publisher")
if publisher_items and len(publisher_items) > 0:
metadata["publisher"] = publisher_items[0][0]
except Exception as e:
logging.warning(f"Error extracting publisher metadata: {e}")
# Try to extract publication year
try:
date_items = self.book.get_metadata("DC", "date")
if date_items and len(date_items) > 0:
date_str = date_items[0][0]
# Try to extract just the year from the date string
year_match = re.search(r"\b(19|20)\d{2}\b", date_str)
if year_match:
metadata["publication_year"] = year_match.group(0)
else:
metadata["publication_year"] = date_str
except Exception as e:
logging.warning(f"Error extracting publication date metadata: {e}")
for item in self.book.get_items_of_type(ebooklib.ITEM_COVER):
metadata["cover_image"] = item.get_content()
break
if not metadata["cover_image"]:
for item in self.book.get_items_of_type(ebooklib.ITEM_IMAGE):
if "cover" in item.get_name().lower():
metadata["cover_image"] = item.get_content()
break
elif self.parser.file_type == "markdown": elif self.parser.file_type == "markdown":
# Extract metadata from markdown frontmatter or first heading return extract_book_metadata_markdown(
if self.markdown_text: self.markdown_text, self.markdown_toc
# Try to extract YAML frontmatter )
frontmatter_match = re.match(
r"^---\s*\n(.*?)\n---\s*\n", self.markdown_text, re.DOTALL
)
if frontmatter_match:
try:
frontmatter = frontmatter_match.group(1)
# Simple YAML-like parsing for common fields
title_match = re.search(
r"^title:\s*(.+)$",
frontmatter,
re.MULTILINE | re.IGNORECASE,
)
if title_match:
metadata["title"] = (
title_match.group(1).strip().strip("\"'")
)
author_match = re.search(
r"^author:\s*(.+)$",
frontmatter,
re.MULTILINE | re.IGNORECASE,
)
if author_match:
metadata["authors"] = [
author_match.group(1).strip().strip("\"'")
]
desc_match = re.search(
r"^description:\s*(.+)$",
frontmatter,
re.MULTILINE | re.IGNORECASE,
)
if desc_match:
metadata["description"] = (
desc_match.group(1).strip().strip("\"'")
)
date_match = re.search(
r"^date:\s*(.+)$", frontmatter, re.MULTILINE | re.IGNORECASE
)
if date_match:
date_str = date_match.group(1).strip().strip("\"'")
year_match = re.search(r"\b(19|20)\d{2}\b", date_str)
if year_match:
metadata["publication_year"] = year_match.group(0)
except Exception as e:
logging.warning(f"Error parsing markdown frontmatter: {e}")
# Fallback: use first H1 header as title if no frontmatter title
if not metadata["title"] and self.markdown_toc:
# Find the first level 1 header
first_h1 = next(
(h for h in self.markdown_toc if h["level"] == 1), None
)
if first_h1:
metadata["title"] = first_h1["name"]
else: else:
pdf_info = self.pdf_doc.metadata return extract_book_metadata_pdf(self.pdf_doc)
if pdf_info:
metadata["title"] = pdf_info.get("title", None)
author = pdf_info.get("author", None)
if author:
metadata["authors"] = [author]
metadata["description"] = pdf_info.get("subject", None)
keywords = pdf_info.get("keywords", None)
if keywords:
if metadata["description"]:
metadata["description"] += f"\n\nKeywords: {keywords}"
else:
metadata["description"] = f"Keywords: {keywords}"
metadata["publisher"] = pdf_info.get("creator", None)
# Try to extract publication date from PDF metadata
if "creationDate" in pdf_info:
date_str = pdf_info["creationDate"]
year_match = re.search(r"D:(\d{4})", date_str)
if year_match:
metadata["publication_year"] = year_match.group(1)
elif "modDate" in pdf_info:
date_str = pdf_info["modDate"]
year_match = re.search(r"D:(\d{4})", date_str)
if year_match:
metadata["publication_year"] = year_match.group(1)
if len(self.pdf_doc) > 0:
try:
pix = self.pdf_doc[0].get_pixmap(matrix=fitz.Matrix(2, 2))
metadata["cover_image"] = pix.tobytes("png")
except Exception:
pass
return metadata
def get_selected_text(self): def get_selected_text(self):
# If a background loader thread is running, wait for it to finish to # If a background loader thread is running, wait for it to finish to
@@ -1136,59 +987,21 @@ class HandlerDialog(QDialog):
def _format_metadata_tags(self): def _format_metadata_tags(self):
"""Format metadata tags for insertion at the beginning of the text""" """Format metadata tags for insertion at the beginning of the text"""
import datetime
from abogen.utils import get_user_cache_path from abogen.utils import get_user_cache_path
metadata = self.book_metadata
filename = os.path.splitext(os.path.basename(self.book_path))[0] filename = os.path.splitext(os.path.basename(self.book_path))[0]
current_year = str(datetime.datetime.now().year) chapter_count = len(self.checked_chapters)
cache_dir = get_user_cache_path()
# Get values with fallbacks return format_metadata_tags(
title = metadata.get("title") or filename self.book_metadata,
authors = metadata.get("authors") or ["Unknown"] filename,
authors_text = ", ".join(authors) chapter_count,
album_artist = authors_text or "Unknown" self.parser.file_type,
year = ( cover_bytes=self.book_metadata.get("cover_image"),
metadata.get("publication_year") or current_year cache_dir=cache_dir,
) # Use publication year if available
# Count chapters/pages
total_chapters = len(self.checked_chapters)
chapter_text = (
f"{total_chapters} {'Chapters' if self.parser.file_type == 'epub' else 'Pages'}"
) )
# Handle cover image
cover_tag = ""
if metadata.get("cover_image"):
try:
import uuid
cache_dir = get_user_cache_path()
cover_path = os.path.join(cache_dir, f"cover_{uuid.uuid4()}.jpg")
cover_path = os.path.normpath(cover_path)
with open(cover_path, "wb") as f:
f.write(metadata["cover_image"])
cover_tag = f"<<METADATA_COVER_PATH:{cover_path}>>"
except Exception as e:
logging.warning(f"Failed to save cover image: {e}")
# Format metadata tags
metadata_tags = [
f"<<METADATA_TITLE:{title}>>",
f"<<METADATA_ARTIST:{authors_text}>>",
f"<<METADATA_ALBUM:{title} ({chapter_text})>>",
f"<<METADATA_YEAR:{year}>>",
f"<<METADATA_ALBUM_ARTIST:{album_artist}>>",
f"<<METADATA_COMPOSER:Narrator>>",
f"<<METADATA_GENRE:Audiobook>>",
]
if cover_tag:
metadata_tags.append(cover_tag)
return "\n".join(metadata_tags)
def _get_markdown_selected_text(self): def _get_markdown_selected_text(self):
"""Get selected text from markdown chapters""" """Get selected text from markdown chapters"""
all_checked_identifiers = set() all_checked_identifiers = set()
+459 -1448
View File
File diff suppressed because it is too large Load Diff
+192
View File
@@ -0,0 +1,192 @@
"""PyQt adapter: ConversionThread -> ConversionRequest.
Converts a PyQt ConversionThread into a ConversionRequest that the application layer can process.
This adapter is the bridge between the PyQt layer and the application/domain layer.
The adapter is responsible for:
- Mapping ConversionThread fields to ConversionRequest fields
- Handling UI-specific state (signals, dialogs, cancellation)
- Providing PipelineProvider and VoiceResolver implementations
Subtitle file/timestamp special paths remain in ConversionThread.run() early return.
"""
from __future__ import annotations
import os
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional
from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_ports import ConversionCancelled, ResolvedVoice
def build_conversion_request_from_thread(thread: Any) -> ConversionRequest:
"""Convert a PyQt ConversionThread into a ConversionRequest.
This is the primary function that maps thread fields to ConversionRequest.
All fields are copied the request is independent of the thread.
Args:
thread: PyQt ConversionThread instance
Returns:
ConversionRequest with all thread data mapped
"""
# Determine source path
source_path = None
is_direct_text = getattr(thread, "is_direct_text", False)
if not is_direct_text and thread.file_name:
source_path = Path(thread.file_name)
# Determine original filename
original_filename = ""
if getattr(thread, "from_queue", False):
base_path = getattr(thread, "save_base_path", None) or thread.file_name
else:
base_path = getattr(thread, "display_path", None) or thread.file_name
if base_path:
original_filename = os.path.basename(base_path)
# Determine output folder
output_folder = None
if thread.output_folder:
output_folder = Path(thread.output_folder)
return ConversionRequest(
# Source
source_path=source_path,
direct_text=thread.file_name if is_direct_text else None,
original_filename=original_filename,
# TTS Settings
language=thread.lang_code,
tts_provider="kokoro", # PyQt uses Kokoro by default
voice=thread.voice,
voice_profile=getattr(thread, "voice_profile", None),
speed=thread.speed,
use_gpu=thread.use_gpu,
supertonic_total_steps=getattr(thread, "supertonic_total_steps", 5),
# Output Format
output_format=thread.output_format,
subtitle_mode=thread.subtitle_mode,
subtitle_format=getattr(thread, "subtitle_format", "srt"),
max_subtitle_words=getattr(thread, "max_subtitle_words", 50),
# Save Options
save_mode=thread.save_option,
output_folder=output_folder,
save_chapters_separately=getattr(thread, "save_chapters_separately", False),
merge_chapters_at_end=getattr(thread, "merge_chapters_at_end", True),
separate_chapters_format=getattr(thread, "separate_chapters_format", "wav"),
save_as_project=getattr(thread, "save_as_project", False),
# Timing
silence_between_chapters=getattr(thread, "silence_duration", 2.0),
chapter_intro_delay=getattr(thread, "chapter_intro_delay", 0.0),
# Content Processing
replace_single_newlines=getattr(thread, "replace_single_newlines", False),
read_title_intro=getattr(thread, "read_title_intro", False),
read_closing_outro=getattr(thread, "read_closing_outro", True),
auto_prefix_chapter_titles=getattr(thread, "auto_prefix_chapter_titles", True),
normalize_chapter_opening_caps=getattr(thread, "normalize_chapter_opening_caps", False),
# Pronunciation / Normalization
pronunciation_overrides=getattr(thread, "pronunciation_overrides", []) or [],
manual_overrides=getattr(thread, "manual_overrides", []) or [],
heteronym_overrides=getattr(thread, "heteronym_overrides", []) or [],
normalization_overrides=getattr(thread, "normalization_overrides", None),
# Chapter/Chunk Configuration
chapter_overrides=[], # PyQt doesn't use chapter overrides from GUI
chunks=[], # PyQt doesn't use chunks from GUI
chunk_level="paragraph",
speaker_mode="single",
speakers={},
# Metadata
metadata_tags=getattr(thread, "metadata_tags", {}) or {},
# Artifacts
cover_image_path=getattr(thread, "cover_image_path", None),
cover_image_mime=getattr(thread, "cover_image_mime", None),
generate_epub3=getattr(thread, "generate_epub3", False),
)
class PyQtEvents:
"""PyQt implementation of ConversionEvents protocol.
Wraps a ConversionThread to provide logging, progress, and cancellation.
"""
def __init__(self, thread: Any):
self._thread = thread
def log(self, message: str, level: str = "info") -> None:
"""Log a message via signal."""
self._thread.log_updated.emit((message, _level_to_color(level)))
def progress(self, pct: int, etr: str) -> None:
"""Update progress via signal."""
self._thread.progress_updated.emit(pct, etr)
def check_cancelled(self) -> None:
"""Check if conversion was cancelled.
Raises:
ConversionCancelled: If cancellation was requested
"""
if self._thread.cancel_requested:
raise ConversionCancelled("Conversion cancelled by user")
class PyQtPipelineProvider:
"""PyQt implementation of PipelineProvider protocol.
Wraps the existing backend from ConversionThread.
"""
def __init__(self, backend: Any):
self._backend = backend
def get(self, provider: str, language: str, use_gpu: bool) -> Any:
"""Get a TTS backend instance.
For PyQt, this returns the pre-initialized backend.
"""
return self._backend
def dispose_all(self) -> None:
"""Dispose all backend resources."""
pass # PyQt manages backend lifecycle in thread
class PyQtVoiceResolver:
"""PyQt implementation of VoiceResolver protocol.
Wraps load_voice_cached from the ConversionThread.
"""
def __init__(self, thread: Any):
self._thread = thread
def resolve(self, voice_spec: str) -> ResolvedVoice:
"""Resolve a voice spec into a loaded voice."""
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
# Use thread's load_voice_cached method
loaded_voice = self._thread.load_voice_cached(voice_spec, self._thread.backend)
return ResolvedVoice(
provider="kokoro",
resolved_spec=voice_spec,
voice=loaded_voice,
speed=self._thread.speed,
supertonic_steps=getattr(self._thread, "supertonic_total_steps", 5),
)
def _level_to_color(level: str) -> str:
"""Map log level to PyQt color string."""
colors = {
"info": "grey",
"warning": "orange",
"error": "red",
"debug": "grey",
}
return colors.get(level, "grey")
+83 -70
View File
@@ -7,6 +7,7 @@ import base64
import re import re
from abogen.pyqt.queue_manager_gui import QueueManager from abogen.pyqt.queue_manager_gui import QueueManager
from abogen.pyqt.queued_item import QueuedItem from abogen.pyqt.queued_item import QueuedItem
import abogen.hf_tracker as hf_tracker import abogen.hf_tracker as hf_tracker
import hashlib # Added for cache path generation import hashlib # Added for cache path generation
from PyQt6.QtWidgets import ( from PyQt6.QtWidgets import (
@@ -82,14 +83,18 @@ from abogen.constants import (
GITHUB_URL, GITHUB_URL,
PROGRAM_DESCRIPTION, PROGRAM_DESCRIPTION,
LANGUAGE_DESCRIPTIONS, LANGUAGE_DESCRIPTIONS,
VOICES_INTERNAL,
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION, SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
COLORS, COLORS,
SUBTITLE_FORMATS, SUBTITLE_FORMATS,
) )
from abogen.tts_plugin.utils import get_voices
import threading import threading
from abogen.pyqt.voice_formula_gui import VoiceFormulaDialog from abogen.pyqt.voice_formula_gui import VoiceFormulaDialog
from abogen.voice_profiles import load_profiles from abogen.voice_profiles import load_profiles
from abogen.domain.settings_core import all_settings_defaults
# Module-level default cache for use outside __init__
_DEFAULTS = all_settings_defaults()
# Import ctypes for Windows-specific taskbar icon # Import ctypes for Windows-specific taskbar icon
if platform.system() == "Windows": if platform.system() == "Windows":
@@ -837,7 +842,7 @@ class WordSubstitutionsDialog(QDialog):
self, self,
) )
instructions.setStyleSheet( instructions.setStyleSheet(
"padding: 10px; background-color: #f0f0f0; border-radius: 5px;" f"padding: 10px; background-color: {COLORS['GREY_BACKGROUND']}; border-radius: 5px;"
) )
instructions.setWordWrap(True) instructions.setWordWrap(True)
layout.addWidget(instructions) layout.addWidget(instructions)
@@ -911,9 +916,10 @@ class abogen(QWidget):
def __init__(self): def __init__(self):
super().__init__() super().__init__()
self.config = load_config() self.config = load_config()
self.apply_theme(self.config.get("theme", "system")) _d = all_settings_defaults()
self.apply_theme(self.config.get("theme", _d["theme"]))
migrate_subtitle_format(self.config) migrate_subtitle_format(self.config)
self.check_updates = self.config.get("check_updates", True) self.check_updates = self.config.get("check_updates", _d["check_updates"])
self.save_option = self.config.get("save_option", "Save next to input file") self.save_option = self.config.get("save_option", "Save next to input file")
self.selected_output_folder = self.config.get("selected_output_folder", None) self.selected_output_folder = self.config.get("selected_output_folder", None)
self.selected_file = self.selected_file_type = self.selected_book_path = None self.selected_file = self.selected_file_type = self.selected_book_path = None
@@ -921,7 +927,7 @@ class abogen(QWidget):
None # Add new variable to track the displayed file path None # Add new variable to track the displayed file path
) )
# Max log lines # Max log lines
self.log_window_max_lines = self.config.get("log_window_max_lines", 2000) self.log_window_max_lines = self.config.get("log_window_max_lines", _d["log_window_max_lines"])
self.selected_chapters = set() self.selected_chapters = set()
self.last_opened_book_path = None # Track the last opened book path self.last_opened_book_path = None # Track the last opened book path
self.last_output_path = None self.last_output_path = None
@@ -936,40 +942,28 @@ class abogen(QWidget):
self.selected_voice = None self.selected_voice = None
self.selected_lang = None self.selected_lang = None
else: else:
self.selected_voice = self.config.get("selected_voice", "af_heart") self.selected_voice = self.config.get("selected_voice", _d["selected_voice"])
self.selected_lang = self.selected_voice[0] if self.selected_voice else None self.selected_lang = self.selected_voice[0] if self.selected_voice else None
self.is_converting = False self.is_converting = False
self.subtitle_mode = self.config.get("subtitle_mode", "Sentence") self.subtitle_mode = self.config.get("subtitle_mode", _d["subtitle_mode"])
self.max_subtitle_words = self.config.get( self.max_subtitle_words = self.config.get("max_subtitle_words", _d["max_subtitle_words"])
"max_subtitle_words", 50 self.silence_duration = self.config.get("silence_duration", _d.get("silence_between_chapters", 2.0))
) # Default max words per subtitle self.selected_format = self.config.get("selected_format", _d["selected_format"])
self.silence_duration = self.config.get( self.separate_chapters_format = self.config.get("separate_chapters_format", _d["separate_chapters_format"])
"silence_duration", 2.0 self.use_gpu = self.config.get("use_gpu", _d["use_gpu"])
) # Default silence duration self.replace_single_newlines = self.config.get("replace_single_newlines", _d.get("replace_single_newlines", True))
self.selected_format = self.config.get("selected_format", "wav") self.use_silent_gaps = self.config.get("use_silent_gaps", _d["use_silent_gaps"])
self.separate_chapters_format = self.config.get( self.subtitle_speed_method = self.config.get("subtitle_speed_method", _d["subtitle_speed_method"])
"separate_chapters_format", "wav" self.use_spacy_segmentation = self.config.get("use_spacy_segmentation", _d["use_spacy_segmentation"])
) # Format for individual chapter files self.read_title_intro = self.config.get("read_title_intro", _d.get("read_title_intro", False))
self.use_gpu = self.config.get( self.read_closing_outro = self.config.get("read_closing_outro", _d.get("read_closing_outro", True))
"use_gpu", True # Load GPU setting with default True
)
self.replace_single_newlines = self.config.get("replace_single_newlines", True)
self.use_silent_gaps = self.config.get("use_silent_gaps", True)
self.subtitle_speed_method = self.config.get("subtitle_speed_method", "tts")
self.use_spacy_segmentation = self.config.get("use_spacy_segmentation", True)
# Word substitution settings # Word substitution settings
self.word_substitutions_enabled = self.config.get( self.word_substitutions_enabled = self.config.get("word_substitutions_enabled", _d["word_substitutions_enabled"])
"word_substitutions_enabled", False self.word_substitutions_list = self.config.get("word_substitutions_list", _d["word_substitutions_list"])
) self.case_sensitive_substitutions = self.config.get("case_sensitive_substitutions", _d["case_sensitive_substitutions"])
self.word_substitutions_list = self.config.get("word_substitutions_list", "") self.replace_all_caps = self.config.get("replace_all_caps", _d["replace_all_caps"])
self.case_sensitive_substitutions = self.config.get( self.replace_numerals = self.config.get("replace_numerals", _d["replace_numerals"])
"case_sensitive_substitutions", False self.fix_nonstandard_punctuation = self.config.get("fix_nonstandard_punctuation", _d["fix_nonstandard_punctuation"])
)
self.replace_all_caps = self.config.get("replace_all_caps", False)
self.replace_numerals = self.config.get("replace_numerals", False)
self.fix_nonstandard_punctuation = self.config.get(
"fix_nonstandard_punctuation", False
)
self._pending_close_event = None self._pending_close_event = None
self.gpu_ok = False # Initialize GPU availability status self.gpu_ok = False # Initialize GPU availability status
@@ -997,7 +991,7 @@ class abogen(QWidget):
self.current_queue_index = 0 self.current_queue_index = 0
self.initUI() self.initUI()
self.speed_slider.setValue(int(self.config.get("speed", 1.00) * 100)) self.speed_slider.setValue(int(self.config.get("speed", _d["speed"]) * 100))
self.update_speed_label() self.update_speed_label()
# Set initial selection: prefer profile, else voice # Set initial selection: prefer profile, else voice
idx = -1 idx = -1
@@ -1873,7 +1867,7 @@ class abogen(QWidget):
for pname in load_profiles().keys(): for pname in load_profiles().keys():
self.voice_combo.addItem(profile_icon, pname, f"profile:{pname}") self.voice_combo.addItem(profile_icon, pname, f"profile:{pname}")
# re-add voices # re-add voices
for v in VOICES_INTERNAL: for v in get_voices("kokoro"):
icon = QIcon() icon = QIcon()
flag_path = get_resource_path("abogen.assets.flags", f"{v[0]}.png") flag_path = get_resource_path("abogen.assets.flags", f"{v[0]}.png")
if flag_path and os.path.exists(flag_path): if flag_path and os.path.exists(flag_path):
@@ -2160,7 +2154,7 @@ class abogen(QWidget):
) )
# CHECK GLOBAL OVERRIDE SETTING # CHECK GLOBAL OVERRIDE SETTING
if not self.config.get("queue_override_settings", False): if not self.config.get("queue_override_settings", _DEFAULTS["queue_override_settings"]):
self.selected_lang = queued_item.lang_code self.selected_lang = queued_item.lang_code
self.speed_slider.setValue(int(queued_item.speed * 100)) self.speed_slider.setValue(int(queued_item.speed * 100))
@@ -2234,11 +2228,10 @@ class abogen(QWidget):
self.current_queue_index = 0 # Reset for next time self.current_queue_index = 0 # Reset for next time
def get_voice_formula(self) -> str: def get_voice_formula(self) -> str:
from abogen.voice_formulas import pairs_to_formula
if self.mixed_voice_state: if self.mixed_voice_state:
formula_components = [ return pairs_to_formula(self.mixed_voice_state) or ""
f"{name}*{weight}" for name, weight in self.mixed_voice_state
]
return " + ".join(filter(None, formula_components))
else: else:
return self.selected_voice return self.selected_voice
@@ -2402,6 +2395,9 @@ class abogen(QWidget):
self.conversion_thread.merge_chapters_at_end = getattr( self.conversion_thread.merge_chapters_at_end = getattr(
self, "merge_chapters_at_end", True self, "merge_chapters_at_end", True
) )
# Pass intro/outro settings
self.conversion_thread.read_title_intro = self.read_title_intro
self.conversion_thread.read_closing_outro = self.read_closing_outro
self.conversion_thread.progress_updated.connect(self.update_progress) self.conversion_thread.progress_updated.connect(self.update_progress)
self.conversion_thread.log_updated.connect(self.update_log) self.conversion_thread.log_updated.connect(self.update_log)
self.conversion_thread.conversion_finished.connect( self.conversion_thread.conversion_finished.connect(
@@ -2426,18 +2422,9 @@ class abogen(QWidget):
self.update_log((gpu_msg, gpu_ok)) self.update_log((gpu_msg, gpu_ok))
self.update_log("Loading modules...") self.update_log("Loading modules...")
# Determine device based on GPU availability
if gpu_ok:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps"
else:
device = "cuda"
else:
device = "cpu"
lang_code = self.selected_lang or "a" lang_code = self.selected_lang or "a"
load_thread = LoadPipelineThread( load_thread = LoadPipelineThread(
pipeline_loaded_callback, lang_code=lang_code, device=device pipeline_loaded_callback, lang_code=lang_code, use_gpu=gpu_ok
) )
load_thread.start() load_thread.start()
@@ -2449,7 +2436,7 @@ class abogen(QWidget):
return return
# Check if override was active (this determines which settings were ACTUALLY used) # Check if override was active (this determines which settings were ACTUALLY used)
override_active = self.config.get("queue_override_settings", False) override_active = self.config.get("queue_override_settings", _DEFAULTS["queue_override_settings"])
# If override is ON, capture the global settings that were used for processing # If override is ON, capture the global settings that were used for processing
if override_active: if override_active:
@@ -2875,18 +2862,9 @@ class abogen(QWidget):
) )
self.loading_movie.start() self.loading_movie.start()
# Determine device based on GPU availability
if self.gpu_ok:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps"
else:
device = "cuda"
else:
device = "cpu"
lang = self.selected_lang or "a" lang = self.selected_lang or "a"
load_thread = LoadPipelineThread( load_thread = LoadPipelineThread(
self._on_pipeline_loaded_for_preview, lang_code=lang, device=device self._on_pipeline_loaded_for_preview, lang_code=lang, use_gpu=self.gpu_ok
) )
load_thread.start() load_thread.start()
@@ -3236,12 +3214,16 @@ class abogen(QWidget):
) )
box.setDefaultButton(QMessageBox.StandardButton.No) box.setDefaultButton(QMessageBox.StandardButton.No)
if box.exec() == QMessageBox.StandardButton.Yes: if box.exec() == QMessageBox.StandardButton.Yes:
from abogen import shutdown
shutdown.request_shutdown()
self.cleanup_conversion_thread() self.cleanup_conversion_thread()
self.cleanup_preview_threads() self.cleanup_preview_threads()
event.accept() event.accept()
else: else:
event.ignore() event.ignore()
else: else:
from abogen import shutdown
shutdown.request_shutdown()
self.cleanup_conversion_thread() self.cleanup_conversion_thread()
self.cleanup_preview_threads() self.cleanup_preview_threads()
event.accept() event.accept()
@@ -3430,7 +3412,7 @@ class abogen(QWidget):
app.installEventFilter(app._dark_titlebar_event_filter) app.installEventFilter(app._dark_titlebar_event_filter)
# Save config if changed # Save config if changed
if self.config.get("theme", "system") != theme: if self.config.get("theme", _DEFAULTS["theme"]) != theme:
self.config["theme"] = theme self.config["theme"] = theme
save_config(self.config) save_config(self.config)
@@ -3452,7 +3434,7 @@ class abogen(QWidget):
] ]
# Get current theme from config, default to "system" # Get current theme from config, default to "system"
current_theme = self.config.get("theme", "system") current_theme = self.config.get("theme", _DEFAULTS["theme"])
for value, text in theme_options: for value, text in theme_options:
theme_action = QAction(text, self) theme_action = QAction(text, self)
theme_action.setCheckable(True) theme_action.setCheckable(True)
@@ -3583,6 +3565,27 @@ class abogen(QWidget):
# Add separator # Add separator
menu.addSeparator() menu.addSeparator()
# Add title intro option
self.title_intro_action = QAction("Read title intro before first chapter", self)
self.title_intro_action.setCheckable(True)
self.title_intro_action.setChecked(self.read_title_intro)
self.title_intro_action.triggered.connect(
lambda checked: self.toggle_read_title_intro(checked)
)
menu.addAction(self.title_intro_action)
# Add closing outro option
self.closing_outro_action = QAction("Read closing outro after last chapter", self)
self.closing_outro_action.setCheckable(True)
self.closing_outro_action.setChecked(self.read_closing_outro)
self.closing_outro_action.triggered.connect(
lambda checked: self.toggle_read_closing_outro(checked)
)
menu.addAction(self.closing_outro_action)
# Add separator
menu.addSeparator()
# Add "Pre-download models and voices for offline use" option # Add "Pre-download models and voices for offline use" option
predownload_action = QAction( predownload_action = QAction(
"Pre-download models and voices for offline use", self "Pre-download models and voices for offline use", self
@@ -3594,7 +3597,7 @@ class abogen(QWidget):
disable_kokoro_action = QAction("Disable Kokoro's internet access", self) disable_kokoro_action = QAction("Disable Kokoro's internet access", self)
disable_kokoro_action.setCheckable(True) disable_kokoro_action.setCheckable(True)
disable_kokoro_action.setChecked( disable_kokoro_action.setChecked(
self.config.get("disable_kokoro_internet", False) self.config.get("disable_kokoro_internet", _DEFAULTS["disable_kokoro_internet"])
) )
disable_kokoro_action.triggered.connect( disable_kokoro_action.triggered.connect(
lambda checked: self.toggle_kokoro_internet_access(checked) lambda checked: self.toggle_kokoro_internet_access(checked)
@@ -3604,7 +3607,7 @@ class abogen(QWidget):
# Add check for updates option # Add check for updates option
check_updates_action = QAction("Check for updates at startup", self) check_updates_action = QAction("Check for updates at startup", self)
check_updates_action.setCheckable(True) check_updates_action.setCheckable(True)
check_updates_action.setChecked(self.config.get("check_updates", True)) check_updates_action.setChecked(self.config.get("check_updates", _DEFAULTS["check_updates"]))
check_updates_action.triggered.connect(self.toggle_check_updates) check_updates_action.triggered.connect(self.toggle_check_updates)
menu.addAction(check_updates_action) menu.addAction(check_updates_action)
@@ -3659,6 +3662,16 @@ class abogen(QWidget):
self.config["use_spacy_segmentation"] = enabled self.config["use_spacy_segmentation"] = enabled
save_config(self.config) save_config(self.config)
def toggle_read_title_intro(self, enabled):
self.read_title_intro = enabled
self.config["read_title_intro"] = enabled
save_config(self.config)
def toggle_read_closing_outro(self, enabled):
self.read_closing_outro = enabled
self.config["read_closing_outro"] = enabled
save_config(self.config)
def restart_app(self): def restart_app(self):
import sys import sys
@@ -4230,7 +4243,7 @@ Categories=AudioVideo;Audio;Utility;
"""Open a dialog to set the maximum words per subtitle""" """Open a dialog to set the maximum words per subtitle"""
from PyQt6.QtWidgets import QInputDialog from PyQt6.QtWidgets import QInputDialog
current_value = self.config.get("max_subtitle_words", 50) current_value = self.config.get("max_subtitle_words", _DEFAULTS["max_subtitle_words"])
value, ok = QInputDialog.getInt( value, ok = QInputDialog.getInt(
self, self,
@@ -4258,7 +4271,7 @@ Categories=AudioVideo;Audio;Utility;
def set_silence_between_chapters(self): def set_silence_between_chapters(self):
"""Open a dialog to set the silence duration between chapters""" """Open a dialog to set the silence duration between chapters"""
current_value = self.config.get("silence_duration", 2.0) current_value = self.config.get("silence_duration", _DEFAULTS.get("silence_between_chapters", 2.0))
dlg = QInputDialog(self) dlg = QInputDialog(self)
dlg.setWindowTitle("Silence Duration (seconds)") dlg.setWindowTitle("Silence Duration (seconds)")
+9 -23
View File
@@ -1,10 +1,10 @@
import os import os
import sys import sys
import platform import platform
import atexit
import signal
from abogen.utils import get_resource_path, load_config, prevent_sleep_end
# Initialise global shutdown handling (atexit, signals, Qt) as early as possible.
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
# Fix PyTorch DLL loading issue ([WinError 1114]) on Windows before importing PyQt6 # Fix PyTorch DLL loading issue ([WinError 1114]) on Windows before importing PyQt6
if platform.system() == "Windows": if platform.system() == "Windows":
@@ -46,6 +46,8 @@ except ImportError:
print("PyQt6 not installed.") print("PyQt6 not installed.")
from abogen.utils import get_resource_path
# Pre-load "libxcb-cursor" on Linux (fixes #101) # Pre-load "libxcb-cursor" on Linux (fixes #101)
if platform.system() == "Linux": if platform.system() == "Linux":
arch = platform.machine().lower() arch = platform.machine().lower()
@@ -94,6 +96,7 @@ os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1" # Disable Hugging Face telemetry
os.environ["HF_HUB_ETAG_TIMEOUT"] = "10" # Metadata request timeout (seconds) os.environ["HF_HUB_ETAG_TIMEOUT"] = "10" # Metadata request timeout (seconds)
os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = "10" # File download timeout (seconds) os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = "10" # File download timeout (seconds)
os.environ["HF_HUB_DISABLE_SYMLINKS_WARNING"] = "1" # Disable symlinks warning os.environ["HF_HUB_DISABLE_SYMLINKS_WARNING"] = "1" # Disable symlinks warning
from abogen.utils import load_config
if load_config().get("disable_kokoro_internet", False): if load_config().get("disable_kokoro_internet", False):
print("INFO: Kokoro's internet access is disabled.") print("INFO: Kokoro's internet access is disabled.")
os.environ["HF_HUB_OFFLINE"] = "1" # Disable Hugging Face Hub internet access os.environ["HF_HUB_OFFLINE"] = "1" # Disable Hugging Face Hub internet access
@@ -105,25 +108,6 @@ from abogen.constants import PROGRAM_NAME, VERSION
os.environ["MIOPEN_FIND_MODE"] = "FAST" os.environ["MIOPEN_FIND_MODE"] = "FAST"
os.environ["MIOPEN_CONV_PRECISE_ROCM_TUNING"] = "0" os.environ["MIOPEN_CONV_PRECISE_ROCM_TUNING"] = "0"
# Reset sleep states
atexit.register(prevent_sleep_end)
# Also handle signals (Ctrl+C, kill, etc.)
def _cleanup_sleep(signum, frame):
prevent_sleep_end()
sys.exit(0)
signal.signal(signal.SIGINT, _cleanup_sleep)
signal.signal(signal.SIGTERM, _cleanup_sleep)
# Ensure sys.stdout and sys.stderr are valid in GUI mode
if sys.stdout is None:
sys.stdout = open(os.devnull, "w")
if sys.stderr is None:
sys.stderr = open(os.devnull, "w")
# Enable MPS GPU acceleration on Mac Apple Silicon # Enable MPS GPU acceleration on Mac Apple Silicon
if platform.system() == "Darwin" and platform.processor() == "arm": if platform.system() == "Darwin" and platform.processor() == "arm":
os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1" os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"
@@ -136,6 +120,8 @@ def qt_message_handler(mode, context, message):
return # Suppress this specific message return # Suppress this specific message
if "setGrabPopup called with a parent, QtWaylandClient" in message: if "setGrabPopup called with a parent, QtWaylandClient" in message:
return return
if "Failed to register with host portal" in message:
return
if mode == QtMsgType.QtWarningMsg: if mode == QtMsgType.QtWarningMsg:
print(f"Qt Warning: {message}") print(f"Qt Warning: {message}")
@@ -184,4 +170,4 @@ def main():
if __name__ == "__main__": if __name__ == "__main__":
main() main()
+5 -4
View File
@@ -21,7 +21,8 @@ from PyQt6.QtWidgets import (
) )
from PyQt6.QtCore import QThread, pyqtSignal from PyQt6.QtCore import QThread, pyqtSignal
from abogen.constants import COLORS, VOICES_INTERNAL from abogen.constants import COLORS
from abogen.tts_plugin.utils import get_voices
from abogen.spacy_utils import SPACY_MODELS from abogen.spacy_utils import SPACY_MODELS
import abogen.hf_tracker import abogen.hf_tracker
@@ -114,7 +115,7 @@ class PreDownloadWorker(QThread):
self._voices_success = False self._voices_success = False
return return
voice_list = VOICES_INTERNAL voice_list = get_voices("kokoro")
for idx, voice in enumerate(voice_list, start=1): for idx, voice in enumerate(voice_list, start=1):
if self._cancelled: if self._cancelled:
self._voices_success = False self._voices_success = False
@@ -462,14 +463,14 @@ class PreDownloadDialog(QDialog):
try: try:
from huggingface_hub import try_to_load_from_cache from huggingface_hub import try_to_load_from_cache
for voice in VOICES_INTERNAL: for voice in get_voices("kokoro"):
if not try_to_load_from_cache( if not try_to_load_from_cache(
repo_id="hexgrad/Kokoro-82M", filename=f"voices/{voice}.pt" repo_id="hexgrad/Kokoro-82M", filename=f"voices/{voice}.pt"
): ):
missing.append(voice) missing.append(voice)
except Exception: except Exception:
# If HF missing, report all as missing # If HF missing, report all as missing
return False, list(VOICES_INTERNAL) return False, list(get_voices("kokoro"))
return (len(missing) == 0), missing return (len(missing) == 0), missing
def _check_kokoro_model(self) -> bool: def _check_kokoro_model(self) -> bool:
+3 -3
View File
@@ -28,11 +28,11 @@ from PyQt6.QtWidgets import (
from PyQt6.QtCore import Qt, QTimer, QPoint, QRect, QSize from PyQt6.QtCore import Qt, QTimer, QPoint, QRect, QSize
from PyQt6.QtGui import QPixmap, QIcon, QAction from PyQt6.QtGui import QPixmap, QIcon, QAction
from abogen.constants import ( from abogen.constants import (
VOICES_INTERNAL,
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION, SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
LANGUAGE_DESCRIPTIONS, LANGUAGE_DESCRIPTIONS,
COLORS, COLORS,
) )
from abogen.tts_plugin.utils import get_voices
import re import re
import platform import platform
from abogen.utils import get_resource_path from abogen.utils import get_resource_path
@@ -179,7 +179,7 @@ class VoiceMixer(QWidget):
layout.addWidget(QLabel(name), alignment=Qt.AlignmentFlag.AlignCenter) layout.addWidget(QLabel(name), alignment=Qt.AlignmentFlag.AlignCenter)
# Voice name label with gender icon # Voice name label with gender icon
is_female = self.voice_name in VOICES_INTERNAL and self.voice_name[1] == "f" is_female = self.voice_name in get_voices("kokoro") and self.voice_name[1] == "f"
# Icons layout (flag and gender) # Icons layout (flag and gender)
icons_layout = QHBoxLayout() icons_layout = QHBoxLayout()
@@ -772,7 +772,7 @@ class VoiceFormulaDialog(QDialog):
def add_voices(self, initial_state): def add_voices(self, initial_state):
first_enabled_voice = None first_enabled_voice = None
for voice in VOICES_INTERNAL: for voice in get_voices("kokoro"):
language_code = voice[0] # First character is the language code language_code = voice[0] # First character is the language code
matching_voice = next( matching_voice = next(
(item for item in initial_state if item[0] == voice), None (item for item in initial_state if item[0] == voice), None
+160
View File
@@ -0,0 +1,160 @@
"""Graceful shutdown - single module, no over-engineering."""
from __future__ import annotations
import atexit
import gc
import signal
import sys
from typing import Callable
_CLEANUP_FUNCS: list[Callable[[], None]] = []
_EXECUTED = False
def register_cleanup(fn: Callable[[], None]) -> None:
"""Register a cleanup function to run on shutdown."""
_CLEANUP_FUNCS.append(fn)
def _run_cleanups() -> None:
global _EXECUTED
if _EXECUTED:
return
_EXECUTED = True
for fn in _CLEANUP_FUNCS:
try:
fn()
except Exception:
pass
# ---- Register built-in cleanup functions ----
# 1. Restore sleep prevention
def _restore_sleep() -> None:
try:
from abogen.utils import prevent_sleep_end
prevent_sleep_end()
except Exception:
pass
register_cleanup(_restore_sleep)
# 2. Shutdown web UI ConversionService
def _shutdown_conversion_service() -> None:
try:
from abogen.webui.service import get_service
svc = get_service()
if svc is not None:
svc.shutdown()
except Exception:
pass
register_cleanup(_shutdown_conversion_service)
# 3. Clear TTS pipelines and GPU memory
def _cleanup_tts_pipelines() -> None:
# Clear web UI pipeline cache
try:
from abogen.webui.conversion_runner import _PIPELINES
_PIPELINES.clear()
except Exception:
pass
# Clear PyQt conversion thread voice cache
try:
from abogen.pyqt.conversion import ConversionThread
if hasattr(ConversionThread, "voice_cache"):
ConversionThread.voice_cache.clear()
except Exception:
pass
gc.collect()
# Release CUDA cache
try:
import torch
if torch.cuda.is_available():
torch.cuda.empty_cache()
torch.cuda.ipc_collect()
except Exception:
pass
register_cleanup(_cleanup_tts_pipelines)
# 4. Clear global voice cache
def _clear_voice_cache() -> None:
try:
from abogen.voice_cache import clear_voice_cache
clear_voice_cache()
except Exception:
pass
register_cleanup(_clear_voice_cache)
# 5. Terminate child processes (ffmpeg, etc.)
def _terminate_subprocesses() -> None:
try:
import psutil
except Exception:
return
try:
current = psutil.Process()
for child in current.children(recursive=True):
try:
child.terminate()
except Exception:
pass
gone, alive = psutil.wait_procs(current.children(recursive=True), timeout=3)
for proc in alive:
try:
proc.kill()
except Exception:
pass
except Exception:
pass
register_cleanup(_terminate_subprocesses)
def register_shutdown() -> None:
"""Install process-wide shutdown hooks (atexit, signals, Qt)."""
if register_shutdown._registered:
return
register_shutdown._registered = True
atexit.register(_run_cleanups)
# POSIX signals
for sig in (signal.SIGINT, signal.SIGTERM):
try:
signal.signal(sig, _on_signal)
except Exception:
pass
# Qt hook
try:
from PyQt6.QtWidgets import QApplication
app = QApplication.instance()
if app is not None:
app.aboutToQuit.connect(_run_cleanups)
except Exception:
pass
register_shutdown._registered = False
def _on_signal(signum: int, _frame) -> None:
_run_cleanups()
sys.exit(0)
def request_shutdown() -> None:
"""Programmatically trigger cleanup (e.g., from GUI closeEvent)."""
_run_cleanups()
__all__ = ["register_shutdown", "request_shutdown", "register_cleanup"]
+36 -11
View File
@@ -2,21 +2,36 @@
Lazy-loaded spaCy utilities for sentence segmentation. Lazy-loaded spaCy utilities for sentence segmentation.
""" """
from abogen.domain.enums import Language
# Cached spaCy module and models (lazy loaded) # Cached spaCy module and models (lazy loaded)
_spacy = None _spacy = None
_nlp_cache = {} _nlp_cache = {}
# Language code to spaCy model mapping # Language code to spaCy model mapping
SPACY_MODELS = { SPACY_MODELS = {
"a": "en_core_web_sm", # American English Language.EN_US: "en_core_web_sm",
"b": "en_core_web_sm", # British English Language.EN_GB: "en_core_web_sm",
"e": "es_core_news_sm", # Spanish Language.ES: "es_core_news_sm",
"f": "fr_core_news_sm", # French Language.FR: "fr_core_news_sm",
"i": "it_core_news_sm", # Italian Language.IT: "it_core_news_sm",
"p": "pt_core_news_sm", # Brazilian Portuguese Language.PT_BR: "pt_core_news_sm",
"z": "zh_core_web_sm", # Mandarin Chinese Language.ZH: "zh_core_web_sm",
"j": "ja_core_news_sm", # Japanese Language.JA: "ja_core_news_sm",
"h": "xx_sent_ud_sm", # Hindi (multi-language model) Language.HI: "xx_sent_ud_sm",
}
# Kokoro single-letter codes -> Language enum (inverse of pipeline_factory._KOKORO_LANG_MAP)
_KOKORO_TO_LANGUAGE = {
"a": Language.EN_US,
"b": Language.EN_GB,
"e": Language.ES,
"f": Language.FR,
"h": Language.HI,
"i": Language.IT,
"j": Language.JA,
"p": Language.PT_BR,
"z": Language.ZH,
} }
@@ -36,10 +51,9 @@ def _load_spacy():
def get_spacy_model(lang_code, log_callback=None): def get_spacy_model(lang_code, log_callback=None):
""" """
Get or load a spaCy model for the given language code. Get or load a spaCy model for the given language code.
Downloads the model automatically if not available.
Args: Args:
lang_code: Language code (a, b, e, f, etc.) lang_code: Language code or Language enum (e.g., "a", "en-US", Language.EN_US)
log_callback: Optional function to log messages log_callback: Optional function to log messages
Returns: Returns:
@@ -58,6 +72,17 @@ def get_spacy_model(lang_code, log_callback=None):
else: else:
print(msg) print(msg)
# Normalize to Language enum
if not isinstance(lang_code, Language):
if isinstance(lang_code, str) and lang_code in _KOKORO_TO_LANGUAGE:
lang_code = _KOKORO_TO_LANGUAGE[lang_code]
else:
try:
lang_code = Language.from_str(lang_code)
except ValueError:
log(f"\nspaCy: Unknown language '{lang_code}'...")
return None
# Check if model is cached # Check if model is cached
if lang_code in _nlp_cache: if lang_code in _nlp_cache:
return _nlp_cache[lang_code] return _nlp_cache[lang_code]
+7 -7
View File
@@ -466,7 +466,7 @@ def sanitize_name_for_os(name, is_folder=True):
def validate_voice_name(voice_name): def validate_voice_name(voice_name):
"""Validate voice name against VOICES_INTERNAL list (case-insensitive). """Validate voice name against available voices (case-insensitive).
Handles both single voices and formulas like 'af_heart*0.5 + am_echo*0.5'. Handles both single voices and formulas like 'af_heart*0.5 + am_echo*0.5'.
Args: Args:
@@ -477,10 +477,10 @@ def validate_voice_name(voice_name):
- is_valid: True if all voices in the name/formula are valid - is_valid: True if all voices in the name/formula are valid
- invalid_voice_name: The first invalid voice found, or None if all valid - invalid_voice_name: The first invalid voice found, or None if all valid
""" """
from abogen.constants import VOICES_INTERNAL from abogen.tts_plugin.utils import get_voices
# Create case-insensitive lookup set (done once per call) # Create case-insensitive lookup set (done once per call)
voice_lookup_lower = {v.lower() for v in VOICES_INTERNAL} voice_lookup_lower = {v.lower() for v in get_voices("kokoro")}
voice_name = voice_name.strip() voice_name = voice_name.strip()
# Check if it's a formula (contains *) # Check if it's a formula (contains *)
@@ -505,7 +505,7 @@ def split_text_by_voice_markers(text, default_voice):
"""Split text by voice markers, returning list of (voice, text) tuples. """Split text by voice markers, returning list of (voice, text) tuples.
IMPORTANT: Returns the last voice used so it can persist across chapters. IMPORTANT: Returns the last voice used so it can persist across chapters.
Voice names are normalized to lowercase to match VOICES_INTERNAL. Voice names are normalized to lowercase to match canonical voice names.
Args: Args:
text: Text potentially containing <<VOICE:name>> markers text: Text potentially containing <<VOICE:name>> markers
@@ -518,7 +518,7 @@ def split_text_by_voice_markers(text, default_voice):
- valid_count: Number of valid voice markers processed - valid_count: Number of valid voice markers processed
- invalid_count: Number of invalid voice markers skipped - invalid_count: Number of invalid voice markers skipped
""" """
from abogen.constants import VOICES_INTERNAL from abogen.tts_plugin.utils import get_voices
voice_splits = list(_VOICE_MARKER_SEARCH_PATTERN.finditer(text)) voice_splits = list(_VOICE_MARKER_SEARCH_PATTERN.finditer(text))
@@ -560,7 +560,7 @@ def split_text_by_voice_markers(text, default_voice):
# Find the canonical (lowercase) voice name # Find the canonical (lowercase) voice name
voice_part_lower = voice_part.strip().lower() voice_part_lower = voice_part.strip().lower()
canonical_voice = next( canonical_voice = next(
(v for v in VOICES_INTERNAL if v.lower() == voice_part_lower), (v for v in get_voices("kokoro") if v.lower() == voice_part_lower),
voice_part.strip() voice_part.strip()
) )
normalized_parts.append(f"{canonical_voice}*{weight.strip()}") normalized_parts.append(f"{canonical_voice}*{weight.strip()}")
@@ -569,7 +569,7 @@ def split_text_by_voice_markers(text, default_voice):
# Find the canonical (lowercase) voice name # Find the canonical (lowercase) voice name
voice_name_lower = voice_name.lower() voice_name_lower = voice_name.lower()
current_voice = next( current_voice = next(
(v for v in VOICES_INTERNAL if v.lower() == voice_name_lower), (v for v in get_voices("kokoro") if v.lower() == voice_name_lower),
voice_name voice_name
) )
valid_markers += 1 valid_markers += 1
-89
View File
@@ -1,89 +0,0 @@
"""
TTS Backend Interface
This module defines the protocol for TTS backends and the
metadata model that describes a backend implementation.
"""
from dataclasses import dataclass
from typing import Protocol, List, Dict, Any
@dataclass(frozen=True)
class TTSBackendMetadata:
"""
Immutable metadata describing a TTS backend implementation.
Attributes:
id: Unique backend identifier (e.g. ``"kokoro"``, ``"supertonic"``).
name: Human-readable display name.
description: Short description of the backend.
voices: Tuple of supported voice identifiers.
"""
id: str
name: str
description: str
voices: tuple[str, ...] = ()
class TTSBackend(Protocol):
"""
Protocol for TTS backends.
All TTS backends must implement this interface to be compatible
with the application.
"""
@property
def metadata(self) -> TTSBackendMetadata:
...
def __init__(self, **kwargs) -> None:
"""
Initialize the TTS backend.
Args:
**kwargs: Backend-specific configuration parameters
"""
...
def synthesize(self, text: str, **kwargs) -> bytes:
"""
Synthesize speech from text.
Args:
text: Text to synthesize
**kwargs: Additional parameters for synthesis
Returns:
Audio data as bytes
"""
...
def get_available_voices(self) -> List[str]:
"""
Get list of available voices.
Returns:
List of voice identifiers
"""
...
def get_supported_formats(self) -> List[str]:
"""
Get list of supported audio formats.
Returns:
List of supported audio formats
"""
...
def get_info(self) -> Dict[str, Any]:
"""
Get backend information.
Returns:
Dictionary with backend information
"""
...
-90
View File
@@ -1,90 +0,0 @@
"""
TTS Backend Registry
Provides a global registry for TTS backend factories.
Backends register themselves with metadata and a factory callable.
The registry is universal and does not know about backend constructors.
"""
from typing import Callable, Any
from abogen.tts_backend import TTSBackend, TTSBackendMetadata
class TTSBackendRegistry:
"""Registry of TTS backend factories.
Stores metadata and factory callables for registered backends.
"""
def __init__(self) -> None:
self._backends: dict[str, TTSBackendMetadata] = {}
self._factories: dict[str, Callable[..., TTSBackend]] = {}
def register(
self,
metadata: TTSBackendMetadata,
factory: Callable[..., TTSBackend],
) -> None:
"""Register a backend with its metadata and factory callable."""
self._backends[metadata.id] = metadata
self._factories[metadata.id] = factory
def list_backends(self) -> list[TTSBackendMetadata]:
"""Return metadata for all registered backends."""
return list(self._backends.values())
def get_metadata(self, backend_id: str) -> TTSBackendMetadata:
"""Get metadata for a specific backend.
Raises:
KeyError: If backend with given id is not registered.
"""
if backend_id not in self._backends:
raise KeyError(f"Unknown backend: {backend_id}")
return self._backends[backend_id]
def create_backend(self, backend_id: str, **kwargs: Any) -> TTSBackend:
"""Create a backend instance by id.
Raises:
KeyError: If backend with given id is not registered.
"""
if backend_id not in self._factories:
raise KeyError(f"Unknown backend: {backend_id}")
return self._factories[backend_id](**kwargs)
_registry = TTSBackendRegistry()
def register_backend(
metadata: TTSBackendMetadata,
factory: Callable[..., TTSBackend],
) -> None:
"""Register a TTS backend in the global registry."""
_registry.register(metadata, factory)
def get_metadata(backend_id: str) -> TTSBackendMetadata:
"""Get metadata for a specific backend by id.
Ensures all backends are registered by importing the tts_backends
package on first access.
Raises:
KeyError: If backend with given id is not registered.
"""
import abogen.tts_backends # noqa: F401 — triggers backend registration
return _registry.get_metadata(backend_id)
def get_default_voice(backend_id: str, fallback: str = "") -> str:
"""Return the first voice of a backend, or *fallback* if none."""
voices = get_metadata(backend_id).voices
return voices[0] if voices else fallback
def create_backend(backend_id: str, **kwargs: Any) -> TTSBackend:
"""Create a TTS backend instance by provider id."""
return _registry.create_backend(backend_id, **kwargs)
-20
View File
@@ -1,20 +0,0 @@
"""TTS backends package.
Backend modules are auto-discovered and imported here.
Each backend module registers itself with the global registry
when imported.
"""
import importlib
import pkgutil
def _discover_backends():
"""Import all modules in this package to trigger their registration."""
package = __name__
for _importer, modname, _ispkg in pkgutil.iter_modules(path=__path__):
importlib.import_module(f"{package}.{modname}")
_discover_backends()
-121
View File
@@ -1,121 +0,0 @@
"""
Kokoro TTS Backend
Encapsulates the Kokoro KPipeline as a TTSBackend implementation.
"""
from __future__ import annotations
from typing import Any, Dict, Iterator, List, Optional
import numpy as np
from abogen.constants import VOICES_INTERNAL
from abogen.tts_backend import TTSBackendMetadata
_KOKORO_METADATA = TTSBackendMetadata(
id="kokoro",
name="Kokoro",
description="Kokoro TTS engine",
voices=tuple(VOICES_INTERNAL),
)
def _load_kpipeline():
"""Lazy-load Kokoro dependencies."""
from kokoro import KPipeline # type: ignore[import-not-found]
return KPipeline
class KokoroBackend:
"""TTSBackend implementation wrapping the Kokoro KPipeline.
All interaction with KPipeline is encapsulated here.
The rest of the project depends only on this class.
"""
def __init__(self, **kwargs: Any) -> None:
lang_code = kwargs["lang_code"]
repo_id = kwargs.get("repo_id", "hexgrad/Kokoro-82M")
device = kwargs.get("device", "cpu")
KPipeline = _load_kpipeline()
self._pipeline = KPipeline(
lang_code=lang_code,
repo_id=repo_id,
device=device,
)
self._lang_code = lang_code
@property
def metadata(self) -> TTSBackendMetadata:
return _KOKORO_METADATA
def __call__(
self,
text: str,
*,
voice: Any,
speed: float = 1.0,
split_pattern: Optional[str] = None,
) -> Iterator[Any]:
"""Delegate to KPipeline's __call__."""
return self._pipeline(
text,
voice=voice,
speed=speed,
split_pattern=split_pattern,
)
def load_single_voice(self, voice_name: str) -> Any:
"""Load a single voice tensor. Used by voice formula system."""
return self._pipeline.load_single_voice(voice_name)
def synthesize(self, text: str, **kwargs: Any) -> bytes:
"""Synthesize speech from text. Returns raw audio bytes."""
voice = kwargs.get("voice", "")
speed = kwargs.get("speed", 1.0)
split_pattern = kwargs.get("split_pattern", None)
audio_parts: list[np.ndarray] = []
for segment in self(text, voice=voice, speed=speed, split_pattern=split_pattern):
audio = segment.audio
if hasattr(audio, "numpy"):
audio = audio.numpy()
audio_parts.append(np.asarray(audio, dtype="float32"))
if not audio_parts:
return b""
combined = np.concatenate(audio_parts).astype("float32", copy=False)
return combined.tobytes()
def get_available_voices(self) -> List[str]:
"""Return known Kokoro voice identifiers."""
return list(self.metadata.voices)
def get_supported_formats(self) -> List[str]:
"""Kokoro outputs raw PCM float32 audio."""
return ["pcm_float32"]
def get_info(self) -> Dict[str, Any]:
return {
"id": "kokoro",
"name": "Kokoro",
"lang_code": self._lang_code,
}
def create_kokoro_backend(**kwargs: Any) -> KokoroBackend:
"""Factory callable registered with TTSBackendRegistry."""
return KokoroBackend(**kwargs)
# --- Registration ---
from abogen.tts_backend_registry import register_backend # noqa: E402
register_backend(
metadata=_KOKORO_METADATA,
factory=create_kokoro_backend,
)
+170
View File
@@ -0,0 +1,170 @@
"""TTS Plugin Architecture - Public API.
This package defines the frozen Plugin API for the TTS Plugin Architecture.
All public interfaces are fully defined but contain no business logic.
Public modules:
- types: Core domain value objects (AudioFormat, Duration, VoiceSelection, etc.)
- errors: Error hierarchy (EngineError and subtypes)
- manifest: Plugin manifest types (PluginManifest, EngineManifest, etc.)
- engine: Engine and EngineSession protocols
- capabilities: Optional capability interfaces (VoiceLister, PreviewGenerator, etc.)
- host_context: HostContext dataclass
- plugin: Plugin contract (create_engine function signature)
- loader: Plugin discovery and loading
- plugin_manager: Plugin management and engine creation
- utils: Direct utility functions (get_voices, create_pipeline, etc.)
Usage:
from abogen.tts_plugin import (
# Types
AudioFormat,
Duration,
VoiceSelection,
ParameterValues,
SynthesisRequest,
SynthesizedAudio,
EngineConfig,
# Errors
EngineError,
ModelNotFoundError,
ModelLoadError,
NetworkError,
InvalidInputError,
ConfigurationError,
CancelledError,
InternalError,
# Manifest
PluginManifest,
EngineManifest,
VoiceSourceManifest,
VoiceManifest,
ParameterManifest,
AudioFormatManifest,
EnumOption,
RequirementManifest,
GpuRequirement,
ModelManifest,
# Engine
Engine,
EngineSession,
# Capabilities
VoiceLister,
PreviewGenerator,
StreamingSynthesizer,
CancelableSession,
# Host Context
HostContext,
HttpClient,
# Plugin Manager
get_plugin_manager,
reset_plugin_manager,
# Utils
get_voices,
get_default_voice,
is_plugin_registered,
resolve_voice_to_plugin,
create_pipeline,
)
"""
from abogen.tts_plugin.capabilities import (
CancelableSession,
PreviewGenerator,
StreamingSynthesizer,
VoiceLister,
)
from abogen.tts_plugin.engine import Engine, EngineSession
from abogen.tts_plugin.errors import (
CancelledError,
ConfigurationError,
EngineError,
InternalError,
InvalidInputError,
ModelLoadError,
ModelNotFoundError,
NetworkError,
)
from abogen.tts_plugin.host_context import HttpClient, HostContext
from abogen.tts_plugin.manifest import (
AudioFormatManifest,
EngineManifest,
EnumOption,
GpuRequirement,
ModelManifest,
ParameterManifest,
PluginManifest,
RequirementManifest,
VoiceManifest,
VoiceSourceManifest,
)
from abogen.tts_plugin.types import (
AudioFormat,
Duration,
EngineConfig,
ParameterValues,
SynthesisRequest,
SynthesizedAudio,
VoiceSelection,
)
# Plugin Manager and Utils
from abogen.tts_plugin.plugin_manager import get_plugin_manager, reset_plugin_manager
from abogen.tts_plugin.utils import (
create_pipeline,
get_default_voice,
get_voices,
is_plugin_registered,
resolve_voice_to_plugin,
)
__all__ = [
# Types
"AudioFormat",
"Duration",
"VoiceSelection",
"ParameterValues",
"SynthesisRequest",
"SynthesizedAudio",
"EngineConfig",
# Errors
"EngineError",
"ModelNotFoundError",
"ModelLoadError",
"NetworkError",
"InvalidInputError",
"ConfigurationError",
"CancelledError",
"InternalError",
# Manifest
"PluginManifest",
"EngineManifest",
"VoiceSourceManifest",
"VoiceManifest",
"ParameterManifest",
"AudioFormatManifest",
"EnumOption",
"RequirementManifest",
"GpuRequirement",
"ModelManifest",
# Engine
"Engine",
"EngineSession",
# Capabilities
"VoiceLister",
"PreviewGenerator",
"StreamingSynthesizer",
"CancelableSession",
# Host Context
"HostContext",
"HttpClient",
# Plugin Manager
"get_plugin_manager",
"reset_plugin_manager",
# Utils
"get_voices",
"get_default_voice",
"is_plugin_registered",
"resolve_voice_to_plugin",
"create_pipeline",
]
+103
View File
@@ -0,0 +1,103 @@
"""Capability interfaces for the TTS Plugin Architecture.
This module defines optional capability interfaces that engines can implement.
Capabilities are additive; implementing new capabilities doesn't break old plugins.
"""
from __future__ import annotations
from typing import Iterator, Protocol, runtime_checkable
from abogen.tts_plugin.manifest import VoiceManifest
from abogen.tts_plugin.types import SynthesisRequest, SynthesizedAudio, VoiceSelection
@runtime_checkable
class VoiceLister(Protocol):
"""Protocol for listing available voices.
Engines that support voice listing should implement this interface.
"""
def listVoices(self, sourceId: str) -> list[VoiceManifest]:
"""List available voices for a given source.
Args:
sourceId: The voice source identifier.
Returns:
List of VoiceManifest describing available voices.
Raises:
EngineError: On failure.
"""
...
@runtime_checkable
class PreviewGenerator(Protocol):
"""Protocol for generating voice previews.
Engines that support voice preview should implement this interface.
"""
def generatePreview(self, voice: VoiceSelection, text: str) -> SynthesizedAudio:
"""Generate a preview audio for a voice.
Args:
voice: Voice selection for the preview.
text: Text to use for the preview.
Returns:
SynthesizedAudio with the preview audio data.
Raises:
EngineError: On failure.
"""
...
@runtime_checkable
class StreamingSynthesizer(Protocol):
"""Protocol for streaming synthesis.
Optional capability of EngineSession, not Engine.
Engines that support streaming synthesis should implement this interface.
"""
def synthesizeStream(self, request: SynthesisRequest) -> Iterator[bytes]:
"""Synthesize audio in streaming mode.
Args:
request: The synthesis request.
Yields:
Audio chunks as they become available.
Raises:
CancelledError: If cancel() is called during iteration.
EngineError: On synthesis failure.
"""
...
# This is a generator function; implementation will use yield
yield b"" # pragma: no cover
@runtime_checkable
class CancelableSession(Protocol):
"""Protocol for cancellation support.
Optional capability for engines that support cancellation.
cancel() causes synthesize() to raise CancelledError.
"""
def cancel(self) -> None:
"""Cancel in-progress synthesis.
After cancellation, synthesize() raises CancelledError.
The session remains usable after cancellation.
Raises:
EngineError: If called after dispose().
"""
...
+95
View File
@@ -0,0 +1,95 @@
"""Engine interfaces for the TTS Plugin Architecture.
This module defines the core Engine and EngineSession protocols.
These are the primary interfaces that plugin implementations must satisfy.
"""
from __future__ import annotations
from typing import Protocol, runtime_checkable
from abogen.tts_plugin.types import SynthesisRequest, SynthesizedAudio
@runtime_checkable
class EngineSession(Protocol):
"""Protocol for a session that owns mutable execution state.
An EngineSession is created by Engine.createSession() and owns
mutable execution state isolated from other concurrent work.
It is NOT thread-safe.
Lifecycle:
1. Created by Engine.createSession()
2. Used for synthesis via synthesize()
3. Disposed via dispose()
After dispose(), all methods except dispose() raise EngineError.
"""
def synthesize(self, request: SynthesisRequest) -> SynthesizedAudio:
"""Synthesize audio from text.
Args:
request: The synthesis request containing text, voice, parameters, and format.
Returns:
SynthesizedAudio with the synthesized audio data.
Raises:
EngineError: On synthesis failure. Session remains usable after error.
EngineError: If called after dispose().
"""
...
def dispose(self) -> None:
"""Release session resources.
This method is idempotent and safe to call multiple times.
It never raises exceptions (catches and logs internally).
After dispose(), all methods except dispose() raise EngineError.
"""
...
@runtime_checkable
class Engine(Protocol):
"""Protocol for a TTS engine that creates sessions.
An Engine is a factory for EngineSession instances. It is stateless
and thread-safe for createSession().
Lifecycle:
1. Created via create_engine() (plugin contract)
2. Sessions created via createSession()
3. Disposed via dispose()
Thread Safety:
- createSession() is thread-safe and can be called from any thread.
- dispose() must be called after all sessions are disposed.
- Disposing engine while sessions are alive violates API contract.
"""
def createSession(self) -> EngineSession:
"""Create a new session for synthesis.
Returns:
A new EngineSession instance. Ownership transfers to caller.
Raises:
EngineError: On failure. No partially initialized session is returned.
"""
...
def dispose(self) -> None:
"""Release engine resources.
Caller must ensure all sessions created by this engine are disposed
before calling dispose(). Disposing an engine while any session is
still alive violates the API contract; behavior is undefined.
This method is idempotent and safe to call multiple times.
It never raises exceptions (catches and logs internally).
After dispose(), all methods except dispose() raise EngineError.
"""
...
+62
View File
@@ -0,0 +1,62 @@
"""Error hierarchy for the TTS Plugin Architecture.
This module defines typed exceptions that engines raise.
Engines should never raise raw exceptions; they must use EngineError or its subtypes.
"""
from __future__ import annotations
class EngineError(Exception):
"""Base exception for all engine errors.
All engine operations that can fail should raise EngineError or one of its subtypes.
After dispose(), all methods except dispose() raise EngineError.
"""
pass
class ModelNotFoundError(EngineError):
"""Raised when a required model is not found."""
pass
class ModelLoadError(EngineError):
"""Raised when a model fails to load."""
pass
class NetworkError(EngineError):
"""Raised when a network operation fails."""
pass
class InvalidInputError(EngineError):
"""Raised when invalid input is provided to the engine."""
pass
class ConfigurationError(EngineError):
"""Raised when there is a configuration error."""
pass
class CancelledError(EngineError):
"""Raised when an operation is cancelled.
This is raised by synthesize() when cancel() is called during synthesis.
"""
pass
class InternalError(EngineError):
"""Raised when an internal engine error occurs."""
pass
+46
View File
@@ -0,0 +1,46 @@
"""Host context for the TTS Plugin Architecture.
This module defines the HostContext dataclass that provides minimal
host services to plugins. It is the only interface through which
plugins can access host functionality.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from pathlib import Path
from typing import Protocol, runtime_checkable
@runtime_checkable
class HttpClient(Protocol):
"""Protocol for HTTP client provided by host.
Plugins can use this for network requests (e.g., API-based engines).
"""
def get(self, url: str, **kwargs: object) -> object:
"""Perform an HTTP GET request."""
...
def post(self, url: str, **kwargs: object) -> object:
"""Perform an HTTP POST request."""
...
@dataclass(frozen=True)
class HostContext:
"""Minimal host context provided to plugins.
Contains only essential host services. No business logic.
Attributes:
config_dir: Directory for API keys, preferences, and configuration.
logger: Logger for plugin logging.
http_client: HTTP client for network requests.
"""
config_dir: Path
logger: logging.Logger
http_client: HttpClient
+365
View File
@@ -0,0 +1,365 @@
"""Plugin loader infrastructure for the TTS Plugin Architecture.
This module provides functionality to discover, import, validate, and load
TTS plugins. It handles both valid and invalid plugins, providing diagnostic
messages for errors.
The loader does NOT:
- Create Engine instances (that's the plugin's create_engine() responsibility)
- Manage plugin lifecycle (that's the Plugin Manager's responsibility)
- Implement any TTS engine functionality
"""
from __future__ import annotations
import importlib.util
import re
import sys
import types
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Callable
from abogen.tts_plugin.manifest import ModelManifest, PluginManifest
# Host API version for compatibility checking
HOST_API_VERSION = "1.0"
@dataclass(frozen=True)
class PluginLoadError:
"""Diagnostic information for a failed plugin load.
Attributes:
plugin_id: Plugin identifier if available, otherwise directory name.
path: Path to the plugin directory.
errors: List of error messages describing what went wrong.
"""
plugin_id: str
path: Path
errors: tuple[str, ...] = field(default_factory=tuple)
@dataclass(frozen=True)
class PluginLoadResult:
"""Result of loading a plugin.
Attributes:
success: Whether the plugin loaded successfully.
manifest: The plugin manifest if successful.
model_requirements: Model requirements if successful.
create_engine: The create_engine function if successful.
module: The plugin module if successful.
error: Error information if failed.
"""
success: bool
manifest: PluginManifest | None = None
model_requirements: tuple[ModelManifest, ...] | None = None
create_engine: Callable[..., Any] | None = None
module: types.ModuleType | None = None
error: PluginLoadError | None = None
def _parse_api_version(version: str) -> tuple[int, int] | None:
"""Parse an api_version string into (major, minor) tuple.
Args:
version: Version string in format "MAJOR.MINOR".
Returns:
Tuple of (major, minor) or None if invalid format.
"""
match = re.match(r"^(\d+)\.(\d+)$", version)
if match:
return int(match.group(1)), int(match.group(2))
return None
def _check_api_version_compatibility(plugin_version: str) -> str | None:
"""Check if plugin api_version is compatible with host.
Architecture spec:
- Format: semver (MAJOR.MINOR)
- Compatibility: Host rejects plugin if major version differs
- Minor version: backward compatible, Host accepts higher minor
Args:
plugin_version: Plugin's api_version string.
Returns:
Error message if incompatible, None if compatible.
"""
plugin_ver = _parse_api_version(plugin_version)
if plugin_ver is None:
return f"Invalid api_version format: '{plugin_version}'. Expected format: MAJOR.MINOR"
host_ver = _parse_api_version(HOST_API_VERSION)
if host_ver is None:
return f"Invalid host api_version format: '{HOST_API_VERSION}'"
if plugin_ver[0] != host_ver[0]:
return (
f"api_version major mismatch: plugin={plugin_ver[0]}, host={host_ver[0]}. "
f"Major version must match for compatibility."
)
return None
def _validate_manifest(module: types.ModuleType, plugin_dir: Path) -> list[str]:
"""Validate that a plugin module has required exports.
Args:
module: The imported plugin module.
plugin_dir: Path to the plugin directory.
Returns:
List of error messages (empty if valid).
"""
errors: list[str] = []
# Check PLUGIN_MANIFEST
manifest = getattr(module, "PLUGIN_MANIFEST", None)
if manifest is None:
errors.append("Missing PLUGIN_MANIFEST export")
elif not isinstance(manifest, PluginManifest):
errors.append(
f"PLUGIN_MANIFEST must be a PluginManifest instance, "
f"got {type(manifest).__name__}"
)
# Check MODEL_REQUIREMENTS
model_reqs = getattr(module, "MODEL_REQUIREMENTS", None)
if model_reqs is None:
errors.append("Missing MODEL_REQUIREMENTS export")
elif not isinstance(model_reqs, list):
errors.append(
f"MODEL_REQUIREMENTS must be a list, got {type(model_reqs).__name__}"
)
else:
for i, req in enumerate(model_reqs):
if not isinstance(req, ModelManifest):
errors.append(
f"MODEL_REQUIREMENTS[{i}] must be a ModelManifest instance, "
f"got {type(req).__name__}"
)
# Check create_engine
create_engine = getattr(module, "create_engine", None)
if create_engine is None:
errors.append("Missing create_engine export")
elif not callable(create_engine):
errors.append(
f"create_engine must be callable, got {type(create_engine).__name__}"
)
return errors
def _validate_capabilities(manifest: PluginManifest) -> list[str]:
"""Validate plugin capabilities.
Args:
manifest: The plugin manifest to validate.
Returns:
List of error messages (empty if valid).
"""
errors: list[str] = []
# Known capabilities (can be extended)
known_capabilities = frozenset({
"voice_list",
"preview",
"voice_clone",
"voice_blend",
"streaming",
"cancel",
})
for cap in manifest.capabilities:
if cap not in known_capabilities:
errors.append(f"Unknown capability: '{cap}'")
return errors
def _validate_api_version(manifest: PluginManifest) -> list[str]:
"""Validate api_version compatibility.
Args:
manifest: The plugin manifest to validate.
Returns:
List of error messages (empty if valid).
"""
errors: list[str] = []
error = _check_api_version_compatibility(manifest.api_version)
if error:
errors.append(error)
return errors
def load_plugin_from_dir(plugin_dir: Path) -> PluginLoadResult:
"""Load and validate a plugin from a directory.
The plugin directory must contain an __init__.py that exports:
- PLUGIN_MANIFEST: PluginManifest
- MODEL_REQUIREMENTS: list[ModelManifest]
- create_engine: Callable
Args:
plugin_dir: Path to the plugin directory.
Returns:
PluginLoadResult with success status and either plugin data or error info.
"""
plugin_id = plugin_dir.name
errors: list[str] = []
# Check if directory exists
if not plugin_dir.exists():
return PluginLoadResult(
success=False,
error=PluginLoadError(
plugin_id=plugin_id,
path=plugin_dir,
errors=(f"Plugin directory does not exist: {plugin_dir}",),
),
)
# Check for __init__.py
init_file = plugin_dir / "__init__.py"
if not init_file.exists():
return PluginLoadResult(
success=False,
error=PluginLoadError(
plugin_id=plugin_id,
path=plugin_dir,
errors=("Missing __init__.py in plugin directory",),
),
)
# Import the module
module_name = f"abogen.tts_plugin._loaded.{plugin_id}"
try:
# Remove from cache if already imported (for testing)
if module_name in sys.modules:
del sys.modules[module_name]
spec = importlib.util.spec_from_file_location(
module_name, init_file, submodule_search_locations=[]
)
if spec is None or spec.loader is None:
return PluginLoadResult(
success=False,
error=PluginLoadError(
plugin_id=plugin_id,
path=plugin_dir,
errors=(f"Failed to create module spec for {init_file}",),
),
)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)
except Exception as e:
# Clean up module from sys.modules on import failure
if module_name in sys.modules:
del sys.modules[module_name]
return PluginLoadResult(
success=False,
error=PluginLoadError(
plugin_id=plugin_id,
path=plugin_dir,
errors=(f"Failed to import plugin module: {e}",),
),
)
# Validate manifest
manifest_errors = _validate_manifest(module, plugin_dir)
errors.extend(manifest_errors)
# If manifest is valid, perform additional validation
manifest = getattr(module, "PLUGIN_MANIFEST", None)
if isinstance(manifest, PluginManifest):
# Validate api_version
api_errors = _validate_api_version(manifest)
errors.extend(api_errors)
# Validate capabilities
cap_errors = _validate_capabilities(manifest)
errors.extend(cap_errors)
# Use manifest id if available
plugin_id = manifest.id
# Check if any errors occurred
if errors:
# Clean up module from sys.modules
if module_name in sys.modules:
del sys.modules[module_name]
return PluginLoadResult(
success=False,
error=PluginLoadError(
plugin_id=plugin_id,
path=plugin_dir,
errors=tuple(errors),
),
)
# Get MODEL_REQUIREMENTS
model_requirements = tuple(getattr(module, "MODEL_REQUIREMENTS", []))
create_engine = getattr(module, "create_engine", None)
return PluginLoadResult(
success=True,
manifest=manifest,
model_requirements=model_requirements,
create_engine=create_engine,
module=module,
)
def discover_plugins(plugin_dirs: list[Path]) -> list[PluginLoadResult]:
"""Discover and load plugins from multiple directories.
Args:
plugin_dirs: List of directories to scan for plugins.
Returns:
List of PluginLoadResult, one per plugin directory found.
"""
results: list[PluginLoadResult] = []
for plugin_dir in plugin_dirs:
if not plugin_dir.exists():
continue
# Scan for subdirectories (each is a potential plugin)
for item in sorted(plugin_dir.iterdir()):
if item.is_dir() and not item.name.startswith("."):
result = load_plugin_from_dir(item)
results.append(result)
return results
def load_plugin(
plugin_dir: Path,
) -> PluginLoadResult:
"""Load a single plugin from a directory.
This is the main entry point for loading a plugin.
Args:
plugin_dir: Path to the plugin directory.
Returns:
PluginLoadResult with success status and either plugin data or error info.
"""
return load_plugin_from_dir(plugin_dir)
+189
View File
@@ -0,0 +1,189 @@
"""Plugin manifest types for the TTS Plugin Architecture.
This module contains static metadata types that describe plugins.
These types have no dependencies and are immutable.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
@dataclass(frozen=True)
class AudioFormatManifest:
"""Manifest describing an audio format.
Attributes:
mime: MIME type of the audio.
extension: File extension.
"""
mime: str
extension: str
@dataclass(frozen=True)
class EnumOption:
"""Manifest describing an enum option for a parameter.
Attributes:
value: The enum value.
label: Human-readable label.
"""
value: str
label: str
@dataclass(frozen=True)
class ParameterManifest:
"""Manifest describing a synthesis parameter.
Attributes:
id: Parameter identifier.
name: Human-readable name.
description: Parameter description.
type: Parameter type ("float", "int", "string", "boolean", "enum").
default: Default value.
min: Minimum value (optional, for numeric types).
max: Maximum value (optional, for numeric types).
step: Step size (optional, for numeric types).
options: Available options (optional, for enum type).
unit: Unit of measurement (optional).
group: Parameter group (optional).
"""
id: str
name: str
description: str
type: str
default: Any
min: float | None = None
max: float | None = None
step: float | None = None
options: tuple[EnumOption, ...] = field(default_factory=tuple)
unit: str | None = None
group: str | None = None
@dataclass(frozen=True)
class VoiceManifest:
"""Manifest describing a voice.
Attributes:
id: Voice identifier.
name: Human-readable name.
tags: Voice tags (e.g., language, style).
"""
id: str
name: str
tags: tuple[str, ...] = field(default_factory=tuple)
@dataclass(frozen=True)
class VoiceSourceManifest:
"""Manifest describing a voice source.
Attributes:
id: Voice source identifier.
name: Human-readable name.
type: Source type ("list", "speaker_id", "clone", "blend", "generate", "none").
config: Source-specific configuration.
"""
id: str
name: str
type: str
config: Any = None
@dataclass(frozen=True)
class EngineManifest:
"""Manifest describing engine capabilities.
Attributes:
voiceSources: Available voice sources.
parameters: Available synthesis parameters.
audioFormats: Supported audio formats.
"""
voiceSources: tuple[VoiceSourceManifest, ...] = field(default_factory=tuple)
parameters: tuple[ParameterManifest, ...] = field(default_factory=tuple)
audioFormats: tuple[AudioFormatManifest, ...] = field(default_factory=tuple)
@dataclass(frozen=True)
class GpuRequirement:
"""Manifest describing GPU requirements.
Attributes:
required: Whether GPU is required.
type: GPU type (e.g., "cuda", "rocm").
memory: Required GPU memory in GB.
"""
required: bool = False
type: str | None = None
memory: float | None = None
@dataclass(frozen=True)
class RequirementManifest:
"""Manifest describing plugin requirements.
Attributes:
gpu: GPU requirements (optional).
memory: Required RAM in GB (optional).
internet: Whether internet is required (optional).
"""
gpu: GpuRequirement | None = None
memory: float | None = None
internet: bool | None = None
@dataclass(frozen=True)
class ModelManifest:
"""Manifest describing a model requirement.
Attributes:
id: Model identifier.
name: Human-readable name.
size: Model size as string (e.g., "100MB", "2GB").
"""
id: str
name: str
size: str
@dataclass(frozen=True)
class PluginManifest:
"""Main manifest for a TTS plugin.
Attributes:
id: Plugin identifier (unique).
name: Human-readable name.
version: Plugin version.
api_version: API version (semver format: MAJOR.MINOR).
description: Plugin description.
author: Plugin author.
capabilities: List of capability identifiers.
requires: Plugin requirements.
engine: Engine manifest.
voices: Optional static voice catalog. None = not declared (use VoiceLister),
empty tuple = explicitly no static voices, non-empty = static catalog.
"""
id: str
name: str
version: str
api_version: str
description: str
author: str
capabilities: tuple[str, ...] = field(default_factory=tuple)
requires: RequirementManifest = field(default_factory=RequirementManifest)
engine: EngineManifest = field(default_factory=EngineManifest)
voices: tuple[VoiceManifest, ...] | None = None
+55
View File
@@ -0,0 +1,55 @@
"""Plugin contract for the TTS Plugin Architecture.
This module defines the plugin contract that all TTS plugins must implement.
Each plugin must export:
- PLUGIN_MANIFEST: PluginManifest instance
- MODEL_REQUIREMENTS: list of ModelManifest instances
- create_engine(): Factory function that creates an Engine
The create_engine() function is the entry point for plugin activation.
It must be atomic: succeed fully or raise and clean up.
"""
from __future__ import annotations
from pathlib import Path
from typing import Protocol, runtime_checkable
from abogen.tts_plugin.engine import Engine
from abogen.tts_plugin.host_context import HostContext
from abogen.tts_plugin.types import EngineConfig
@runtime_checkable
class Plugin(Protocol):
"""Protocol defining the plugin contract.
Every TTS plugin must implement this protocol by exporting:
- PLUGIN_MANIFEST: PluginManifest
- MODEL_REQUIREMENTS: list[ModelManifest]
- create_engine: Callable[[HostContext, Path | None, EngineConfig], Engine]
"""
def create_engine(
self,
context: HostContext,
model_path: Path | None,
config: EngineConfig,
) -> Engine:
"""Create an engine instance.
This is the factory function that creates an Engine from a plugin.
It must be atomic: succeed fully or raise EngineError and clean up.
Args:
context: Host services (config dir, logger, http client).
model_path: Resolved model path, or None for cloud/no-model engines.
config: Engine initialization settings.
Returns:
A fully initialized Engine instance.
Raises:
EngineError: On failure. Cleans up partially created resources.
"""
...
+156
View File
@@ -0,0 +1,156 @@
"""Plugin Manager
Provides a simple interface for consumers to access TTS engines via the
new Plugin Architecture. Discovers, loads, and manages plugins from the
plugins directory.
Usage:
from abogen.tts_plugin.plugin_manager import get_plugin_manager
manager = get_plugin_manager()
engine = manager.create_engine("kokoro", lang_code="a", device="cpu")
session = engine.create_session()
try:
result = session.synthesize("Hello world")
finally:
session.dispose()
"""
from typing import Any, Dict, List, Optional, Type
from abogen.tts_plugin.engine import Engine, EngineSession
from abogen.tts_plugin.manifest import PluginManifest
from abogen.tts_plugin.types import AudioFormat
class PluginManager:
"""Manages TTS plugins and provides a simple interface for consumers."""
def __init__(self) -> None:
self._plugins: Dict[str, dict] = {}
self._engines: Dict[str, Engine] = {}
self._loaded = False
def discover(self, plugins_dir: str = "plugins") -> None:
"""Discover and load all plugins from the given directory."""
import os
from pathlib import Path
from abogen.tts_plugin.loader import load_plugin_from_dir
self._plugins.clear()
self._engines.clear()
plugins_path = Path(plugins_dir)
if not plugins_path.exists():
if plugins_dir == "plugins":
plugins_path = Path(__file__).resolve().parent.parent.parent / "plugins"
if not plugins_path.exists():
self._loaded = True
return
for entry in plugins_path.iterdir():
if entry.is_dir() and (entry / "__init__.py").exists():
try:
result = load_plugin_from_dir(entry)
if result.success and result.manifest is not None:
self._plugins[result.manifest.id] = {
"manifest": result.manifest,
"create_engine": result.create_engine,
"module": result.module,
}
except Exception as e:
# Log error but continue with other plugins
print(f"Warning: Failed to load plugin from {entry}: {e}")
self._loaded = True
def _ensure_loaded(self) -> None:
"""Ensure plugins have been discovered."""
if not self._loaded:
self.discover()
def list_plugins(self) -> List[PluginManifest]:
"""Return manifests for all loaded plugins."""
self._ensure_loaded()
return [info["manifest"] for info in self._plugins.values()]
def get_plugin(self, plugin_id: str) -> Optional[dict]:
"""Get plugin info by ID."""
self._ensure_loaded()
return self._plugins.get(plugin_id)
def has_plugin(self, plugin_id: str) -> bool:
"""Check if a plugin is loaded."""
self._ensure_loaded()
return plugin_id in self._plugins
def create_engine(self, plugin_id: str, **kwargs: Any) -> Engine:
"""Create an engine instance for the given plugin.
Args:
plugin_id: The plugin identifier (e.g., "kokoro")
**kwargs: Arguments passed to the engine constructor
Returns:
An Engine instance
Raises:
KeyError: If plugin_id is not found
Exception: If engine creation fails
"""
self._ensure_loaded()
if plugin_id not in self._plugins:
raise KeyError(f"Plugin not found: {plugin_id}")
plugin_info = self._plugins[plugin_id]
create_engine_func = plugin_info["create_engine"]
# Create engine using the plugin's factory
engine = create_engine_func(**kwargs)
return engine
def get_or_create_engine(self, plugin_id: str, **kwargs: Any) -> Engine:
"""Get an existing engine or create a new one.
Engines are cached by plugin_id. If you need multiple instances
with different parameters, use create_engine() directly.
"""
self._ensure_loaded()
cache_key = plugin_id
if cache_key in self._engines:
return self._engines[cache_key]
engine = self.create_engine(plugin_id, **kwargs)
self._engines[cache_key] = engine
return engine
def dispose_all(self) -> None:
"""Dispose all cached engines."""
for engine in self._engines.values():
try:
engine.dispose()
except Exception:
pass # dispose() should never raise
self._engines.clear()
# Global singleton
_manager: Optional[PluginManager] = None
def get_plugin_manager() -> PluginManager:
"""Get the global PluginManager instance."""
global _manager
if _manager is None:
_manager = PluginManager()
return _manager
def reset_plugin_manager() -> None:
"""Reset the global PluginManager (for testing)."""
global _manager
if _manager is not None:
_manager.dispose_all()
_manager = None
+111
View File
@@ -0,0 +1,111 @@
"""Core domain types for the TTS Plugin Architecture.
This module contains immutable value objects that form the core domain.
These types have zero dependencies and are used across the plugin system.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Mapping
@dataclass(frozen=True)
class AudioFormat:
"""Immutable value object representing an audio format.
Attributes:
mime: MIME type of the audio (e.g., "audio/wav", "audio/mpeg").
extension: File extension (e.g., "wav", "mp3").
"""
mime: str
extension: str
@dataclass(frozen=True)
class Duration:
"""Immutable value object representing a time duration.
Attributes:
seconds: Duration in seconds.
"""
seconds: float
@dataclass(frozen=True)
class VoiceSelection:
"""Immutable value object for voice selection. Opaque to engine.
Attributes:
source: Voice source identifier (e.g., "builtin", "clone").
key: Voice key within the source.
payload: Optional payload for clone/blend sources.
"""
source: str
key: str
payload: Any = None
@dataclass(frozen=True)
class ParameterValues:
"""Immutable value object for synthesis parameters. Behaves like Mapping[str, Any].
Attributes:
values: Mapping of parameter names to their values.
"""
values: Mapping[str, Any] = field(default_factory=dict)
@dataclass(frozen=True)
class SynthesisRequest:
"""Immutable value object for a synthesis request.
Attributes:
text: Text to synthesize.
voice: Voice selection.
parameters: Synthesis parameters.
format: Desired audio output format.
"""
text: str
voice: VoiceSelection
parameters: ParameterValues
format: AudioFormat
@dataclass(frozen=True)
class SynthesizedAudio:
"""Immutable value object for synthesized audio result.
Attributes:
data: Raw audio bytes.
format: Audio format of the result.
duration: Duration of the audio.
"""
data: bytes
format: AudioFormat
duration: Duration
@dataclass(frozen=True)
class EngineConfig:
"""Immutable configuration of an Engine instance.
Contains parameters that define how a particular Engine instance is
created and that remain constant throughout the lifetime of that Engine.
Plugin implementations may ignore fields that are not applicable to them.
Attributes:
device: Device to use (e.g., "cpu", "cuda:0").
lang_code: Language code for the engine (e.g., "a" for Kokoro English).
Plugins that do not require a language code ignore this field.
"""
device: str = "cpu"
lang_code: str = "a"
+241
View File
@@ -0,0 +1,241 @@
"""TTS Plugin Architecture — direct utility functions.
Provides helpers that replace the former compatibility adapter by
calling the Plugin Manager directly.
"""
from __future__ import annotations
from typing import Any, Iterator
import numpy as np
from abogen.tts_plugin.plugin_manager import get_plugin_manager
def get_voices(plugin_id: str) -> tuple[str, ...]:
"""Return the voice-id tuple for *plugin_id*.
Uses the official Plugin Architecture: PluginManager Engine VoiceLister.
First checks plugin manifest for static voice catalog.
"""
import logging
import tempfile
from pathlib import Path
from abogen.tts_plugin.host_context import HostContext
from abogen.tts_plugin.types import EngineConfig
manager = get_plugin_manager()
if not manager.has_plugin(plugin_id):
return ()
# Check manifest for static voice catalog
plugin_info = manager.get_plugin(plugin_id)
if plugin_info is not None:
manifest = plugin_info.get("manifest")
if manifest is not None and manifest.voices is not None:
return tuple(v.id for v in manifest.voices)
ctx = HostContext(
config_dir=Path(tempfile.gettempdir()),
logger=logging.getLogger(f"abogen.utils.{plugin_id}"),
http_client=type("_StubHttpClient", (), {
"get": staticmethod(lambda url, **kw: None),
"post": staticmethod(lambda url, **kw: None),
})(),
)
try:
engine = manager.create_engine(
plugin_id,
context=ctx,
model_path=None,
config=EngineConfig(device="cpu"),
)
except Exception:
return ()
try:
from abogen.tts_plugin.capabilities import VoiceLister
if isinstance(engine, VoiceLister):
manifests = engine.listVoices("builtin")
return tuple(v.id for v in manifests)
return ()
except Exception:
return ()
finally:
engine.dispose()
def get_default_voice(plugin_id: str, fallback: str = "") -> str:
"""Return the first voice of *plugin_id*, or *fallback*."""
voices = get_voices(plugin_id)
return voices[0] if voices else fallback
def is_plugin_registered(plugin_id: str) -> bool:
"""Check whether *plugin_id* is loaded by the Plugin Manager."""
return get_plugin_manager().has_plugin(plugin_id)
def resolve_voice_to_plugin(spec: str, fallback: str = "kokoro") -> str:
"""Determine which plugin owns the given voice specification.
Resolution rules:
1. Empty spec -> fallback
2. Kokoro formula (contains '*' or '+') -> "kokoro"
3. Exact voice-id match against loaded plugins -> plugin id
4. Unknown voice -> fallback
"""
raw = str(spec or "").strip()
if not raw:
return fallback
if "*" in raw or "+" in raw:
return "kokoro"
upper = raw.upper()
manager = get_plugin_manager()
for manifest in manager.list_plugins():
for voice_source in manifest.engine.voiceSources:
if voice_source.type == "list" and isinstance(voice_source.config, dict):
try:
engine = manager.create_engine(manifest.id)
try:
if hasattr(engine, "listVoices"):
voice_manifests = engine.listVoices(voice_source.id)
voice_ids = [v.id.upper() for v in voice_manifests]
if upper in voice_ids:
return manifest.id
finally:
engine.dispose()
except Exception:
continue
return fallback
class Pipeline:
"""Callable wrapper around Engine / EngineSession.
Presents the same interface that old callers expect::
pipeline = create_pipeline("kokoro", lang_code="a", device="cpu")
for segment in pipeline(text, voice="af_nova", speed=1.0):
audio = segment.audio
"""
def __init__(self, engine: Any, **engine_kwargs: Any) -> None:
self._engine = engine
self._engine_kwargs = engine_kwargs
self._session: Any = None
def _ensure_session(self) -> Any:
if self._session is None:
self._session = self._engine.createSession()
return self._session
def __call__(
self,
text: str,
voice: str = "default",
speed: float = 1.0,
split_pattern: str | None = None,
**kwargs: Any,
) -> Iterator[Any]:
from abogen.tts_plugin.types import (
AudioFormat,
ParameterValues,
SynthesisRequest,
VoiceSelection,
)
session = self._ensure_session()
params: dict[str, Any] = {"speed": speed}
if split_pattern is not None:
params["split_pattern"] = split_pattern
params.update(kwargs)
request = SynthesisRequest(
text=text,
voice=VoiceSelection(source="builtin", key=voice),
parameters=ParameterValues(values=params),
format=AudioFormat(mime="audio/wav", extension="wav"),
)
result = session.synthesize(request)
audio_array = np.frombuffer(result.data, dtype=np.float32)
from dataclasses import dataclass
@dataclass
class Segment:
graphemes: str
audio: np.ndarray
yield Segment(graphemes=text, audio=audio_array)
def load_single_voice(self, voice_name: str) -> Any:
engine_pipeline = getattr(self._engine, '_pipeline', None)
if engine_pipeline is not None and hasattr(engine_pipeline, 'load_single_voice'):
return engine_pipeline.load_single_voice(voice_name)
raise AttributeError(f"load_single_voice not available on {type(self._engine).__name__}")
def dispose(self) -> None:
if self._session is not None:
try:
self._session.dispose()
except Exception:
pass
self._session = None
def __del__(self) -> None:
self.dispose()
def create_pipeline(
plugin_id: str,
*,
lang_code: str = "a",
device: str = "cpu",
) -> Pipeline:
"""Create a callable TTS pipeline via the Plugin Architecture.
Builds a proper HostContext and EngineConfig, then delegates to the
PluginManager to create the engine. Returns a :class:`Pipeline` whose
``__call__`` interface matches the callable protocol used by consumers.
Args:
plugin_id: Plugin identifier (e.g., "kokoro", "supertonic").
lang_code: Language code for the engine.
device: Device to use (e.g., "cpu", "cuda:0").
Returns:
A callable Pipeline instance.
"""
import logging
import tempfile
from pathlib import Path
from abogen.tts_plugin.host_context import HostContext
from abogen.tts_plugin.types import EngineConfig
manager = get_plugin_manager()
ctx = HostContext(
config_dir=Path(tempfile.gettempdir()),
logger=logging.getLogger(f"abogen.pipeline.{plugin_id}"),
http_client=type("_StubHttpClient", (), {
"get": staticmethod(lambda url, **kw: None),
"post": staticmethod(lambda url, **kw: None),
})(),
)
config = EngineConfig(device=device, lang_code=lang_code)
engine = manager.create_engine(plugin_id, context=ctx, model_path=None, config=config)
return Pipeline(engine)
+5 -5
View File
@@ -530,18 +530,18 @@ def prevent_sleep_end():
class LoadPipelineThread(Thread): class LoadPipelineThread(Thread):
def __init__(self, callback, lang_code="a", device="cpu"): def __init__(self, callback, lang_code="a", use_gpu=True):
super().__init__() super().__init__()
self.callback = callback self.callback = callback
self.lang_code = lang_code self.lang_code = lang_code
self.device = device self.use_gpu = use_gpu
def run(self): def run(self):
try: try:
from abogen.tts_backend_registry import create_backend from abogen.domain.pipeline_factory import create_pipeline_for_job
backend = create_backend( backend = create_pipeline_for_job(
"kokoro", lang_code=self.lang_code, device=self.device "kokoro", language=self.lang_code, use_gpu=self.use_gpu
) )
self.callback(backend, None) self.callback(backend, None)
except Exception as e: except Exception as e:
+10 -2
View File
@@ -17,7 +17,7 @@ if LocalEntryNotFoundError is None: # pragma: no cover - fallback for tests
pass pass
from abogen.tts_backend_registry import get_metadata from abogen.tts_plugin.utils import get_voices
_CACHE_LOCK = threading.Lock() _CACHE_LOCK = threading.Lock()
_CACHED_VOICES: Set[str] = set() _CACHED_VOICES: Set[str] = set()
@@ -26,7 +26,7 @@ _BOOTSTRAPPED = False
def _normalize_targets(voices: Optional[Iterable[str]]) -> Set[str]: def _normalize_targets(voices: Optional[Iterable[str]]) -> Set[str]:
kokoro_voices = get_metadata("kokoro").voices kokoro_voices = get_voices("kokoro")
if not voices: if not voices:
return set(kokoro_voices) return set(kokoro_voices)
normalized: Set[str] = set() normalized: Set[str] = set()
@@ -144,3 +144,11 @@ def _ensure_single_voice_asset(
hf_hub_download(resume_download=True, **common_kwargs) hf_hub_download(resume_download=True, **common_kwargs)
return True return True
def clear_voice_cache() -> None:
"""Clear the inprocess voice cache (used during shutdown)."""
with _CACHE_LOCK:
_CACHED_VOICES.clear()
global _BOOTSTRAPPED
_BOOTSTRAPPED = False
+30 -3
View File
@@ -1,7 +1,7 @@
import re import re
from typing import List, Tuple from typing import Iterable, List, Optional, Tuple
from abogen.tts_backend_registry import get_metadata from abogen.tts_plugin.utils import get_voices
# Calls parsing and loads the voice to gpu or cpu # Calls parsing and loads the voice to gpu or cpu
@@ -22,7 +22,7 @@ def parse_formula_terms(formula: str) -> List[Tuple[str, float]]:
raise ValueError("Empty voice formula") raise ValueError("Empty voice formula")
terms: List[Tuple[str, float]] = [] terms: List[Tuple[str, float]] = []
kokoro_voices = get_metadata("kokoro").voices kokoro_voices = get_voices("kokoro")
for segment in formula.split("+"): for segment in formula.split("+"):
part = segment.strip() part = segment.strip()
if not part: if not part:
@@ -72,6 +72,33 @@ def parse_voice_formula(pipeline, formula):
return weighted_sum return weighted_sum
def pairs_to_formula(pairs: Iterable[Tuple[str, float]]) -> Optional[str]:
"""Build a voice formula string from (voice_name, weight) pairs.
Normalizes weights to sum to 1.0 and formats as "voice1*0.5+voice2*0.5".
Args:
pairs: Iterable of (voice_name, weight) tuples. Zero-weight entries
are filtered out.
Returns:
Formula string, or None if no valid entries.
"""
voices = [(voice, float(weight)) for voice, weight in pairs if weight is not None and float(weight) > 0]
if not voices:
return None
total = sum(weight for _, weight in voices)
if total <= 0:
return None
def _format_value(value: float) -> str:
normalized = value / total if total else 0.0
return (f"{normalized:.4f}").rstrip("0").rstrip(".") or "0"
parts = [f"{voice}*{_format_value(weight)}" for voice, weight in voices]
return "+".join(parts)
def calculate_sum_from_formula(formula): def calculate_sum_from_formula(formula):
weights = re.findall(r"\* *([\d.]+)", formula) weights = re.findall(r"\* *([\d.]+)", formula)
total_sum = sum(float(weight) for weight in weights) total_sum = sum(float(weight) for weight in weights)
+4 -4
View File
@@ -2,7 +2,7 @@ import json
import os import os
from typing import Any, Dict, Iterable, List, Tuple from typing import Any, Dict, Iterable, List, Tuple
from abogen.tts_backend_registry import get_metadata from abogen.tts_plugin.utils import get_voices, is_plugin_registered
from abogen.utils import get_user_config_path from abogen.utils import get_user_config_path
@@ -69,7 +69,7 @@ def serialize_profiles() -> Dict[str, Dict[str, Iterable[Tuple[str, float]]]]:
def _normalize_supertonic_voice(value: Any) -> str: def _normalize_supertonic_voice(value: Any) -> str:
raw = str(value or "").strip().upper() raw = str(value or "").strip().upper()
supertonic_voices = get_metadata("supertonic").voices supertonic_voices = get_voices("supertonic")
return raw if raw in supertonic_voices else "M1" return raw if raw in supertonic_voices else "M1"
@@ -101,7 +101,7 @@ def normalize_profile_entry(entry: Any) -> Dict[str, Any]:
return {} return {}
provider = str(entry.get("provider") or "kokoro").strip().lower() provider = str(entry.get("provider") or "kokoro").strip().lower()
if provider not in {"kokoro", "supertonic"}: if not is_plugin_registered(provider):
provider = "kokoro" provider = "kokoro"
language = str(entry.get("language") or "a").strip().lower() or "a" language = str(entry.get("language") or "a").strip().lower() or "a"
@@ -135,7 +135,7 @@ def normalize_profile_entry(entry: Any) -> Dict[str, Any]:
def _normalize_voice_entries(entries: Iterable) -> List[Tuple[str, float]]: def _normalize_voice_entries(entries: Iterable) -> List[Tuple[str, float]]:
normalized: List[Tuple[str, float]] = [] normalized: List[Tuple[str, float]] = []
kokoro_voices = get_metadata("kokoro").voices kokoro_voices = get_voices("kokoro")
for item in entries or []: for item in entries or []:
if isinstance(item, dict): if isinstance(item, dict):
voice = item.get("id") or item.get("voice") voice = item.get("id") or item.get("voice")
+6 -7
View File
@@ -2,7 +2,6 @@ FROM nvidia/cuda:12.6.3-cudnn-runtime-ubuntu22.04
ENV PYTHONDONTWRITEBYTECODE=1 \ ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \ PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1 \
VIRTUAL_ENV=/opt/venv \ VIRTUAL_ENV=/opt/venv \
PATH=/opt/venv/bin:$PATH PATH=/opt/venv/bin:$PATH
@@ -27,22 +26,22 @@ RUN python3 -m venv "$VIRTUAL_ENV"
WORKDIR /app WORKDIR /app
COPY pyproject.toml README.md ./ COPY pyproject.toml README.md ./
RUN pip install --upgrade pip \ RUN pip install uv \
&& if [ -n "$TORCH_VERSION" ]; then \ && if [ -n "$TORCH_VERSION" ]; then \
pip install torch=="$TORCH_VERSION" torchvision=="$TORCH_VERSION" torchaudio=="$TORCH_VERSION" --index-url "$TORCH_INDEX_URL"; \ uv pip install --system torch=="$TORCH_VERSION" torchvision=="$TORCH_VERSION" torchaudio=="$TORCH_VERSION" --index-url "$TORCH_INDEX_URL"; \
else \ else \
pip install torch torchvision torchaudio --index-url "$TORCH_INDEX_URL"; \ uv pip install --system torch torchvision torchaudio --index-url "$TORCH_INDEX_URL"; \
fi \ fi \
&& pip install --no-cache-dir . \ && uv pip install --system . \
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl \ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl \
&& pip install --no-cache-dir "mutagen>=1.47.0" && uv pip install --system "mutagen>=1.47.0"
COPY abogen ./abogen COPY abogen ./abogen
# Install onnxruntime-gpu for CUDA acceleration (supertonic uses ONNX Runtime) # Install onnxruntime-gpu for CUDA acceleration (supertonic uses ONNX Runtime)
# Set USE_GPU=false to skip this for CPU-only deployments # Set USE_GPU=false to skip this for CPU-only deployments
RUN if [ "$USE_GPU" = "true" ]; then \ RUN if [ "$USE_GPU" = "true" ]; then \
pip install --no-cache-dir onnxruntime-gpu; \ uv pip install --system onnxruntime-gpu; \
fi fi
ENV ABOGEN_HOST=0.0.0.0 \ ENV ABOGEN_HOST=0.0.0.0 \
+9 -4
View File
@@ -1,6 +1,5 @@
from __future__ import annotations from __future__ import annotations
import atexit
import logging import logging
import os import os
from pathlib import Path from pathlib import Path
@@ -8,6 +7,8 @@ from typing import Any, Optional
from flask import Flask from flask import Flask
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
from abogen.utils import get_user_cache_path, get_user_output_path, get_user_settings_dir from abogen.utils import get_user_cache_path, get_user_output_path, get_user_settings_dir
from .conversion_runner import run_conversion_job from .conversion_runner import run_conversion_job
@@ -83,6 +84,12 @@ def create_app(config: Optional[dict[str, Any]] = None) -> Flask:
"UPLOAD_FOLDER": str(uploads_dir), "UPLOAD_FOLDER": str(uploads_dir),
"OUTPUT_FOLDER": str(outputs_dir), "OUTPUT_FOLDER": str(outputs_dir),
"MAX_CONTENT_LENGTH": 1024 * 1024 * 400, # 400 MB uploads "MAX_CONTENT_LENGTH": 1024 * 1024 * 400, # 400 MB uploads
# Large books can submit four form fields per chapter. Werkzeug's
# defaults reject those requests before the wizard route can process
# them, even though the encoded payload is much smaller than the upload
# limit above.
"MAX_FORM_MEMORY_SIZE": 10 * 1024 * 1024,
"MAX_FORM_PARTS": 10_000,
} }
if config: if config:
base_config.update(config) base_config.update(config)
@@ -113,8 +120,6 @@ def create_app(config: Optional[dict[str, Any]] = None) -> Flask:
app.register_blueprint(books_bp, url_prefix="/find-books") app.register_blueprint(books_bp, url_prefix="/find-books")
app.register_blueprint(api_bp, url_prefix="/api") app.register_blueprint(api_bp, url_prefix="/api")
atexit.register(service.shutdown)
global _access_log_filter_attached global _access_log_filter_attached
if not _access_log_filter_attached: if not _access_log_filter_attached:
logging.getLogger("werkzeug").addFilter(_SuppressSuccessfulAccessFilter()) logging.getLogger("werkzeug").addFilter(_SuppressSuccessfulAccessFilter())
@@ -132,4 +137,4 @@ def main() -> None:
if __name__ == "__main__": # pragma: no cover if __name__ == "__main__": # pragma: no cover
main() main()
+162
View File
@@ -0,0 +1,162 @@
"""WebUI adapter: Job -> ConversionRequest.
Converts a WebUI Job into a ConversionRequest that the application layer can process.
This adapter is the bridge between the WebUI layer and the application/domain layer.
The adapter is responsible for:
- Mapping Job fields to ConversionRequest fields
- Handling UI-specific state (logs, progress, cancellation)
- Providing PipelineProvider and VoiceResolver implementations
All conversions happen through this adapter the application layer
never accesses Job directly.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional
from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_ports import ConversionCancelled, ResolvedVoice
def build_conversion_request_from_job(job: Any) -> ConversionRequest:
"""Convert a WebUI Job into a ConversionRequest.
This is the primary function that maps Job fields to ConversionRequest.
All fields are copied the request is independent of the Job.
Args:
job: WebUI Job instance
Returns:
ConversionRequest with all Job data mapped
"""
return ConversionRequest(
# Source
source_path=Path(job.stored_path) if job.stored_path else None,
original_filename=job.original_filename,
# TTS Settings
language=job.language,
tts_provider=job.tts_provider,
voice=job.voice,
voice_profile=job.voice_profile,
speed=job.speed,
use_gpu=job.use_gpu,
supertonic_total_steps=job.supertonic_total_steps,
# Output Format
output_format=job.output_format,
subtitle_mode=job.subtitle_mode,
subtitle_format=job.subtitle_format,
max_subtitle_words=job.max_subtitle_words,
# Save Options
save_mode=job.save_mode,
output_folder=Path(job.output_folder) if job.output_folder else None,
save_chapters_separately=job.save_chapters_separately,
merge_chapters_at_end=job.merge_chapters_at_end,
separate_chapters_format=job.separate_chapters_format,
save_as_project=job.save_as_project,
# Timing
silence_between_chapters=job.silence_between_chapters,
chapter_intro_delay=job.chapter_intro_delay,
# Content Processing
replace_single_newlines=job.replace_single_newlines,
read_title_intro=job.read_title_intro,
read_closing_outro=job.read_closing_outro,
auto_prefix_chapter_titles=job.auto_prefix_chapter_titles,
normalize_chapter_opening_caps=job.normalize_chapter_opening_caps,
# Pronunciation / Normalization
pronunciation_overrides=job.pronunciation_overrides or [],
manual_overrides=job.manual_overrides or [],
heteronym_overrides=job.heteronym_overrides or [],
normalization_overrides=job.normalization_overrides or {},
# Chapter/Chunk Configuration
chapter_overrides=job.chapters or [],
chunks=job.chunks or [],
chunk_level=job.chunk_level,
speaker_mode=job.speaker_mode,
speakers=job.speakers or {},
# Metadata
metadata_tags=job.metadata_tags or {},
# Artifacts
cover_image_path=Path(job.cover_image_path) if job.cover_image_path else None,
cover_image_mime=job.cover_image_mime,
generate_epub3=job.generate_epub3,
)
class WebJobEvents:
"""WebUI implementation of ConversionEvents protocol.
Wraps a Job to provide logging, progress, and cancellation.
"""
def __init__(self, job: Any):
self._job = job
def log(self, message: str, level: str = "info") -> None:
"""Log a message to the Job."""
self._job.add_log(message, level=level)
def progress(self, pct: int, etr: str) -> None:
"""Update progress on the Job."""
self._job.progress = pct / 100.0
self._job.etr_str = etr
def check_cancelled(self) -> None:
"""Check if the Job was cancelled.
Raises:
ConversionCancelled: If cancellation was requested
"""
if self._job.cancel_requested:
raise ConversionCancelled("Job cancelled by user")
class WebPipelineProvider:
"""WebUI implementation of PipelineProvider protocol.
Wraps PipelinePool to provide TTS backends.
"""
def __init__(self, pipeline_pool: Any):
self._pool = pipeline_pool
def get(self, provider: str, language: str, use_gpu: bool) -> Any:
"""Get a TTS backend instance."""
return self._pool.get(provider, language, use_gpu)
def dispose_all(self) -> None:
"""Dispose all backend resources."""
self._pool.dispose_all()
class WebVoiceResolver:
"""WebUI implementation of VoiceResolver protocol.
Wraps the voice resolution logic from conversion_runner.py.
"""
def __init__(
self,
resolve_fn: Callable[[str], tuple[str, str, Any, Optional[float], Optional[int]]],
):
"""Initialize with a voice resolution function.
Args:
resolve_fn: Function that takes a voice_spec and returns
(provider, resolved_spec, voice_choice, speed, steps)
"""
self._resolve_fn = resolve_fn
def resolve(self, voice_spec: str) -> ResolvedVoice:
"""Resolve a voice spec into a loaded voice."""
provider, resolved_spec, voice, speed, steps = self._resolve_fn(voice_spec)
return ResolvedVoice(
provider=provider,
resolved_spec=resolved_spec,
voice=voice,
speed=speed or 1.0,
supertonic_steps=steps or 5,
)
File diff suppressed because it is too large Load Diff
+11 -5
View File
@@ -14,8 +14,10 @@ from abogen.kokoro_text_normalization import normalize_for_pipeline
from abogen.normalization_settings import build_apostrophe_config from abogen.normalization_settings import build_apostrophe_config
from abogen.text_extractor import extract_from_path from abogen.text_extractor import extract_from_path
from abogen.voice_cache import ensure_voice_assets from abogen.voice_cache import ensure_voice_assets
from abogen.webui.conversion_runner import SAMPLE_RATE, SPLIT_PATTERN, _select_device, _to_float32, _resolve_voice, _spec_to_voice_ids from abogen.webui.conversion_runner import SAMPLE_RATE, _select_device, _to_float32, _spec_to_voice_ids
from abogen.tts_backend_registry import create_backend from abogen.domain.voice_loader import resolve_voice
from abogen.domain.split_pattern import get_split_pattern
from abogen.tts_plugin.utils import create_pipeline
_MARKER_RE = re.compile(re.escape(MARKER_PREFIX) + r"(?P<code>[A-Z0-9_]+)" + re.escape(MARKER_SUFFIX)) _MARKER_RE = re.compile(re.escape(MARKER_PREFIX) + r"(?P<code>[A-Z0-9_]+)" + re.escape(MARKER_SUFFIX))
@@ -45,7 +47,7 @@ def _load_pipeline(language: str, use_gpu: bool) -> Any:
device = "cpu" device = "cpu"
if use_gpu: if use_gpu:
device = _select_device() device = _select_device()
return create_backend("kokoro", lang_code=language, device=device) return create_pipeline("kokoro", lang_code=language, device=device)
def _extract_cases_from_text(text: str) -> List[Tuple[str, str]]: def _extract_cases_from_text(text: str) -> List[Tuple[str, str]]:
@@ -175,7 +177,7 @@ def run_debug_tts_wavs(
pass pass
pipeline = _load_pipeline(language, use_gpu) pipeline = _load_pipeline(language, use_gpu)
voice_choice = _resolve_voice(pipeline, voice_spec, use_gpu) voice_choice = resolve_voice(voice_spec, pipeline, use_gpu)
apostrophe_config = build_apostrophe_config(settings=settings) apostrophe_config = build_apostrophe_config(settings=settings)
normalization_settings = dict(settings) normalization_settings = dict(settings)
@@ -200,7 +202,7 @@ def run_debug_tts_wavs(
normalized, normalized,
voice=voice_choice, voice=voice_choice,
speed=speed, speed=speed,
split_pattern=SPLIT_PATTERN, split_pattern=get_split_pattern(language, "Disabled"),
): ):
audio = _to_float32(getattr(segment, "audio", None)) audio = _to_float32(getattr(segment, "audio", None))
if audio.size: if audio.size:
@@ -247,4 +249,8 @@ def run_debug_tts_wavs(
"sample_rate": SAMPLE_RATE, "sample_rate": SAMPLE_RATE,
} }
(run_dir / "manifest.json").write_text(json.dumps(manifest, indent=2), encoding="utf-8") (run_dir / "manifest.json").write_text(json.dumps(manifest, indent=2), encoding="utf-8")
try:
pipeline.dispose()
except Exception:
pass
return manifest return manifest
+5 -80
View File
@@ -25,7 +25,7 @@ from abogen.voice_profiles import (
normalize_profile_entry, normalize_profile_entry,
) )
from abogen.webui.routes.utils.common import split_profile_spec from abogen.webui.routes.utils.common import split_profile_spec
from abogen.webui.routes.utils.preview import synthesize_preview, generate_preview_audio from abogen.webui.routes.utils.synthesize import synthesize_preview, generate_preview_audio
from abogen.webui.routes.utils.voice import formula_from_profile from abogen.webui.routes.utils.voice import formula_from_profile
from abogen.normalization_settings import ( from abogen.normalization_settings import (
build_llm_configuration, build_llm_configuration,
@@ -34,6 +34,7 @@ from abogen.normalization_settings import (
) )
from abogen.llm_client import list_models, LLMClientError from abogen.llm_client import list_models, LLMClientError
from abogen.kokoro_text_normalization import normalize_for_pipeline from abogen.kokoro_text_normalization import normalize_for_pipeline
from abogen.tts_plugin.utils import is_plugin_registered
from abogen.integrations.audiobookshelf import AudiobookshelfClient, AudiobookshelfConfig from abogen.integrations.audiobookshelf import AudiobookshelfClient, AudiobookshelfConfig
from abogen.integrations.calibre_opds import ( from abogen.integrations.calibre_opds import (
CalibreOPDSClient, CalibreOPDSClient,
@@ -63,7 +64,7 @@ def api_save_voice_profile() -> ResponseReturnValue:
if profile is None: if profile is None:
# Speaker Studio payload format # Speaker Studio payload format
provider = str(payload.get("provider") or "kokoro").strip().lower() provider = str(payload.get("provider") or "kokoro").strip().lower()
if provider not in {"kokoro", "supertonic"}: if not is_plugin_registered(provider):
provider = "kokoro" provider = "kokoro"
if provider == "supertonic": if provider == "supertonic":
profile = { profile = {
@@ -230,7 +231,7 @@ def api_speaker_preview() -> ResponseReturnValue:
use_gpu = settings.get("use_gpu", False) use_gpu = settings.get("use_gpu", False)
base_spec, speaker_name = split_profile_spec(voice) base_spec, speaker_name = split_profile_spec(voice)
resolved_provider = tts_provider if tts_provider in {"kokoro", "supertonic"} else "" resolved_provider = tts_provider if is_plugin_registered(tts_provider) else ""
if speaker_name: if speaker_name:
entry = normalize_profile_entry(load_profiles().get(speaker_name)) entry = normalize_profile_entry(load_profiles().get(speaker_name))
@@ -280,83 +281,7 @@ def api_speaker_preview() -> ResponseReturnValue:
# --- Integration Routes --- # --- Integration Routes ---
def _opds_metadata_overrides(metadata_payload: Mapping[str, Any]) -> Dict[str, Any]: from abogen.domain.metadata_overrides import normalize_opds_metadata as _opds_metadata_overrides
metadata_overrides: Dict[str, Any] = {}
def _stringify_metadata_value(value: Any) -> str:
if value is None:
return ""
if isinstance(value, (list, tuple, set)):
parts = [str(item).strip() for item in value if item is not None]
parts = [part for part in parts if part]
return ", ".join(parts)
return str(value).strip()
raw_series = metadata_payload.get("series") or metadata_payload.get("series_name")
series_name = str(raw_series or "").strip()
if series_name:
metadata_overrides["series"] = series_name
metadata_overrides.setdefault("series_name", series_name)
series_index_value = (
metadata_payload.get("series_index")
or metadata_payload.get("series_position")
or metadata_payload.get("series_sequence")
or metadata_payload.get("book_number")
)
if series_index_value is not None:
series_index_text = str(series_index_value).strip()
if series_index_text:
metadata_overrides.setdefault("series_index", series_index_text)
metadata_overrides.setdefault("series_position", series_index_text)
metadata_overrides.setdefault("series_sequence", series_index_text)
metadata_overrides.setdefault("book_number", series_index_text)
tags_value = metadata_payload.get("tags") or metadata_payload.get("keywords")
if tags_value:
tags_text = _stringify_metadata_value(tags_value)
if tags_text:
metadata_overrides.setdefault("tags", tags_text)
metadata_overrides.setdefault("keywords", tags_text)
metadata_overrides.setdefault("genre", tags_text)
description_value = metadata_payload.get("description") or metadata_payload.get("summary")
if description_value:
description_text = _stringify_metadata_value(description_value)
if description_text:
metadata_overrides.setdefault("description", description_text)
metadata_overrides.setdefault("summary", description_text)
subtitle_value = (
metadata_payload.get("subtitle")
or metadata_payload.get("sub_title")
or metadata_payload.get("calibre_subtitle")
)
if subtitle_value:
subtitle_text = _stringify_metadata_value(subtitle_value)
if subtitle_text:
metadata_overrides.setdefault("subtitle", subtitle_text)
publisher_value = metadata_payload.get("publisher")
if publisher_value:
publisher_text = _stringify_metadata_value(publisher_value)
if publisher_text:
metadata_overrides.setdefault("publisher", publisher_text)
# Author mapping: Abogen templates look for either 'authors' or 'author'.
authors_value = (
metadata_payload.get("authors")
or metadata_payload.get("author")
or metadata_payload.get("creator")
or metadata_payload.get("dc_creator")
)
if authors_value:
authors_text = _stringify_metadata_value(authors_value)
if authors_text:
metadata_overrides.setdefault("authors", authors_text)
metadata_overrides.setdefault("author", authors_text)
return metadata_overrides
@api_bp.get("/integrations/calibre-opds/feed") @api_bp.get("/integrations/calibre-opds/feed")
def api_calibre_opds_feed() -> ResponseReturnValue: def api_calibre_opds_feed() -> ResponseReturnValue:
+43 -76
View File
@@ -8,8 +8,8 @@ from flask.typing import ResponseReturnValue
from abogen.webui.service import ( from abogen.webui.service import (
JobStatus, JobStatus,
load_audiobookshelf_chapters,
build_audiobookshelf_metadata, build_audiobookshelf_metadata,
load_audiobookshelf_chapters,
) )
from abogen.webui.routes.utils.service import get_service from abogen.webui.routes.utils.service import get_service
from abogen.webui.routes.utils.form import render_jobs_panel from abogen.webui.routes.utils.form import render_jobs_panel
@@ -22,15 +22,22 @@ from abogen.webui.routes.utils.epub import (
from abogen.webui.routes.utils.settings import ( from abogen.webui.routes.utils.settings import (
stored_integration_config, stored_integration_config,
build_audiobookshelf_config, build_audiobookshelf_config,
coerce_bool,
) )
from abogen.webui.routes.utils.common import existing_paths from abogen.webui.routes.utils.common import existing_paths
from abogen.integrations.audiobookshelf import AudiobookshelfClient, AudiobookshelfUploadError from abogen.infrastructure.exporters import ExportService
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
jobs_bp = Blueprint("jobs", __name__) jobs_bp = Blueprint("jobs", __name__)
def _resolve_cover(job: Any, config: Any) -> Optional[Path]:
"""Resolve cover image path if enabled."""
if not config.send_cover or not job.cover_image_path:
return None
cover = job.cover_image_path if isinstance(job.cover_image_path, Path) else Path(str(job.cover_image_path))
return cover if cover.exists() else None
@jobs_bp.get("/<job_id>") @jobs_bp.get("/<job_id>")
def job_detail(job_id: str) -> ResponseReturnValue: def job_detail(job_id: str) -> ResponseReturnValue:
job = get_service().get_job(job_id) job = get_service().get_job(job_id)
@@ -98,24 +105,18 @@ def send_job_to_audiobookshelf(job_id: str) -> ResponseReturnValue:
return _panel_response() return _panel_response()
settings = stored_integration_config("audiobookshelf") settings = stored_integration_config("audiobookshelf")
if not settings or not coerce_bool(settings.get("enabled"), False): if not settings or not settings.get("enabled"):
job.add_log("Audiobookshelf upload skipped: integration is disabled.", level="warning") job.add_log("Audiobookshelf upload skipped: integration is disabled.", level="warning")
service._persist_state() service._persist_state()
return _panel_response() return _panel_response()
config = build_audiobookshelf_config(settings) config = build_audiobookshelf_config(settings)
if config is None: if config is None:
job.add_log( job.add_log("Audiobookshelf upload skipped: configure base URL, API token, and library ID first.", level="warning")
"Audiobookshelf upload skipped: configure base URL, API token, and library ID first.",
level="warning",
)
service._persist_state() service._persist_state()
return _panel_response() return _panel_response()
if not config.folder_id: if not config.folder_id:
job.add_log( job.add_log("Audiobookshelf upload skipped: enter the folder name or ID in the Audiobookshelf settings.", level="warning")
"Audiobookshelf upload skipped: enter the folder name or ID in the Audiobookshelf settings.",
level="warning",
)
service._persist_state() service._persist_state()
return _panel_response() return _panel_response()
@@ -125,83 +126,49 @@ def send_job_to_audiobookshelf(job_id: str) -> ResponseReturnValue:
service._persist_state() service._persist_state()
return _panel_response() return _panel_response()
cover_path = None
if config.send_cover and job.cover_image_path:
cover_candidate = job.cover_image_path
if not isinstance(cover_candidate, Path):
cover_candidate = Path(str(cover_candidate))
if cover_candidate.exists():
cover_path = cover_candidate
subtitles = existing_paths(job.result.subtitle_paths) if config.send_subtitles else None
chapters = load_audiobookshelf_chapters(job) if config.send_chapters else None
metadata = build_audiobookshelf_metadata(job)
display_title = metadata.get("title") or audio_path.stem
overwrite_requested = request.form.get("overwrite") == "true" or request.args.get("overwrite") == "true" overwrite_requested = request.form.get("overwrite") == "true" or request.args.get("overwrite") == "true"
try: if not overwrite_requested:
client = AudiobookshelfClient(config) from abogen.integrations.audiobookshelf import AudiobookshelfClient, AudiobookshelfUploadError
except ValueError as exc: metadata = build_audiobookshelf_metadata(job)
job.add_log(f"Audiobookshelf configuration error: {exc}", level="error") display_title = metadata.get("title") or audio_path.stem
service._persist_state()
return _panel_response()
try:
existing_items = client.find_existing_items(display_title, folder_id=config.folder_id)
except AudiobookshelfUploadError as exc:
job.add_log(f"Audiobookshelf lookup failed: {exc}", level="error")
service._persist_state()
return _panel_response()
if existing_items and not overwrite_requested:
job.add_log(
f"Audiobookshelf already contains '{display_title}'. Awaiting overwrite confirmation.",
level="warning",
)
service._persist_state()
if request.headers.get("HX-Request"):
detail = {
"jobId": job.id,
"title": display_title,
"url": url_for("jobs.send_job_to_audiobookshelf", job_id=job.id),
"target": request.headers.get("HX-Target") or "#jobs-panel",
"message": f'Audiobookshelf already contains "{display_title}". Overwrite?',
}
headers = {"HX-Trigger": json.dumps({"audiobookshelf-overwrite-prompt": detail})}
return Response("", status=204, headers=headers)
return _panel_response()
if existing_items and overwrite_requested:
try: try:
client.delete_items(existing_items) existing_items = AudiobookshelfClient(config).find_existing_items(display_title, folder_id=config.folder_id)
except AudiobookshelfUploadError as exc: except AudiobookshelfUploadError as exc:
job.add_log(f"Audiobookshelf overwrite aborted: {exc}", level="error") job.add_log(f"Audiobookshelf lookup failed: {exc}", level="error")
service._persist_state() service._persist_state()
return _panel_response() return _panel_response()
else: if existing_items:
job.add_log( job.add_log(f"Audiobookshelf already contains '{display_title}'. Awaiting overwrite confirmation.", level="warning")
f"Removed {len(existing_items)} existing Audiobookshelf item(s) prior to overwrite.", service._persist_state()
level="info", if request.headers.get("HX-Request"):
) detail = {
"jobId": job.id,
"title": display_title,
"url": url_for("jobs.send_job_to_audiobookshelf", job_id=job.id),
"target": request.headers.get("HX-Target") or "#jobs-panel",
"message": f'Audiobookshelf already contains "{display_title}". Overwrite?',
}
headers = {"HX-Trigger": json.dumps({"audiobookshelf-overwrite-prompt": detail})}
return Response("", status=204, headers=headers)
return _panel_response()
job.add_log("Audiobookshelf upload triggered manually.", level="info") job.add_log("Audiobookshelf upload triggered manually.", level="info")
export_svc = ExportService()
try: try:
client.upload_audiobook( export_svc.upload_audiobookshelf(
job,
audio_path, audio_path,
metadata=metadata, existing_paths(job.result.subtitle_paths),
cover_path=cover_path, load_audiobookshelf_chapters(job) if config.send_chapters else None,
chapters=chapters, build_audiobookshelf_metadata(job),
subtitles=subtitles, cover_path=_resolve_cover(job, config),
config=config,
log_callback=lambda msg, lvl="info": job.add_log(msg, level=lvl),
) )
except AudiobookshelfUploadError as exc:
job.add_log(f"Audiobookshelf upload failed: {exc}", level="error")
except Exception as exc: except Exception as exc:
job.add_log(f"Audiobookshelf integration error: {exc}", level="error") job.add_log(f"Audiobookshelf integration error: {exc}", level="error")
else: service._persist_state()
job.add_log("Audiobookshelf upload queued.", level="success")
finally:
service._persist_state()
return _panel_response() return _panel_response()
@jobs_bp.post("/clear-finished") @jobs_bp.post("/clear-finished")
+4 -153
View File
@@ -7,22 +7,17 @@ from flask import Blueprint, current_app, render_template, request, redirect, ur
from flask.typing import ResponseReturnValue from flask.typing import ResponseReturnValue
from abogen.webui.routes.utils.settings import ( from abogen.webui.routes.utils.settings import (
load_settings,
load_integration_settings, load_integration_settings,
load_settings,
save_settings, save_settings,
stored_integration_config,
coerce_bool,
coerce_int,
SAVE_MODE_LABELS, SAVE_MODE_LABELS,
llm_ready, llm_ready,
_NORMALIZATION_BOOLEAN_KEYS,
_NORMALIZATION_STRING_KEYS,
_DEFAULT_ANALYSIS_THRESHOLD,
) )
from abogen.webui.routes.utils.voice import template_options from abogen.webui.routes.utils.voice import template_options
from abogen.webui.services.settings_service import apply_form_to_settings
from abogen.webui.debug_tts_runner import run_debug_tts_wavs from abogen.webui.debug_tts_runner import run_debug_tts_wavs
from abogen.debug_tts_samples import DEBUG_TTS_SAMPLES from abogen.debug_tts_samples import DEBUG_TTS_SAMPLES
from abogen.utils import get_user_output_path, load_config from abogen.utils import get_user_output_path
settings_bp = Blueprint("settings", __name__) settings_bp = Blueprint("settings", __name__)
@@ -37,151 +32,7 @@ _NORMALIZATION_SAMPLES = {
@settings_bp.post("/update") @settings_bp.post("/update")
def update_settings() -> ResponseReturnValue: def update_settings() -> ResponseReturnValue:
current = load_settings() current = load_settings()
form = request.form apply_form_to_settings(current, request.form)
# General settings
current["language"] = (form.get("language") or "en").strip()
current["default_speaker"] = (form.get("default_speaker") or "").strip()
current["default_voice"] = (form.get("default_voice") or "").strip()
try:
current["supertonic_total_steps"] = max(2, min(15, int(form.get("supertonic_total_steps", current.get("supertonic_total_steps", 5)))))
except (TypeError, ValueError):
pass
try:
current["supertonic_speed"] = max(0.7, min(2.0, float(form.get("supertonic_speed", current.get("supertonic_speed", 1.0)))))
except (TypeError, ValueError):
pass
current["output_format"] = (form.get("output_format") or "mp3").strip()
current["subtitle_mode"] = (form.get("subtitle_mode") or "Disabled").strip()
current["subtitle_format"] = (form.get("subtitle_format") or "srt").strip()
current["save_mode"] = (form.get("save_mode") or "save_next_to_input").strip()
current["replace_single_newlines"] = coerce_bool(form.get("replace_single_newlines"), False)
current["use_gpu"] = coerce_bool(form.get("use_gpu"), False)
current["save_chapters_separately"] = coerce_bool(form.get("save_chapters_separately"), False)
current["merge_chapters_at_end"] = coerce_bool(form.get("merge_chapters_at_end"), True)
current["save_as_project"] = coerce_bool(form.get("save_as_project"), False)
current["separate_chapters_format"] = (form.get("separate_chapters_format") or "wav").strip()
try:
current["silence_between_chapters"] = max(0.0, float(form.get("silence_between_chapters", 2.0)))
except ValueError:
pass
try:
current["chapter_intro_delay"] = max(0.0, float(form.get("chapter_intro_delay", 0.5)))
except ValueError:
pass
current["read_title_intro"] = coerce_bool(form.get("read_title_intro"), False)
current["read_closing_outro"] = coerce_bool(form.get("read_closing_outro"), True)
current["normalize_chapter_opening_caps"] = coerce_bool(form.get("normalize_chapter_opening_caps"), True)
current["auto_prefix_chapter_titles"] = coerce_bool(form.get("auto_prefix_chapter_titles"), True)
try:
current["max_subtitle_words"] = max(1, int(form.get("max_subtitle_words", 50)))
except ValueError:
pass
current["chunk_level"] = (form.get("chunk_level") or "paragraph").strip()
current["generate_epub3"] = coerce_bool(form.get("generate_epub3"), False)
current["speaker_analysis_threshold"] = coerce_int(
form.get("speaker_analysis_threshold"),
_DEFAULT_ANALYSIS_THRESHOLD,
minimum=1,
maximum=25,
)
def _extract_checkbox(name: str, default: bool) -> bool:
values = form.getlist(name) if hasattr(form, "getlist") else []
if values:
return coerce_bool(values[-1], default)
if hasattr(form, "__contains__") and name in form:
return False
return default
# Normalization settings
for key in _NORMALIZATION_BOOLEAN_KEYS:
current[key] = _extract_checkbox(key, bool(current.get(key, True)))
for key in _NORMALIZATION_STRING_KEYS:
if hasattr(form, "__contains__") and key in form:
current[key] = (form.get(key) or "").strip()
# Integrations
# `load_settings()` returns only the general settings subset and intentionally
# does not include stored integrations. Seed them from the stored config so
# saving unrelated settings cannot wipe credentials/tokens.
current_integrations: dict[str, dict[str, Any]] = {}
cfg = load_config() or {}
stored_integrations = cfg.get("integrations")
if isinstance(stored_integrations, Mapping):
for name, payload in stored_integrations.items():
if isinstance(name, str) and isinstance(payload, Mapping):
current_integrations[name] = dict(payload)
# Ensure known integrations are loaded even if the config is still in legacy format.
for name in ("audiobookshelf", "calibre_opds"):
stored = stored_integration_config(name)
if stored and name not in current_integrations:
current_integrations[name] = dict(stored)
current["integrations"] = current_integrations
# Audiobookshelf
abs_enabled = coerce_bool(form.get("audiobookshelf_enabled"), False)
abs_url = (form.get("audiobookshelf_base_url") or "").strip()
abs_token = (form.get("audiobookshelf_api_token") or "").strip()
abs_library = (form.get("audiobookshelf_library_id") or "").strip()
abs_folder = (form.get("audiobookshelf_folder_id") or "").strip()
abs_verify = coerce_bool(form.get("audiobookshelf_verify_ssl"), True)
abs_auto_send = coerce_bool(form.get("audiobookshelf_auto_send"), False)
abs_cover = coerce_bool(form.get("audiobookshelf_send_cover"), True)
abs_chapters = coerce_bool(form.get("audiobookshelf_send_chapters"), True)
abs_subtitles = coerce_bool(form.get("audiobookshelf_send_subtitles"), False)
try:
abs_timeout = max(1.0, float(form.get("audiobookshelf_timeout", 30.0)))
except ValueError:
abs_timeout = 30.0
# Preserve existing token if not provided and not cleared
if not abs_token and not coerce_bool(form.get("audiobookshelf_api_token_clear"), False):
existing_abs = current["integrations"].get("audiobookshelf", {})
abs_token = existing_abs.get("api_token", "")
current["integrations"]["audiobookshelf"] = {
"enabled": abs_enabled,
"base_url": abs_url,
"api_token": abs_token,
"library_id": abs_library,
"folder_id": abs_folder,
"verify_ssl": abs_verify,
"auto_send": abs_auto_send,
"send_cover": abs_cover,
"send_chapters": abs_chapters,
"send_subtitles": abs_subtitles,
"timeout": abs_timeout,
}
# Calibre OPDS
calibre_enabled = coerce_bool(form.get("calibre_opds_enabled"), False)
calibre_url = (form.get("calibre_opds_base_url") or "").strip()
calibre_user = (form.get("calibre_opds_username") or "").strip()
calibre_pass = (form.get("calibre_opds_password") or "").strip()
calibre_verify = coerce_bool(form.get("calibre_opds_verify_ssl"), True)
# Preserve existing password if not provided and not cleared
if not calibre_pass and not coerce_bool(form.get("calibre_opds_password_clear"), False):
existing_calibre = current["integrations"].get("calibre_opds", {})
calibre_pass = existing_calibre.get("password", "")
current["integrations"]["calibre_opds"] = {
"enabled": calibre_enabled,
"base_url": calibre_url,
"username": calibre_user,
"password": calibre_pass,
"verify_ssl": calibre_verify,
}
save_settings(current) save_settings(current)
flash("Settings updated successfully.", "success") flash("Settings updated successfully.", "success")
return redirect(url_for("settings.settings_page")) return redirect(url_for("settings.settings_page"))
+26 -12
View File
@@ -1,24 +1,38 @@
from typing import Any, Optional, Tuple, Iterable, List from typing import Any, Optional, Tuple, Iterable, List, Mapping
from pathlib import Path from pathlib import Path
def split_profile_spec(value: Any) -> Tuple[str, Optional[str]]: from abogen.domain.settings_core import coerce_bool, split_profile_spec # noqa: F401
text = str(value or "").strip()
if not text:
return "", None
lowered = text.lower()
if lowered.startswith("profile:") or lowered.startswith("speaker:"):
_, _, remainder = text.partition(":")
name = remainder.strip()
return "", name or None
return text, None
def split_speaker_spec(value: Any) -> Tuple[str, Optional[str]]: def split_speaker_spec(value: Any) -> Tuple[str, Optional[str]]:
"""Preferred alias for split_profile_spec (supports 'speaker:' and legacy 'profile:').""" """Preferred alias for split_profile_spec (supports 'speaker:' and legacy 'profile:')."""
return split_profile_spec(value) return split_profile_spec(value)
def existing_paths(paths: Optional[Iterable[Path]]) -> List[Path]: def existing_paths(paths: Optional[Iterable[Path]]) -> List[Path]:
if not paths: if not paths:
return [] return []
return [p for p in paths if p.exists()] return [p for p in paths if p.exists()]
def extract_checkbox(form: Mapping[str, Any], name: str, default: bool) -> bool:
"""Extract a boolean checkbox value from a form-like mapping.
Handles both multi-value forms (Flask's `getlist`) and simple mappings.
If the checkbox name is present but has no value, it means unchecked (False).
"""
values: List[str] = []
getter = getattr(form, "getlist", None)
if callable(getter):
raw_values = getter(name)
if raw_values:
values = list(raw_values)
else:
raw_flag = form.get(name)
if raw_flag is not None:
values = [raw_flag]
if values:
return coerce_bool(values[-1], default)
if name in form:
return False
return default
+11 -141
View File
@@ -1,12 +1,17 @@
import re
import time import time
import uuid import uuid
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
from flask import request, render_template, jsonify from flask import request, render_template, jsonify
from flask.typing import ResponseReturnValue from flask.typing import ResponseReturnValue
from abogen.domain.chapter_classification import (
supplement_score,
should_preselect_chapter,
ensure_at_least_one_chapter_enabled,
)
from abogen.webui.service import PendingJob, JobStatus from abogen.webui.service import PendingJob, JobStatus
from abogen.webui.routes.utils.service import get_service from abogen.webui.routes.utils.service import get_service
from abogen.tts_plugin.utils import is_plugin_registered
from abogen.webui.routes.utils.settings import ( from abogen.webui.routes.utils.settings import (
load_settings, load_settings,
coerce_bool, coerce_bool,
@@ -28,11 +33,11 @@ from abogen.webui.routes.utils.voice import (
) )
from abogen.webui.routes.utils.entity import sync_pronunciation_overrides from abogen.webui.routes.utils.entity import sync_pronunciation_overrides
from abogen.webui.routes.utils.epub import job_download_flags from abogen.webui.routes.utils.epub import job_download_flags
from abogen.webui.routes.utils.common import split_profile_spec from abogen.webui.routes.utils.common import split_profile_spec, extract_checkbox
from abogen.utils import calculate_text_length from abogen.utils import calculate_text_length
from abogen.voice_profiles import serialize_profiles, normalize_profile_entry from abogen.voice_profiles import serialize_profiles, normalize_profile_entry
from abogen.chunking import ChunkLevel, build_chunks_for_chapters from abogen.chunking import ChunkLevel, build_chunks_for_chapters
from abogen.tts_backend_registry import get_default_voice from abogen.tts_plugin.utils import get_default_voice
from abogen.speaker_configs import get_config from abogen.speaker_configs import get_config
from abogen.kokoro_text_normalization import normalize_roman_numeral_titles from abogen.kokoro_text_normalization import normalize_roman_numeral_titles
from dataclasses import dataclass from dataclasses import dataclass
@@ -65,109 +70,6 @@ _WIZARD_STEP_META = {
}, },
} }
_SUPPLEMENT_TITLE_PATTERNS: List[tuple[re.Pattern[str], float]] = [
(re.compile(r"\btitle\s+page\b"), 3.0),
(re.compile(r"\bcopyright\b"), 2.4),
(re.compile(r"\btable\s+of\s+contents\b"), 2.8),
(re.compile(r"\bcontents\b"), 2.0),
(re.compile(r"\backnowledg(e)?ments?\b"), 2.0),
(re.compile(r"\bdedication\b"), 2.0),
(re.compile(r"\babout\s+the\s+author(s)?\b"), 2.4),
(re.compile(r"\balso\s+by\b"), 2.0),
(re.compile(r"\bpraise\s+for\b"), 2.0),
(re.compile(r"\bcolophon\b"), 2.2),
(re.compile(r"\bpublication\s+data\b"), 2.2),
(re.compile(r"\btranscriber'?s?\s+note\b"), 2.2),
(re.compile(r"\bglossary\b"), 2.0),
(re.compile(r"\bindex\b"), 2.0),
(re.compile(r"\bbibliograph(y|ies)\b"), 2.0),
(re.compile(r"\breferences\b"), 1.8),
(re.compile(r"\bappendix\b"), 1.9),
]
_CONTENT_TITLE_PATTERNS: List[re.Pattern[str]] = [
re.compile(r"\bchapter\b"),
re.compile(r"\bbook\b"),
re.compile(r"\bpart\b"),
re.compile(r"\bsection\b"),
re.compile(r"\bscene\b"),
re.compile(r"\bprologue\b"),
re.compile(r"\bepilogue\b"),
re.compile(r"\bintroduction\b"),
re.compile(r"\bstory\b"),
]
_SUPPLEMENT_TEXT_KEYWORDS: List[tuple[str, float]] = [
("copyright", 1.2),
("all rights reserved", 1.1),
("isbn", 0.9),
("library of congress", 1.0),
("table of contents", 1.0),
("dedicated to", 0.8),
("acknowledg", 0.8),
("printed in", 0.6),
("permission", 0.6),
("publisher", 0.5),
("praise for", 0.9),
("also by", 0.9),
("glossary", 0.8),
("index", 0.8),
("newsletter", 3.2),
("mailing list", 2.6),
("sign-up", 2.2),
]
def supplement_score(title: str, text: str, index: int) -> float:
normalized_title = (title or "").lower()
score = 0.0
for pattern, weight in _SUPPLEMENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score += weight
for pattern in _CONTENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score -= 2.0
stripped_text = (text or "").strip()
length = len(stripped_text)
if length <= 150:
score += 0.9
elif length <= 400:
score += 0.6
elif length <= 800:
score += 0.35
lowercase_text = stripped_text.lower()
for keyword, weight in _SUPPLEMENT_TEXT_KEYWORDS:
if keyword in lowercase_text:
score += weight
if index == 0 and score > 0:
score += 0.25
return score
def should_preselect_chapter(
title: str,
text: str,
index: int,
total_count: int,
) -> bool:
if total_count <= 1:
return True
score = supplement_score(title, text, index)
return score < 1.9
def ensure_at_least_one_chapter_enabled(chapters: List[Dict[str, Any]]) -> None:
if not chapters:
return
if any(chapter.get("enabled") for chapter in chapters):
return
best_index = max(range(len(chapters)), key=lambda idx: chapters[idx].get("characters", 0))
chapters[best_index]["enabled"] = True
def apply_prepare_form( def apply_prepare_form(
pending: PendingJob, form: Mapping[str, Any] pending: PendingJob, form: Mapping[str, Any]
@@ -536,28 +438,11 @@ def apply_book_step_form(
else: else:
pending.normalize_chapter_opening_caps = caps_default pending.normalize_chapter_opening_caps = caps_default
def _extract_checkbox(name: str, default: bool) -> bool:
values: List[str] = []
getter = getattr(form, "getlist", None)
if callable(getter):
raw_values = getter(name)
if raw_values:
values = list(cast(Iterable[str], raw_values))
else:
raw_flag = form.get(name)
if raw_flag is not None:
values = [raw_flag]
if values:
return coerce_bool(values[-1], default)
if hasattr(form, "__contains__") and name in form:
return False
return default
overrides_existing = getattr(pending, "normalization_overrides", None) overrides_existing = getattr(pending, "normalization_overrides", None)
overrides: Dict[str, Any] = dict(overrides_existing or {}) overrides: Dict[str, Any] = dict(overrides_existing or {})
for key in _NORMALIZATION_BOOLEAN_KEYS: for key in _NORMALIZATION_BOOLEAN_KEYS:
default_toggle = overrides.get(key, bool(settings.get(key, True))) default_toggle = overrides.get(key, bool(settings.get(key, True)))
overrides[key] = _extract_checkbox(key, default_toggle) overrides[key] = extract_checkbox(form, key, default_toggle)
for key in _NORMALIZATION_STRING_KEYS: for key in _NORMALIZATION_STRING_KEYS:
default_val = overrides.get(key, str(settings.get(key, ""))) default_val = overrides.get(key, str(settings.get(key, "")))
val = form.get(key) val = form.get(key)
@@ -579,7 +464,7 @@ def apply_book_step_form(
# spec (e.g. "speaker:Name" for saved speakers, or a Kokoro mix formula). # spec (e.g. "speaker:Name" for saved speakers, or a Kokoro mix formula).
# This enables mixed-provider conversions (e.g. narrator=SuperTonic, characters=Kokoro). # This enables mixed-provider conversions (e.g. narrator=SuperTonic, characters=Kokoro).
provider_value = str(form.get("tts_provider") or "").strip().lower() provider_value = str(form.get("tts_provider") or "").strip().lower()
if provider_value in {"kokoro", "supertonic"}: if is_plugin_registered(provider_value):
pending.tts_provider = provider_value pending.tts_provider = provider_value
# Determine the base speaker selection (saved speaker ref or raw voice). # Determine the base speaker selection (saved speaker ref or raw voice).
@@ -885,25 +770,10 @@ def build_pending_job_from_extraction(
apply_config=bool(speaker_config_payload), apply_config=bool(speaker_config_payload),
) )
def _extract_checkbox(name: str, default: bool) -> bool:
values: List[str] = []
getter = getattr(form, "getlist", None)
if callable(getter):
raw_values = getter(name)
if raw_values:
values = list(cast(Iterable[str], raw_values))
else:
raw_flag = form.get(name)
if raw_flag is not None:
values = [raw_flag]
if values:
return coerce_bool(values[-1], default)
return default
normalization_overrides = {} normalization_overrides = {}
for key in _NORMALIZATION_BOOLEAN_KEYS: for key in _NORMALIZATION_BOOLEAN_KEYS:
default_val = bool(settings.get(key, True)) default_val = bool(settings.get(key, True))
normalization_overrides[key] = _extract_checkbox(key, default_val) normalization_overrides[key] = extract_checkbox(form, key, default_val)
for key in _NORMALIZATION_STRING_KEYS: for key in _NORMALIZATION_STRING_KEYS:
default_val = str(settings.get(key, "")) default_val = str(settings.get(key, ""))
+25 -339
View File
@@ -1,108 +1,24 @@
import os import os
import re
from typing import Any, Dict, Mapping, Optional from typing import Any, Dict, Mapping, Optional
from abogen.constants import (
LANGUAGE_DESCRIPTIONS,
SUBTITLE_FORMATS,
SUPPORTED_SOUND_FORMATS,
)
from abogen.tts_backend_registry import get_default_voice
from abogen.normalization_settings import (
DEFAULT_LLM_PROMPT,
environment_llm_defaults,
)
from abogen.utils import load_config, save_config
from abogen.integrations.calibre_opds import CalibreOPDSClient from abogen.integrations.calibre_opds import CalibreOPDSClient
from abogen.integrations.audiobookshelf import AudiobookshelfConfig from abogen.integrations.audiobookshelf import AudiobookshelfConfig
from abogen.webui.routes.utils.common import split_profile_spec from abogen.utils import load_config, save_config
from abogen.domain.settings_core import (
SAVE_MODE_LABELS = { CHUNK_LEVEL_OPTIONS,
"save_next_to_input": "Save next to input file", CHUNK_LEVEL_VALUES,
"save_to_desktop": "Save to Desktop", DEFAULT_ANALYSIS_THRESHOLD,
"choose_output_folder": "Choose output folder", SAVE_MODE_LABELS,
"default_output": "Use default save location", _NORMALIZATION_BOOLEAN_KEYS,
} _NORMALIZATION_STRING_KEYS,
coerce_bool,
LEGACY_SAVE_MODE_MAP = {label: key for key, label in SAVE_MODE_LABELS.items()} coerce_float,
coerce_int,
_CHUNK_LEVEL_OPTIONS = [ integration_defaults,
{"value": "paragraph", "label": "Paragraphs"}, load_settings,
{"value": "sentence", "label": "Sentences"}, llm_ready,
] settings_defaults,
)
_CHUNK_LEVEL_VALUES = {option["value"] for option in _CHUNK_LEVEL_OPTIONS}
_DEFAULT_ANALYSIS_THRESHOLD = 3
_APOSTROPHE_MODE_OPTIONS = [
{"value": "off", "label": "Off"},
{"value": "spacy", "label": "spaCy (built-in)"},
{"value": "llm", "label": "LLM assisted"},
]
_NORMALIZATION_BOOLEAN_KEYS = {
"normalization_numbers",
"normalization_titles",
"normalization_terminal",
"normalization_phoneme_hints",
"normalization_caps_quotes",
"normalization_currency",
"normalization_footnotes",
"normalization_internet_slang",
"normalization_apostrophes_contractions",
"normalization_apostrophes_plural_possessives",
"normalization_apostrophes_sibilant_possessives",
"normalization_apostrophes_decades",
"normalization_apostrophes_leading_elisions",
"normalization_contraction_aux_be",
"normalization_contraction_aux_have",
"normalization_contraction_modal_will",
"normalization_contraction_modal_would",
"normalization_contraction_negation_not",
"normalization_contraction_let_us",
}
_NORMALIZATION_STRING_KEYS = {
"normalization_numbers_year_style",
"normalization_apostrophe_mode",
}
BOOLEAN_SETTINGS = {
"replace_single_newlines",
"use_gpu",
"save_chapters_separately",
"merge_chapters_at_end",
"save_as_project",
"generate_epub3",
"enable_entity_recognition",
"read_title_intro",
"read_closing_outro",
"auto_prefix_chapter_titles",
"normalize_chapter_opening_caps",
"normalization_numbers",
"normalization_titles",
"normalization_terminal",
"normalization_phoneme_hints",
"normalization_caps_quotes",
"normalization_currency",
"normalization_footnotes",
"normalization_internet_slang",
"normalization_apostrophes_contractions",
"normalization_apostrophes_plural_possessives",
"normalization_apostrophes_sibilant_possessives",
"normalization_apostrophes_decades",
"normalization_apostrophes_leading_elisions",
"normalization_contraction_aux_be",
"normalization_contraction_aux_have",
"normalization_contraction_modal_will",
"normalization_contraction_modal_would",
"normalization_contraction_negation_not",
"normalization_contraction_let_us",
}
FLOAT_SETTINGS = {"silence_between_chapters", "chapter_intro_delay", "llm_timeout"}
INT_SETTINGS = {"max_subtitle_words", "speaker_analysis_threshold"}
_NORMALIZATION_GROUPS = [ _NORMALIZATION_GROUPS = [
{ {
@@ -136,246 +52,16 @@ _NORMALIZATION_GROUPS = [
} }
] ]
_APOSTROPHE_MODE_OPTIONS = [
{"value": "off", "label": "Off"},
{"value": "spacy", "label": "spaCy (built-in)"},
{"value": "llm", "label": "LLM assisted"},
]
def integration_defaults() -> Dict[str, Dict[str, Any]]: # Backward-compatible aliases for modules still referencing old underscore-prefixed names
return { _DEFAULT_ANALYSIS_THRESHOLD = DEFAULT_ANALYSIS_THRESHOLD
"calibre_opds": { _CHUNK_LEVEL_OPTIONS = CHUNK_LEVEL_OPTIONS
"enabled": False, _CHUNK_LEVEL_VALUES = CHUNK_LEVEL_VALUES
"base_url": "",
"username": "",
"password": "",
"verify_ssl": True,
},
"audiobookshelf": {
"enabled": False,
"base_url": "",
"api_token": "",
"library_id": "",
"collection_id": "",
"folder_id": "",
"verify_ssl": True,
"send_cover": True,
"send_chapters": True,
"send_subtitles": False,
"auto_send": False,
"timeout": 30.0,
},
}
def has_output_override() -> bool:
return bool(os.environ.get("ABOGEN_OUTPUT_DIR") or os.environ.get("ABOGEN_OUTPUT_ROOT"))
def settings_defaults() -> Dict[str, Any]:
llm_env_defaults = environment_llm_defaults()
return {
"output_format": "wav",
"subtitle_format": "srt",
"save_mode": "default_output" if has_output_override() else "save_next_to_input",
"default_speaker": "",
"default_voice": get_default_voice("kokoro"),
"supertonic_total_steps": 5,
"supertonic_speed": 1.0,
"replace_single_newlines": False,
"use_gpu": True,
"save_chapters_separately": False,
"merge_chapters_at_end": True,
"save_as_project": False,
"separate_chapters_format": "wav",
"silence_between_chapters": 2.0,
"chapter_intro_delay": 0.5,
"read_title_intro": False,
"read_closing_outro": True,
"normalize_chapter_opening_caps": True,
"max_subtitle_words": 50,
"chunk_level": "paragraph",
"enable_entity_recognition": True,
"generate_epub3": False,
"auto_prefix_chapter_titles": True,
"speaker_analysis_threshold": _DEFAULT_ANALYSIS_THRESHOLD,
"speaker_pronunciation_sentence": "This is {{name}} speaking.",
"speaker_random_languages": [],
"llm_base_url": llm_env_defaults.get("llm_base_url", ""),
"llm_api_key": llm_env_defaults.get("llm_api_key", ""),
"llm_model": llm_env_defaults.get("llm_model", ""),
"llm_timeout": llm_env_defaults.get("llm_timeout", 30.0),
"llm_prompt": llm_env_defaults.get("llm_prompt", DEFAULT_LLM_PROMPT),
"llm_context_mode": llm_env_defaults.get("llm_context_mode", "sentence"),
"normalization_numbers": True,
"normalization_currency": True,
"normalization_footnotes": True,
"normalization_titles": True,
"normalization_terminal": True,
"normalization_phoneme_hints": True,
"normalization_caps_quotes": True,
"normalization_internet_slang": False,
"normalization_apostrophes_contractions": True,
"normalization_apostrophes_plural_possessives": True,
"normalization_apostrophes_sibilant_possessives": True,
"normalization_apostrophes_decades": True,
"normalization_apostrophes_leading_elisions": True,
"normalization_apostrophe_mode": "spacy",
"normalization_numbers_year_style": "american",
"normalization_contraction_aux_be": True,
"normalization_contraction_aux_have": True,
"normalization_contraction_modal_will": True,
"normalization_contraction_modal_would": True,
"normalization_contraction_negation_not": True,
"normalization_contraction_let_us": True,
}
def llm_ready(settings: Mapping[str, Any]) -> bool:
base_url = str(settings.get("llm_base_url") or "").strip()
return bool(base_url)
_PROMPT_TOKEN_RE = re.compile(r"{{\s*([a-zA-Z0-9_]+)\s*}}")
def render_prompt_template(template: str, context: Mapping[str, str]) -> str:
if not template:
return ""
def _replace(match: re.Match[str]) -> str:
key = match.group(1)
return context.get(key, "")
return _PROMPT_TOKEN_RE.sub(_replace, template)
def coerce_bool(value: Any, default: bool) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() in {"true", "1", "yes", "on"}
if value is None:
return default
return bool(value)
def coerce_float(value: Any, default: float) -> float:
try:
return max(0.0, float(value))
except (TypeError, ValueError):
return default
def coerce_int(value: Any, default: int, *, minimum: int = 1, maximum: int = 200) -> int:
try:
parsed = int(value)
except (TypeError, ValueError):
return default
return max(minimum, min(parsed, maximum))
def normalize_save_mode(value: Any, default: str) -> str:
if isinstance(value, str):
if value in SAVE_MODE_LABELS:
return value
if value in LEGACY_SAVE_MODE_MAP:
return LEGACY_SAVE_MODE_MAP[value]
return default
def normalize_setting_value(key: str, value: Any, defaults: Dict[str, Any]) -> Any:
if key in BOOLEAN_SETTINGS:
return coerce_bool(value, defaults[key])
if key in FLOAT_SETTINGS:
return coerce_float(value, defaults[key])
if key in INT_SETTINGS:
return coerce_int(value, defaults[key])
if key == "save_mode":
return normalize_save_mode(value, defaults[key])
if key == "output_format":
return value if value in SUPPORTED_SOUND_FORMATS else defaults[key]
if key == "subtitle_format":
valid = {item[0] for item in SUBTITLE_FORMATS}
return value if value in valid else defaults[key]
if key == "separate_chapters_format":
if isinstance(value, str):
normalized = value.lower()
if normalized in {"wav", "flac", "mp3", "opus"}:
return normalized
return defaults[key]
if key == "default_voice":
if isinstance(value, str):
text = value.strip()
if not text:
return defaults[key]
spec, profile_name = split_profile_spec(text)
if profile_name:
return f"speaker:{profile_name}"
return spec
return defaults[key]
if key == "default_speaker":
if isinstance(value, str):
text = value.strip()
if not text:
return ""
spec, profile_name = split_profile_spec(text)
if profile_name:
return f"speaker:{profile_name}"
return spec
return ""
if key == "chunk_level":
if isinstance(value, str) and value in _CHUNK_LEVEL_VALUES:
return value
return defaults[key]
if key == "normalization_apostrophe_mode":
if isinstance(value, str):
normalized_mode = value.strip().lower()
if normalized_mode in {"off", "spacy", "llm"}:
return normalized_mode
return defaults[key]
if key == "normalization_numbers_year_style":
if isinstance(value, str):
normalized_style = value.strip().lower()
if normalized_style in {"american", "off"}:
return normalized_style
return defaults[key]
if key == "llm_context_mode":
if isinstance(value, str):
normalized_scope = value.strip().lower()
if normalized_scope == "sentence":
return normalized_scope
return defaults[key]
if key == "llm_prompt":
candidate = str(value or "").strip()
return candidate if candidate else defaults[key]
if key in {"llm_base_url", "llm_api_key", "llm_model"}:
return str(value or "").strip()
if key == "speaker_random_languages":
if isinstance(value, (list, tuple, set)):
return [code for code in value if isinstance(code, str) and code in LANGUAGE_DESCRIPTIONS]
if isinstance(value, str):
parts = [item.strip().lower() for item in value.split(",") if item.strip()]
return [code for code in parts if code in LANGUAGE_DESCRIPTIONS]
return defaults.get(key, [])
if key == "supertonic_total_steps":
try:
steps = int(value)
except (TypeError, ValueError):
return defaults.get(key, 5)
return max(2, min(15, steps))
if key == "supertonic_speed":
try:
speed = float(value)
except (TypeError, ValueError):
return defaults.get(key, 1.0)
return max(0.7, min(2.0, speed))
return value if value is not None else defaults.get(key)
def load_settings() -> Dict[str, Any]:
defaults = settings_defaults()
cfg = load_config() or {}
settings: Dict[str, Any] = {}
for key, default in defaults.items():
raw_value = cfg.get(key, default)
settings[key] = normalize_setting_value(key, raw_value, defaults)
return settings
def load_integration_settings() -> Dict[str, Dict[str, Any]]: def load_integration_settings() -> Dict[str, Dict[str, Any]]:
@@ -6,37 +6,39 @@ import soundfile as sf
from flask import current_app, send_file from flask import current_app, send_file
from flask.typing import ResponseReturnValue from flask.typing import ResponseReturnValue
from abogen.domain.device import select_device as _select_device
from abogen.domain.enums import Language
from abogen.domain.split_pattern import get_split_pattern
# Kokoro-specific language mapping (engine's responsibility)
_KOKORO_LANG_MAP = {
Language.EN_US: "a",
Language.EN_GB: "b",
Language.ES: "e",
Language.FR: "f",
Language.HI: "h",
Language.IT: "i",
Language.JA: "j",
Language.PT_BR: "p",
Language.ZH: "z",
}
SPLIT_PATTERN = r"\n+"
SAMPLE_RATE = 24000 SAMPLE_RATE = 24000
_preview_pipelines: Dict[Tuple[str, str], Any] = {} _preview_pipelines: Dict[Tuple[str, str], Any] = {}
_preview_pipeline_lock = threading.Lock() _preview_pipeline_lock = threading.Lock()
def _select_device() -> str: def clear_preview_pipelines() -> None:
import platform """Dispose all cached preview pipelines and clear the cache."""
with _preview_pipeline_lock:
try: for pipeline in _preview_pipelines.values():
import torch # type: ignore[import-not-found] try:
except Exception: pipeline.dispose()
return "cpu" except Exception:
pass
system = platform.system() _preview_pipelines.clear()
if system == "Darwin" and platform.processor() == "arm":
try:
if torch.backends.mps.is_available():
return "mps"
except Exception:
pass
return "cpu"
try:
if torch.cuda.is_available():
return "cuda"
except Exception:
pass
return "cpu"
def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]: def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]:
@@ -56,31 +58,22 @@ def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]:
raise RuntimeError("Preview pipeline is unavailable") from last_error raise RuntimeError("Preview pipeline is unavailable") from last_error
def _to_float32(audio_segment) -> np.ndarray:
if audio_segment is None:
return np.zeros(0, dtype="float32")
tensor = audio_segment
if hasattr(tensor, "detach"):
tensor = tensor.detach()
if hasattr(tensor, "cpu"):
try:
tensor = tensor.cpu()
except Exception:
pass
if hasattr(tensor, "numpy"):
return np.asarray(tensor.numpy(), dtype="float32").reshape(-1)
return np.asarray(tensor, dtype="float32").reshape(-1)
def get_preview_pipeline(language: str, device: str) -> Any: def get_preview_pipeline(language: str, device: str) -> Any:
key = (language, device) # Convert Language enum to Kokoro single-letter code
try:
lang = Language.from_str(language) if not isinstance(language, Language) else language
except ValueError:
lang = Language.EN_US
kokoro_code = _KOKORO_LANG_MAP.get(lang, "a")
key = (kokoro_code, device)
with _preview_pipeline_lock: with _preview_pipeline_lock:
pipeline = _preview_pipelines.get(key) pipeline = _preview_pipelines.get(key)
if pipeline is not None: if pipeline is not None:
return pipeline return pipeline
from abogen.tts_backend_registry import create_backend from abogen.tts_plugin.utils import create_pipeline
pipeline = create_backend("kokoro", lang_code=language, device=device) pipeline = create_pipeline("kokoro", lang_code=kokoro_code, device=device)
_preview_pipelines[key] = pipeline _preview_pipelines[key] = pipeline
return pipeline return pipeline
@@ -135,15 +128,17 @@ def generate_preview_audio(
current_app.logger.exception("Preview normalization failed; using raw text") current_app.logger.exception("Preview normalization failed; using raw text")
normalized_text = source_text normalized_text = source_text
if provider == "supertonic": preview_split = get_split_pattern(str(language or "a"), "Disabled")
from abogen.tts_backend_registry import create_backend
pipeline = create_backend("supertonic", sample_rate=SAMPLE_RATE, auto_download=True, total_steps=supertonic_total_steps) if provider == "supertonic":
from abogen.tts_plugin.utils import create_pipeline
pipeline = create_pipeline("supertonic")
segments = pipeline( segments = pipeline(
normalized_text, normalized_text,
voice=voice_spec, voice=voice_spec,
speed=speed, speed=speed,
split_pattern=SPLIT_PATTERN, split_pattern=preview_split,
total_steps=supertonic_total_steps, total_steps=supertonic_total_steps,
) )
else: else:
@@ -161,7 +156,7 @@ def generate_preview_audio(
normalized_text, normalized_text,
voice=voice_choice, voice=voice_choice,
speed=speed, speed=speed,
split_pattern=SPLIT_PATTERN, split_pattern=preview_split,
) )
audio_chunks: List[np.ndarray] = [] audio_chunks: List[np.ndarray] = []
+9 -106
View File
@@ -1,6 +1,4 @@
import threading
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
import numpy as np
from abogen.speaker_configs import slugify_label from abogen.speaker_configs import slugify_label
from abogen.speaker_analysis import analyze_speakers from abogen.speaker_analysis import analyze_speakers
@@ -10,7 +8,7 @@ from abogen.voice_profiles import (
load_profiles, load_profiles,
serialize_profiles, serialize_profiles,
) )
from abogen.voice_formulas import get_new_voice, parse_formula_terms from abogen.voice_formulas import parse_formula_terms
from abogen.constants import ( from abogen.constants import (
LANGUAGE_DESCRIPTIONS, LANGUAGE_DESCRIPTIONS,
SUBTITLE_FORMATS, SUBTITLE_FORMATS,
@@ -18,13 +16,9 @@ from abogen.constants import (
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION, SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
SAMPLE_VOICE_TEXTS, SAMPLE_VOICE_TEXTS,
) )
from abogen.tts_backend_registry import get_metadata from abogen.tts_plugin.utils import get_voices
from abogen.speaker_configs import list_configs from abogen.speaker_configs import list_configs
from abogen.tts_backend_registry import create_backend
from abogen.webui.conversion_runner import _select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN
_preview_pipeline_lock = threading.RLock()
_preview_pipelines: Dict[Tuple[str, str], Any] = {}
def build_narrator_roster( def build_narrator_roster(
voice: str, voice: str,
@@ -285,7 +279,7 @@ def filter_voice_catalog(
def build_voice_catalog() -> List[Dict[str, str]]: def build_voice_catalog() -> List[Dict[str, str]]:
catalog: List[Dict[str, str]] = [] catalog: List[Dict[str, str]] = []
gender_map = {"f": "Female", "m": "Male"} gender_map = {"f": "Female", "m": "Male"}
for voice_id in get_metadata("kokoro").voices: for voice_id in get_voices("kokoro"):
prefix, _, rest = voice_id.partition("_") prefix, _, rest = voice_id.partition("_")
language_code = prefix[0] if prefix else "a" language_code = prefix[0] if prefix else "a"
gender_code = prefix[1] if len(prefix) > 1 else "" gender_code = prefix[1] if len(prefix) > 1 else ""
@@ -554,19 +548,12 @@ def prepare_speaker_metadata(
def formula_from_profile(entry: Dict[str, Any]) -> Optional[str]: def formula_from_profile(entry: Dict[str, Any]) -> Optional[str]:
from abogen.voice_formulas import pairs_to_formula
voices = entry.get("voices") or [] voices = entry.get("voices") or []
if not voices: if not voices:
return None return None
total = sum(weight for _, weight in voices) return pairs_to_formula(voices)
if total <= 0:
return None
def _format_weight(value: float) -> str:
normalized = value / total if total else 0.0
return (f"{normalized:.4f}").rstrip("0").rstrip(".") or "0"
parts = [f"{name}*{_format_weight(weight)}" for name, weight in voices if weight > 0]
return "+".join(parts) if parts else None
def template_options() -> Dict[str, Any]: def template_options() -> Dict[str, Any]:
@@ -590,7 +577,7 @@ def template_options() -> Dict[str, Any]:
voice_catalog = build_voice_catalog() voice_catalog = build_voice_catalog()
return { return {
"languages": LANGUAGE_DESCRIPTIONS, "languages": LANGUAGE_DESCRIPTIONS,
"voices": get_metadata("kokoro").voices, "voices": get_voices("kokoro"),
"subtitle_formats": SUBTITLE_FORMATS, "subtitle_formats": SUBTITLE_FORMATS,
"supported_langs_for_subs": SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION, "supported_langs_for_subs": SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
"output_formats": SUPPORTED_SOUND_FORMATS, "output_formats": SUPPORTED_SOUND_FORMATS,
@@ -716,93 +703,9 @@ def sanitize_voice_entries(entries: Iterable[Any]) -> List[Dict[str, Any]]:
def pairs_to_formula(pairs: Iterable[Tuple[str, float]]) -> Optional[str]: def pairs_to_formula(pairs: Iterable[Tuple[str, float]]) -> Optional[str]:
voices = [(voice, float(weight)) for voice, weight in pairs if float(weight) > 0] from abogen.voice_formulas import pairs_to_formula as _pairs_to_formula
if not voices: return _pairs_to_formula(pairs)
return None
total = sum(weight for _, weight in voices)
if total <= 0:
return None
def _format_value(value: float) -> str:
normalized = value / total if total else 0.0
return (f"{normalized:.4f}").rstrip("0").rstrip(".") or "0"
parts = [f"{voice}*{_format_value(weight)}" for voice, weight in voices]
return "+".join(parts)
def profiles_payload() -> Dict[str, Any]: def profiles_payload() -> Dict[str, Any]:
return {"profiles": serialize_profiles()} return {"profiles": serialize_profiles()}
def get_preview_pipeline(language: str, device: str):
key = (language, device)
with _preview_pipeline_lock:
pipeline = _preview_pipelines.get(key)
if pipeline is not None:
return pipeline
pipeline = create_backend("kokoro", lang_code=language, device=device)
_preview_pipelines[key] = pipeline
return pipeline
def synthesize_audio_from_normalized(
*,
normalized_text: str,
voice_spec: str,
language: str,
speed: float,
use_gpu: bool,
max_seconds: float,
) -> np.ndarray:
if not normalized_text.strip():
raise ValueError("Preview text is required")
device = "cpu"
if use_gpu:
try:
device = _select_device()
except Exception:
device = "cpu"
use_gpu = False
pipeline = get_preview_pipeline(language, device)
if pipeline is None:
raise RuntimeError("Preview pipeline is unavailable")
voice_choice: Any = voice_spec
if voice_spec and "*" in voice_spec:
voice_choice = get_new_voice(pipeline, voice_spec, use_gpu)
segments = pipeline(
normalized_text,
voice=voice_choice,
speed=speed,
split_pattern=SPLIT_PATTERN,
)
audio_chunks: List[np.ndarray] = []
accumulated = 0
max_samples = int(max(1.0, max_seconds) * SAMPLE_RATE)
for segment in segments:
graphemes = getattr(segment, "graphemes", "").strip()
if not graphemes:
continue
audio = _to_float32(getattr(segment, "audio", None))
if audio.size == 0:
continue
remaining = max_samples - accumulated
if remaining <= 0:
break
if audio.shape[0] > remaining:
audio = audio[:remaining]
audio_chunks.append(audio)
accumulated += audio.shape[0]
if accumulated >= max_samples:
break
if not audio_chunks:
raise RuntimeError("Preview could not be generated")
return np.concatenate(audio_chunks)
+1 -1
View File
@@ -9,7 +9,7 @@ from abogen.webui.routes.utils.voice import (
parse_voice_formula, parse_voice_formula,
) )
from abogen.webui.routes.utils.settings import load_settings, coerce_bool from abogen.webui.routes.utils.settings import load_settings, coerce_bool
from abogen.webui.routes.utils.preview import synthesize_preview from abogen.webui.routes.utils.synthesize import synthesize_preview
from abogen.speaker_configs import ( from abogen.speaker_configs import (
list_configs, list_configs,
get_config, get_config,
+35 -266
View File
@@ -2,9 +2,7 @@ from __future__ import annotations
import json import json
import logging import logging
import math
import os import os
import re
import shutil import shutil
import sys import sys
import threading import threading
@@ -14,7 +12,7 @@ import traceback
from dataclasses import dataclass, field from dataclasses import dataclass, field
from enum import Enum from enum import Enum
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Dict, Iterable, List, Optional, Mapping, Tuple from typing import Any, Callable, Dict, Iterable, List, Optional, Mapping
from abogen.utils import get_internal_cache_path, get_user_settings_dir, load_config from abogen.utils import get_internal_cache_path, get_user_settings_dir, load_config
from abogen.voice_cache import bootstrap_voice_cache from abogen.voice_cache import bootstrap_voice_cache
@@ -23,6 +21,17 @@ from abogen.integrations.audiobookshelf import (
AudiobookshelfConfig, AudiobookshelfConfig,
AudiobookshelfUploadError, AudiobookshelfUploadError,
) )
from abogen.domain.metadata_helpers import (
normalize_metadata_casefold as _normalize_metadata_casefold,
split_people_field as _split_people_field,
split_simple_list as _split_simple_list,
first_nonempty as _first_nonempty,
extract_year as _extract_year,
normalize_series_sequence as _normalize_series_sequence,
build_audiobookshelf_metadata as _build_abs_metadata,
load_audiobookshelf_chapters as _load_abs_chapters,
_SERIES_SEQUENCE_TAG_KEYS,
)
def _create_set_event() -> threading.Event: def _create_set_event() -> threading.Event:
@@ -53,9 +62,6 @@ _JOB_LEVEL_MAP: Dict[str, int] = {
} }
_PEOPLE_SPLIT_RE = re.compile(r"[;,/&]|\band\b", re.IGNORECASE)
def _emit_job_log(job_id: str, level: str, message: str) -> None: def _emit_job_log(job_id: str, level: str, message: str) -> None:
normalized = (level or "info").lower() normalized = (level or "info").lower()
log_level = _JOB_LEVEL_MAP.get(normalized, logging.INFO) log_level = _JOB_LEVEL_MAP.get(normalized, logging.INFO)
@@ -131,6 +137,7 @@ class Job:
progress: float = 0.0 progress: float = 0.0
total_characters: int = 0 total_characters: int = 0
processed_characters: int = 0 processed_characters: int = 0
etr_str: str = ""
logs: List[JobLog] = field(default_factory=list) logs: List[JobLog] = field(default_factory=list)
error: Optional[str] = None error: Optional[str] = None
result: JobResult = field(default_factory=JobResult) result: JobResult = field(default_factory=JobResult)
@@ -162,20 +169,25 @@ class Job:
@property @property
def estimated_time_remaining(self) -> Optional[float]: def estimated_time_remaining(self) -> Optional[float]:
""" """
Returns the estimated seconds remaining based on current progress and elapsed time. Returns the estimated seconds remaining.
Returns None if the job hasn't started, is finished, or progress is 0. Uses the same calc_etr_str from domain/progress.py as the PyQt desktop GUI.
""" """
if self.status != JobStatus.RUNNING or not self.started_at or self.progress <= 0: from abogen.domain.progress import calc_etr_str
if self.status != JobStatus.RUNNING or not self.started_at or self.total_characters <= 0:
return None return None
elapsed = time.time() - self.started_at elapsed = time.time() - self.started_at
if elapsed <= 0: if elapsed <= 0:
return None return None
# Estimate total time based on current progress etr = calc_etr_str(elapsed, self.processed_characters, self.total_characters)
total_estimated = elapsed / self.progress if etr == "Processing...":
remaining = total_estimated - elapsed return None
return max(0.0, remaining)
# Parse "HH:MM:SS" back to seconds for backward compatibility
parts = etr.split(":")
return int(parts[0]) * 3600 + int(parts[1]) * 60 + int(parts[2])
def add_log(self, message: str, level: str = "info") -> None: def add_log(self, message: str, level: str = "info") -> None:
entry = JobLog(timestamp=time.time(), message=message, level=level) entry = JobLog(timestamp=time.time(), message=message, level=level)
@@ -194,6 +206,7 @@ class Job:
"progress": self.progress, "progress": self.progress,
"total_characters": self.total_characters, "total_characters": self.total_characters,
"processed_characters": self.processed_characters, "processed_characters": self.processed_characters,
"etr_str": self.etr_str,
"error": self.error, "error": self.error,
"logs": [log.__dict__ for log in self.logs], "logs": [log.__dict__ for log in self.logs],
"result": { "result": {
@@ -252,234 +265,13 @@ class Job:
} }
def _normalize_metadata_casefold(values: Optional[Mapping[str, Any]]) -> Dict[str, Any]:
normalized: Dict[str, Any] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
key_text = str(key).strip().lower()
if not key_text:
continue
if isinstance(value, (list, tuple, set)):
normalized[key_text] = value
else:
text = str(value).strip()
if text:
normalized[key_text] = text
return normalized
def _split_people_field(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(_split_people_field(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _PEOPLE_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
_LIST_SPLIT_RE = re.compile(r"[;,\n]")
_SERIES_SEQUENCE_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?")
_SERIES_SEQUENCE_TAG_KEYS: Tuple[str, ...] = (
"series_index",
"series_position",
"series_sequence",
"series_number",
"seriesnumber",
"book_number",
"booknumber",
)
def _split_simple_list(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(_split_simple_list(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _LIST_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def _first_nonempty(*values: Any) -> Optional[str]:
for value in values:
if value is None:
continue
if isinstance(value, (list, tuple, set)):
items = list(value)
if not items:
continue
value = items[0]
text = str(value).strip()
if text:
return text
return None
def _extract_year(raw: Optional[str]) -> Optional[int]:
if not raw:
return None
text = str(raw).strip()
if not text:
return None
match = re.search(r"(19|20)\d{2}", text)
if match:
try:
return int(match.group(0))
except ValueError:
return None
try:
parsed = int(text)
except ValueError:
return None
if 0 < parsed < 3000:
return parsed
return None
def build_audiobookshelf_metadata(job: Job) -> Dict[str, Any]: def build_audiobookshelf_metadata(job: Job) -> Dict[str, Any]:
tags = _normalize_metadata_casefold(job.metadata_tags)
filename = Path(job.original_filename or "").stem or job.original_filename or "Audiobook" filename = Path(job.original_filename or "").stem or job.original_filename or "Audiobook"
title = _first_nonempty( return _build_abs_metadata(
tags.get("title"), job.metadata_tags,
tags.get("book_title"), language=job.language or "",
tags.get("name"), filename=filename,
tags.get("album"),
filename,
) )
authors = _split_people_field(
tags.get("authors")
or tags.get("author")
or tags.get("album_artist")
or tags.get("artist")
)
narrators = _split_people_field(tags.get("narrators") or tags.get("narrator"))
description = _first_nonempty(tags.get("description"), tags.get("summary"), tags.get("comment"))
genres = _split_simple_list(tags.get("genre"))
keywords = _split_simple_list(tags.get("tags") or tags.get("keywords"))
language = _first_nonempty(tags.get("language"), tags.get("lang")) or job.language or ""
series_name = _first_nonempty(
tags.get("series"),
tags.get("series_name"),
tags.get("seriesname"),
tags.get("series_title"),
tags.get("seriestitle"),
)
series_sequence = None
for key in _SERIES_SEQUENCE_TAG_KEYS:
raw_value = tags.get(key)
normalized_sequence = _normalize_series_sequence(raw_value)
if normalized_sequence:
series_sequence = normalized_sequence
break
if not series_name:
series_sequence = None
data: Dict[str, Any] = {
"title": title,
"subtitle": tags.get("subtitle"),
"authors": authors,
"narrators": narrators,
"description": description,
"publisher": tags.get("publisher"),
"genres": genres,
"tags": keywords,
"language": language,
"publishedYear": _extract_year(tags.get("published") or tags.get("publication_year") or tags.get("date") or tags.get("year")),
"seriesName": series_name,
"seriesSequence": series_sequence,
"isbn": _first_nonempty(tags.get("isbn"), tags.get("asin")),
}
published_date = _first_nonempty(tags.get("published"), tags.get("publication_date"), tags.get("date"))
if published_date:
data["publishedDate"] = published_date
rating_text = _first_nonempty(tags.get("rating"), tags.get("my_rating"))
if rating_text:
try:
data["rating"] = float(str(rating_text).strip())
except ValueError:
pass
rating_max_text = _first_nonempty(tags.get("rating_max"), tags.get("rating_scale"))
if rating_max_text:
try:
data["ratingMax"] = float(str(rating_max_text).strip())
except ValueError:
pass
# Remove empty values
cleaned: Dict[str, Any] = {}
for key, value in data.items():
if value is None:
continue
if isinstance(value, str) and not value.strip():
continue
if isinstance(value, (list, tuple)) and not value:
continue
cleaned[key] = value
return cleaned
def _normalize_series_sequence(raw: Any) -> Optional[str]:
if raw is None:
return None
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return None
text = str(raw)
else:
text = str(raw).strip()
if not text:
return None
candidate = text.replace(",", ".")
match = _SERIES_SEQUENCE_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
def load_audiobookshelf_chapters(job: Job) -> Optional[List[Dict[str, Any]]]: def load_audiobookshelf_chapters(job: Job) -> Optional[List[Dict[str, Any]]]:
@@ -487,32 +279,7 @@ def load_audiobookshelf_chapters(job: Job) -> Optional[List[Dict[str, Any]]]:
if not metadata_ref: if not metadata_ref:
return None return None
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref)) metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
if not metadata_path.exists(): return _load_abs_chapters(metadata_path)
return None
try:
payload = json.loads(metadata_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
chapters = payload.get("chapters")
if not isinstance(chapters, list):
return None
cleaned: List[Dict[str, Any]] = []
for entry in chapters:
if not isinstance(entry, Mapping):
continue
title = _first_nonempty(entry.get("title"), entry.get("original_title"))
start = entry.get("start")
end = entry.get("end")
if title is None or not isinstance(start, (int, float)):
continue
chapter_payload: Dict[str, Any] = {
"title": title,
"start": float(start),
}
if isinstance(end, (int, float)):
chapter_payload["end"] = float(end)
cleaned.append(chapter_payload)
return cleaned or None
def _existing_paths(paths: Iterable[Any]) -> List[Path]: def _existing_paths(paths: Iterable[Any]) -> List[Path]:
@@ -1609,10 +1376,12 @@ def build_service(
output_root: Optional[Path] = None, output_root: Optional[Path] = None,
uploads_root: Optional[Path] = None, uploads_root: Optional[Path] = None,
) -> ConversionService: ) -> ConversionService:
global _service_instance
output_root = output_root or default_storage_root() output_root = output_root or default_storage_root()
service = ConversionService( service = ConversionService(
output_root=output_root, output_root=output_root,
uploads_root=uploads_root, uploads_root=uploads_root,
runner=runner, runner=runner,
) )
_service_instance = service
return service return service
+165
View File
@@ -0,0 +1,165 @@
"""Settings form-to-dict mapping.
Pure functions that convert form data into a settings dict.
No Flask dependencies testable without a request context.
"""
from __future__ import annotations
from collections.abc import Mapping
from typing import Any
def apply_form_to_settings(current: dict, form: Mapping[str, Any]) -> dict:
"""Apply form data to a settings dict.
Pure function: takes a current settings dict and a form-like mapping,
returns the updated settings dict. No Flask dependencies.
Args:
current: Current settings dict (will be mutated).
form: Form-like mapping (e.g. request.form.to_dict()).
Returns:
Updated settings dict (same object as input).
"""
from abogen.domain.settings_core import (
coerce_bool,
coerce_int,
DEFAULT_ANALYSIS_THRESHOLD,
_NORMALIZATION_BOOLEAN_KEYS,
_NORMALIZATION_STRING_KEYS,
)
from abogen.webui.routes.utils.settings import stored_integration_config
from abogen.webui.routes.utils.common import extract_checkbox
from abogen.utils import load_config
# General settings
current["language"] = (form.get("language") or "en").strip()
current["default_speaker"] = (form.get("default_speaker") or "").strip()
current["default_voice"] = (form.get("default_voice") or "").strip()
try:
current["supertonic_total_steps"] = max(2, min(15, int(form.get("supertonic_total_steps", current.get("supertonic_total_steps", 5)))))
except (TypeError, ValueError):
pass
try:
current["supertonic_speed"] = max(0.7, min(2.0, float(form.get("supertonic_speed", current.get("supertonic_speed", 1.0)))))
except (TypeError, ValueError):
pass
current["output_format"] = (form.get("output_format") or "mp3").strip()
current["subtitle_mode"] = (form.get("subtitle_mode") or "Disabled").strip()
current["subtitle_format"] = (form.get("subtitle_format") or "srt").strip()
current["save_mode"] = (form.get("save_mode") or "save_next_to_input").strip()
current["replace_single_newlines"] = coerce_bool(form.get("replace_single_newlines"), False)
current["use_gpu"] = coerce_bool(form.get("use_gpu"), False)
current["save_chapters_separately"] = coerce_bool(form.get("save_chapters_separately"), False)
current["merge_chapters_at_end"] = coerce_bool(form.get("merge_chapters_at_end"), True)
current["save_as_project"] = coerce_bool(form.get("save_as_project"), False)
current["separate_chapters_format"] = (form.get("separate_chapters_format") or "wav").strip()
try:
current["silence_between_chapters"] = max(0.0, float(form.get("silence_between_chapters", 2.0)))
except ValueError:
pass
try:
current["chapter_intro_delay"] = max(0.0, float(form.get("chapter_intro_delay", 0.5)))
except ValueError:
pass
current["read_title_intro"] = coerce_bool(form.get("read_title_intro"), False)
current["read_closing_outro"] = coerce_bool(form.get("read_closing_outro"), True)
current["normalize_chapter_opening_caps"] = coerce_bool(form.get("normalize_chapter_opening_caps"), True)
current["auto_prefix_chapter_titles"] = coerce_bool(form.get("auto_prefix_chapter_titles"), True)
try:
current["max_subtitle_words"] = max(1, int(form.get("max_subtitle_words", 50)))
except ValueError:
pass
current["chunk_level"] = (form.get("chunk_level") or "paragraph").strip()
current["generate_epub3"] = coerce_bool(form.get("generate_epub3"), False)
current["speaker_analysis_threshold"] = coerce_int(
form.get("speaker_analysis_threshold"),
DEFAULT_ANALYSIS_THRESHOLD,
minimum=1,
maximum=25,
)
# Normalization settings
for key in _NORMALIZATION_BOOLEAN_KEYS:
current[key] = extract_checkbox(form, key, bool(current.get(key, True)))
for key in _NORMALIZATION_STRING_KEYS:
if key in form:
current[key] = (form.get(key) or "").strip()
# Integrations — seed from stored config to prevent wiping credentials
current_integrations: dict[str, dict[str, Any]] = {}
cfg = load_config() or {}
stored_integrations = cfg.get("integrations")
if isinstance(stored_integrations, Mapping):
for name, payload in stored_integrations.items():
if isinstance(name, str) and isinstance(payload, Mapping):
current_integrations[name] = dict(payload)
for name in ("audiobookshelf", "calibre_opds"):
stored = stored_integration_config(name)
if stored and name not in current_integrations:
current_integrations[name] = dict(stored)
current["integrations"] = current_integrations
# Audiobookshelf
abs_enabled = coerce_bool(form.get("audiobookshelf_enabled"), False)
abs_url = (form.get("audiobookshelf_base_url") or "").strip()
abs_token = (form.get("audiobookshelf_api_token") or "").strip()
abs_library = (form.get("audiobookshelf_library_id") or "").strip()
abs_folder = (form.get("audiobookshelf_folder_id") or "").strip()
abs_verify = coerce_bool(form.get("audiobookshelf_verify_ssl"), True)
abs_auto_send = coerce_bool(form.get("audiobookshelf_auto_send"), False)
abs_cover = coerce_bool(form.get("audiobookshelf_send_cover"), True)
abs_chapters = coerce_bool(form.get("audiobookshelf_send_chapters"), True)
abs_subtitles = coerce_bool(form.get("audiobookshelf_send_subtitles"), False)
try:
abs_timeout = max(1.0, float(form.get("audiobookshelf_timeout", 30.0)))
except ValueError:
abs_timeout = 30.0
if not abs_token and not coerce_bool(form.get("audiobookshelf_api_token_clear"), False):
existing_abs = current["integrations"].get("audiobookshelf", {})
abs_token = existing_abs.get("api_token", "")
current["integrations"]["audiobookshelf"] = {
"enabled": abs_enabled,
"base_url": abs_url,
"api_token": abs_token,
"library_id": abs_library,
"folder_id": abs_folder,
"verify_ssl": abs_verify,
"auto_send": abs_auto_send,
"send_cover": abs_cover,
"send_chapters": abs_chapters,
"send_subtitles": abs_subtitles,
"timeout": abs_timeout,
}
# Calibre OPDS
calibre_enabled = coerce_bool(form.get("calibre_opds_enabled"), False)
calibre_url = (form.get("calibre_opds_base_url") or "").strip()
calibre_user = (form.get("calibre_opds_username") or "").strip()
calibre_pass = (form.get("calibre_opds_password") or "").strip()
calibre_verify = coerce_bool(form.get("calibre_opds_verify_ssl"), True)
if not calibre_pass and not coerce_bool(form.get("calibre_opds_password_clear"), False):
existing_calibre = current["integrations"].get("calibre_opds", {})
calibre_pass = existing_calibre.get("password", "")
current["integrations"]["calibre_opds"] = {
"enabled": calibre_enabled,
"base_url": calibre_url,
"username": calibre_user,
"password": calibre_pass,
"verify_ssl": calibre_verify,
}
return current
+2 -2
View File
@@ -28,8 +28,8 @@
</div> </div>
<div class="job-card__progress-meta"> <div class="job-card__progress-meta">
<small>{{ progress_value }}% · {{ job.processed_characters }} / {{ job.total_characters or '—' }}</small> <small>{{ progress_value }}% · {{ job.processed_characters }} / {{ job.total_characters or '—' }}</small>
{% if job.estimated_time_remaining %} {% if job.etr_str and job.etr_str != 'Processing...' %}
<small class="job-card__eta">~{{ job.estimated_time_remaining | durationformat }} remaining</small> <small class="job-card__eta">~{{ job.etr_str }} remaining</small>
{% endif %} {% endif %}
</div> </div>
</div> </div>

Some files were not shown because too many files have changed in this diff Show More