Commit Graph
762 Commits
Author SHA1 Message Date
Artem Akymenko 804517f5b2 extract subtitle writer creation: resolve_subtitle_format() + make_subtitle_writer()
- infrastructure/subtitle_writer.py: add resolve_subtitle_format() that
  maps format strings (e.g. 'ass_centered_narrow') to (extension, alignment),
  and make_subtitle_writer() convenience that resolves + creates writer or None
- conversion_runner.py: replace _create_subtitle_writer() with make_subtitle_writer()
- pyqt/conversion.py: replace _subtitle_alignment_from_format() and 3 manual
  create_subtitle_writer() call sites with resolve_subtitle_format()/make_subtitle_writer()
2026-07-20 07:52:15 +00:00
Artem Akymenko 5d30903149 extract conversion_engine: shared TTS segment iteration loop for WebUI and PyQt
- domain/conversion_engine.py: run_tts_segment_loop() with CancelChecker,
  SegmentStats, SegmentInfo protocols; on_segment callback for per-segment
  subtitle processing; process_and_write_subtitles() helper
- conversion_runner.py: emit_text() delegates TTS iteration to engine
- pyqt/conversion.py: inner tts_segments loop replaced with engine call,
  on_segment handles dual merged+chapter subtitle writers
- routes/utils/settings.py: re-exports load_settings, coerce_int/float,
  llm_ready, settings_defaults from domain for backward compat
2026-07-20 07:35:58 +00:00
Artem Akymenko 476063bc3d refactor: move load_settings() to domain, simplify settings.py
- load_settings() now in domain/settings_core.py (shared by all UIs)
- settings.py delegates to domain instead of reimplementing
- settings.py: 456 → 430 lines
2026-07-20 06:56:42 +00:00
Artem Akymenko 079e185108 refactor: simplify normalize_setting_value() via Setting.normalizer
- Added normalizer callable to Setting dataclass
- Moved special-case logic (_norm_save_mode, _norm_voice_spec, etc.)
  into registry entries as normalizers
- normalize_setting_value() reduced from 25 lines to 10 lines
- Single dispatch: normalizer → coerce → fallback
2026-07-20 06:48:06 +00:00
Artem Akymenko 69c398ebf0 refactor(pyqt): replace hardcoded config defaults with SETTINGS_REGISTRY
gui.py now reads defaults from all_settings_defaults() instead of
hardcoding values like 50, True, 'wav', etc. One source of truth
for all settings across Web UI and Desktop GUI.
2026-07-19 15:57:49 +00:00
Artem Akymenko dbe73254a4 refactor: add SETTINGS_REGISTRY contract to domain/settings_core.py
- Setting dataclass: key, type, default, min/max, valid_values, scope
- 72 settings total: 54 shared, 18 PyQt-only
- validate_setting() checks types and ranges
- Setting.coerce() handles type conversion with bounds
- settings_defaults() / all_settings_defaults() derived from registry
- BOOLEAN_SETTINGS, FLOAT_SETTINGS, INT_SETTINGS now auto-derived
- 17 tests validating schema, coercion, and validation
2026-07-19 13:24:52 +00:00
Artem Akymenko 64e8a8f4e6 refactor: extract shared settings_core and Flask route logic to domain/services 2026-07-19 15:52:35 +03:00
Artem Akymenko aec3462f1f refactor: both UIs use shared tts_segments() from domain/conversion_pipeline.py
- domain/conversion_pipeline.py: add tts_segments() for pre-normalized text;
  emit_text_segments() now delegates to tts_segments() internally
- pyqt/conversion.py: inner TTS loop replaced with tts_segments() iterator;
  removed FakeToken import (handled by domain)
- webui/conversion_runner.py: emit_text() inner loop replaced with
  tts_segments() iterator; removed FakeToken import
- Both UIs now share the same TTS emission logic — normalization + backend
  invocation + segment iteration + token extraction
- +3 tests for tts_segments (no-normalization, chunk_start, basic)
- 1188 tests pass
2026-07-19 11:47:14 +00:00
Artem Akymenko 1193185833 refactor: create domain/conversion_pipeline.py with shared TTS emission loop
- domain/conversion_pipeline.py: emit_text_segments() — generator yielding
  SegmentResult for each TTS segment; emit_text_to_sinks() — convenience
  wrapper handling audio writing + token accumulation + subtitle flushing
- Both WebUI and PyQt can call these instead of reimplementing the TTS loop
- Caller provides backend, voice, speed, split_pattern; domain handles
  normalization, TTS invocation, token extraction
- +7 tests (segment yielding, empty audio skip, chunk_start, tokens, fallback)
- 1185 tests pass
2026-07-19 10:31:58 +00:00
Artem Akymenko a99cf58c79 refactor: consolidate voice formula building into voice_formulas.py
- voice_formulas.py: add pairs_to_formula() as canonical implementation
- webui/routes/utils/voice.py: formula_from_profile() and pairs_to_formula()
  now delegate to voice_formulas.pairs_to_formula()
- pyqt/gui.py: get_voice_formula() now uses voice_formulas.pairs_to_formula()
  instead of inline string formatting
- Eliminates 3 duplicate implementations of voice*weight formula building
- +9 tests
- 1178 tests pass
2026-07-19 10:02:14 +00:00
Artem Akymenko fe62b6b44c refactor: extract book metadata logic from PyQt to domain
- domain/metadata_extraction.py: add format_metadata_tags(),
  extract_book_metadata_epub(), extract_book_metadata_pdf(),
  extract_book_metadata_markdown(), _save_cover_to_cache()
- pyqt/book_handler.py: _extract_book_metadata() reduced from ~165 lines
  to ~10 lines by delegating to domain; _format_metadata_tags() reduced
  from ~55 lines to ~15 lines; ebooklib/fitz imports moved to domain
- +16 tests (format_metadata_tags, save_cover, markdown extraction)
- 1169 tests pass
2026-07-19 09:51:50 +00:00
Artem Akymenko 0e216f3786 refactor: extract _process_subtitle_file domain logic to shared modules
- domain/subtitle_processor.py: parse_subtitle_file, format_time_range,
  speed_up_audio, fit_audio_to_duration (moved from audio_buffer),
  process_subtitle_entries (core TTS loop with cancel/log/progress callbacks)
- domain/audio_buffer.py: add fit_audio_to_duration, ffmpeg_time_stretch
- domain/output_paths.py: add resolve_unique_path (collision-safe filename)
- pyqt/conversion.py: _process_subtitle_file reduced from ~350 to ~100 lines
  by delegating to domain functions; removed 4 unused subtitle parser imports
- +33 tests (8 resolve_unique_path, 14 subtitle_processor, 8 audio_buffer,
  3 format_time_range)
- 1153 tests pass
2026-07-19 09:34:17 +00:00
Artem Akymenko 380cdee0cb refactor(pyqt): use create_pipeline_for_job() in LoadPipelineThread
Replace direct create_pipeline() call with domain function for
consistent provider validation and device resolution.
2026-07-19 07:57:09 +00:00
Artem Akymenko c76cf74efc refactor: unify duplicated logic between WebUI and PyQt
domain/output_paths.py:
- Add sanitize_filename_for_chapter() with OS safety + smart truncation
- Keep existing slugify() for backward compatibility

domain/text_chapters.py (NEW):
- parse_chapters_from_text() combines intro preservation (PyQt) + clean_text (WebUI)

PyQt/conversion.py:
- Replace 3x copy-pasted ASS/SRT headers with create_subtitle_writer()
- Replace inline M4B muxing with ExportService.embed_m4b_metadata()
- Replace inline chapter splitting with parse_chapters_from_text()
- Replace inline chapter filename sanitization with sanitize_filename_for_chapter()
- Remove unused _CHAPTER_MARKER_SEARCH_PATTERN import

Tests: 1152 passed (+21 new)
2026-07-19 07:50:02 +00:00
Artem Akymenko a299947bb1 refactor(webui): replace inline get_pipeline/resolve_voice_target closures with domain modules
- Replace get_pipeline() closure with PipelinePool from domain/pipeline_factory
- Replace resolve_voice_target() closure with domain function from voice_utils
- Remove dead _load_pipeline() function and unused is_plugin_registered import
- Add 33 tests for resolve_voice_target and PipelinePool
- Add 10 regression tests verifying domain extraction preserves behavior
- 1131 tests pass (+61 new)
2026-07-18 14:13:17 +03:00
Artem Akymenko 957c6778f6 refactor(pyt): dedup voice formula resolution in PyQt
Replace 2 inline 'if * in voice: get_new_voice(...)' patterns with
resolve_voice() from domain/voice_loader.py. Removes unused import
of get_new_voice.
2026-07-18 14:13:16 +03:00
Artem Akymenko fcdaf2b2a8 refactor(domain): extract FakeToken to domain/tokens.py
Shared token stub used by both WebUI and PyQt for languages
without per-word token support.
2026-07-18 14:12:18 +03:00
Artem Akymenko d8634f812d refactor(domain): extract audio sink abstraction to domain layer
- New domain/audio_sink.py: AudioSink context manager + open_audio_sink() factory
  - Supports WAV/FLAC (soundfile) and MP3/Opus/M4B (ffmpeg pipe)
  - cancel_check, extra_ffmpeg_args, ffmpeg_cmd parameters
  - 17 tests in test_domain_audio_sink.py

- WebUI: replaced local AudioSink + _open_audio_sink with domain module
  - Removed 35 lines, 3 call sites now use open_audio_sink()

- PyQt: replaced 3 inline audio output setups with domain module
  - New _open_merged_sink() helper encapsulates m4b cover art logic
  - ExitStack for automatic cleanup in main conversion
  - Removed ~140 lines of duplicated ffmpeg/soundfile boilerplate
2026-07-18 14:12:01 +03:00
Artem Akymenko 85b5851786 refactor(pyt): replace all inline float32 conversion with to_float32()
PyQt had 7 inline float32 conversion patterns:
  hasattr(x, 'numpy') ? x.numpy().astype('float32') : x.astype('float32')
spread across TTS loop, subtitle processing, and streaming.

All replaced with domain.audio_helpers.to_float32() which handles:
- None → zeros
- PyTorch tensors → .detach().cpu().numpy()
- Plain numpy → asarray(dtype=float32)
- reshape(-1) for consistent 1D output

The old inline code missed .detach() and .cpu() on GPU tensors,
causing potential crashes. Now both UIs use the same robust conversion.

1053 tests pass.
2026-07-18 06:58:21 +00:00
Artem Akymenko e77c8b3372 fix: review cleanup — imports, _FakeToken, use_spacy_segmentation
- Move pronunciation imports from inside run() to top-level imports
- Extract _FakeToken to module level (was redefined every loop iteration)
- use_spacy_segmentation now mirrors PyQt logic: pass the flag,
  let process_subtitle_tokens filter by language internally
2026-07-18 06:52:28 +00:00
Artem Akymenko 294069e53e refactor(webui): token-level subtitle processing via process_subtitle_tokens — P0
Before: WebUI wrote one subtitle entry per TTS segment (no sentence
grouping, no comma splitting, no karaoke highlighting). The subtitle
modes 'Sentence', 'Sentence + Comma', and 'Sentence + Highlighting'
produced broken output.

After: emit_text() accumulates tokens_with_timestamps from each
segment's .tokens attribute, then flushes them through
domain.subtitle_generation.process_subtitle_tokens() at the end.
This gives the WebUI the same subtitle quality as the PyQt desktop GUI:
- Sentence mode: groups tokens into sentences
- Sentence + Comma: splits on commas within sentences
- Sentence + Highlighting: karaoke timing per word
- Word-count mode: groups by N words

Also removed the duplicate _to_float32 function from synthesize.py
(now imports from domain.audio_helpers).

1053 tests pass.
2026-07-18 06:36:19 +00:00
Artem Akymenko 4ff09be664 refactor(pyt): add text normalization via prepare_text_for_tts — P0
PyQt desktop GUI now calls the shared normalization pipeline before
TTS synthesis, matching the Web UI's behavior:

1. Heteronym sentence rules (context-dependent pronunciation)
2. Pronunciation rules (token-level replacements)
3. Pipeline normalization (apostrophe handling, LLM)

Before: PyQt passed raw text to the backend — no normalization at all,
resulting in inferior audio quality compared to the Web UI.

The normalization rules are compiled once at the start of run() from
pronunciation_overrides and heteronym_overrides (currently None since
the PyQt GUI doesn't expose these settings yet — basic apostrophe
normalization still applies).

1053 tests pass.
2026-07-18 09:28:53 +03:00
Artem Akymenko a1d93820b1 refactor(domain): unify ETR calculation — both UIs now use calc_etr_str
Before:
  - PyQt: inline ETR using chars-based formula
  - WebUI: Job.estimated_time_remaining using progress-based formula
  (different formulas → different ETR estimates)

After:
  - Both UIs call domain.progress.calc_etr_str(elapsed, done, total)
  - Same formula, same ETR, single source of truth
  - WebUI now stores etr_str on Job and displays it directly
  - Job.estimated_time_remaining property kept for backward compat

domain/progress.py: ProgressTracker class + calc_etr_str function
1053 tests pass.
2026-07-16 09:31:26 +00:00
Artem Akymenko 0c1a3c1904 refactor(webui): use shared get_split_pattern instead of hardcoded \\n+
All three Web UI consumers now call domain.split_pattern.get_split_pattern()
which selects the correct split pattern based on language and subtitle mode.

Before: WebUI always split on \\n+ regardless of language (CJK missed
punctuation-based splitting that PyQt already had).
After: Both UIs share identical language-aware splitting logic.

1038 tests pass.
2026-07-16 09:12:46 +00:00
Artem Akymenko 2228f37c06 refactor(domain): add prepare_text_for_tts — unified normalization pipeline
New function chains all three normalization stages:
  1. Heteronym sentence rules (context-dependent pronunciation)
  2. Pronunciation rules (token-level replacements)
  3. Pipeline normalization (apostrophe, LLM)

This is the single entry point that both Web UI and PyQt should call
before TTS synthesis. Currently only Web UI uses it; PyQt has NO
normalization — this unlocks that capability.

Updated conversion_runner.emit_text to use the new function.
1038 tests pass.
2026-07-16 08:53:15 +00:00
Artem Akymenko 832e2c5197 refactor(domain): extract chapter classification heuristics from form.py
Moved supplement_score, should_preselect_chapter, and
ensure_at_least_one_chapter_enabled to domain/chapter_classification.py.

Also moved coerce_bool from settings.py to common.py to break circular
import introduced in previous commit.

1025 tests pass.
2026-07-16 08:13:37 +00:00
Artem Akymenko 17229b2390 refactor(webui): deduplicate _extract_checkbox (3 copies → 1)
Extracted extract_checkbox from settings.py, form.py (x2) into
webui/routes/utils/common.py. Moved coerce_bool from settings.py
to common.py to break circular import.

Fixed bug in second form.py copy (was missing __contains__ check).
1006 tests pass.
2026-07-16 07:46:51 +00:00
Artem Akymenko c2c584e741 refactor(domain): deduplicate metadata helpers across 3 layers
Extracted 8 metadata functions from service.py, exporters.py, and
audiobookshelf.py into domain/metadata_helpers.py:
- normalize_metadata_casefold, split_people_field, split_simple_list
- first_nonempty, extract_year, normalize_series_sequence
- build_audiobookshelf_metadata, load_audiobookshelf_chapters

service.py, exporters.py, and audiobookshelf.py now import from domain
instead of maintaining separate copies. Thin wrappers adapt to layer
interfaces (Job objects, etc.).

Net -231 lines. 1006 tests pass.
2026-07-16 07:34:58 +00:00
Artem Akymenko 8ccdc85ccb refactor(webui): rename preview.py to synthesize.py and remove dead code
- Rename abogen/webui/routes/utils/preview.py → synthesize.py
  The file contains the core TTS synthesis pipeline (generate_preview_audio,
  synthesize_preview), not just preview logic. Name now matches responsibility.
- Remove dead code from voice.py: get_preview_pipeline(), synthesize_audio_from_normalized(),
  _preview_pipeline_lock, _preview_pipelines, and unused imports (threading, numpy,
  create_pipeline, get_new_voice, _select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN).
  These were never called — identical logic lives in synthesize.py.
- Update imports in api.py, voices.py, and test_preview_applies_manual_overrides.py
- 7 new tests in test_synthesize_module.py enforce file naming and import rules
- 7 tests in test_domain_imports.py updated for renamed module
2026-07-16 10:19:01 +03:00
Artem Akymenko ef07a8b5b2 fix(tests): mock spacy in plugin tests to fix externally-managed-environment failures 2026-07-16 10:02:01 +03:00
Artem Akymenko 1268a83cff fix(webui): voice.py imports from domain modules instead of conversion_runner
Replace private imports (_select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN)
from abogen.webui.conversion_runner with proper domain imports:
- select_device from abogen.domain.device
- to_float32, SAMPLE_RATE from abogen.domain.audio_helpers
- SPLIT_PATTERN defined locally (r'\n+')

Also verifies preview.py already uses domain imports correctly.

7 new tests in tests/test_domain_imports.py enforce the architecture rule.
2026-07-16 06:57:12 +00:00
Artem Akymenko ef6faff2e8 refactor: extract metadata processing logic to domain layer
- Replace manual metadata extraction with regex in pyqt/conversion.py
  with calls to domain/metadata_extraction.py functions
- Remove duplicate _embed_m4b_metadata and _apply_m4b_chapters_with_mutagen
  functions from webui/conversion_runner.py
- Use ExportService.embed_m4b_metadata for m4b metadata embedding
- Reduce code duplication between PyQt and WebUI interfaces
2026-07-15 20:44:19 +03:00
Artem Akymenko da9d5e7eb9 fix(tests): audio_buffer and subtitle_generation tests
- Fix mix_audio to return target buffer (was not modifying in-place)
- Fix samples_for_duration to return 0 for negative durations
- Fix test assertions for numpy 2.x compatibility (share_memory -> shares_memory)
- Adjust subtitle_generation tests to match actual behavior
2026-07-15 20:26:31 +03:00
Artem Akymenko acb000b9e6 refactor: extract voice loading logic to domain layer
- Add abogen/domain/voice_loader.py with:
  - VoiceCache class: unified cache for loaded voices
  - resolve_voice(): load voice with optional caching
  - load_voice_cached(): compatibility wrapper for PyQt

- Update abogen/pyqt/conversion.py:
  - Replace load_voice_cached method body with call to domain function
  - Maintain backward compatibility with existing interface

- Add tests/test_voice_loader.py with unit tests for VoiceCache and voice loading
2026-07-15 20:20:18 +03:00
Artem Akymenko d6c66dc18a refactor: extract subtitle token processing to domain layer
- Add abogen/domain/subtitle_generation.py with:
  - process_subtitle_tokens(): main function for converting TTS tokens to subtitles
  - Support for all subtitle modes: Line, Sentence, Sentence + Comma, Sentence + Highlighting
  - Support for word-count based grouping (e.g., '5' for 5 words per entry)
  - spaCy integration for English sentence boundary detection
  - Karaoke highlighting tags for Sentence + Highlighting mode
  - Punctuation constants for sentence splitting

- Update abogen/pyqt/conversion.py:
  - Replace _process_subtitle_tokens method body with call to domain function
  - Remove ~260 lines of duplicate logic

- Add tests/test_subtitle_generation.py with comprehensive unit tests
2026-07-15 20:14:02 +03:00
Artem Akymenko 0d46076bf6 refactor: extract audio buffer operations to domain layer
- Add abogen/domain/audio_buffer.py with core audio operations:
  - create_silence(): create silence audio buffer
  - mix_audio(): mix source into target buffer with auto-resize
  - normalize_audio(): normalize to prevent clipping
  - ensure_buffer_size(): extend buffer to minimum size
  - concatenate_audio(): join multiple audio buffers
  - audio_duration(): calculate duration from samples
  - samples_for_duration(): calculate samples from duration
  - SAMPLE_RATE constant (24000)

- Update abogen/pyqt/conversion.py:
  - Import and use create_silence for chapter silence
  - Use mix_audio for subtitle file mixing
  - Use normalize_audio for clipping prevention
  - Use create_silence for padding in subtitle processing

- Update abogen/webui/conversion_runner.py:
  - Import and use create_silence in append_silence
  - Replace np.zeros with domain function

- Add tests/test_audio_buffer.py with comprehensive unit tests
2026-07-15 20:02:33 +03:00
Artem Akymenko 7fef9c1d93 extract normalize_text_for_pipeline to domain/normalization.py 2026-07-15 15:19:01 +00:00
Artem Akymenko 56cfd0810d extract resolve_fallback_voice_spec to domain/voice_resolution.py; fix missing get_default_voice import and __custom_mix reset bug 2026-07-15 15:01:17 +00:00
Artem Akymenko 7bd3177241 extract select_device to domain/device.py; fix bug where conversion_runner didn't check torch availability 2026-07-15 14:45:38 +00:00
Artem AkymenkoandGitHub d5c2a81733 Merge pull request #191 from hydraxman/fix/large-chapter-form-limits
fix(webui): allow large chapter forms
2026-07-15 17:35:52 +03:00
Artem Akymenko 514e29a761 extract apply_chapter_text_transforms to domain/chapter_titles.py 2026-07-15 14:29:29 +00:00
Artem Akymenko 86042a3315 fix bugs, remove dead code and unused imports in conversion_runner.py 2026-07-15 13:50:18 +00:00
Artem Akymenko 50d75eb2fc standardize m4b encoding to VBR -q:a 2; replace remaining ffmpeg blocks in desktop GUI with domain modules 2026-07-15 13:28:21 +00:00
Artem Akymenko ae9ab70421 refactor: extract audio helpers to domain/audio_helpers.py
- Extract build_ffmpeg_command, to_float32, apply_m4b_chapters_with_mutagen
- _apply_m4b_chapters_with_mutagen becomes thin wrapper with error handling
- Add tests/test_audio_helpers.py (12 tests)
- conversion_runner.py: 1410 → 1320 lines
- All tests pass
2026-07-15 12:15:10 +00:00
Artem Akymenko 4364276a5b refactor: extract output path utilities to domain/output_paths.py
- Extract slugify, sanitize_output_stem, output_timestamp_token, build_output_path
- Extract apply_newline_policy, resolve_output_directory, resolve_project_layout
- _prepare_output_dir and _prepare_project_layout become thin wrappers with mkdir
- Add tests/test_output_paths.py (21 tests)
- conversion_runner.py: 1443 → 1410 lines
- All tests pass
2026-07-15 11:56:06 +00:00
Artem Akymenko 914e77de46 refactor: wire up domain/voice_utils.py and remove duplicates
- Import supertonic_voice_from_spec, split_speaker_reference, formula_from_kokoro_entry
- Import infer_provider_from_spec, coerce_truthy from domain/voice_utils.py
- Remove duplicate function bodies from conversion_runner.py
- conversion_runner.py: 1518 → 1443 lines
- All tests pass
2026-07-15 11:06:29 +00:00
Artem Akymenko 1d7a2aeed6 refactor: extract chunk utils to domain/chunk_utils.py
- Extract safe_int, group_chunks_by_chapter, record_override_usage, chunk_text_for_tts
- Add tests/test_chunk_utils.py (15 tests)
- Update test_chunk_helpers.py and test_chunk_text_for_tts_prefers_raw.py imports
- conversion_runner.py: 1574 → 1518 lines
- All tests pass
2026-07-15 11:00:18 +00:00
Artem Akymenko a26e02b017 refactor: wire up domain/chapter_overrides.py and domain/metadata_merge.py
- Update chapter_overrides.py to return tuple matching original signature
- Import apply_chapter_overrides and merge_metadata from domain modules
- Remove old function bodies from conversion_runner.py
- Add tests/test_chapter_merge_normalize.py (19 tests)
- conversion_runner.py: 1677 → 1574 lines
- All tests pass
2026-07-15 10:36:31 +00:00
Artem Akymenko c94347b33b refactor: extract voice resolution to domain/voice_resolution.py
- Extract spec_to_voice_ids, job_voice_fallback, collect_required_voice_ids
- Extract initialize_voice_cache, chapter_voice_spec, chunk_voice_spec
- Add tests/test_voice_resolution.py (29 tests)
- conversion_runner.py: 1822 → 1677 lines
- All tests pass
2026-07-15 10:20:58 +00:00
Artem Akymenko b7a48e3204 refactor: extract pronunciation rules to domain/pronunciation.py
- Extract compile_pronunciation_rules, compile_heteronym_sentence_rules
- Extract apply_pronunciation_rules, apply_heteronym_sentence_rules
- Extract merge_pronunciation_overrides
- Add tests/test_pronunciation.py (31 tests)
- All tests pass
2026-07-14 18:27:22 +00:00