Commit Graph
154 Commits
Author SHA1 Message Date
Artem Akymenko a1241ee9ca refactor: word substitution in planner, add planner tests 2026-07-24 19:17:37 +03:00
Artem Akymenko 0ee5bb0496 refactor: config objects for feature toggles in ConversionRequest 2026-07-24 19:17:36 +03:00
Artem Akymenko 7d28b7eb52 test: mock subprocess.Popen in test_executor_m4b_forces_merge to eliminate ffmpeg dependency 2026-07-24 19:17:36 +03:00
Artem Akymenko fcec4e9fe5 fix: test_stretch_reduces_duration — remove stale self param, fix mock data size, fix atempo assertion 2026-07-22 15:32:18 +03:00
Artem Akymenko d3682e7672 refactor: dynamic ConversionRequest validation, remove 'or default' from adapters
- __post_init__: _apply_none_defaults() iterates dataclasses.fields() dynamically
- _NUMERIC_CONSTRAINTS and _ENUM_CONSTRAINTS dicts replace per-field if chains
- Both adapters pass values as-is (no 'or default' fallbacks)
- 18 validation tests + updated adapter tests for Enum assertions
2026-07-22 11:33:44 +00:00
Artem Akymenko 0805e9fdae refactor: Language Enum with ISO codes
- Language enum: en-US, en-GB, es, fr, hi, it, ja, pt-BR, zh
- Engine-specific mappings (kokoro → single-letter) live in pipeline_factory and synthesize
- spacy_utils uses Language enum keys for model mapping
- split_pattern uses Language enum properties (is_cjk)
- Updated all tests to use ISO codes
2026-07-22 10:54:39 +00:00
Artem Akymenko f6a8008f51 refactor: typed Enums for format/mode fields
- SubtitleMode, OutputFormat, SaveMode, SubtitleFormat, InputFormat
- Properties: dot_ext, is_lossless, is_book, is_subtitle
- from_str/from_path class methods with normalization
- Updated domain and application layers to use Enums
- 17 new tests for enum validation and properties
2026-07-22 09:01:50 +00:00
Artem Akymenko c4cebb8822 refactor: SynthParams dataclass for synthesize_text
- Frozen dataclass in domain/conversion_engine.py with common params
- synthesize_text now takes params=SynthParams + unique kwargs
- Executor, PyQt legacy, WebUI legacy, and tests updated
- Adding new common params now only requires changing the dataclass
2026-07-22 08:21:45 +00:00
Artem Akymenko 7f317ca784 test: adapter field mapping + import layering tests
- 28 adapter tests (WebUI + PyQt): field mapping, events, provider, resolver
- 15 import/layering tests: no PyQt/WebUI in app layer, all models importable
2026-07-21 11:47:46 +00:00
Artem Akymenko 71916aa39f fix: conversion_service import bug + coverage tests
Fix import error in _prepare_tts_context:
- apply_normalization_overrides doesn't exist in domain.normalization
- merge_pronunciation_overrides expects job-like object, not two lists
- Use _MockJob adapter to bridge ConversionRequest to existing API

Add test_application_coverage.py (23 tests):
- ConversionService: simple, logs, cancellation, empty text, multi-chapter, intro/outro, error
- OutputLayoutService: custom folder, source path, project, merged path, chapter path, should_merge
- Executor gaps: no layout, m4b, separate chapters, no intro/outro, voice fallback, silence

Coverage: 80% -> 92%
2026-07-21 09:34:47 +00:00
Artem Akymenko 75a3ad517a test: executor tests with fake backend/sink/ports
7 tests for the unified conversion executor:
- simple text conversion
- multi-chapter with separate chapter output
- voice markers
- intro/outro
- cancellation behavior
- progress reporting
- metadata preservation

Uses FakeBackend, FakeAudioSink, FakeSubtitleWriter, FakeEvents,
FakePipelineProvider, FakeVoiceResolver to test without real TTS.
2026-07-21 11:16:41 +03:00
Artem Akymenko 680418fa1d test: planner tests + domain regression tests
51 tests for the unified conversion planner:
- build_conversion_plan: direct text, voice markers, chunks, chapters, intro/outro, output layout
- Domain regression: chapter parsing, voice markers, TTSContext, voice resolution, intro/outro, output paths, subtitles
- All tests use domain functions only (no UI, no TTS, no audio I/O)
2026-07-21 11:16:41 +03:00
Artem Akymenko e1e49e8a0f test: regression tests for conversion flow unification
Three new test files describing expected behavior before refactoring:
- test_conversion_planner.py: chapter parsing, voice markers, TTSContext, intro/outro, output paths, subtitles (30 tests)
- test_conversion_request.py: settings, context building, chapter selection, cancellation/logging protocols (15 tests)
- test_conversion_executor.py: synthesize_text, process_and_write_subtitles, full pipeline with fake backend/sink (12 tests)

All 1310 tests pass (1253 existing + 57 new).
2026-07-21 11:16:40 +03:00
Artem Akymenko dbe73254a4 refactor: add SETTINGS_REGISTRY contract to domain/settings_core.py
- Setting dataclass: key, type, default, min/max, valid_values, scope
- 72 settings total: 54 shared, 18 PyQt-only
- validate_setting() checks types and ranges
- Setting.coerce() handles type conversion with bounds
- settings_defaults() / all_settings_defaults() derived from registry
- BOOLEAN_SETTINGS, FLOAT_SETTINGS, INT_SETTINGS now auto-derived
- 17 tests validating schema, coercion, and validation
2026-07-19 13:24:52 +00:00
Artem Akymenko 64e8a8f4e6 refactor: extract shared settings_core and Flask route logic to domain/services 2026-07-19 15:52:35 +03:00
Artem Akymenko aec3462f1f refactor: both UIs use shared tts_segments() from domain/conversion_pipeline.py
- domain/conversion_pipeline.py: add tts_segments() for pre-normalized text;
  emit_text_segments() now delegates to tts_segments() internally
- pyqt/conversion.py: inner TTS loop replaced with tts_segments() iterator;
  removed FakeToken import (handled by domain)
- webui/conversion_runner.py: emit_text() inner loop replaced with
  tts_segments() iterator; removed FakeToken import
- Both UIs now share the same TTS emission logic — normalization + backend
  invocation + segment iteration + token extraction
- +3 tests for tts_segments (no-normalization, chunk_start, basic)
- 1188 tests pass
2026-07-19 11:47:14 +00:00
Artem Akymenko 1193185833 refactor: create domain/conversion_pipeline.py with shared TTS emission loop
- domain/conversion_pipeline.py: emit_text_segments() — generator yielding
  SegmentResult for each TTS segment; emit_text_to_sinks() — convenience
  wrapper handling audio writing + token accumulation + subtitle flushing
- Both WebUI and PyQt can call these instead of reimplementing the TTS loop
- Caller provides backend, voice, speed, split_pattern; domain handles
  normalization, TTS invocation, token extraction
- +7 tests (segment yielding, empty audio skip, chunk_start, tokens, fallback)
- 1185 tests pass
2026-07-19 10:31:58 +00:00
Artem Akymenko a99cf58c79 refactor: consolidate voice formula building into voice_formulas.py
- voice_formulas.py: add pairs_to_formula() as canonical implementation
- webui/routes/utils/voice.py: formula_from_profile() and pairs_to_formula()
  now delegate to voice_formulas.pairs_to_formula()
- pyqt/gui.py: get_voice_formula() now uses voice_formulas.pairs_to_formula()
  instead of inline string formatting
- Eliminates 3 duplicate implementations of voice*weight formula building
- +9 tests
- 1178 tests pass
2026-07-19 10:02:14 +00:00
Artem Akymenko fe62b6b44c refactor: extract book metadata logic from PyQt to domain
- domain/metadata_extraction.py: add format_metadata_tags(),
  extract_book_metadata_epub(), extract_book_metadata_pdf(),
  extract_book_metadata_markdown(), _save_cover_to_cache()
- pyqt/book_handler.py: _extract_book_metadata() reduced from ~165 lines
  to ~10 lines by delegating to domain; _format_metadata_tags() reduced
  from ~55 lines to ~15 lines; ebooklib/fitz imports moved to domain
- +16 tests (format_metadata_tags, save_cover, markdown extraction)
- 1169 tests pass
2026-07-19 09:51:50 +00:00
Artem Akymenko 0e216f3786 refactor: extract _process_subtitle_file domain logic to shared modules
- domain/subtitle_processor.py: parse_subtitle_file, format_time_range,
  speed_up_audio, fit_audio_to_duration (moved from audio_buffer),
  process_subtitle_entries (core TTS loop with cancel/log/progress callbacks)
- domain/audio_buffer.py: add fit_audio_to_duration, ffmpeg_time_stretch
- domain/output_paths.py: add resolve_unique_path (collision-safe filename)
- pyqt/conversion.py: _process_subtitle_file reduced from ~350 to ~100 lines
  by delegating to domain functions; removed 4 unused subtitle parser imports
- +33 tests (8 resolve_unique_path, 14 subtitle_processor, 8 audio_buffer,
  3 format_time_range)
- 1153 tests pass
2026-07-19 09:34:17 +00:00
Artem Akymenko c76cf74efc refactor: unify duplicated logic between WebUI and PyQt
domain/output_paths.py:
- Add sanitize_filename_for_chapter() with OS safety + smart truncation
- Keep existing slugify() for backward compatibility

domain/text_chapters.py (NEW):
- parse_chapters_from_text() combines intro preservation (PyQt) + clean_text (WebUI)

PyQt/conversion.py:
- Replace 3x copy-pasted ASS/SRT headers with create_subtitle_writer()
- Replace inline M4B muxing with ExportService.embed_m4b_metadata()
- Replace inline chapter splitting with parse_chapters_from_text()
- Replace inline chapter filename sanitization with sanitize_filename_for_chapter()
- Remove unused _CHAPTER_MARKER_SEARCH_PATTERN import

Tests: 1152 passed (+21 new)
2026-07-19 07:50:02 +00:00
Artem Akymenko a299947bb1 refactor(webui): replace inline get_pipeline/resolve_voice_target closures with domain modules
- Replace get_pipeline() closure with PipelinePool from domain/pipeline_factory
- Replace resolve_voice_target() closure with domain function from voice_utils
- Remove dead _load_pipeline() function and unused is_plugin_registered import
- Add 33 tests for resolve_voice_target and PipelinePool
- Add 10 regression tests verifying domain extraction preserves behavior
- 1131 tests pass (+61 new)
2026-07-18 14:13:17 +03:00
Artem Akymenko d8634f812d refactor(domain): extract audio sink abstraction to domain layer
- New domain/audio_sink.py: AudioSink context manager + open_audio_sink() factory
  - Supports WAV/FLAC (soundfile) and MP3/Opus/M4B (ffmpeg pipe)
  - cancel_check, extra_ffmpeg_args, ffmpeg_cmd parameters
  - 17 tests in test_domain_audio_sink.py

- WebUI: replaced local AudioSink + _open_audio_sink with domain module
  - Removed 35 lines, 3 call sites now use open_audio_sink()

- PyQt: replaced 3 inline audio output setups with domain module
  - New _open_merged_sink() helper encapsulates m4b cover art logic
  - ExitStack for automatic cleanup in main conversion
  - Removed ~140 lines of duplicated ffmpeg/soundfile boilerplate
2026-07-18 14:12:01 +03:00
Artem Akymenko a1d93820b1 refactor(domain): unify ETR calculation — both UIs now use calc_etr_str
Before:
  - PyQt: inline ETR using chars-based formula
  - WebUI: Job.estimated_time_remaining using progress-based formula
  (different formulas → different ETR estimates)

After:
  - Both UIs call domain.progress.calc_etr_str(elapsed, done, total)
  - Same formula, same ETR, single source of truth
  - WebUI now stores etr_str on Job and displays it directly
  - Job.estimated_time_remaining property kept for backward compat

domain/progress.py: ProgressTracker class + calc_etr_str function
1053 tests pass.
2026-07-16 09:31:26 +00:00
Artem Akymenko 2228f37c06 refactor(domain): add prepare_text_for_tts — unified normalization pipeline
New function chains all three normalization stages:
  1. Heteronym sentence rules (context-dependent pronunciation)
  2. Pronunciation rules (token-level replacements)
  3. Pipeline normalization (apostrophe, LLM)

This is the single entry point that both Web UI and PyQt should call
before TTS synthesis. Currently only Web UI uses it; PyQt has NO
normalization — this unlocks that capability.

Updated conversion_runner.emit_text to use the new function.
1038 tests pass.
2026-07-16 08:53:15 +00:00
Artem Akymenko 832e2c5197 refactor(domain): extract chapter classification heuristics from form.py
Moved supplement_score, should_preselect_chapter, and
ensure_at_least_one_chapter_enabled to domain/chapter_classification.py.

Also moved coerce_bool from settings.py to common.py to break circular
import introduced in previous commit.

1025 tests pass.
2026-07-16 08:13:37 +00:00
Artem Akymenko c2c584e741 refactor(domain): deduplicate metadata helpers across 3 layers
Extracted 8 metadata functions from service.py, exporters.py, and
audiobookshelf.py into domain/metadata_helpers.py:
- normalize_metadata_casefold, split_people_field, split_simple_list
- first_nonempty, extract_year, normalize_series_sequence
- build_audiobookshelf_metadata, load_audiobookshelf_chapters

service.py, exporters.py, and audiobookshelf.py now import from domain
instead of maintaining separate copies. Thin wrappers adapt to layer
interfaces (Job objects, etc.).

Net -231 lines. 1006 tests pass.
2026-07-16 07:34:58 +00:00
Artem Akymenko 8ccdc85ccb refactor(webui): rename preview.py to synthesize.py and remove dead code
- Rename abogen/webui/routes/utils/preview.py → synthesize.py
  The file contains the core TTS synthesis pipeline (generate_preview_audio,
  synthesize_preview), not just preview logic. Name now matches responsibility.
- Remove dead code from voice.py: get_preview_pipeline(), synthesize_audio_from_normalized(),
  _preview_pipeline_lock, _preview_pipelines, and unused imports (threading, numpy,
  create_pipeline, get_new_voice, _select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN).
  These were never called — identical logic lives in synthesize.py.
- Update imports in api.py, voices.py, and test_preview_applies_manual_overrides.py
- 7 new tests in test_synthesize_module.py enforce file naming and import rules
- 7 tests in test_domain_imports.py updated for renamed module
2026-07-16 10:19:01 +03:00
Artem Akymenko ef07a8b5b2 fix(tests): mock spacy in plugin tests to fix externally-managed-environment failures 2026-07-16 10:02:01 +03:00
Artem Akymenko 1268a83cff fix(webui): voice.py imports from domain modules instead of conversion_runner
Replace private imports (_select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN)
from abogen.webui.conversion_runner with proper domain imports:
- select_device from abogen.domain.device
- to_float32, SAMPLE_RATE from abogen.domain.audio_helpers
- SPLIT_PATTERN defined locally (r'\n+')

Also verifies preview.py already uses domain imports correctly.

7 new tests in tests/test_domain_imports.py enforce the architecture rule.
2026-07-16 06:57:12 +00:00
Artem Akymenko ef6faff2e8 refactor: extract metadata processing logic to domain layer
- Replace manual metadata extraction with regex in pyqt/conversion.py
  with calls to domain/metadata_extraction.py functions
- Remove duplicate _embed_m4b_metadata and _apply_m4b_chapters_with_mutagen
  functions from webui/conversion_runner.py
- Use ExportService.embed_m4b_metadata for m4b metadata embedding
- Reduce code duplication between PyQt and WebUI interfaces
2026-07-15 20:44:19 +03:00
Artem Akymenko da9d5e7eb9 fix(tests): audio_buffer and subtitle_generation tests
- Fix mix_audio to return target buffer (was not modifying in-place)
- Fix samples_for_duration to return 0 for negative durations
- Fix test assertions for numpy 2.x compatibility (share_memory -> shares_memory)
- Adjust subtitle_generation tests to match actual behavior
2026-07-15 20:26:31 +03:00
Artem Akymenko acb000b9e6 refactor: extract voice loading logic to domain layer
- Add abogen/domain/voice_loader.py with:
  - VoiceCache class: unified cache for loaded voices
  - resolve_voice(): load voice with optional caching
  - load_voice_cached(): compatibility wrapper for PyQt

- Update abogen/pyqt/conversion.py:
  - Replace load_voice_cached method body with call to domain function
  - Maintain backward compatibility with existing interface

- Add tests/test_voice_loader.py with unit tests for VoiceCache and voice loading
2026-07-15 20:20:18 +03:00
Artem Akymenko d6c66dc18a refactor: extract subtitle token processing to domain layer
- Add abogen/domain/subtitle_generation.py with:
  - process_subtitle_tokens(): main function for converting TTS tokens to subtitles
  - Support for all subtitle modes: Line, Sentence, Sentence + Comma, Sentence + Highlighting
  - Support for word-count based grouping (e.g., '5' for 5 words per entry)
  - spaCy integration for English sentence boundary detection
  - Karaoke highlighting tags for Sentence + Highlighting mode
  - Punctuation constants for sentence splitting

- Update abogen/pyqt/conversion.py:
  - Replace _process_subtitle_tokens method body with call to domain function
  - Remove ~260 lines of duplicate logic

- Add tests/test_subtitle_generation.py with comprehensive unit tests
2026-07-15 20:14:02 +03:00
Artem Akymenko 0d46076bf6 refactor: extract audio buffer operations to domain layer
- Add abogen/domain/audio_buffer.py with core audio operations:
  - create_silence(): create silence audio buffer
  - mix_audio(): mix source into target buffer with auto-resize
  - normalize_audio(): normalize to prevent clipping
  - ensure_buffer_size(): extend buffer to minimum size
  - concatenate_audio(): join multiple audio buffers
  - audio_duration(): calculate duration from samples
  - samples_for_duration(): calculate samples from duration
  - SAMPLE_RATE constant (24000)

- Update abogen/pyqt/conversion.py:
  - Import and use create_silence for chapter silence
  - Use mix_audio for subtitle file mixing
  - Use normalize_audio for clipping prevention
  - Use create_silence for padding in subtitle processing

- Update abogen/webui/conversion_runner.py:
  - Import and use create_silence in append_silence
  - Replace np.zeros with domain function

- Add tests/test_audio_buffer.py with comprehensive unit tests
2026-07-15 20:02:33 +03:00
Artem Akymenko 7fef9c1d93 extract normalize_text_for_pipeline to domain/normalization.py 2026-07-15 15:19:01 +00:00
Artem Akymenko 56cfd0810d extract resolve_fallback_voice_spec to domain/voice_resolution.py; fix missing get_default_voice import and __custom_mix reset bug 2026-07-15 15:01:17 +00:00
Artem Akymenko 7bd3177241 extract select_device to domain/device.py; fix bug where conversion_runner didn't check torch availability 2026-07-15 14:45:38 +00:00
Artem Akymenko d5c2a81733 Merge pull request #191 from hydraxman/fix/large-chapter-form-limits
fix(webui): allow large chapter forms
2026-07-15 17:35:52 +03:00
Artem Akymenko 514e29a761 extract apply_chapter_text_transforms to domain/chapter_titles.py 2026-07-15 14:29:29 +00:00
Artem Akymenko 50d75eb2fc standardize m4b encoding to VBR -q:a 2; replace remaining ffmpeg blocks in desktop GUI with domain modules 2026-07-15 13:28:21 +00:00
Artem Akymenko ae9ab70421 refactor: extract audio helpers to domain/audio_helpers.py
- Extract build_ffmpeg_command, to_float32, apply_m4b_chapters_with_mutagen
- _apply_m4b_chapters_with_mutagen becomes thin wrapper with error handling
- Add tests/test_audio_helpers.py (12 tests)
- conversion_runner.py: 1410 → 1320 lines
- All tests pass
2026-07-15 12:15:10 +00:00
Artem Akymenko 4364276a5b refactor: extract output path utilities to domain/output_paths.py
- Extract slugify, sanitize_output_stem, output_timestamp_token, build_output_path
- Extract apply_newline_policy, resolve_output_directory, resolve_project_layout
- _prepare_output_dir and _prepare_project_layout become thin wrappers with mkdir
- Add tests/test_output_paths.py (21 tests)
- conversion_runner.py: 1443 → 1410 lines
- All tests pass
2026-07-15 11:56:06 +00:00
Artem Akymenko 1d7a2aeed6 refactor: extract chunk utils to domain/chunk_utils.py
- Extract safe_int, group_chunks_by_chapter, record_override_usage, chunk_text_for_tts
- Add tests/test_chunk_utils.py (15 tests)
- Update test_chunk_helpers.py and test_chunk_text_for_tts_prefers_raw.py imports
- conversion_runner.py: 1574 → 1518 lines
- All tests pass
2026-07-15 11:00:18 +00:00
Artem Akymenko a26e02b017 refactor: wire up domain/chapter_overrides.py and domain/metadata_merge.py
- Update chapter_overrides.py to return tuple matching original signature
- Import apply_chapter_overrides and merge_metadata from domain modules
- Remove old function bodies from conversion_runner.py
- Add tests/test_chapter_merge_normalize.py (19 tests)
- conversion_runner.py: 1677 → 1574 lines
- All tests pass
2026-07-15 10:36:31 +00:00
Artem Akymenko c94347b33b refactor: extract voice resolution to domain/voice_resolution.py
- Extract spec_to_voice_ids, job_voice_fallback, collect_required_voice_ids
- Extract initialize_voice_cache, chapter_voice_spec, chunk_voice_spec
- Add tests/test_voice_resolution.py (29 tests)
- conversion_runner.py: 1822 → 1677 lines
- All tests pass
2026-07-15 10:20:58 +00:00
Artem Akymenko b7a48e3204 refactor: extract pronunciation rules to domain/pronunciation.py
- Extract compile_pronunciation_rules, compile_heteronym_sentence_rules
- Extract apply_pronunciation_rules, apply_heteronym_sentence_rules
- Extract merge_pronunciation_overrides
- Add tests/test_pronunciation.py (31 tests)
- All tests pass
2026-07-14 18:27:22 +00:00
Artem Akymenko f63590932d refactor: extract voice utils to domain/voice_utils.py
- Extract infer_provider_from_spec, supertonic_voice_from_spec, split_speaker_reference, formula_from_kokoro_entry, coerce_truthy to domain/voice_utils.py
- Add tests/test_voice_utils.py with 24 tests
- All tests match old behavior
2026-07-14 11:02:34 +00:00
Artem Akymenko 7777e58f1d refactor: extract title/outro builders into domain/title_builder.py
- Extract build_title_intro_text and build_outro_text into domain/title_builder.py
- Uses metadata_helpers for metadata processing
- Remove _build_title_intro_text and _build_outro_text from conversion_runner.py
- Add tests/test_title_builder.py with 12 tests
- All tests match old behavior
2026-07-14 10:27:48 +00:00
Artem Akymenko 364c179bd6 refactor: extract metadata helpers into domain/metadata_helpers.py
- Extract normalize_metadata_map, format_author_sentence, ensure_sentence
- Extract normalize_series_number, extract_series_metadata, format_series_sentence
- Remove _SERIES_NAME_KEYS, _SERIES_NUMBER_KEYS, _SERIES_NUMBER_RE from conversion_runner.py
- Add tests/test_metadata_helpers.py with 32 tests
- All tests match old behavior
2026-07-14 10:26:05 +00:00