fix(subtitles): enhance ellipsis and paragraph break handling in subtitle processing
feat(gui): implement background update check for new versions
test(tests): add tests for ellipsis handling and paragraph breaks preservation
- kokoro_text_normalization:
- refactor _cleanup_spacing to contextually distinguish opening vs closing
straight quotes (" and ') so leading quotes do not have spaces added after
them and spaces before opening quotes are preserved
- recognize non-English opening delimiters (¡, ¿, «, 「, etc.) and closing
delimiters (», 」, etc.) in spacing normalization
- in normalize_apostrophes, reconstruct text using original token offsets
to preserve inter-token whitespace instead of blindly joining with spaces
- subtitle_generation:
- add _is_sentence_boundary to recognize sentence punctuation followed by
closing quotes/brackets (.", !", ?") in regex and karaoke modes
- support proportional splitting for single FakeToken multi-sentence segments
in spaCy mode (Supertonic / non-English Kokoro)
- support newline splitting for Line mode in FakeToken fallbacks
- safely convert language and subtitle_mode inputs
- tests:
- add tests/test_subtitle_scenarios.py covering quotes, dialogues,
paragraphs, contractions, all subtitle modes, and all TTS token styles
Three related fixes:
- make_subtitle_writer: accept word-count modes ("N words") in
create_subtitle_writer. SubtitleMode("5 words") raised ValueError,
which make_subtitle_writer swallowed and returned None, so PyQt never
created or wrote the subtitle file when an "N words" mode was selected.
Unknown modes now fall back to SubtitleMode.SENTENCE (writers only
branch on SENTENCE_HIGHLIGHT). Regression test added.
- gui.py: refresh subtitle combo item availability after initial
voice/profile selection. update_subtitle_options_availability() ran
during initUI with selected_lang=None for a profile, taking the
non-English branch and disabling Highlighting/N-words items; the fix-up
never re-ran because setCurrentIndex on an already-current index emits
no signal. Now called once more after the profile/voice language is
resolved.
- AssWriter: stop discarding per-word karaoke timing. _add_karaoke_tags
unconditionally replaced entry text with uniform {\\k100} tags,
destroying the real per-word {\\kf} timings from
_process_karaoke_highlighting. Only synthesize simplified tags when the
text has no karaoke tags. Regression test added.
Sentence modes processed all text as a whole: Pipeline.__call__ merged every
engine segment back into one (whole text, no per-token timings), producing a
single giant subtitle and whole-text progress logs.
- tts_plugin/types: add TokenTiming, AudioSegment, SynthesizedAudio.segments
- tts_plugin/utils: Pipeline yields one Segment per engine segment (with
tokens); merged fallback only when engine provides none
- kokoro engine: expose per-segment graphemes/audio + per-word token timings
- supertonic engine: expose per-segment graphemes/audio (no tokens)
- split_pattern: English Sentence/Sentence+Comma engine split is newline-only
(boundaries applied at subtitle time via spaCy); non-English Sentence+Comma
with spaCy ON uses spaCy pre-segmentation + newline engine split (no
commas); spaCy-off fallback keeps comma pattern
- tts_segments: restore inter-segment whitespace on real per-word token
boundaries only (never FakeToken fallbacks)
- _to_language_enum: accept Language enum input (str(enum) is "Language.ES",
silently resolved to EN_US and disabled spaCy pre-TTS for every language
in WebUI)
- pyqt/conversion, utils: replace print with logging
- add AGENTS.md documenting the segmentation/subtitle contract for future
sessions
- tests: update English split-pattern expectations (1566 passing)
Complete the Language enum unification (commit 0dc491e) for the PyQt
GUI, which was left resolving languages from kokoro letter codes
(voice[0]) while domain and WebUI already used Language enums.
- selected_lang is now always a Language enum: voices resolve via
language_for_voice_id(), profiles via resolve_profile_language()
- kokoro letter->Language mapping stays in the engine (new public
language_for_code()); the Language.from_code() domain shim is removed
- legacy profile files with letter codes ("a", "e", ...) are tolerated
only at the profile read boundary; new profiles save ISO codes
- conversion.py letter comparisons replaced with enum comparisons, and
SynthParams lang_code= -> language= (would TypeError at runtime)
- subtitle dropdown enabled for all languages; word-count and
highlighting modes restricted to English with auto-switch to Sentence
- tests: engine mapping + profile language resolution added; 1566 pass
- Create domain/text_utils.py with canonical calculate_text_length
(strips chapter markers, voice markers, metadata tags)
- Remove calculate_text_length from utils.py and subtitle_utils.py
- Update all imports to use domain.text_utils directly
- Create application/chapter_selection.py with build_chapter_payload()
(orchestrates preselection + character count + safety net)
- Replace manual chapter logic in form.py with build_chapter_payload()
- 22 new tests for text_utils and chapter_selection (1550 total)
- Add expand_metadata_aliases() for concept fan-out (series→5, author→2,
description→2, tags→3 keys)
- Replace _normalize_metadata_tags in webui/service.py with
normalize_metadata_map from domain
- Replace _normalize_metadata in epub3/exporter.py with
normalize_metadata_map from domain
- Replace manual fan-out in form.py with expand_metadata_aliases()
- Rewrite metadata_overrides.py to use expand_metadata_aliases()
- Remove unused normalize_metadata_casefold import from exporters.py
- 19 new tests for expand_metadata_aliases (1528 total)
Move build_narrator_roster, build_speaker_roster, match_configured_speaker,
apply_speaker_config_to_roster, prepare_speaker_metadata from
webui/routes/utils/voice.py to domain/speaker_metadata.py.
- prepare_speaker_metadata now accepts optional inject_recommended callback
for UI-specific voice enrichment
- 47 new tests in test_domain_speaker_metadata.py
- All 1509 tests pass
- Add formula_from_profile, resolve_profile_voice, resolve_voice_setting, resolve_voice_choice to domain/voice_resolution.py
- Add build_voice_catalog, filter_voice_catalog to domain/voice_catalog.py
- Update webui/routes/utils/voice.py to import from domain
- Update webui/routes/utils/form.py and voices.py to import from domain directly
- Update synthesize.py to use domain resolve_voice
- Add 29 tests for voice resolution functions
- Move stored_integration_config(), build_audiobookshelf_config() to domain/settings_core.py
- Remove legacy fallback from stored_integration_config() (only config[integrations])
- Add load_audiobookshelf_config() as combined entry point
- PostConversionHooks reads config directly via stored_integration_config()
- WebUI imports from domain/settings_core instead of webui/routes/utils/settings
- Remove duplicate _build_abs_config() from PostConversionHooks
- Remove dead audiobookshelf code from infrastructure/exporters.py
- Remove duplicate audiobookshelf functions from webui/service.py
- Fix OutputFormat enum serialization in output_layout_service.py and conversion_executor.py
(f'{enum}' gave 'OutputFormat.WAV' instead of '.wav')
- Mock spaCy in test_returns_at_least_one_segment instead of loading real model
- Add 28 tests for PostConversionHooks and build_audiobookshelf_config
- Deleted _prepare_tts_context() (104 lines of duplicated logic)
- Replaced with direct build_tts_context() call in run_conversion()
- Fixed _finalize() to use raw fields (generate_epub3, epub3_book_id)
- Removed _MockJob antipattern
- Updated tests to use raw normalization_overrides field
- ConversionRequest uses raw fields instead of PronunciationConfig/Epub3ExportConfig
- Added speakers field to ConversionRequest
- Rewrote collect_required_voice_ids(), initialize_voice_cache(),
job_voice_fallback(), chapter_voice_spec(), chunk_voice_spec()
to accept ConversionRequest instead of job
- PipelinePool.get() now takes request= instead of job=
- Updated all tests to use ConversionRequest interface
- 1493 tests passing
- conversion_pipeline.py: new spacy_pre_tts_segmentation() function
- Handles: condition checks, language exclusion, split_pattern override
- Executor: integrated spaCy pre-TTS before each synthesize_text() call
- 6 new tests for condition checks and fallback behavior
- English excluded from pre-TTS (spaCy only for post-TTS subtitles)
- Fallback to regex when spaCy unavailable
- MarkerCollector: SRP extraction from executor (observation, not execution)
- Outro marker: executor now records outro as chapter marker
- Voice format: chapter voices [{provider, voice}], chunk voice {provider, voice}
- _finalize(): build_metadata_payload() + metadata.json + record_override_usage()
- ffmetadata: voices list → comma-separated string
- EPUB3: ChunkOverlay.voice = dict, _render_chunk_inline handles dict format
- 8 new tests: multi-speaker, ffmetadata format, EPUB3 format
- Updated existing tests for new voice format
- Remove 77 unused imports across domain/application/runner files via ruff
- Add # noqa: F401 to re-exports used by tests and debug_tts_runner
- Fix build_tts_context: usage_counter uses 'is not None' instead of truthiness
- Fix test assertions: compiled rules use 'replacement' key not 'pronunciation'
- Add re-exports: _compile_pronunciation_rules, _merge_pronunciation_overrides
- Language enum: en-US, en-GB, es, fr, hi, it, ja, pt-BR, zh
- Engine-specific mappings (kokoro → single-letter) live in pipeline_factory and synthesize
- spacy_utils uses Language enum keys for model mapping
- split_pattern uses Language enum properties (is_cjk)
- Updated all tests to use ISO codes
- SubtitleMode, OutputFormat, SaveMode, SubtitleFormat, InputFormat
- Properties: dot_ext, is_lossless, is_book, is_subtitle
- from_str/from_path class methods with normalization
- Updated domain and application layers to use Enums
- 17 new tests for enum validation and properties
- Frozen dataclass in domain/conversion_engine.py with common params
- synthesize_text now takes params=SynthParams + unique kwargs
- Executor, PyQt legacy, WebUI legacy, and tests updated
- Adding new common params now only requires changing the dataclass
7 tests for the unified conversion executor:
- simple text conversion
- multi-chapter with separate chapter output
- voice markers
- intro/outro
- cancellation behavior
- progress reporting
- metadata preservation
Uses FakeBackend, FakeAudioSink, FakeSubtitleWriter, FakeEvents,
FakePipelineProvider, FakeVoiceResolver to test without real TTS.
- voice_formulas.py: add pairs_to_formula() as canonical implementation
- webui/routes/utils/voice.py: formula_from_profile() and pairs_to_formula()
now delegate to voice_formulas.pairs_to_formula()
- pyqt/gui.py: get_voice_formula() now uses voice_formulas.pairs_to_formula()
instead of inline string formatting
- Eliminates 3 duplicate implementations of voice*weight formula building
- +9 tests
- 1178 tests pass
Apply the book form after extraction-backed pending job creation so explicit title and author values take precedence over fallback metadata. Add a focused regression test for the upload path.
- Replace get_pipeline() closure with PipelinePool from domain/pipeline_factory
- Replace resolve_voice_target() closure with domain function from voice_utils
- Remove dead _load_pipeline() function and unused is_plugin_registered import
- Add 33 tests for resolve_voice_target and PipelinePool
- Add 10 regression tests verifying domain extraction preserves behavior
- 1131 tests pass (+61 new)
Before:
- PyQt: inline ETR using chars-based formula
- WebUI: Job.estimated_time_remaining using progress-based formula
(different formulas → different ETR estimates)
After:
- Both UIs call domain.progress.calc_etr_str(elapsed, done, total)
- Same formula, same ETR, single source of truth
- WebUI now stores etr_str on Job and displays it directly
- Job.estimated_time_remaining property kept for backward compat
domain/progress.py: ProgressTracker class + calc_etr_str function
1053 tests pass.
Deniz Şafak