Compare commits

...
46 Commits
Author SHA1 Message Date
Artem Akymenko a299947bb1 refactor(webui): replace inline get_pipeline/resolve_voice_target closures with domain modules
- Replace get_pipeline() closure with PipelinePool from domain/pipeline_factory
- Replace resolve_voice_target() closure with domain function from voice_utils
- Remove dead _load_pipeline() function and unused is_plugin_registered import
- Add 33 tests for resolve_voice_target and PipelinePool
- Add 10 regression tests verifying domain extraction preserves behavior
- 1131 tests pass (+61 new)
2026-07-18 14:13:17 +03:00
Artem Akymenko 957c6778f6 refactor(pyt): dedup voice formula resolution in PyQt
Replace 2 inline 'if * in voice: get_new_voice(...)' patterns with
resolve_voice() from domain/voice_loader.py. Removes unused import
of get_new_voice.
2026-07-18 14:13:16 +03:00
Artem Akymenko fcdaf2b2a8 refactor(domain): extract FakeToken to domain/tokens.py
Shared token stub used by both WebUI and PyQt for languages
without per-word token support.
2026-07-18 14:12:18 +03:00
Artem Akymenko d8634f812d refactor(domain): extract audio sink abstraction to domain layer
- New domain/audio_sink.py: AudioSink context manager + open_audio_sink() factory
  - Supports WAV/FLAC (soundfile) and MP3/Opus/M4B (ffmpeg pipe)
  - cancel_check, extra_ffmpeg_args, ffmpeg_cmd parameters
  - 17 tests in test_domain_audio_sink.py

- WebUI: replaced local AudioSink + _open_audio_sink with domain module
  - Removed 35 lines, 3 call sites now use open_audio_sink()

- PyQt: replaced 3 inline audio output setups with domain module
  - New _open_merged_sink() helper encapsulates m4b cover art logic
  - ExitStack for automatic cleanup in main conversion
  - Removed ~140 lines of duplicated ffmpeg/soundfile boilerplate
2026-07-18 14:12:01 +03:00
Artem Akymenko 85b5851786 refactor(pyt): replace all inline float32 conversion with to_float32()
PyQt had 7 inline float32 conversion patterns:
  hasattr(x, 'numpy') ? x.numpy().astype('float32') : x.astype('float32')
spread across TTS loop, subtitle processing, and streaming.

All replaced with domain.audio_helpers.to_float32() which handles:
- None → zeros
- PyTorch tensors → .detach().cpu().numpy()
- Plain numpy → asarray(dtype=float32)
- reshape(-1) for consistent 1D output

The old inline code missed .detach() and .cpu() on GPU tensors,
causing potential crashes. Now both UIs use the same robust conversion.

1053 tests pass.
2026-07-18 06:58:21 +00:00
Artem Akymenko e77c8b3372 fix: review cleanup — imports, _FakeToken, use_spacy_segmentation
- Move pronunciation imports from inside run() to top-level imports
- Extract _FakeToken to module level (was redefined every loop iteration)
- use_spacy_segmentation now mirrors PyQt logic: pass the flag,
  let process_subtitle_tokens filter by language internally
2026-07-18 06:52:28 +00:00
Artem Akymenko 294069e53e refactor(webui): token-level subtitle processing via process_subtitle_tokens — P0
Before: WebUI wrote one subtitle entry per TTS segment (no sentence
grouping, no comma splitting, no karaoke highlighting). The subtitle
modes 'Sentence', 'Sentence + Comma', and 'Sentence + Highlighting'
produced broken output.

After: emit_text() accumulates tokens_with_timestamps from each
segment's .tokens attribute, then flushes them through
domain.subtitle_generation.process_subtitle_tokens() at the end.
This gives the WebUI the same subtitle quality as the PyQt desktop GUI:
- Sentence mode: groups tokens into sentences
- Sentence + Comma: splits on commas within sentences
- Sentence + Highlighting: karaoke timing per word
- Word-count mode: groups by N words

Also removed the duplicate _to_float32 function from synthesize.py
(now imports from domain.audio_helpers).

1053 tests pass.
2026-07-18 06:36:19 +00:00
Artem Akymenko 4ff09be664 refactor(pyt): add text normalization via prepare_text_for_tts — P0
PyQt desktop GUI now calls the shared normalization pipeline before
TTS synthesis, matching the Web UI's behavior:

1. Heteronym sentence rules (context-dependent pronunciation)
2. Pronunciation rules (token-level replacements)
3. Pipeline normalization (apostrophe handling, LLM)

Before: PyQt passed raw text to the backend — no normalization at all,
resulting in inferior audio quality compared to the Web UI.

The normalization rules are compiled once at the start of run() from
pronunciation_overrides and heteronym_overrides (currently None since
the PyQt GUI doesn't expose these settings yet — basic apostrophe
normalization still applies).

1053 tests pass.
2026-07-18 09:28:53 +03:00
Artem Akymenko a1d93820b1 refactor(domain): unify ETR calculation — both UIs now use calc_etr_str
Before:
  - PyQt: inline ETR using chars-based formula
  - WebUI: Job.estimated_time_remaining using progress-based formula
  (different formulas → different ETR estimates)

After:
  - Both UIs call domain.progress.calc_etr_str(elapsed, done, total)
  - Same formula, same ETR, single source of truth
  - WebUI now stores etr_str on Job and displays it directly
  - Job.estimated_time_remaining property kept for backward compat

domain/progress.py: ProgressTracker class + calc_etr_str function
1053 tests pass.
2026-07-16 09:31:26 +00:00
Artem Akymenko 0c1a3c1904 refactor(webui): use shared get_split_pattern instead of hardcoded \\n+
All three Web UI consumers now call domain.split_pattern.get_split_pattern()
which selects the correct split pattern based on language and subtitle mode.

Before: WebUI always split on \\n+ regardless of language (CJK missed
punctuation-based splitting that PyQt already had).
After: Both UIs share identical language-aware splitting logic.

1038 tests pass.
2026-07-16 09:12:46 +00:00
Artem Akymenko 2228f37c06 refactor(domain): add prepare_text_for_tts — unified normalization pipeline
New function chains all three normalization stages:
  1. Heteronym sentence rules (context-dependent pronunciation)
  2. Pronunciation rules (token-level replacements)
  3. Pipeline normalization (apostrophe, LLM)

This is the single entry point that both Web UI and PyQt should call
before TTS synthesis. Currently only Web UI uses it; PyQt has NO
normalization — this unlocks that capability.

Updated conversion_runner.emit_text to use the new function.
1038 tests pass.
2026-07-16 08:53:15 +00:00
Artem Akymenko 832e2c5197 refactor(domain): extract chapter classification heuristics from form.py
Moved supplement_score, should_preselect_chapter, and
ensure_at_least_one_chapter_enabled to domain/chapter_classification.py.

Also moved coerce_bool from settings.py to common.py to break circular
import introduced in previous commit.

1025 tests pass.
2026-07-16 08:13:37 +00:00
Artem Akymenko 17229b2390 refactor(webui): deduplicate _extract_checkbox (3 copies → 1)
Extracted extract_checkbox from settings.py, form.py (x2) into
webui/routes/utils/common.py. Moved coerce_bool from settings.py
to common.py to break circular import.

Fixed bug in second form.py copy (was missing __contains__ check).
1006 tests pass.
2026-07-16 07:46:51 +00:00
Artem Akymenko c2c584e741 refactor(domain): deduplicate metadata helpers across 3 layers
Extracted 8 metadata functions from service.py, exporters.py, and
audiobookshelf.py into domain/metadata_helpers.py:
- normalize_metadata_casefold, split_people_field, split_simple_list
- first_nonempty, extract_year, normalize_series_sequence
- build_audiobookshelf_metadata, load_audiobookshelf_chapters

service.py, exporters.py, and audiobookshelf.py now import from domain
instead of maintaining separate copies. Thin wrappers adapt to layer
interfaces (Job objects, etc.).

Net -231 lines. 1006 tests pass.
2026-07-16 07:34:58 +00:00
Artem Akymenko 8ccdc85ccb refactor(webui): rename preview.py to synthesize.py and remove dead code
- Rename abogen/webui/routes/utils/preview.py → synthesize.py
  The file contains the core TTS synthesis pipeline (generate_preview_audio,
  synthesize_preview), not just preview logic. Name now matches responsibility.
- Remove dead code from voice.py: get_preview_pipeline(), synthesize_audio_from_normalized(),
  _preview_pipeline_lock, _preview_pipelines, and unused imports (threading, numpy,
  create_pipeline, get_new_voice, _select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN).
  These were never called — identical logic lives in synthesize.py.
- Update imports in api.py, voices.py, and test_preview_applies_manual_overrides.py
- 7 new tests in test_synthesize_module.py enforce file naming and import rules
- 7 tests in test_domain_imports.py updated for renamed module
2026-07-16 10:19:01 +03:00
Artem Akymenko ef07a8b5b2 fix(tests): mock spacy in plugin tests to fix externally-managed-environment failures 2026-07-16 10:02:01 +03:00
Artem Akymenko 1268a83cff fix(webui): voice.py imports from domain modules instead of conversion_runner
Replace private imports (_select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN)
from abogen.webui.conversion_runner with proper domain imports:
- select_device from abogen.domain.device
- to_float32, SAMPLE_RATE from abogen.domain.audio_helpers
- SPLIT_PATTERN defined locally (r'\n+')

Also verifies preview.py already uses domain imports correctly.

7 new tests in tests/test_domain_imports.py enforce the architecture rule.
2026-07-16 06:57:12 +00:00
Artem Akymenko ef6faff2e8 refactor: extract metadata processing logic to domain layer
- Replace manual metadata extraction with regex in pyqt/conversion.py
  with calls to domain/metadata_extraction.py functions
- Remove duplicate _embed_m4b_metadata and _apply_m4b_chapters_with_mutagen
  functions from webui/conversion_runner.py
- Use ExportService.embed_m4b_metadata for m4b metadata embedding
- Reduce code duplication between PyQt and WebUI interfaces
2026-07-15 20:44:19 +03:00
Artem Akymenko da9d5e7eb9 fix(tests): audio_buffer and subtitle_generation tests
- Fix mix_audio to return target buffer (was not modifying in-place)
- Fix samples_for_duration to return 0 for negative durations
- Fix test assertions for numpy 2.x compatibility (share_memory -> shares_memory)
- Adjust subtitle_generation tests to match actual behavior
2026-07-15 20:26:31 +03:00
Artem Akymenko acb000b9e6 refactor: extract voice loading logic to domain layer
- Add abogen/domain/voice_loader.py with:
  - VoiceCache class: unified cache for loaded voices
  - resolve_voice(): load voice with optional caching
  - load_voice_cached(): compatibility wrapper for PyQt

- Update abogen/pyqt/conversion.py:
  - Replace load_voice_cached method body with call to domain function
  - Maintain backward compatibility with existing interface

- Add tests/test_voice_loader.py with unit tests for VoiceCache and voice loading
2026-07-15 20:20:18 +03:00
Artem Akymenko d6c66dc18a refactor: extract subtitle token processing to domain layer
- Add abogen/domain/subtitle_generation.py with:
  - process_subtitle_tokens(): main function for converting TTS tokens to subtitles
  - Support for all subtitle modes: Line, Sentence, Sentence + Comma, Sentence + Highlighting
  - Support for word-count based grouping (e.g., '5' for 5 words per entry)
  - spaCy integration for English sentence boundary detection
  - Karaoke highlighting tags for Sentence + Highlighting mode
  - Punctuation constants for sentence splitting

- Update abogen/pyqt/conversion.py:
  - Replace _process_subtitle_tokens method body with call to domain function
  - Remove ~260 lines of duplicate logic

- Add tests/test_subtitle_generation.py with comprehensive unit tests
2026-07-15 20:14:02 +03:00
Artem Akymenko 0d46076bf6 refactor: extract audio buffer operations to domain layer
- Add abogen/domain/audio_buffer.py with core audio operations:
  - create_silence(): create silence audio buffer
  - mix_audio(): mix source into target buffer with auto-resize
  - normalize_audio(): normalize to prevent clipping
  - ensure_buffer_size(): extend buffer to minimum size
  - concatenate_audio(): join multiple audio buffers
  - audio_duration(): calculate duration from samples
  - samples_for_duration(): calculate samples from duration
  - SAMPLE_RATE constant (24000)

- Update abogen/pyqt/conversion.py:
  - Import and use create_silence for chapter silence
  - Use mix_audio for subtitle file mixing
  - Use normalize_audio for clipping prevention
  - Use create_silence for padding in subtitle processing

- Update abogen/webui/conversion_runner.py:
  - Import and use create_silence in append_silence
  - Replace np.zeros with domain function

- Add tests/test_audio_buffer.py with comprehensive unit tests
2026-07-15 20:02:33 +03:00
Artem Akymenko 7fef9c1d93 extract normalize_text_for_pipeline to domain/normalization.py 2026-07-15 15:19:01 +00:00
Artem Akymenko 56cfd0810d extract resolve_fallback_voice_spec to domain/voice_resolution.py; fix missing get_default_voice import and __custom_mix reset bug 2026-07-15 15:01:17 +00:00
Artem Akymenko 7bd3177241 extract select_device to domain/device.py; fix bug where conversion_runner didn't check torch availability 2026-07-15 14:45:38 +00:00
Artem AkymenkoandGitHub d5c2a81733 Merge pull request #191 from hydraxman/fix/large-chapter-form-limits
fix(webui): allow large chapter forms
2026-07-15 17:35:52 +03:00
Artem Akymenko 514e29a761 extract apply_chapter_text_transforms to domain/chapter_titles.py 2026-07-15 14:29:29 +00:00
Artem Akymenko 86042a3315 fix bugs, remove dead code and unused imports in conversion_runner.py 2026-07-15 13:50:18 +00:00
Artem Akymenko 50d75eb2fc standardize m4b encoding to VBR -q:a 2; replace remaining ffmpeg blocks in desktop GUI with domain modules 2026-07-15 13:28:21 +00:00
Artem Akymenko ae9ab70421 refactor: extract audio helpers to domain/audio_helpers.py
- Extract build_ffmpeg_command, to_float32, apply_m4b_chapters_with_mutagen
- _apply_m4b_chapters_with_mutagen becomes thin wrapper with error handling
- Add tests/test_audio_helpers.py (12 tests)
- conversion_runner.py: 1410 → 1320 lines
- All tests pass
2026-07-15 12:15:10 +00:00
Artem Akymenko 4364276a5b refactor: extract output path utilities to domain/output_paths.py
- Extract slugify, sanitize_output_stem, output_timestamp_token, build_output_path
- Extract apply_newline_policy, resolve_output_directory, resolve_project_layout
- _prepare_output_dir and _prepare_project_layout become thin wrappers with mkdir
- Add tests/test_output_paths.py (21 tests)
- conversion_runner.py: 1443 → 1410 lines
- All tests pass
2026-07-15 11:56:06 +00:00
Artem Akymenko 914e77de46 refactor: wire up domain/voice_utils.py and remove duplicates
- Import supertonic_voice_from_spec, split_speaker_reference, formula_from_kokoro_entry
- Import infer_provider_from_spec, coerce_truthy from domain/voice_utils.py
- Remove duplicate function bodies from conversion_runner.py
- conversion_runner.py: 1518 → 1443 lines
- All tests pass
2026-07-15 11:06:29 +00:00
Artem Akymenko 1d7a2aeed6 refactor: extract chunk utils to domain/chunk_utils.py
- Extract safe_int, group_chunks_by_chapter, record_override_usage, chunk_text_for_tts
- Add tests/test_chunk_utils.py (15 tests)
- Update test_chunk_helpers.py and test_chunk_text_for_tts_prefers_raw.py imports
- conversion_runner.py: 1574 → 1518 lines
- All tests pass
2026-07-15 11:00:18 +00:00
Artem Akymenko a26e02b017 refactor: wire up domain/chapter_overrides.py and domain/metadata_merge.py
- Update chapter_overrides.py to return tuple matching original signature
- Import apply_chapter_overrides and merge_metadata from domain modules
- Remove old function bodies from conversion_runner.py
- Add tests/test_chapter_merge_normalize.py (19 tests)
- conversion_runner.py: 1677 → 1574 lines
- All tests pass
2026-07-15 10:36:31 +00:00
Artem Akymenko c94347b33b refactor: extract voice resolution to domain/voice_resolution.py
- Extract spec_to_voice_ids, job_voice_fallback, collect_required_voice_ids
- Extract initialize_voice_cache, chapter_voice_spec, chunk_voice_spec
- Add tests/test_voice_resolution.py (29 tests)
- conversion_runner.py: 1822 → 1677 lines
- All tests pass
2026-07-15 10:20:58 +00:00
Artem Akymenko b7a48e3204 refactor: extract pronunciation rules to domain/pronunciation.py
- Extract compile_pronunciation_rules, compile_heteronym_sentence_rules
- Extract apply_pronunciation_rules, apply_heteronym_sentence_rules
- Extract merge_pronunciation_overrides
- Add tests/test_pronunciation.py (31 tests)
- All tests pass
2026-07-14 18:27:22 +00:00
Artem Akymenko feb38a24ec fix: create missing domain/file_type.py from previous incomplete refactoring 2026-07-14 18:27:13 +00:00
Artem Akymenko f63590932d refactor: extract voice utils to domain/voice_utils.py
- Extract infer_provider_from_spec, supertonic_voice_from_spec, split_speaker_reference, formula_from_kokoro_entry, coerce_truthy to domain/voice_utils.py
- Add tests/test_voice_utils.py with 24 tests
- All tests match old behavior
2026-07-14 11:02:34 +00:00
Artem Akymenko 7777e58f1d refactor: extract title/outro builders into domain/title_builder.py
- Extract build_title_intro_text and build_outro_text into domain/title_builder.py
- Uses metadata_helpers for metadata processing
- Remove _build_title_intro_text and _build_outro_text from conversion_runner.py
- Add tests/test_title_builder.py with 12 tests
- All tests match old behavior
2026-07-14 10:27:48 +00:00
Artem Akymenko 364c179bd6 refactor: extract metadata helpers into domain/metadata_helpers.py
- Extract normalize_metadata_map, format_author_sentence, ensure_sentence
- Extract normalize_series_number, extract_series_metadata, format_series_sentence
- Remove _SERIES_NAME_KEYS, _SERIES_NUMBER_KEYS, _SERIES_NUMBER_RE from conversion_runner.py
- Add tests/test_metadata_helpers.py with 32 tests
- All tests match old behavior
2026-07-14 10:26:05 +00:00
Artem Akymenko 60ba01557e refactor: extract chapter title processing into domain/chapter_titles.py
- Extract simplify_heading_text, headings_equivalent, strip_duplicate_heading_line
- Extract normalize_caps_word, normalize_chapter_opening_caps
- Extract format_spoken_chapter_title
- Remove _HEADING_SANITIZE_RE, _HEADING_NUMBER_PREFIX_RE, _ACRONYM_ALLOWLIST, _ROMAN_NUMERAL_CHARS, _CAPS_WORD_RE from conversion_runner.py
- Add tests/test_chapter_titles.py with 31 tests
- All tests match old behavior
2026-07-14 10:17:20 +00:00
Artem Akymenko 39eac9b032 refactor: replace _srt_time/_ass_time with _format_timestamp from infrastructure/subtitle_writer.py
- Remove _srt_time() and _ass_time() methods from ConversionThread
- Use _format_timestamp() from infrastructure/subtitle_writer.py instead
- Supports both SRT (ass=False) and ASS (ass=True) formats
- All existing tests pass
2026-07-14 10:14:58 +00:00
Artem Akymenko 1499a3b426 refactor: extract _get_split_pattern into domain/split_pattern.py
- Extract unified split pattern logic to domain/split_pattern.py
- Add get_split_pattern() function with language and subtitle_mode support
- Remove duplicated logic from pyqt/conversion.py
- Update pyqt/conversion.py to use domain.split_pattern.get_split_pattern
- Add tests/test_split_pattern.py with 20 tests covering English, CJK, Spanish, French, and pattern structure
2026-07-14 10:11:52 +00:00
Artem Akymenko 013c80b92c refactor: migrate FFmpeg metadata functions to infrastructure/exporters.py
- Extract FFmpeg metadata functions to infrastructure/exporters.py as ExportService
- _escape_ffmetadata_value → _escape_ffmetadata_value
- _render_ffmetadata → render_ffmetadata
- _write_ffmetadata_file → write_ffmetadata_file
- _metadata_to_ffmpeg_args → _metadata_to_ffmpeg_args
- _apply_m4b_chapters_with_mutagen → _apply_m4b_chapters_mutagen
- _embed_m4b_metadata → embed_m4b_metadata
- Add tests/test_exporters.py with 28 tests for ExportService
- Update tests/test_ffmetadata.py to use ExportService
- Update conversion_runner.py to use ExportService
- All tests pass with new implementation matching old behavior
2026-07-14 10:09:27 +00:00
Artem Akymenko 62f42a9f79 refactor: migrate SubtitleWriter to infrastructure/subtitle_writer.py
- Extract SubtitleWriter classes (SrtWriter, AssWriter, VttWriter) to infrastructure/subtitle_writer.py
- Add create_subtitle_writer() factory function
- Remove old SubtitleWriter class and _format_timestamp from conversion_runner.py
- Use create_subtitle_writer() factory from infrastructure layer
- Add tests/test_subtitle_writer.py with 28 tests covering SrtWriter, AssWriter, VttWriter
- All tests match old _format_timestamp behavior
2026-07-14 10:06:51 +00:00
Bryan Nathan 2c4d13bf56 fix(webui): allow large chapter forms 2026-07-14 08:51:40 +08:00
79 changed files with 9397 additions and 3366 deletions
+172
View File
@@ -0,0 +1,172 @@
"""Audio buffer operations for audiobook generation.
This module provides core audio buffer manipulation functions including:
- Silence generation
- Audio mixing
- Audio normalization
- Audio buffer resizing
"""
from __future__ import annotations
from typing import Optional
import numpy as np
# Standard sample rate used throughout the application
SAMPLE_RATE = 24000
def create_silence(duration_seconds: float) -> np.ndarray:
"""Create a silence audio buffer.
Args:
duration_seconds: Duration of silence in seconds.
Returns:
Numpy array of float32 zeros with length = duration_seconds * SAMPLE_RATE.
Returns empty array if duration is <= 0.
"""
if duration_seconds <= 0:
return np.array([], dtype="float32")
samples = int(round(duration_seconds * SAMPLE_RATE))
if samples <= 0:
return np.array([], dtype="float32")
return np.zeros(samples, dtype="float32")
def mix_audio(
target: np.ndarray,
source: np.ndarray,
start_sample: int,
end_sample: Optional[int] = None,
) -> np.ndarray:
"""Mix source audio into target buffer at specified position.
This performs additive mixing (target += source). The target buffer
is extended if necessary to accommodate the source audio.
Args:
target: The target audio buffer to mix into.
source: The source audio buffer to mix.
start_sample: Starting sample index in target buffer.
end_sample: Optional end sample index. If None, calculated from source length.
Returns:
The target buffer (possibly extended). If target was extended, returns new array.
"""
if source.size == 0:
return target
if end_sample is None:
end_sample = start_sample + len(source)
# Extend target buffer if needed
if end_sample > len(target):
new_length = end_sample
new_target = np.concatenate([
target,
np.zeros(new_length - len(target), dtype="float32")
])
target = new_target
# Perform the mix (additive)
target[start_sample:end_sample] += source
return target
def normalize_audio(
audio: np.ndarray,
target_peak: float = 1.0,
) -> np.ndarray:
"""Normalize audio buffer to prevent clipping.
If the audio exceeds the target peak (default 1.0), it is scaled down
proportionally to prevent distortion.
Args:
audio: Input audio buffer.
target_peak: Target maximum amplitude (default 1.0).
Returns:
Normalized audio buffer (new array, original is not modified).
"""
if audio.size == 0:
return audio.copy()
max_amplitude = float(np.abs(audio).max())
if max_amplitude <= target_peak:
return audio.copy()
# Scale down to prevent clipping
scale_factor = target_peak / max_amplitude
return (audio * scale_factor).astype("float32")
def ensure_buffer_size(
buffer: np.ndarray,
min_samples: int,
) -> np.ndarray:
"""Ensure audio buffer is at least min_samples long.
If buffer is shorter, it is extended with zeros.
Args:
buffer: Input audio buffer.
min_samples: Minimum required length in samples.
Returns:
Buffer of at least min_samples length (new array if extended).
"""
if len(buffer) >= min_samples:
return buffer
new_buffer = np.zeros(min_samples, dtype="float32")
new_buffer[:len(buffer)] = buffer
return new_buffer
def concatenate_audio(*buffers: np.ndarray) -> np.ndarray:
"""Concatenate multiple audio buffers.
Args:
*buffers: Audio buffers to concatenate.
Returns:
Single concatenated audio buffer.
"""
non_empty = [b for b in buffers if b.size > 0]
if not non_empty:
return np.array([], dtype="float32")
return np.concatenate(non_empty)
def audio_duration(audio: np.ndarray, sample_rate: int = SAMPLE_RATE) -> float:
"""Calculate duration of audio buffer in seconds.
Args:
audio: Audio buffer.
sample_rate: Sample rate in Hz (default SAMPLE_RATE).
Returns:
Duration in seconds.
"""
return len(audio) / sample_rate
def samples_for_duration(duration_seconds: float, sample_rate: int = SAMPLE_RATE) -> int:
"""Calculate number of samples for a given duration.
Args:
duration_seconds: Duration in seconds.
sample_rate: Sample rate in Hz (default SAMPLE_RATE).
Returns:
Number of samples (rounded to nearest integer), or 0 if duration is <= 0.
"""
if duration_seconds <= 0:
return 0
return int(round(duration_seconds * sample_rate))
+118
View File
@@ -0,0 +1,118 @@
"""Audio helper utilities.
Functions for building ffmpeg commands, converting audio formats,
and applying chapter metadata to MP4 files.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Dict, List, Optional
import numpy as np
SAMPLE_RATE = 24000
def build_ffmpeg_command(path: Path, fmt: str, metadata: Optional[Dict[str, str]] = None) -> list[str]:
from abogen.infrastructure.exporters import ExportService
base = [
"ffmpeg",
"-y",
"-f",
"f32le",
"-ar",
str(SAMPLE_RATE),
"-ac",
"1",
"-i",
"pipe:0",
]
if fmt == "mp3":
base += ["-c:a", "libmp3lame", "-qscale:a", "2"]
elif fmt == "opus":
base += ["-c:a", "libopus", "-b:a", "24000"]
elif fmt == "m4b":
base += ["-c:a", "aac", "-q:a", "2", "-movflags", "+faststart+use_metadata_tags"]
else:
base += ["-c:a", "copy"]
if metadata:
svc = ExportService()
base.extend(svc._metadata_to_ffmpeg_args(metadata))
base.append(str(path))
return base
def to_float32(audio_segment) -> np.ndarray:
if audio_segment is None:
return np.zeros(0, dtype="float32")
tensor = audio_segment
if hasattr(tensor, "detach"):
tensor = tensor.detach()
if hasattr(tensor, "cpu"):
try:
tensor = tensor.cpu()
except Exception:
pass
if hasattr(tensor, "numpy"):
return np.asarray(tensor.numpy(), dtype="float32").reshape(-1)
return np.asarray(tensor, dtype="float32").reshape(-1)
def apply_m4b_chapters_with_mutagen(
audio_path: Path,
chapters: List[Dict[str, Any]],
) -> bool:
"""Apply chapter atoms to an MP4/M4B file using mutagen.
Returns True if chapters were written, False otherwise.
Raises ImportError if mutagen is not installed.
"""
if not chapters:
return False
from fractions import Fraction
from mutagen.mp4 import MP4, MP4Chapter # type: ignore[import]
mp4 = MP4(str(audio_path))
chapter_objects: List[MP4Chapter] = []
for index, entry in enumerate(sorted(chapters, key=lambda item: float(item.get("start") or 0.0))):
start_raw = entry.get("start")
if start_raw is None:
continue
try:
start_seconds = max(0.0, float(start_raw))
except (TypeError, ValueError):
continue
title_value = entry.get("title")
title_text = str(title_value) if title_value else f"Chapter {index + 1}"
start_fraction = Fraction(int(round(start_seconds * 1000)), 1000)
chapter_atom = MP4Chapter(start_fraction, title_text)
end_raw = entry.get("end")
if end_raw is not None:
try:
end_seconds = float(end_raw)
except (TypeError, ValueError):
end_seconds = None
if end_seconds is not None and end_seconds > start_seconds:
chapter_atom.end = Fraction(int(round(end_seconds * 1000)), 1000)
chapter_objects.append(chapter_atom)
if not chapter_objects:
return False
from typing import cast
mp4.chapters = cast(Any, chapter_objects)
mp4.save()
return True
+131
View File
@@ -0,0 +1,131 @@
"""Audio sink abstraction for unified audio output.
Provides a context-manager-based abstraction for writing audio data
to various output formats (WAV, FLAC via soundfile; compressed via ffmpeg).
Usage:
with open_audio_sink(path, "wav") as sink:
sink.write(audio_data)
"""
from __future__ import annotations
import os
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Optional
import numpy as np
from abogen.domain.audio_buffer import SAMPLE_RATE
from abogen.domain.audio_helpers import build_ffmpeg_command
@dataclass(frozen=True)
class AudioSink:
"""Represents an open audio output target."""
write: Callable[[np.ndarray], None]
close: Callable[[], None]
def __enter__(self) -> AudioSink:
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.close()
def _ensure_ffmpeg() -> None:
"""Ensure static ffmpeg binaries are on PATH."""
import static_ffmpeg # type: ignore
ffmpeg_cache_root = _get_ffmpeg_cache_root()
platform_cache = os.path.join(ffmpeg_cache_root, sys.platform)
os.makedirs(platform_cache, exist_ok=True)
try:
import static_ffmpeg.run as static_ffmpeg_run # type: ignore
static_ffmpeg_run.LOCK_FILE = os.path.join(ffmpeg_cache_root, "lock.file")
except Exception:
pass
static_ffmpeg.add_paths(weak=True, download_dir=platform_cache)
def _get_ffmpeg_cache_root() -> str:
from abogen.infrastructure.cache import get_internal_cache_path
return get_internal_cache_path("ffmpeg")
def open_audio_sink(
path: Path,
fmt: str,
*,
metadata: Optional[dict[str, str]] = None,
cancel_check: Optional[Callable[[], bool]] = None,
extra_ffmpeg_args: Optional[list[str]] = None,
ffmpeg_cmd: Optional[list[str]] = None,
) -> AudioSink:
"""Open an audio output sink for writing raw float32 PCM samples.
Args:
path: Output file path.
fmt: Output format ("wav", "flac", "mp3", "opus", "m4b").
metadata: Optional metadata dict (ignored when ffmpeg_cmd is provided).
cancel_check: Optional callable; if it returns True, writes are silently skipped.
extra_ffmpeg_args: Optional extra args inserted after ffmpeg header (ignored when ffmpeg_cmd is provided).
ffmpeg_cmd: Optional pre-built ffmpeg command list (for m4b with cover art etc.).
Returns:
AudioSink with write() and close() methods.
"""
fmt = fmt.lower()
if fmt in {"wav", "flac"}:
import soundfile as sf
soundfile_obj = sf.SoundFile(
path,
mode="w",
samplerate=SAMPLE_RATE,
channels=1,
format=fmt.upper(),
)
def _write_wav(data: np.ndarray) -> None:
if cancel_check and cancel_check():
return
soundfile_obj.write(data)
def _close_wav() -> None:
soundfile_obj.close()
return AudioSink(write=_write_wav, close=_close_wav)
# Compressed formats: pipe through ffmpeg
_ensure_ffmpeg()
if ffmpeg_cmd is not None:
cmd = list(ffmpeg_cmd)
else:
cmd = build_ffmpeg_command(path, fmt, metadata=metadata)
if extra_ffmpeg_args:
cmd[2:2] = extra_ffmpeg_args
process = subprocess.Popen(
cmd, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
)
def _write_compressed(data: np.ndarray) -> None:
if (cancel_check and cancel_check()) or process.stdin is None or process.stdin.closed:
return
process.stdin.write(data.tobytes())
def _close_compressed() -> None:
if process.stdin and not process.stdin.closed:
process.stdin.close()
process.wait()
return AudioSink(write=_write_compressed, close=_close_compressed)
+131
View File
@@ -0,0 +1,131 @@
"""Heuristics for classifying chapters as content vs. supplements.
A 'supplement' is any non-story material that a listener would typically
skip: title page, copyright, table of contents, acknowledgements, etc.
The scoring functions return a float; higher ⇒ more likely to be a
supplement. ``should_preselect_chapter`` turns that score into a
boolean suitable for a web form default.
"""
from __future__ import annotations
import re
from typing import Any, Dict, List, Tuple
# Compiled once at module load these are immutable.
_SUPPLEMENT_TITLE_PATTERNS: List[Tuple[re.Pattern[str], float]] = [
(re.compile(r"\btitle\s+page\b"), 3.0),
(re.compile(r"\bcopyright\b"), 2.4),
(re.compile(r"\btable\s+of\s+contents\b"), 2.8),
(re.compile(r"\bcontents\b"), 2.0),
(re.compile(r"\backnowledg(e)?ments?\b"), 2.0),
(re.compile(r"\bdedication\b"), 2.0),
(re.compile(r"\babout\s+the\s+author(s)?\b"), 2.4),
(re.compile(r"\balso\s+by\b"), 2.0),
(re.compile(r"\bpraise\s+for\b"), 2.0),
(re.compile(r"\bcolophon\b"), 2.2),
(re.compile(r"\bpublication\s+data\b"), 2.2),
(re.compile(r"\btranscriber'?s?\s+note\b"), 2.2),
(re.compile(r"\bglossary\b"), 2.2),
(re.compile(r"\bindex\b"), 2.0),
(re.compile(r"\bbibliograph(y|ies)\b"), 2.0),
(re.compile(r"\breferences\b"), 1.8),
(re.compile(r"\bappendix\b"), 1.9),
]
_CONTENT_TITLE_PATTERNS: List[re.Pattern[str]] = [
re.compile(r"\bchapter\b"),
re.compile(r"\bbook\b"),
re.compile(r"\bpart\b"),
re.compile(r"\bsection\b"),
re.compile(r"\bscene\b"),
re.compile(r"\bprologue\b"),
re.compile(r"\bepilogue\b"),
re.compile(r"\bintroduction\b"),
re.compile(r"\bstory\b"),
]
_SUPPLEMENT_TEXT_KEYWORDS: List[Tuple[str, float]] = [
("copyright", 1.2),
("all rights reserved", 1.1),
("isbn", 0.9),
("library of congress", 1.0),
("table of contents", 1.0),
("dedicated to", 0.8),
("acknowledg", 0.8),
("printed in", 0.6),
("permission", 0.6),
("publisher", 0.5),
("praise for", 0.9),
("also by", 0.9),
("glossary", 0.8),
("index", 0.8),
("newsletter", 3.2),
("mailing list", 2.6),
("sign-up", 2.2),
]
def supplement_score(title: str, text: str, index: int) -> float:
"""Return a score indicating how likely *title*/*text* is a supplement.
Higher values ⇒ more likely to be non-story material (title page,
copyright, acknowledgements, etc.).
"""
normalized_title = (title or "").lower()
score = 0.0
for pattern, weight in _SUPPLEMENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score += weight
for pattern in _CONTENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score -= 2.0
stripped_text = (text or "").strip()
length = len(stripped_text)
if length <= 150:
score += 0.9
elif length <= 400:
score += 0.6
elif length <= 800:
score += 0.35
lowercase_text = stripped_text.lower()
for keyword, weight in _SUPPLEMENT_TEXT_KEYWORDS:
if keyword in lowercase_text:
score += weight
if index == 0 and score > 0:
score += 0.25
return score
def should_preselect_chapter(
title: str,
text: str,
index: int,
total_count: int,
) -> bool:
"""Return True if the chapter should be *enabled* by default in the form.
A single chapter is always preselected. For multi-chapter books, the
chapter is preselected when its supplement score is below 1.9.
"""
if total_count <= 1:
return True
score = supplement_score(title, text, index)
return score < 1.9
def ensure_at_least_one_chapter_enabled(chapters: List[Dict[str, Any]]) -> None:
"""Mutate *chapters* in-place so that at least one has ``enabled=True``."""
if not chapters:
return
if any(chapter.get("enabled") for chapter in chapters):
return
best_index = max(range(len(chapters)), key=lambda idx: chapters[idx].get("characters", 0))
chapters[best_index]["enabled"] = True
+92
View File
@@ -0,0 +1,92 @@
from __future__ import annotations
from typing import Any, Dict, List, Optional, Tuple
from abogen.text_extractor import ExtractedChapter
from abogen.domain.voice_utils import coerce_truthy
def apply_chapter_overrides(
extracted: List[ExtractedChapter],
overrides: List[Dict[str, Any]],
) -> Tuple[List[ExtractedChapter], Dict[str, str], List[str]]:
if not overrides:
return [], {}, []
selected: List[ExtractedChapter] = []
metadata_updates: Dict[str, str] = {}
diagnostics: List[str] = []
for position, payload in enumerate(overrides):
if not isinstance(payload, dict):
diagnostics.append(
f"Skipped chapter override at position {position + 1}: unsupported payload type {type(payload).__name__}."
)
continue
enabled = coerce_truthy(payload.get("enabled", True))
payload["enabled"] = enabled
if not enabled:
continue
metadata_payload = payload.get("metadata") or {}
if isinstance(metadata_payload, dict):
for key, value in metadata_payload.items():
if value is None:
continue
metadata_updates[str(key)] = str(value)
base: Optional[ExtractedChapter] = None
idx_candidate = payload.get("index")
idx_normalized: Optional[int] = None
if isinstance(idx_candidate, int):
idx_normalized = idx_candidate
elif isinstance(idx_candidate, str):
try:
idx_normalized = int(idx_candidate)
except ValueError:
idx_normalized = None
if idx_normalized is not None and 0 <= idx_normalized < len(extracted):
base = extracted[idx_normalized]
payload["index"] = idx_normalized
if base is None:
source_title = payload.get("source_title")
if isinstance(source_title, str):
base = next((chapter for chapter in extracted if chapter.title == source_title), None)
if base is None:
candidate_title = payload.get("title")
if isinstance(candidate_title, str):
base = next((chapter for chapter in extracted if chapter.title == candidate_title), None)
text_override = payload.get("text")
if text_override is not None:
text_value = str(text_override)
elif base is not None:
text_value = base.text
else:
diagnostics.append(
f"Skipped chapter override at position {position + 1}: no text provided and no matching source chapter found."
)
continue
title_override = payload.get("title")
if title_override is not None:
title_value = str(title_override)
elif base is not None:
title_value = base.title
else:
title_value = f"Chapter {position + 1}"
if base and not payload.get("source_title"):
payload["source_title"] = base.title
payload["title"] = title_value
payload["text"] = text_value
payload["characters"] = len(text_value)
payload.setdefault("order", payload.get("order", position))
selected.append(ExtractedChapter(title=title_value, text=text_value))
return selected, metadata_updates, diagnostics
+204
View File
@@ -0,0 +1,204 @@
from __future__ import annotations
import re
from typing import List, Tuple
_HEADING_SANITIZE_RE = re.compile(r"[^a-z0-9]+")
_HEADING_NUMBER_PREFIX_RE = re.compile(
r"^\s*(?P<number>(?:\d+|[ivxlcdm]+))(?P<suffix>(?:[\s.:;-].*)?)$",
re.IGNORECASE,
)
_ACRONYM_ALLOWLIST = {
"AI", "API", "CPU", "DIY", "GPU", "HTML", "HTTP", "HTTPS", "ID",
"JSON", "MP3", "MP4", "M4B", "NASA", "OCR", "PDF", "SQL", "TV",
"TTS", "UK", "UN", "UFO", "OK", "URL", "USA", "US", "VR",
}
_ROMAN_NUMERAL_CHARS = frozenset("IVXLCDM")
_CAPS_WORD_RE = re.compile(r"[A-Z][A-Z0-9'\u2019-]*")
def simplify_heading_text(text: str) -> str:
raw = str(text or "").strip().lower()
if not raw:
return ""
simplified = _HEADING_SANITIZE_RE.sub("", raw)
if simplified.startswith("chapter"):
simplified = simplified[7:]
return simplified
def headings_equivalent(left: str, right: str) -> bool:
simple_left = simplify_heading_text(left)
simple_right = simplify_heading_text(right)
if not simple_left or not simple_right:
return False
if simple_left == simple_right:
return True
if simple_right.startswith(simple_left):
return True
if simple_left.startswith(simple_right):
return True
if len(simple_left) > 5 and simple_left in simple_right:
return True
return False
def strip_duplicate_heading_line(text: str, heading: str) -> Tuple[str, bool]:
source_text = str(text or "")
if not source_text:
return source_text, False
normalized_heading = simplify_heading_text(heading)
if not normalized_heading:
return source_text, False
lines = source_text.splitlines()
new_lines: List[str] = []
removed = False
for line in lines:
stripped = line.strip()
if not removed and stripped:
if headings_equivalent(stripped, heading):
removed = True
continue
new_lines.append(line)
if not removed:
return source_text, False
while new_lines and not new_lines[0].strip():
new_lines.pop(0)
return "\n".join(new_lines), True
def normalize_caps_word(word: str) -> str:
upper = word.upper()
letters = [char for char in upper if char.isalpha()]
if not letters:
return word
if upper in _ACRONYM_ALLOWLIST:
return word
if len(letters) <= 1:
return word
if all(char in _ROMAN_NUMERAL_CHARS for char in letters) and len(letters) <= 7:
return word
parts = re.split(r"(['\-\u2019])", word)
normalized_parts: List[str] = []
for part in parts:
if part in {"'", "-", "\u2019"}:
normalized_parts.append(part)
continue
if not part:
continue
normalized_parts.append(part[0].upper() + part[1:].lower())
return "".join(normalized_parts) or word
def normalize_chapter_opening_caps(text: str) -> Tuple[str, bool]:
if not text:
return text, False
leading_len = len(text) - len(text.lstrip())
leading = text[:leading_len]
working = text[leading_len:]
if not working:
return text, False
builder: List[str] = []
pos = 0
changed = False
while pos < len(working):
char = working[pos]
if char in "\r\n":
builder.append(working[pos:])
pos = len(working)
break
if char.isspace():
builder.append(char)
pos += 1
continue
if char.islower():
builder.append(working[pos:])
pos = len(working)
break
if not char.isalpha():
builder.append(char)
pos += 1
continue
match = _CAPS_WORD_RE.match(working, pos)
if not match:
builder.append(char)
pos += 1
continue
word = match.group(0)
if any(ch.islower() for ch in word):
builder.append(working[pos:])
pos = len(working)
break
normalized = normalize_caps_word(word)
if normalized != word:
changed = True
builder.append(normalized)
pos = match.end()
if pos < len(working):
builder.append(working[pos:])
if not changed:
return text, False
return leading + "".join(builder), True
def format_spoken_chapter_title(title: str, index: int, apply_prefix: bool) -> str:
base = str(title or "").strip()
if not base:
return f"Chapter {index}" if apply_prefix else ""
if not apply_prefix:
return base
lowered = base.lower()
if lowered.startswith("chapter") and (len(lowered) == 7 or not lowered[7].isalpha()):
return base
match = _HEADING_NUMBER_PREFIX_RE.match(base)
if match:
number = match.group("number") or ""
suffix = match.group("suffix") or ""
cleaned_suffix = suffix.lstrip(" .,:;-_ \t\u2013\u2014\u00b7\u2022")
if cleaned_suffix:
return f"Chapter {number}. {cleaned_suffix}"
return f"Chapter {number}"
return base
def apply_chapter_text_transforms(
text: str,
*,
heading_text: str,
raw_title: str,
strip_heading: bool,
normalize_caps: bool,
) -> Tuple[str, bool, bool]:
"""Strip duplicate heading and normalize opening caps.
Returns ``(text, heading_removed, caps_changed)``.
The caller is responsible for state updates (pending flags, logging,
dict mutation, ``continue``).
"""
heading_removed = False
caps_changed = False
if strip_heading and heading_text:
text, heading_removed = strip_duplicate_heading_line(text, heading_text)
if not heading_removed and raw_title:
match = _HEADING_NUMBER_PREFIX_RE.match(raw_title)
if match:
number = match.group("number")
if number:
text, heading_removed = strip_duplicate_heading_line(text, number)
if normalize_caps and text:
text, caps_changed = normalize_chapter_opening_caps(text)
return text, heading_removed, caps_changed
+75
View File
@@ -0,0 +1,75 @@
"""Chunk processing utilities.
Functions for grouping chunks, recording override usage, and selecting
text for TTS synthesis.
"""
from __future__ import annotations
from collections import defaultdict
from typing import Any, Dict, Iterable, Mapping, Optional
from abogen.pronunciation_store import increment_usage
def safe_int(value: Any, default: int = 0) -> int:
try:
return int(value)
except (TypeError, ValueError):
return default
def group_chunks_by_chapter(chunks: Iterable[Dict[str, Any]]) -> Dict[int, List[Dict[str, Any]]]:
grouped: Dict[int, List[Dict[str, Any]]] = defaultdict(list)
for entry in chunks or []:
if not isinstance(entry, dict):
continue
try:
chapter_index = int(entry.get("chapter_index", 0))
except (TypeError, ValueError):
chapter_index = 0
grouped[chapter_index].append(dict(entry))
for chapter_index, items in grouped.items():
items.sort(key=lambda payload: safe_int(payload.get("chunk_index")))
return grouped
def record_override_usage(
job: Any,
usage_counter: Mapping[str, int],
token_map: Mapping[str, str],
) -> None:
if not usage_counter:
return
language = getattr(job, "language", "") or "a"
for normalized, amount in usage_counter.items():
if amount <= 0:
continue
token_value = token_map.get(normalized, normalized)
try:
increment_usage(language=language, token=token_value, amount=int(amount))
except Exception: # pragma: no cover - defensive logging
job.add_log(f"Failed to record usage for override {token_value}", level="warning")
def chunk_text_for_tts(entry: Mapping[str, Any]) -> str:
"""Choose the best source text for synthesis.
We must prefer the raw chunk text (``text`` / ``original_text``) so
manual/pronunciation overrides can match against the original tokens
(e.g. censored words like ``Unfu*k``). ``normalized_text`` may have
already been run through ``normalize_for_pipeline``, which can remove
punctuation and prevent overrides from triggering.
"""
if not isinstance(entry, Mapping):
return ""
return str(
entry.get("text")
or entry.get("original_text")
or entry.get("normalized_text")
or ""
).strip()
+31
View File
@@ -0,0 +1,31 @@
from __future__ import annotations
import platform as _platform
def select_device() -> str:
"""Return the best available compute device (``"mps"``, ``"cuda"``, or ``"cpu"``).
Checks ``torch`` availability at runtime so this can be called from
any context without requiring torch at import time.
"""
try:
import torch # type: ignore[import-not-found]
except Exception:
return "cpu"
system = _platform.system()
if system == "Darwin" and _platform.processor() == "arm":
try:
if torch.backends.mps.is_available(): # type: ignore[union-attr]
return "mps"
except Exception:
pass
return "cpu"
try:
if torch.cuda.is_available(): # type: ignore[union-attr]
return "cuda"
except Exception:
pass
return "cpu"
+136
View File
@@ -0,0 +1,136 @@
from __future__ import annotations
import re
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Tuple
from abogen.text_extractor import ExtractedChapter
_SIGNIFICANT_LENGTH_THRESHOLDS: Dict[str, int] = {"epub": 1000, "markdown": 500}
_MIN_SHORT_CONTENT: Dict[str, int] = {"epub": 240, "markdown": 160}
_STRUCTURAL_KEYWORDS = (
"preface",
"prologue",
"introduction",
"foreword",
"epilogue",
"afterword",
"appendix",
"acknowledgment",
"acknowledgement",
)
_STRUCTURAL_MIN_LENGTH = 120
_MAX_SHORT_CHAPTERS = 2
@dataclass
class ChapterFilterResult:
kept: List[ExtractedChapter]
skipped: List[Tuple[str, int]]
def infer_file_type(path: Path) -> str:
suffix = path.suffix.lower()
if suffix == ".epub":
return "epub"
if suffix in {".md", ".markdown"}:
return "markdown"
if suffix == ".pdf":
return "pdf"
if suffix == ".txt":
return "text"
return suffix.lstrip(".") or "text"
def looks_structural(title: str) -> bool:
lowered = title.strip().lower()
if not lowered:
return False
return any(keyword in lowered for keyword in _STRUCTURAL_KEYWORDS)
def chapter_label(file_type: str) -> str:
return "chapters" if file_type.lower() in {"epub", "markdown"} else "pages"
def auto_select_relevant_chapters(
chapters: List[ExtractedChapter],
file_type: str,
) -> ChapterFilterResult:
if not chapters:
return ChapterFilterResult(kept=[], skipped=[])
normalized = file_type.lower()
threshold = _SIGNIFICANT_LENGTH_THRESHOLDS.get(normalized, 0)
min_short = _MIN_SHORT_CONTENT.get(normalized, 0)
kept: List[ExtractedChapter] = []
skipped: List[Tuple[str, int]] = []
short_kept = 0
for chapter in chapters:
stripped = chapter.text.strip()
length = len(stripped)
if length == 0:
skipped.append((chapter.title, length))
continue
keep = False
if threshold == 0:
keep = True
elif length >= threshold:
keep = True
elif not kept:
keep = True
elif min_short and length >= min_short and short_kept < _MAX_SHORT_CHAPTERS:
keep = True
short_kept += 1
elif looks_structural(chapter.title) and length >= _STRUCTURAL_MIN_LENGTH:
keep = True
if keep:
kept.append(chapter)
else:
skipped.append((chapter.title, length))
if kept:
return ChapterFilterResult(kept=kept, skipped=skipped)
longest_idx = None
longest_length = 0
for idx, chapter in enumerate(chapters):
stripped = chapter.text.strip()
if stripped and len(stripped) > longest_length:
longest_length = len(stripped)
longest_idx = idx
if longest_idx is not None:
longest = chapters[longest_idx]
fallback_skipped = [
(chapter.title, len(chapter.text.strip()))
for idx, chapter in enumerate(chapters)
if idx != longest_idx and chapter.text.strip()
]
return ChapterFilterResult(kept=[longest], skipped=fallback_skipped)
return ChapterFilterResult(kept=[], skipped=skipped)
def update_metadata_for_chapter_count(
metadata: Dict[str, Any], count: int, file_type: str
) -> None:
if not metadata or count <= 0:
return
label = "Chapters" if file_type.lower() in {"epub", "markdown"} else "Pages"
metadata["chapter_count"] = str(count)
pattern = re.compile(r"\(\d+\s+(Chapters?|Pages?)\)")
replacement = f"({count} {label})"
for key in ("album", "ALBUM"):
value = metadata.get(key)
if not isinstance(value, str):
continue
metadata[key] = pattern.sub(replacement, value)
+191
View File
@@ -0,0 +1,191 @@
"""Metadata extraction and processing utilities.
This module provides functions for extracting metadata from text content
and generating ffmpeg metadata arguments.
"""
from __future__ import annotations
import datetime
import os
import re
from pathlib import Path
from typing import Dict, List, Optional, Tuple
def extract_metadata_from_text(text: str) -> Dict[str, Optional[str]]:
"""Extract metadata tags from text content.
Looks for tags in format: <<METADATA_KEY:value>>
Supported tags:
- TITLE, ARTIST, ALBUM, YEAR
- ALBUM_ARTIST, COMPOSER, GENRE
- COVER_PATH
Args:
text: Text content to search for metadata tags.
Returns:
Dictionary with extracted metadata values (None if not found).
"""
metadata = {}
patterns = {
"title": r"<<METADATA_TITLE:([^>]*)>>",
"artist": r"<<METADATA_ARTIST:([^>]*)>>",
"album": r"<<METADATA_ALBUM:([^>]*)>>",
"year": r"<<METADATA_YEAR:([^>]*)>>",
"album_artist": r"<<METADATA_ALBUM_ARTIST:([^>]*)>>",
"composer": r"<<METADATA_COMPOSER:([^>]*)>>",
"genre": r"<<METADATA_GENRE:([^>]*)>>",
"cover_path": r"<<METADATA_COVER_PATH:([^>]*)>>",
}
for key, pattern in patterns.items():
match = re.search(pattern, text)
if match:
metadata[key] = match.group(1).strip()
else:
metadata[key] = None
return metadata
def get_filename_from_path(
file_path: str,
display_path: Optional[str] = None,
from_queue: bool = False,
) -> str:
"""Extract filename (without extension) from path.
Args:
file_path: The file path to extract from.
display_path: Optional display path (used if from_queue is False).
from_queue: Whether the file is from queue.
Returns:
Filename without extension.
"""
if from_queue:
base_path = file_path
else:
base_path = display_path if display_path else file_path
filename = os.path.splitext(os.path.basename(base_path))[0]
return filename
def build_ffmpeg_metadata_args(
metadata: Dict[str, Optional[str]],
filename: str,
) -> List[str]:
"""Build ffmpeg metadata arguments from metadata dictionary.
Args:
metadata: Dictionary with metadata keys and values.
filename: Fallback filename for title/album if not specified.
Returns:
List of ffmpeg metadata arguments.
"""
args = []
# Default values
defaults = {
"title": filename,
"artist": "Unknown",
"album": filename,
"date": str(datetime.datetime.now().year),
"album_artist": "Unknown",
"composer": "Narrator",
"genre": "Audiobook",
}
# Map of metadata keys to ffmpeg metadata keys
key_mapping = {
"title": "title",
"artist": "artist",
"album": "album",
"year": "date", # year -> date for ffmpeg
"album_artist": "album_artist",
"composer": "composer",
"genre": "genre",
}
for metadata_key, ffmpeg_key in key_mapping.items():
value = metadata.get(metadata_key)
if value is None:
value = defaults.get(metadata_key, "")
if value:
args.extend(["-metadata", f"{ffmpeg_key}={value}"])
return args
def extract_metadata_and_build_args(
text: str,
filename: str,
display_path: Optional[str] = None,
from_queue: bool = False,
) -> Tuple[List[str], Optional[str]]:
"""Extract metadata from text and build ffmpeg arguments.
Convenience function that combines extract_metadata_from_text and
build_ffmpeg_metadata_args.
Args:
text: Text content to search for metadata tags.
filename: Fallback filename for title/album.
display_path: Optional display path.
from_queue: Whether the file is from queue.
Returns:
Tuple of (ffmpeg_metadata_args, cover_path).
"""
metadata = extract_metadata_from_text(text)
cover_path = metadata.get("cover_path")
# Get actual filename from path
actual_filename = get_filename_from_path(
file_path=filename,
display_path=display_path,
from_queue=from_queue,
)
args = build_ffmpeg_metadata_args(metadata, actual_filename)
return args, cover_path
def read_text_for_metadata(
file_path: str,
is_direct_text: bool,
direct_text: Optional[str] = None,
encoding: Optional[str] = None,
) -> str:
"""Read text content for metadata extraction.
Args:
file_path: Path to file (or text if is_direct_text).
is_direct_text: Whether file_path contains direct text.
direct_text: Optional direct text (used if is_direct_text).
encoding: File encoding (detected if not provided).
Returns:
Text content for metadata extraction.
"""
if is_direct_text:
return direct_text or file_path
# Read from file
actual_path = direct_text if direct_text else file_path
try:
if encoding is None:
from abogen.utils import detect_encoding
encoding = detect_encoding(actual_path)
with open(actual_path, "r", encoding=encoding, errors="replace") as f:
return f.read()
except Exception:
return ""
+405
View File
@@ -0,0 +1,405 @@
from __future__ import annotations
import json
import math
import re
from pathlib import Path
from typing import Any, Dict, List, Mapping, Optional, Tuple
_SERIES_NAME_KEYS = (
"series",
"series_name",
"series_title",
)
_SERIES_NUMBER_KEYS = (
"series_index",
"series_position",
"series_sequence",
"book_number",
"series_number",
)
_SERIES_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?")
def normalize_metadata_map(values: Optional[Mapping[str, Any]]) -> Dict[str, str]:
normalized: Dict[str, str] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
text = str(value).strip()
if not text:
continue
normalized[str(key).casefold()] = text
return normalized
def format_author_sentence(raw: Optional[str]) -> str:
if raw is None:
return ""
normalized = str(raw).strip()
if not normalized:
return ""
lowered = normalized.casefold()
if lowered in {"unknown", "various"}:
return ""
working = normalized.replace("&", " and ")
segments = [segment.strip() for segment in working.split(",") if segment.strip()]
tokens: List[str] = []
if segments:
for segment in segments:
parts = [part.strip() for part in re.split(r"\band\b", segment, flags=re.IGNORECASE) if part.strip()]
if parts:
tokens.extend(parts)
else:
tokens.append(segment)
else:
parts = [part.strip() for part in re.split(r"\band\b", working, flags=re.IGNORECASE) if part.strip()]
tokens.extend(parts or [normalized])
cleaned = [token for token in tokens if token and token.casefold() not in {"unknown", "various"}]
if not cleaned:
return ""
if len(cleaned) == 1:
return f"By {cleaned[0]}"
if len(cleaned) == 2:
return f"By {cleaned[0]} and {cleaned[1]}"
return f"By {', '.join(cleaned[:-1])}, and {cleaned[-1]}"
def ensure_sentence(text: str) -> str:
cleaned = text.strip()
if not cleaned:
return ""
if cleaned[-1] in ".!?":
return cleaned
return f"{cleaned}."
def normalize_series_number(value: Any) -> Optional[str]:
text = str(value or "").strip()
if not text:
return None
candidate = text.replace(",", ".")
if candidate.replace(".", "", 1).isdigit():
if "." in candidate:
normalized = candidate.rstrip("0").rstrip(".")
return normalized or "0"
try:
return str(int(candidate))
except ValueError:
pass
match = _SERIES_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
return normalized or "0"
try:
return str(int(normalized))
except ValueError:
return normalized
def extract_series_metadata(values: Mapping[str, str]) -> Tuple[Optional[str], Optional[str]]:
series_name: Optional[str] = None
for key in _SERIES_NAME_KEYS:
raw = values.get(key)
if raw:
cleaned = str(raw).strip()
if cleaned:
series_name = cleaned
break
series_number: Optional[str] = None
for key in _SERIES_NUMBER_KEYS:
raw = values.get(key)
if raw is None:
continue
normalized = normalize_series_number(raw)
if normalized:
series_number = normalized
break
return series_name, series_number
def format_series_sentence(series_name: Optional[str], series_number: Optional[str]) -> str:
if not series_name or not series_number:
return ""
name = series_name.strip()
number = series_number.strip()
if not name or not number:
return ""
article = "the " if not name.lower().startswith("the ") else ""
phrase = f"Book {number} of {article}{name}"
return re.sub(r"\s+", " ", phrase).strip()
_PEOPLE_SPLIT_RE = re.compile(r"[;,/&]|\band\b", re.IGNORECASE)
_LIST_SPLIT_RE = re.compile(r"[;,\n]")
_SERIES_SEQUENCE_TAG_KEYS: Tuple[str, ...] = (
"series_index",
"series_position",
"series_sequence",
"series_number",
"seriesnumber",
"book_number",
"booknumber",
)
def normalize_metadata_casefold(values: Optional[Mapping[str, Any]]) -> Dict[str, Any]:
normalized: Dict[str, Any] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
key_text = str(key).strip().lower()
if not key_text:
continue
if isinstance(value, (list, tuple, set)):
normalized[key_text] = value
else:
text = str(value).strip()
if text:
normalized[key_text] = text
return normalized
def split_people_field(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(split_people_field(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _PEOPLE_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def split_simple_list(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(split_simple_list(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _LIST_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def first_nonempty(*values: Any) -> Optional[str]:
for value in values:
if value is None:
continue
if isinstance(value, (list, tuple, set)):
items = list(value)
if not items:
continue
value = items[0]
text = str(value).strip()
if text:
return text
return None
def extract_year(raw: Optional[str]) -> Optional[int]:
if not raw:
return None
text = str(raw).strip()
if not text:
return None
match = re.search(r"(19|20)\d{2}", text)
if match:
try:
return int(match.group(0))
except ValueError:
return None
try:
parsed = int(text)
except ValueError:
return None
if 0 < parsed < 3000:
return parsed
return None
def normalize_series_sequence(raw: Any) -> Optional[str]:
if raw is None:
return None
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return None
text = str(raw)
else:
text = str(raw).strip()
if not text:
return None
candidate = text.replace(",", ".")
match = _SERIES_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
def build_audiobookshelf_metadata(
tags: Mapping[str, Any],
*,
language: str = "",
filename: str = "",
) -> Dict[str, Any]:
normalized = normalize_metadata_casefold(tags)
title = first_nonempty(
normalized.get("title"),
normalized.get("book_title"),
normalized.get("name"),
normalized.get("album"),
filename,
)
authors = split_people_field(
normalized.get("authors")
or normalized.get("author")
or normalized.get("album_artist")
or normalized.get("artist")
)
narrators = split_people_field(normalized.get("narrators") or normalized.get("narrator"))
description = first_nonempty(
normalized.get("description"), normalized.get("summary"), normalized.get("comment")
)
genres = split_simple_list(normalized.get("genre"))
keywords = split_simple_list(normalized.get("tags") or normalized.get("keywords"))
lang = first_nonempty(normalized.get("language"), normalized.get("lang")) or language or ""
series_name = first_nonempty(
normalized.get("series"),
normalized.get("series_name"),
normalized.get("seriesname"),
normalized.get("series_title"),
normalized.get("seriestitle"),
)
series_sequence = None
for key in _SERIES_SEQUENCE_TAG_KEYS:
raw_value = normalized.get(key)
seq = normalize_series_sequence(raw_value)
if seq:
series_sequence = seq
break
if not series_name:
series_sequence = None
data: Dict[str, Any] = {
"title": title,
"subtitle": normalized.get("subtitle"),
"authors": authors,
"narrators": narrators,
"description": description,
"publisher": normalized.get("publisher"),
"genres": genres,
"tags": keywords,
"language": lang,
"publishedYear": extract_year(
normalized.get("published")
or normalized.get("publication_year")
or normalized.get("date")
or normalized.get("year")
),
"seriesName": series_name,
"seriesSequence": series_sequence,
"isbn": first_nonempty(normalized.get("isbn"), normalized.get("asin")),
}
published_date = first_nonempty(
normalized.get("published"), normalized.get("publication_date"), normalized.get("date")
)
if published_date:
data["publishedDate"] = published_date
rating_text = first_nonempty(normalized.get("rating"), normalized.get("my_rating"))
if rating_text:
try:
data["rating"] = float(str(rating_text).strip())
except ValueError:
pass
rating_max_text = first_nonempty(
normalized.get("rating_max"), normalized.get("rating_scale")
)
if rating_max_text:
try:
data["ratingMax"] = float(str(rating_max_text).strip())
except ValueError:
pass
cleaned: Dict[str, Any] = {}
for key, value in data.items():
if value is None:
continue
if isinstance(value, str) and not value.strip():
continue
if isinstance(value, (list, tuple)) and not value:
continue
cleaned[key] = value
return cleaned
def load_audiobookshelf_chapters(
metadata_path: Path,
) -> Optional[List[Dict[str, Any]]]:
if not metadata_path.exists():
return None
try:
payload = json.loads(metadata_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
chapters = payload.get("chapters")
if not isinstance(chapters, list):
return None
cleaned: List[Dict[str, Any]] = []
for entry in chapters:
if not isinstance(entry, Mapping):
continue
title = first_nonempty(entry.get("title"), entry.get("original_title"))
start = entry.get("start")
end = entry.get("end")
if title and start is not None and end is not None:
cleaned.append({"title": str(title), "start": start, "end": end})
return cleaned or None
+23
View File
@@ -0,0 +1,23 @@
from __future__ import annotations
from typing import Any, Dict, Optional
def merge_metadata(
extracted: Optional[Dict[str, Any]],
overrides: Optional[Dict[str, Any]],
) -> Dict[str, str]:
merged: Dict[str, str] = {}
if extracted:
for key, value in extracted.items():
if value is None:
continue
merged[str(key)] = str(value)
if overrides:
for key, value in overrides.items():
key_str = str(key)
if value is None:
merged.pop(key_str, None)
else:
merged[key_str] = str(value)
return merged
+96
View File
@@ -0,0 +1,96 @@
"""Text normalization convenience helpers.
Provides both the simple ``normalize_text_for_pipeline`` (apostrophe + LLM only)
and the comprehensive ``prepare_text_for_tts`` that chains all three normalization
stages used during conversion: heteronym rules → pronunciation rules → pipeline
normalization. The latter is the single entry point that both the Web UI and
PyQt Desktop GUI should use.
"""
from __future__ import annotations
from typing import Any, Dict, List, Mapping, Optional
from abogen.kokoro_text_normalization import (
ApostropheConfig,
normalize_for_pipeline as _normalize_for_pipeline,
)
from abogen.normalization_settings import (
build_apostrophe_config,
get_runtime_settings,
apply_overrides as _apply_overrides,
)
_BASE_APOSTROPHE_CONFIG = ApostropheConfig()
def normalize_text_for_pipeline(
text: str,
*,
normalization_overrides: Optional[Mapping[str, Any]] = None,
) -> str:
"""Normalize text using runtime settings with optional overrides."""
runtime_settings = get_runtime_settings()
if normalization_overrides:
runtime_settings = _apply_overrides(runtime_settings, normalization_overrides)
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
return _normalize_for_pipeline(text, config=apostrophe_config, settings=runtime_settings)
def prepare_text_for_tts(
text: str,
*,
heteronym_rules: Optional[List[Dict[str, Any]]] = None,
pronunciation_rules: Optional[List[Dict[str, Any]]] = None,
normalization_overrides: Optional[Mapping[str, Any]] = None,
usage_counter: Optional[Dict[str, int]] = None,
) -> str:
"""Apply the full text normalization pipeline before TTS synthesis.
Chains three stages in order:
1. Heteronym sentence rules (context-dependent pronunciation)
2. Pronunciation rules (token-level replacements)
3. Pipeline normalization (apostrophe handling, LLM normalization)
This is the **single entry point** that both the Web UI conversion runner
and the PyQt conversion thread should call before passing text to the TTS
backend.
Parameters
----------
text:
Raw text to normalize.
heteronym_rules:
Compiled heteronym rules from ``compile_heteronym_sentence_rules``.
pronunciation_rules:
Compiled pronunciation rules from ``compile_pronunciation_rules``.
normalization_overrides:
User-level overrides for normalization settings (apostrophe mode, etc.).
usage_counter:
Mutable dict that tracks how many times each pronunciation override was
applied. Passed through to ``apply_pronunciation_rules``.
Returns
-------
str
Fully normalized text ready for TTS.
"""
from abogen.domain.pronunciation import (
apply_heteronym_sentence_rules,
apply_pronunciation_rules,
)
result = str(text or "")
if heteronym_rules:
result = apply_heteronym_sentence_rules(result, heteronym_rules)
if pronunciation_rules:
result = apply_pronunciation_rules(result, pronunciation_rules, usage_counter)
runtime_settings = get_runtime_settings()
if normalization_overrides:
runtime_settings = _apply_overrides(runtime_settings, normalization_overrides)
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
return _normalize_for_pipeline(result, config=apostrophe_config, settings=runtime_settings)
+91
View File
@@ -0,0 +1,91 @@
"""Output path resolution utilities.
Pure functions for resolving output directories, building file paths,
and computing project folder layouts.
"""
from __future__ import annotations
import re
from datetime import datetime
from pathlib import Path
from typing import Any, Callable, List, Optional, Tuple
from abogen.text_extractor import ExtractedChapter
_OUTPUT_SANITIZE_RE = re.compile(r"[^\w\-_.]+")
def slugify(title: str, index: int) -> str:
sanitized = re.sub(r"[^\w\-]+", "_", title.lower()).strip("_")
if not sanitized:
sanitized = f"chapter_{index:02d}"
return sanitized[:80]
def sanitize_output_stem(name: str) -> str:
base = Path(name or "").stem
sanitized = _OUTPUT_SANITIZE_RE.sub("_", base).strip("_")
return sanitized or "output"
def output_timestamp_token() -> str:
return datetime.now().strftime("%Y%m%d-%H%M%S")
def build_output_path(directory: Path, original_name: str, extension: str) -> Path:
sanitized = sanitize_output_stem(original_name)
return directory / f"{sanitized}.{extension}"
def apply_newline_policy(chapters: List[ExtractedChapter], replace_single_newlines: bool) -> None:
if not replace_single_newlines:
return
newline_regex = re.compile(r"(?<!\n)\n(?!\n)")
for chapter in chapters:
chapter.text = newline_regex.sub(" ", chapter.text)
def resolve_output_directory(
*,
save_mode: str,
stored_path: Path,
output_folder: Optional[str],
desktop_dir: Optional[Path],
user_output_path: Optional[Path],
user_cache_outputs: Optional[Path],
) -> Path:
if save_mode == "Save to Desktop" and desktop_dir:
return desktop_dir
if save_mode == "Save next to input file":
return stored_path.parent
if save_mode == "Choose output folder" and output_folder:
return Path(output_folder)
if save_mode == "Use default save location" and user_output_path:
return user_output_path
return user_cache_outputs or Path(".")
def resolve_project_layout(
*,
original_filename: str,
save_as_project: bool,
base_dir: Path,
timestamp_fn: Callable[[], str] = output_timestamp_token,
sanitize_fn: Callable[[str, int], str] = sanitize_output_stem,
) -> Tuple[Path, Path, Path, Optional[Path]]:
sanitized = sanitize_fn(original_filename, 0)
folder_name = f"{timestamp_fn()}_{sanitized}"
project_root = base_dir / folder_name
project_root.mkdir(parents=True, exist_ok=True)
if save_as_project:
audio_dir = project_root / "audio"
subtitle_dir = project_root / "subtitles"
metadata_dir = project_root / "metadata"
for directory in (audio_dir, subtitle_dir, metadata_dir):
directory.mkdir(parents=True, exist_ok=True)
return project_root, audio_dir, subtitle_dir, metadata_dir
return project_root, project_root, project_root, None
+72
View File
@@ -0,0 +1,72 @@
from __future__ import annotations
"""Progress and ETR (estimated time remaining) calculation.
Shared by Web UI and PyQt desktop GUI. Pure math, no UI dependencies.
"""
import time
from dataclasses import dataclass, field
@dataclass
class ProgressTracker:
"""Tracks character-based progress with ETR calculation.
Usage:
tracker = ProgressTracker(total_chars=50000)
# ... as processing occurs:
tracker.update(chars_done=5000)
print(tracker.etr_str) # "00:04:30"
print(tracker.percent) # 10
"""
total_chars: int
_start_time: float = field(default_factory=time.time, repr=False)
_chars_done: int = field(default=0, repr=False)
def update(self, chars_done: int) -> None:
self._chars_done = chars_done
@property
def percent(self) -> int:
if self.total_chars <= 0:
return 0
return min(int(self._chars_done / self.total_chars * 100), 99)
@property
def etr_str(self) -> str:
elapsed = time.time() - self._start_time
if self._chars_done <= 0 or elapsed <= 0.5:
return "Processing..."
avg_time_per_char = elapsed / self._chars_done
remaining = self.total_chars - self._chars_done
if remaining <= 0:
return "00:00:00"
secs = avg_time_per_char * remaining
h = int(secs // 3600)
m = int((secs % 3600) // 60)
s = int(secs % 60)
return f"{h:02d}:{m:02d}:{s:02d}"
def calc_etr_str(elapsed: float, done: int, total: int) -> str:
"""Standalone ETR string calculation (matches PyQt original logic).
Args:
elapsed: seconds since processing started
done: items/characters processed so far
total: total items/characters to process
Returns:
ETR string like "01:23:45" or "Processing..."
"""
if done <= 0 or elapsed <= 0.5:
return "Processing..."
avg_time_per_item = elapsed / done
remaining = total - done
if remaining <= 0:
return "00:00:00"
secs = avg_time_per_item * remaining
h = int(secs // 3600)
m = int((secs % 3600) // 60)
s = int(secs % 60)
return f"{h:02d}:{m:02d}:{s:02d}"
+261
View File
@@ -0,0 +1,261 @@
"""Pronunciation rule compilation and application.
Pure functions for compiling token-level and sentence-level pronunciation
overrides into regex patterns, applying them to text, and merging multiple
override sources with precedence rules.
"""
from __future__ import annotations
import re
from typing import Any, Dict, Iterable, List, Mapping, Optional
from abogen.entity_analysis import normalize_token as normalize_entity_token
from abogen.entity_analysis import normalize_manual_override_token
def compile_pronunciation_rules(
overrides: Optional[Iterable[Mapping[str, Any]]],
) -> List[Dict[str, Any]]:
if not overrides:
return []
candidates: List[Dict[str, Any]] = []
seen: set[str] = set()
for entry in overrides:
if not isinstance(entry, Mapping):
continue
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not pronunciation_value:
continue
token_values: List[str] = []
token_raw = entry.get("token")
if token_raw:
token_value = str(token_raw).strip()
if token_value:
token_values.append(token_value)
normalized_raw = entry.get("normalized")
if normalized_raw:
normalized_value = str(normalized_raw).strip()
if normalized_value:
token_values.append(normalized_value)
if token_raw and not token_values:
fallback = normalize_entity_token(str(token_raw))
if fallback:
token_values.append(fallback)
if not token_values:
continue
usage_normalized = str(entry.get("normalized") or "").strip()
if not usage_normalized and token_values:
usage_normalized = normalize_entity_token(token_values[0]) or token_values[0]
usage_token = str(entry.get("token") or token_values[0])
for token_value in token_values:
key = token_value.casefold()
if key in seen:
continue
seen.add(key)
candidates.append(
{
"token": token_value,
"normalized": usage_normalized,
"replacement": pronunciation_value,
}
)
if not candidates:
return []
candidates.sort(key=lambda item: len(item["token"]), reverse=True)
compiled: List[Dict[str, Any]] = []
for candidate in candidates:
token_value = candidate["token"]
pronunciation_value = candidate["replacement"]
escaped = re.escape(token_value)
pattern = re.compile(rf"(?i)(?<!\w){escaped}(?P<possessive>'s|\u2019s|\u2019)?(?!\w)")
compiled.append(
{
"pattern": pattern,
"replacement": pronunciation_value,
"normalized": candidate.get("normalized") or token_value,
"token": candidate.get("token") or token_value,
}
)
return compiled
def compile_heteronym_sentence_rules(
overrides: Optional[Iterable[Mapping[str, Any]]],
) -> List[Dict[str, Any]]:
if not overrides:
return []
compiled: List[Dict[str, Any]] = []
seen: set[str] = set()
for entry in overrides:
if not isinstance(entry, Mapping):
continue
sentence = str(entry.get("sentence") or "").strip()
if not sentence:
continue
choice = str(entry.get("choice") or "").strip()
if not choice:
continue
replacement_sentence = ""
options = entry.get("options")
if isinstance(options, list):
for opt in options:
if not isinstance(opt, Mapping):
continue
if str(opt.get("key") or "").strip() == choice:
replacement_sentence = str(opt.get("replacement_sentence") or "").strip()
break
if not replacement_sentence:
continue
rule_key = f"{sentence}\n{choice}".casefold()
if rule_key in seen:
continue
seen.add(rule_key)
parts = [p for p in re.split(r"\s+", sentence) if p]
if not parts:
continue
pattern_text = r"\s+".join(re.escape(p) for p in parts)
pattern = re.compile(pattern_text)
compiled.append({"pattern": pattern, "replacement": replacement_sentence})
compiled.sort(key=lambda item: len(item["pattern"].pattern), reverse=True)
return compiled
def apply_heteronym_sentence_rules(text: str, rules: List[Dict[str, Any]]) -> str:
if not text or not rules:
return text
result = text
for rule in rules:
pattern = rule["pattern"]
replacement = rule["replacement"]
result = pattern.sub(replacement, result)
return result
def apply_pronunciation_rules(
text: str,
rules: List[Dict[str, Any]],
usage_counter: Optional[Dict[str, int]] = None,
) -> str:
if not text or not rules:
return text
result = text
for rule in rules:
pattern = rule["pattern"]
pronunciation_value = rule["replacement"]
usage_key = str(rule.get("normalized") or "").strip()
def _replacement(match: re.Match[str]) -> str:
suffix = match.group("possessive") or ""
if usage_counter is not None and usage_key:
usage_counter[usage_key] = usage_counter.get(usage_key, 0) + 1
return pronunciation_value + suffix
result = pattern.sub(_replacement, result)
return result
def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
"""Return pronunciation override entries, ensuring manual overrides are included.
Pending jobs keep both ``manual_overrides`` and ``pronunciation_overrides``, but the
latter can be stale if the UI didn't resync before enqueue. During conversion,
we must merge manual overrides so they always apply (before TTS).
Precedence: manual overrides win over existing entries for the same normalized key.
"""
collected: Dict[str, Dict[str, Any]] = {}
existing = getattr(job, "pronunciation_overrides", None)
if isinstance(existing, list):
for entry in existing:
if not isinstance(entry, Mapping):
continue
token_value = str(entry.get("token") or "").strip()
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = str(entry.get("normalized") or "").strip() or normalize_entity_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(entry.get("voice") or "").strip() or None,
"notes": str(entry.get("notes") or "").strip() or None,
"context": str(entry.get("context") or "").strip() or None,
"source": str(entry.get("source") or "pronunciation"),
"language": getattr(job, "language", None),
}
speakers = getattr(job, "speakers", None)
if isinstance(speakers, dict):
for payload in speakers.values():
if not isinstance(payload, Mapping):
continue
token_value = str(payload.get("token") or "").strip()
pronunciation_value = str(payload.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = normalize_entity_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(
payload.get("resolved_voice")
or payload.get("voice")
or getattr(job, "voice", "")
).strip()
or None,
"notes": None,
"context": None,
"source": "speaker",
"language": getattr(job, "language", None),
}
manual = getattr(job, "manual_overrides", None)
if isinstance(manual, list):
for entry in manual:
if not isinstance(entry, Mapping):
continue
token_value = str(entry.get("token") or "").strip()
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = str(entry.get("normalized") or "").strip() or normalize_manual_override_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(entry.get("voice") or "").strip() or None,
"notes": str(entry.get("notes") or "").strip() or None,
"context": str(entry.get("context") or "").strip() or None,
"source": str(entry.get("source") or "manual"),
"language": getattr(job, "language", None),
}
return list(collected.values())
+40
View File
@@ -0,0 +1,40 @@
from __future__ import annotations
"""Unified split pattern logic extracted from 3 copies."""
import re
PUNCTUATION_SENTENCE = r".!?。!?"
PUNCTUATION_SENTENCE_COMMA = r".!?,。!?、,"
def get_split_pattern(language: str, subtitle_mode: str) -> str:
"""Get the appropriate split pattern based on language and subtitle mode.
Args:
language: Language code (a, b, e, f, etc.)
subtitle_mode: Subtitle mode ("Sentence", "Sentence + Comma", "Line", etc.)
Returns:
Split pattern string
"""
# For English, always use newline splitting only
if language in ("a", "b"):
return "\n"
# Determine spacing pattern based on language
spacing = r"\s*" if language in ("z", "j") else r"\s+"
# For CJK languages, when subtitle mode is Disabled or Line, prefer
# punctuation-based splitting instead of plain newline splitting.
if subtitle_mode in ("Disabled", "Line") and language in ("z", "j"):
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
if subtitle_mode == "Line":
return "\n"
elif subtitle_mode == "Sentence":
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
elif subtitle_mode == "Sentence + Comma":
return rf"(?<=[{PUNCTUATION_SENTENCE_COMMA}]){spacing}|\n+"
else:
return r"\n+"
+358
View File
@@ -0,0 +1,358 @@
"""Subtitle generation utilities for audiobook generation.
This module provides functions for processing TTS tokens into subtitle entries
according to various subtitle modes (Line, Sentence, Sentence + Comma,
Sentence + Highlighting).
"""
from __future__ import annotations
import re
from typing import List, Optional, Tuple
# Punctuation constants for sentence splitting
PUNCTUATION_SENTENCE = ".!?\u061f\u3002\uff01\uff1f" # .!? .?. ??
PUNCTUATION_SENTENCE_COMMA = ".!?,\u3001\u061f\u3002\uff01\uff0c\uff1f" # .!?, ,. ??
def process_subtitle_tokens(
tokens_with_timestamps: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
lang_code: str,
use_spacy_segmentation: bool = False,
fallback_end_time: Optional[float] = None,
) -> None:
"""Process TTS tokens into subtitle entries according to the subtitle mode.
This function modifies subtitle_entries in-place by appending new entries.
Args:
tokens_with_timestamps: List of token dictionaries with 'start', 'end', 'text',
and 'whitespace' keys.
subtitle_entries: List to append subtitle entries to (modified in-place).
Each entry is a tuple of (start_time, end_time, text).
max_subtitle_words: Maximum number of words per subtitle entry.
subtitle_mode: One of "Disabled", "Line", "Sentence", "Sentence + Comma",
"Sentence + Highlighting", or a string like "5" for word-count mode.
lang_code: Language code for spaCy processing (e.g., "a" for English).
use_spacy_segmentation: Whether to use spaCy for sentence boundary detection.
fallback_end_time: Fallback end time for the last entry if none is available.
"""
if not tokens_with_timestamps:
return
processed_tokens = tokens_with_timestamps
# For English with spaCy enabled and sentence-based modes, use spaCy for sentence boundaries
# spaCy is disabled when subtitle mode is "Disabled" or "Line"
use_spacy_for_english = (
use_spacy_segmentation
and subtitle_mode not in ["Disabled", "Line"]
and lang_code in ["a", "b"]
and subtitle_mode in ["Sentence", "Sentence + Comma"]
)
if subtitle_mode == "Sentence + Highlighting":
_process_karaoke_highlighting(
processed_tokens, subtitle_entries, max_subtitle_words, fallback_end_time
)
elif subtitle_mode in ["Sentence", "Sentence + Comma", "Line"]:
if use_spacy_for_english and subtitle_mode != "Line":
_process_spacy_sentences(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, lang_code, fallback_end_time
)
else:
_process_regex_sentences(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
else:
# Word count-based grouping (e.g., "5" for 5-word groups)
_process_word_count(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
def _process_karaoke_highlighting(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens for Sentence + Highlighting mode (karaoke effect)."""
separator = rf"[{re.escape(PUNCTUATION_SENTENCE)}]"
current_sentence = []
word_count = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
# Split sentences based on separator or word count
if (
re.search(separator, token["text"]) and token.get("whitespace") == " "
) or word_count >= max_subtitle_words:
if current_sentence:
# Create karaoke subtitle entry for this sentence
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Generate karaoke text with timing
karaoke_text = ""
for t in current_sentence:
# Calculate duration in centiseconds
duration = (
t["end"] - t["start"]
if t.get("end") is not None and t.get("start") is not None
else 0.5
)
duration_cs = int(duration * 100)
# Add karaoke effect
karaoke_text += f"{{\\kf{duration_cs}}}{t['text']}{t.get('whitespace', '') or ''}"
subtitle_entries.append(
(start_time, end_time, karaoke_text.strip())
)
current_sentence = []
word_count = 0
# Add any remaining tokens as a sentence
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Generate karaoke text for remaining tokens
karaoke_text = ""
for t in current_sentence:
duration = t["end"] - t["start"] if t.get("end") and t.get("start") else 0.5
duration_cs = int(duration * 100)
karaoke_text += f"{{\\kf{duration_cs}}}{t['text']}{t.get('whitespace', '') or ''}"
subtitle_entries.append((start_time, end_time, karaoke_text.strip()))
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_spacy_sentences(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
lang_code: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens using spaCy for sentence boundary detection."""
try:
from abogen.spacy_utils import get_spacy_model
except ImportError:
# Fall back to regex if spaCy is not available
_process_regex_sentences(
tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
return
nlp = get_spacy_model(lang_code)
if not nlp:
_process_regex_sentences(
tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
return
# Build full text and track character positions to token indices
full_text = ""
for token in tokens:
text_part = token["text"] + (token.get("whitespace") or "")
full_text += text_part
# Get sentence boundaries from spaCy
doc = nlp(full_text)
sentence_boundaries = [sent.end_char for sent in doc.sents]
# For "Sentence + Comma" mode, also split on commas
if subtitle_mode == "Sentence + Comma":
comma_positions = [
i + 1 for i, c in enumerate(full_text) if c == ","
]
sentence_boundaries = sorted(
set(sentence_boundaries + comma_positions)
)
# Group tokens by sentence boundaries
current_sentence = []
word_count = 0
current_char_pos = 0
boundary_idx = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
text_len = len(token["text"]) + len(token.get("whitespace") or "")
current_char_pos += text_len
# Check if we've hit a sentence boundary or max words
at_boundary = (
boundary_idx < len(sentence_boundaries)
and current_char_pos >= sentence_boundaries[boundary_idx]
)
if at_boundary or word_count >= max_subtitle_words:
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = "".join(
t["text"] + (t.get("whitespace") or "")
for t in current_sentence
)
subtitle_entries.append(
(start_time, end_time, sentence_text.strip())
)
current_sentence = []
word_count = 0
if at_boundary:
boundary_idx += 1
# Add remaining tokens
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = "".join(
t["text"] + (t.get("whitespace") or "")
for t in current_sentence
)
subtitle_entries.append(
(start_time, end_time, sentence_text.strip())
)
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_regex_sentences(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens using regex for sentence boundary detection."""
# Define separator pattern based on mode
if subtitle_mode == "Line":
separator = r"\n"
elif subtitle_mode == "Sentence":
# Use punctuation without comma
separator = rf"[{re.escape(PUNCTUATION_SENTENCE)}]"
else: # Sentence + Comma
# Use punctuation with comma
separator = rf"[{re.escape(PUNCTUATION_SENTENCE_COMMA)}]"
current_sentence = []
word_count = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
# Split sentences based on separator or word count
if (
re.search(separator, token["text"]) and token.get("whitespace") == " "
) or word_count >= max_subtitle_words:
if current_sentence:
# Create subtitle entry for this sentence
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Simplified text joining logic
sentence_text = ""
for t in current_sentence:
sentence_text += t["text"] + (t.get("whitespace") or "")
subtitle_entries.append(
(start_time, end_time, sentence_text.strip())
)
current_sentence = []
word_count = 0
# Add any remaining tokens as a sentence
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Simplified text joining logic
sentence_text = ""
for t in current_sentence:
sentence_text += t["text"] + (t.get("whitespace") or "")
subtitle_entries.append((start_time, end_time, sentence_text.strip()))
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_word_count(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens by counting spaces (word count mode)."""
try:
word_count = int(subtitle_mode.split()[0])
word_count = min(word_count, max_subtitle_words)
except (ValueError, IndexError):
word_count = 1
current_group = []
space_count = 0
for token in tokens:
current_group.append(token)
# Count spaces after tokens (in the whitespace field)
if token.get("whitespace", "") == " ":
space_count += 1
# Split after counting N spaces
if space_count >= word_count:
text = "".join(
t["text"] + (t.get("whitespace") or "")
for t in current_group
)
subtitle_entries.append(
(
current_group[0]["start"],
current_group[-1]["end"],
text.strip(),
)
)
current_group = []
space_count = 0
# Add any remaining tokens
if current_group:
text = "".join(
t["text"] + (t.get("whitespace") or "") for t in current_group
)
subtitle_entries.append(
(current_group[0]["start"], current_group[-1]["end"], text.strip())
)
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _apply_fallback_end_time(
subtitle_entries: List[Tuple[float, float, str]],
fallback_end_time: Optional[float],
) -> None:
"""Apply fallback end time to the last entry if needed."""
if subtitle_entries and fallback_end_time is not None:
last_entry = subtitle_entries[-1]
start, end, text = last_entry
if end is None or end <= start or end <= 0:
subtitle_entries[-1] = (start, fallback_end_time, text)
+97
View File
@@ -0,0 +1,97 @@
from __future__ import annotations
from pathlib import Path
from typing import Any, Dict, List, Mapping, Optional
from .metadata_helpers import (
ensure_sentence,
extract_series_metadata,
format_author_sentence,
format_series_sentence,
normalize_metadata_map,
)
def build_title_intro_text(
metadata: Optional[Mapping[str, Any]],
fallback_basename: str,
) -> str:
"""Build the title introduction text from metadata."""
normalized = normalize_metadata_map(metadata)
fallback_title = Path(fallback_basename).stem if fallback_basename else ""
title = (
normalized.get("title")
or normalized.get("book_title")
or normalized.get("album")
or fallback_title
)
if not title:
title = fallback_title
subtitle = normalized.get("subtitle") or normalized.get("sub_title")
if subtitle and title and subtitle.casefold() == title.casefold():
subtitle = ""
author_value = ""
for candidate in ("artist", "album_artist", "author", "authors", "writer", "composer"):
value = normalized.get(candidate)
if value:
author_value = value
break
series_name, series_number = extract_series_metadata(normalized)
series_sentence = format_series_sentence(series_name, series_number)
sentences: List[str] = []
if series_sentence:
sentences.append(ensure_sentence(series_sentence))
if title:
sentences.append(ensure_sentence(title))
if subtitle:
sentences.append(ensure_sentence(subtitle))
author_sentence = format_author_sentence(author_value)
if author_sentence:
sentences.append(ensure_sentence(author_sentence))
return " ".join(sentences).strip()
def build_outro_text(
metadata: Optional[Mapping[str, Any]],
fallback_basename: str,
) -> str:
"""Build the outro/closing text from metadata."""
normalized = normalize_metadata_map(metadata)
fallback_title = Path(fallback_basename).stem if fallback_basename else ""
title = (
normalized.get("title")
or normalized.get("book_title")
or normalized.get("album")
or fallback_title
)
author_value = ""
for candidate in ("authors", "author", "album_artist", "artist", "writer", "composer"):
value = normalized.get(candidate)
if value:
author_value = value
break
author_sentence = format_author_sentence(author_value)
authors_fragment = (
author_sentence[3:].strip() if author_sentence.lower().startswith("by ") else author_sentence.strip()
)
if title and authors_fragment:
closing_line = f"The end of {title} from {authors_fragment}"
elif title:
closing_line = f"The end of {title}"
elif authors_fragment:
closing_line = f"The end from {authors_fragment}"
else:
closing_line = "The end"
series_name, series_number = extract_series_metadata(normalized)
series_sentence = format_series_sentence(series_name, series_number)
sentences: List[str] = [ensure_sentence(closing_line)]
if series_sentence:
sentences.append(ensure_sentence(series_sentence))
return " ".join(sentence for sentence in sentences if sentence).strip()
+13
View File
@@ -0,0 +1,13 @@
"""Shared token stubs for TTS processing."""
from __future__ import annotations
class FakeToken:
"""Minimal token stub for languages without per-word token support."""
def __init__(self, text: str, start: float, end: float):
self.text = text
self.start_ts = start
self.end_ts = end
self.whitespace = ""
+116
View File
@@ -0,0 +1,116 @@
"""Voice loading and caching utilities.
This module provides unified voice loading with caching support for both
PyQt and WebUI interfaces.
"""
from __future__ import annotations
from typing import Any, Dict, Optional, Tuple
from abogen.voice_formulas import get_new_voice
class VoiceCache:
"""Thread-safe voice cache for loaded voice tensors."""
def __init__(self):
self._cache: Dict[str, Any] = {}
def get(self, voice_spec: str) -> Optional[Any]:
"""Get cached voice by spec."""
return self._cache.get(voice_spec)
def set(self, voice_spec: str, voice: Any) -> None:
"""Cache a loaded voice."""
self._cache[voice_spec] = voice
def contains(self, voice_spec: str) -> bool:
"""Check if voice is in cache."""
return voice_spec in self._cache
def clear(self) -> None:
"""Clear all cached voices."""
self._cache.clear()
def __contains__(self, voice_spec: str) -> bool:
return self.contains(voice_spec)
def resolve_voice(
voice_spec: str,
pipeline: Any,
use_gpu: bool,
cache: Optional[VoiceCache] = None,
) -> Any:
"""Resolve voice spec to actual voice tensor or name.
If voice_spec contains '*' (formula), loads the voice using get_new_voice.
Otherwise, returns the voice_spec as-is (it's a voice name).
Uses optional cache to avoid reloading same voice multiple times.
Args:
voice_spec: Voice specification (name or formula string with '*').
pipeline: TTS pipeline instance for loading formula voices.
use_gpu: Whether to use GPU for voice loading.
cache: Optional VoiceCache instance for caching loaded voices.
Returns:
Loaded voice tensor (for formulas) or voice name string.
"""
# Check cache first
if cache and cache.contains(voice_spec):
return cache.get(voice_spec)
# Load voice
if "*" in voice_spec:
if pipeline is None:
return voice_spec
loaded_voice = get_new_voice(pipeline, voice_spec, use_gpu)
else:
loaded_voice = voice_spec
# Cache it
if cache:
cache.set(voice_spec, loaded_voice)
return loaded_voice
def load_voice_cached(
voice_name: str,
pipeline: Any,
use_gpu: bool,
cache: Optional[Dict[str, Any]] = None,
) -> Any:
"""Load voice with caching (compatibility wrapper for PyQt).
This function maintains backward compatibility with the PyQt interface
while using the unified voice loading logic.
Args:
voice_name: Voice name or formula string.
pipeline: TTS pipeline instance.
use_gpu: Whether to use GPU.
cache: Optional dict to use as cache (instead of VoiceCache).
Returns:
Loaded voice tensor or voice name string.
"""
# Use dict cache if provided (for backward compatibility)
if cache is not None:
if voice_name in cache:
return cache[voice_name]
# Load voice
if "*" in voice_name:
loaded_voice = get_new_voice(pipeline, voice_name, use_gpu)
else:
loaded_voice = voice_name
# Cache it
if cache is not None:
cache[voice_name] = loaded_voice
return loaded_voice
+190
View File
@@ -0,0 +1,190 @@
"""Voice resolution helpers.
Functions for resolving voice specifications, collecting required voice IDs,
and determining the voice to use for chapters and chunks.
"""
from __future__ import annotations
from typing import Any, Dict, Optional, Set
from abogen.tts_plugin.utils import get_voices, get_default_voice
from abogen.voice_formulas import extract_voice_ids
from abogen.voice_cache import ensure_voice_assets
def spec_to_voice_ids(spec: Any) -> Set[str]:
text = str(spec or "").strip()
if not text:
return set()
if text == "__custom_mix":
return set()
if "*" in text:
try:
return set(extract_voice_ids(text))
except ValueError:
return set()
if text in get_voices("kokoro"):
return {text}
return set()
def job_voice_fallback(job: Any) -> str:
base = str(getattr(job, "voice", "") or "").strip()
if base and base != "__custom_mix":
return base
speakers = getattr(job, "speakers", None)
if isinstance(speakers, dict):
narrator = speakers.get("narrator")
if isinstance(narrator, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = narrator.get(key)
candidate = str(value or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
for payload in speakers.values() or []:
if not isinstance(payload, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
value = payload.get(key)
candidate = str(value or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
for chapter in getattr(job, "chapters", []) or []:
if not isinstance(chapter, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
candidate = str(chapter.get(key) or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
return ""
def collect_required_voice_ids(job: Any) -> Set[str]:
voices: Set[str] = set()
voices.update(spec_to_voice_ids(job.voice))
voices.update(spec_to_voice_ids(job_voice_fallback(job)))
for chapter in getattr(job, "chapters", []) or []:
if not isinstance(chapter, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(chapter.get(key)))
for chunk in getattr(job, "chunks", []) or []:
if not isinstance(chunk, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(chunk.get(key)))
speakers = getattr(job, "speakers", {})
if isinstance(speakers, dict):
for payload in speakers.values() or []:
if not isinstance(payload, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(payload.get(key)))
voices.update(get_voices("kokoro"))
return voices
def initialize_voice_cache(job: Any) -> None:
try:
targets = collect_required_voice_ids(job)
downloaded, errors = ensure_voice_assets(
targets,
on_progress=lambda message: job.add_log(message, level="debug"),
)
except RuntimeError as exc:
job.add_log(f"Voice cache unavailable: {exc}", level="warning")
return
if downloaded:
job.add_log(
f"Cached {len(downloaded)} voice asset{'s' if len(downloaded) != 1 else ''} locally.",
level="info",
)
for voice_id, error in errors.items():
job.add_log(f"Failed to cache voice '{voice_id}': {error}", level="warning")
def chapter_voice_spec(job: Any, override: Optional[Dict[str, Any]]) -> str:
if not override:
return job_voice_fallback(job)
resolved = str(override.get("resolved_voice", "")).strip()
if resolved:
return resolved
formula = str(override.get("voice_formula", "")).strip()
if formula:
return formula
voice = str(override.get("voice", "")).strip()
if voice:
return voice
return job_voice_fallback(job)
def chunk_voice_spec(job: Any, chunk: Dict[str, Any], fallback: str) -> str:
for key in ("resolved_voice", "voice_formula", "voice"):
value = chunk.get(key)
if value:
return str(value)
speaker_id = chunk.get("speaker_id")
speakers = getattr(job, "speakers", None)
if isinstance(speakers, dict) and speaker_id in speakers:
speaker_entry = speakers.get(speaker_id) or {}
if isinstance(speaker_entry, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = speaker_entry.get(key)
if value:
return str(value)
profile_formula = speaker_entry.get("voice_formula")
if profile_formula:
return str(profile_formula)
profile_name = chunk.get("voice_profile")
if profile_name:
if isinstance(speakers, dict):
speaker_entry = speakers.get(profile_name)
if isinstance(speaker_entry, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = speaker_entry.get(key)
if value:
return str(value)
if fallback:
return fallback
return job_voice_fallback(job)
def resolve_fallback_voice_spec(
base_spec: str,
job_voice: str,
voice_cache_keys: list[str],
provider: str = "kokoro",
) -> str:
"""Resolve the voice spec for intro/outro with a priority fallback chain.
Priority: base_spec → job_voice → first voice_cache key → default voice.
``"__custom_mix"`` is treated as empty (it is not a usable voice spec).
"""
spec = base_spec or job_voice
if spec == "__custom_mix":
spec = job_voice or ""
if not spec:
for key in voice_cache_keys:
if key and key != "__custom_mix":
spec = key.split(":", 1)[-1]
break
if not spec:
spec = get_default_voice(provider)
return spec
+97
View File
@@ -0,0 +1,97 @@
from __future__ import annotations
from typing import Any, Mapping, Optional, Tuple, Set
from abogen.voice_formulas import extract_voice_ids, get_new_voice
from abogen.tts_plugin.utils import get_voices
def infer_provider_from_spec(value: Any, fallback: str = "kokoro") -> str:
"""Infer TTS provider from voice specification."""
raw = str(value or "").strip()
if not raw:
return fallback
if raw.upper() == raw and raw.replace("_", "").isalnum():
return "supertonic"
if raw == "__custom_mix" or "*" in raw or "+" in raw:
return "kokoro"
if raw in get_voices("kokoro"):
return "kokoro"
return fallback
def supertonic_voice_from_spec(spec: Any, fallback: str) -> str:
"""Normalize a voice specification for Supertonic.
This function only performs Supertonic-specific normalization (uppercase conversion
and fallback handling). Backend resolution is handled by the registry.
"""
raw = str(spec or "").strip()
fallback_raw = str(fallback or "").strip()
# Normalize to uppercase for Supertonic voice IDs
upper = raw.upper() if raw else ""
# If empty or contains formula characters, use fallback
if not upper or "*" in upper or "+" in upper:
upper = fallback_raw.upper() if fallback_raw else ""
# If still empty, use default Supertonic voice
if not upper or "*" in upper or "+" in upper:
upper = "M1"
return upper
def split_speaker_reference(value: Any) -> Tuple[Optional[str], str]:
"""Parse speaker/profile reference from string.
Expected format: "speaker:name" or "profile:name"
Returns (name, original) or (None, original) if not a valid reference.
"""
raw = str(value or "").strip()
if not raw or ":" not in raw:
return None, raw
prefix, remainder = raw.split(":", 1)
prefix = prefix.strip().lower()
if prefix not in {"speaker", "profile"}:
return None, raw
name = remainder.strip()
return (name or None), raw
def formula_from_kokoro_entry(entry: Mapping[str, Any]) -> str:
"""Build voice formula string from kokoro entry."""
voices = entry.get("voices") or []
if not voices:
return ""
total = 0.0
parts: list[tuple[str, float]] = []
for item in voices:
if not isinstance(item, (list, tuple)) or len(item) < 2:
continue
name = str(item[0] or "").strip()
try:
weight = float(item[1])
except (TypeError, ValueError):
continue
if name and weight > 0:
parts.append((name, weight))
total += weight
if not parts:
return ""
normalized = [(name, weight / total) for name, weight in parts]
return " + ".join(f"{name}*{weight:.6f}" for name, weight in normalized)
def coerce_truthy(value: Any, default: bool = True) -> bool:
"""Coerce a value to boolean with default."""
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() not in {"false", "0", "no", "off", ""}
if value is None:
return default
return bool(value)
+448
View File
@@ -0,0 +1,448 @@
from __future__ import annotations
import json
import logging
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Optional, Mapping, Sequence
import static_ffmpeg
from abogen.domain.metadata_helpers import (
normalize_metadata_casefold,
split_people_field,
split_simple_list,
first_nonempty,
extract_year,
normalize_series_sequence,
build_audiobookshelf_metadata as _build_abs_metadata,
load_audiobookshelf_chapters as _load_abs_chapters,
_SERIES_SEQUENCE_TAG_KEYS,
)
from abogen.epub3.exporter import build_epub3_package
from abogen.integrations.audiobookshelf import (
AudiobookshelfClient,
AudiobookshelfConfig,
AudiobookshelfUploadError,
)
from abogen.utils import create_process
logger = logging.getLogger(__name__)
@dataclass
class ExportConfig:
"""Configuration for export operations."""
ffmpeg_path: str = "ffmpeg"
verify_ssl: bool = True
class ExportService:
"""Unified service for audiobook exports (M4B, FFMETADATA, EPUB3, Audiobookshelf)."""
def __init__(self, config: Optional[ExportConfig] = None):
self.config = config or ExportConfig()
static_ffmpeg.add_paths()
# ----------------------------------------------------------------------
# FFMETADATA
# ----------------------------------------------------------------------
def render_ffmetadata(
self,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
) -> str:
"""Render FFMETADATA content."""
lines = [";FFMETADATA1"]
for key, value in (metadata or {}).items():
if value is None:
continue
key_str = str(key).strip()
if not key_str:
continue
lines.append(f"{key_str}={self._escape_ffmetadata_value(value)}")
for chapter in chapters or []:
start = chapter.get("start")
end = chapter.get("end")
if start is None or end is None:
continue
try:
start_ms = max(0, int(round(float(start) * 1000)))
end_ms = int(round(float(end) * 1000))
except (TypeError, ValueError):
continue
if end_ms <= start_ms:
end_ms = start_ms + 1
lines.append("[CHAPTER]")
lines.append("TIMEBASE=1/1000")
lines.append(f"START={start_ms}")
lines.append(f"END={end_ms}")
title = chapter.get("title")
if title:
lines.append(f"title={self._escape_ffmetadata_value(title)}")
voice = chapter.get("voice")
if voice:
lines.append(f"voice={self._escape_ffmetadata_value(voice)}")
return "\n".join(lines) + "\n"
@staticmethod
def _escape_ffmetadata_value(value: Any) -> str:
escaped = str(value).replace("\\", "\\\\").replace("\n", "\\n")
escaped = escaped.replace("=", "\\=").replace(";", "\\;").replace("#", "\\#")
return escaped
def write_ffmetadata_file(
self,
audio_path: Path,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
) -> Optional[Path]:
"""Write FFMETADATA file to temp location."""
content = self.render_ffmetadata(metadata, chapters)
if content.strip() == ";FFMETADATA1":
return None
directory = audio_path.parent if audio_path.parent.exists() else Path(tempfile.gettempdir())
with tempfile.NamedTemporaryFile(
mode="w",
encoding="utf-8",
suffix=".ffmeta",
delete=False,
dir=str(directory),
) as handle:
handle.write(content)
return Path(handle.name)
# ----------------------------------------------------------------------
# M4B Export
# ----------------------------------------------------------------------
def embed_m4b_metadata(
self,
audio_path: Path,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
cover_path: Optional[Path] = None,
cover_mime: Optional[str] = None,
log_callback: Optional[callable] = None,
) -> None:
"""Embed metadata and chapters into M4B file using FFmpeg + Mutagen."""
ffmetadata_path = self.write_ffmetadata_file(audio_path, metadata, chapters)
metadata_args = self._metadata_to_ffmpeg_args(metadata)
cmd = ["ffmpeg", "-y", "-i", str(audio_path)]
if ffmetadata_path:
cmd.extend(["-f", "ffmetadata", "-i", str(ffmetadata_path)])
if cover_path and cover_path.exists():
cmd.extend(["-i", str(cover_path)])
cmd.extend(["-map", "0:a"])
cmd.extend(["-map", "1:v:0", "-c:v:0", "mjpeg", "-disposition:v:0", "attached_pic"])
if cover_mime:
cmd.extend(["-metadata:s:v:0", f"mimetype={cover_mime}"])
cmd.extend(["-metadata:s:v:0", "title=Cover Art"])
else:
cmd.extend(["-map", "0:a"])
cmd.extend(["-c:a", "copy"])
if ffmetadata_path:
cmd.extend(["-map_metadata", "1", "-map_chapters", "1"])
else:
cmd.extend(["-map_metadata", "0"])
if metadata_args:
cmd.extend(metadata_args)
cmd.extend(["-movflags", "+faststart+use_metadata_tags"])
temp_output = audio_path.with_suffix(audio_path.suffix + ".tmp")
if audio_path.suffix.lower() in {".m4b", ".mp4", ".m4a"}:
cmd.extend(["-f", "mp4"])
cmd.append(str(temp_output))
if log_callback:
log_callback("Embedding metadata into M4B output")
process = create_process(cmd, text=True)
return_code = process.wait()
if ffmetadata_path and ffmetadata_path.exists():
try:
ffmetadata_path.unlink()
except OSError:
pass
if return_code != 0:
if temp_output.exists():
temp_output.unlink(missing_ok=True)
raise RuntimeError(f"ffmpeg failed to embed metadata (exit code {return_code})")
temp_output.replace(audio_path)
if log_callback:
log_callback("Embedded metadata and chapters into M4B output", "info")
# Apply chapters via Mutagen for better compatibility
self._apply_m4b_chapters_mutagen(audio_path, chapters, log_callback)
@staticmethod
def _metadata_to_ffmpeg_args(metadata: Dict[str, Any]) -> List[str]:
args = []
for key, value in (metadata or {}).items():
if value in (None, ""):
continue
key_str = str(key).strip()
if not key_str:
continue
normalized_key = key_str.lower()
if normalized_key == "year":
ffmpeg_key = "date"
else:
ffmpeg_key = key_str
args.extend(["-metadata", f"{ffmpeg_key}={value}"])
return args
def _apply_m4b_chapters_mutagen(
self,
audio_path: Path,
chapters: List[Dict[str, Any]],
log_callback: Optional[callable] = None,
) -> bool:
"""Apply chapter atoms using Mutagen."""
if not chapters:
return False
try:
from fractions import Fraction
from mutagen.mp4 import MP4, MP4Chapter
except ImportError:
if log_callback:
log_callback("Unable to write MP4 chapter atoms because mutagen is not installed.", "warning")
return False
try:
mp4 = MP4(str(audio_path))
except Exception as exc:
if log_callback:
log_callback(f"Failed to open m4b for chapter embedding: {exc}", "warning")
return False
chapter_objects = []
for index, entry in enumerate(sorted(chapters, key=lambda item: float(item.get("start") or 0.0))):
start_raw = entry.get("start")
if start_raw is None:
continue
try:
start_seconds = max(0.0, float(start_raw))
except (TypeError, ValueError):
continue
title_value = entry.get("title")
title_text = str(title_value) if title_value else f"Chapter {index + 1}"
start_fraction = Fraction(int(round(start_seconds * 1000)), 1000)
chapter_atom = MP4Chapter(start_fraction, title_text)
end_raw = entry.get("end")
if end_raw is not None:
try:
end_seconds = float(end_raw)
except (TypeError, ValueError):
end_seconds = None
if end_seconds is not None and end_seconds > start_seconds:
chapter_atom.end = Fraction(int(round(end_seconds * 1000)), 1000)
chapter_objects.append(chapter_atom)
if not chapter_objects:
return False
try:
mp4.chapters = chapter_objects
mp4.save()
except Exception as exc:
if log_callback:
log_callback(f"Failed to persist MP4 chapter atoms: {exc}", "warning")
return False
if log_callback:
log_callback(f"Applied {len(chapter_objects)} chapter markers via mutagen", "info")
return True
# ----------------------------------------------------------------------
# EPUB3 Export
# ----------------------------------------------------------------------
def export_epub3(
self,
output_path: Path,
book_id: str,
extraction: Any, # ExtractionResult
metadata_tags: Dict[str, Any],
chapter_markers: Sequence[Dict[str, Any]],
chunk_markers: Sequence[Dict[str, Any]],
chunks: Iterable[Dict[str, Any]],
audio_path: Path,
speaker_mode: str = "single",
cover_path: Optional[Path] = None,
cover_mime: Optional[str] = None,
) -> Path:
"""Export EPUB3 with media overlays."""
return build_epub3_package(
output_path=output_path,
book_id=book_id,
extraction=extraction,
metadata_tags=metadata_tags,
chapter_markers=chapter_markers,
chunk_markers=chunk_markers,
chunks=chunks,
audio_path=audio_path,
speaker_mode=speaker_mode,
cover_image_path=cover_path,
cover_image_mime=cover_mime,
)
# ----------------------------------------------------------------------
# Audiobookshelf Integration
# ----------------------------------------------------------------------
def build_audiobookshelf_metadata(self, job: Any) -> Dict[str, Any]:
"""Build Audiobookshelf metadata from job."""
filename = Path(getattr(job, "original_filename", "") or "").stem or "Audiobook"
return _build_abs_metadata(
getattr(job, "metadata_tags", {}),
language=getattr(job, "language", "") or "",
filename=filename,
)
def load_audiobookshelf_chapters(self, job: Any) -> Optional[List[Dict[str, Any]]]:
"""Load chapters from job artifacts for Audiobookshelf."""
metadata_ref = job.result.artifacts.get("metadata") if getattr(job, "result", None) else None
if not metadata_ref:
return None
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
return _load_abs_chapters(metadata_path)
def upload_audiobookshelf(
self,
job: Any,
audio_path: Path,
subtitle_paths: List[Path],
chapters: List[Dict[str, Any]],
metadata: Dict[str, Any],
cover_path: Optional[Path] = None,
config: Optional[AudiobookshelfConfig] = None,
log_callback: Optional[callable] = None,
) -> None:
"""Upload to Audiobookshelf."""
if config is None:
# Load from job or global config
cfg = getattr(job, "_abs_config", None)
if cfg is None:
from abogen.utils import load_config
global_cfg = load_config() or {}
abs_cfg = global_cfg.get("audiobookshelf")
if isinstance(abs_cfg, Mapping):
config = AudiobookshelfConfig(
base_url=str(abs_cfg.get("base_url") or "").strip(),
api_token=str(abs_cfg.get("api_token") or "").strip(),
library_id=str(abs_cfg.get("library_id") or "").strip(),
collection_id=(str(abs_cfg.get("collection_id") or "").strip() or None),
folder_id=str(abs_cfg.get("folder_id") or "").strip(),
verify_ssl=self._coerce_bool(abs_cfg.get("verify_ssl"), True),
send_cover=self._coerce_bool(abs_cfg.get("send_cover"), True),
send_chapters=self._coerce_bool(abs_cfg.get("send_chapters"), True),
send_subtitles=self._coerce_bool(abs_cfg.get("send_subtitles"), False),
timeout=float(abs_cfg.get("timeout", 3600.0)),
)
else:
if log_callback:
log_callback("Audiobookshelf upload skipped: not configured", "warning")
return
if not config.base_url or not config.api_token or not config.library_id:
if log_callback:
log_callback("Audiobookshelf upload skipped: configure base URL, API token, and library ID first", "warning")
return
if not config.folder_id:
if log_callback:
log_callback("Audiobookshelf upload skipped: enter folder name or ID in settings", "warning")
return
if not audio_path.exists():
if log_callback:
log_callback("Audiobookshelf upload skipped: audio output not found", "warning")
return
existing_subtitles = [p for p in subtitle_paths if p.exists()] if config.send_subtitles else None
chapters_to_send = chapters if config.send_chapters else None
client = AudiobookshelfClient(config)
display_title = metadata.get("title") or audio_path.stem
try:
existing_items = client.find_existing_items(display_title, folder_id=config.folder_id)
except AudiobookshelfUploadError as exc:
if log_callback:
log_callback(f"Audiobookshelf lookup failed: {exc}", "error")
return
if existing_items:
if log_callback:
log_callback(f"Removing existing Audiobookshelf item(s) for '{display_title}' before upload.", "info")
try:
client.delete_items(existing_items)
except Exception as exc:
if log_callback:
log_callback(f"Failed to remove existing item(s): {exc}", "warning")
cover_to_send = cover_path
if config.send_cover and cover_to_send:
if isinstance(cover_to_send, str):
cover_to_send = Path(cover_to_send)
if not cover_to_send.exists():
cover_to_send = None
client.upload_audiobook(
audio_path,
metadata=metadata,
cover_path=cover_to_send,
chapters=chapters_to_send,
subtitles=existing_subtitles,
)
if log_callback:
log_callback("Audiobookshelf upload queued.", "info")
# ----------------------------------------------------------------------
# Helpers
# ----------------------------------------------------------------------
@staticmethod
def _coerce_bool(value: Any, default: bool = True) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
lowered = value.strip().lower()
if lowered in {"true", "1", "yes", "on"}:
return True
if lowered in {"false", "0", "no", "off"}:
return False
return default
if value is None:
return default
return bool(value)
__all__ = [
"ExportConfig",
"ExportService",
]
+303
View File
@@ -0,0 +1,303 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass
from enum import Enum
from pathlib import Path
from typing import List, Optional, TextIO
from abogen.subtitle_utils import clean_subtitle_text
class SubtitleFormat(Enum):
SRT = "srt"
ASS = "ass"
VTT = "vtt"
class SubtitleMode(Enum):
DISABLED = "Disabled"
LINE = "Line"
SENTENCE = "Sentence"
SENTENCE_COMMA = "Sentence + Comma"
SENTENCE_HIGHLIGHT = "Sentence + Highlighting"
class SubtitleAlignment(Enum):
LEFT = "left"
CENTER = "center"
NARROW = "narrow"
CENTER_NARROW = "center_narrow"
@dataclass
class SubtitleConfig:
"""Configuration for subtitle writer."""
format: SubtitleFormat
mode: SubtitleMode
alignment: SubtitleAlignment = SubtitleAlignment.LEFT
max_words: int = 50
highlight_color: str = "&H00FFFF00" # ASS highlight color
class SubtitleWriter(ABC):
"""Abstract base class for subtitle writers."""
def __init__(self, path: Path, config: SubtitleConfig):
self.path = path
self.config = config
self._file: Optional[TextIO] = None
self._index = 0
self._opened = False
def open(self) -> None:
"""Open the subtitle file and write header."""
if self._opened:
return
self._file = open(self.path, "w", encoding="utf-8", errors="replace")
self._write_header()
self._opened = True
@abstractmethod
def _write_header(self) -> None:
pass
def write_entry(
self,
start: float,
end: float,
text: str,
voice: Optional[str] = None,
) -> None:
"""Write a subtitle entry."""
if not self._opened:
self.open()
text = clean_subtitle_text(text)
if not text:
return
self._index += 1
self._write_entry(self._index, start, end, text, voice)
@abstractmethod
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
pass
def close(self) -> None:
"""Close the subtitle file."""
if self._file:
self._file.close()
self._file = None
self._opened = False
def __enter__(self) -> "SubtitleWriter":
self.open()
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.close()
class SrtWriter(SubtitleWriter):
"""SRT subtitle writer."""
def _write_header(self) -> None:
pass # SRT has no header
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
self._file.write(f"{index}\n")
self._file.write(f"{start_str} --> {end_str}\n")
self._file.write(f"{text}\n\n")
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = int(seconds % 60)
millis = int((seconds - int(seconds)) * 1000)
return f"{hours:02d}:{minutes:02d}:{secs:02d},{millis:03d}"
class VttWriter(SubtitleWriter):
"""WebVTT subtitle writer."""
def _write_header(self) -> None:
self._file.write("WEBVTT\n\n")
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
self._file.write(f"{index}\n")
self._file.write(f"{start_str} --> {end_str}\n")
self._file.write(f"{text}\n\n")
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = seconds % 60
return f"{hours:02d}:{minutes:02d}:{secs:06.3f}".replace(".", ".")
class AssWriter(SubtitleWriter):
"""ASS subtitle writer with karaoke highlighting support."""
def __init__(self, path: Path, config: SubtitleConfig):
super().__init__(path, config)
self._is_centered = config.alignment in (SubtitleAlignment.CENTER, SubtitleAlignment.CENTER_NARROW)
self._is_narrow = config.alignment in (SubtitleAlignment.NARROW, SubtitleAlignment.CENTER_NARROW)
def _write_header(self) -> None:
margin = "90" if self._is_narrow else "10"
alignment = "5" if self._is_centered else "2"
self._file.write("[Script Info]\n")
self._file.write("Title: Generated by Abogen\n")
self._file.write("ScriptType: v4.00+\n\n")
# Styles
self._file.write("[V4+ Styles]\n")
self._file.write(
"Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, "
"OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, "
"ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, "
"Alignment, MarginL, MarginR, MarginV, Encoding\n"
)
if self.config.mode == SubtitleMode.SENTENCE_HIGHLIGHT:
# Karaoke style with highlighting
self._file.write(
f"Style: Default,Arial,24,&H00FFFFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n"
)
self._file.write(
f"Style: Highlight,Arial,24,&H0000FFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n\n"
)
else:
self._file.write(
f"Style: Default,Arial,24,&H00FFFFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n\n"
)
self._file.write("[Events]\n")
self._file.write(
"Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text\n"
)
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
style = "Default"
if self.config.mode == SubtitleMode.SENTENCE_HIGHLIGHT:
# Add karaoke tags for highlighting
text = self._add_karaoke_tags(text)
style = "Highlight"
alignment_tag = r"{\an5}" if self._is_centered else ""
self._file.write(
f"Dialogue: 0,{start_str},{end_str},{style},,0,0,0,,{alignment_tag}{text}\n"
)
def _add_karaoke_tags(self, text: str) -> str:
"""Add karaoke highlighting tags to text."""
# Simple word-level karaoke timing
words = text.split()
if not words:
return text
# This is a simplified version - real karaoke needs per-word timing
# For now, just return the text with the highlight color
return r"{\k100}" + r"{\k100}".join(words) + r"{\k0}"
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = seconds % 60
return f"{hours}:{minutes:02d}:{secs:05.2f}"
def create_subtitle_writer(
path: Path,
format: str,
mode: str,
alignment: str = "left",
max_words: int = 50,
) -> SubtitleWriter:
"""Factory function to create subtitle writer."""
fmt = SubtitleFormat(format.lower())
mode = SubtitleMode(mode)
align = SubtitleAlignment(alignment.lower())
config = SubtitleConfig(
format=fmt,
mode=mode,
alignment=align,
max_words=max_words,
)
if fmt == SubtitleFormat.SRT:
return SrtWriter(path, config)
elif fmt == SubtitleFormat.VTT:
return VttWriter(path, config)
elif fmt == SubtitleFormat.ASS:
return AssWriter(path, config)
else:
raise ValueError(f"Unsupported subtitle format: {format}")
__all__ = [
"SubtitleFormat",
"SubtitleMode",
"SubtitleAlignment",
"SubtitleConfig",
"SubtitleWriter",
"SrtWriter",
"VttWriter",
"AssWriter",
"create_subtitle_writer",
]
+3 -36
View File
@@ -2,9 +2,7 @@ from __future__ import annotations
import json import json
import logging import logging
import math
import mimetypes import mimetypes
import re
from contextlib import ExitStack from contextlib import ExitStack
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
@@ -12,6 +10,8 @@ from typing import Any, Dict, Iterable, List, Mapping, Optional, Sequence, Tuple
import httpx import httpx
from abogen.domain.metadata_helpers import normalize_series_sequence
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -641,40 +641,7 @@ class AudiobookshelfClient:
for key in preferred_keys: for key in preferred_keys:
if key not in metadata: if key not in metadata:
continue continue
normalized = AudiobookshelfClient._normalize_series_sequence(metadata.get(key)) normalized = normalize_series_sequence(metadata.get(key))
if normalized: if normalized:
return normalized return normalized
return "" return ""
@staticmethod
def _normalize_series_sequence(raw: Any) -> str:
if raw is None:
return ""
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return ""
text = str(raw)
else:
text = str(raw).strip()
if not text:
return ""
candidate = text.replace(",", ".")
match = re.search(r"\d+(?:\.\d+)?", candidate)
if not match:
return ""
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
+225 -786
View File
File diff suppressed because it is too large Load Diff
+3 -8
View File
@@ -7,6 +7,7 @@ import base64
import re import re
from abogen.pyqt.queue_manager_gui import QueueManager from abogen.pyqt.queue_manager_gui import QueueManager
from abogen.pyqt.queued_item import QueuedItem from abogen.pyqt.queued_item import QueuedItem
from abogen.domain.device import select_device as _select_device
import abogen.hf_tracker as hf_tracker import abogen.hf_tracker as hf_tracker
import hashlib # Added for cache path generation import hashlib # Added for cache path generation
from PyQt6.QtWidgets import ( from PyQt6.QtWidgets import (
@@ -2428,10 +2429,7 @@ class abogen(QWidget):
# Determine device based on GPU availability # Determine device based on GPU availability
if gpu_ok: if gpu_ok:
if platform.system() == "Darwin" and platform.processor() == "arm": device = _select_device()
device = "mps"
else:
device = "cuda"
else: else:
device = "cpu" device = "cpu"
@@ -2877,10 +2875,7 @@ class abogen(QWidget):
# Determine device based on GPU availability # Determine device based on GPU availability
if self.gpu_ok: if self.gpu_ok:
if platform.system() == "Darwin" and platform.processor() == "arm": device = _select_device()
device = "mps"
else:
device = "cuda"
else: else:
device = "cpu" device = "cpu"
+6
View File
@@ -84,6 +84,12 @@ def create_app(config: Optional[dict[str, Any]] = None) -> Flask:
"UPLOAD_FOLDER": str(uploads_dir), "UPLOAD_FOLDER": str(uploads_dir),
"OUTPUT_FOLDER": str(outputs_dir), "OUTPUT_FOLDER": str(outputs_dir),
"MAX_CONTENT_LENGTH": 1024 * 1024 * 400, # 400 MB uploads "MAX_CONTENT_LENGTH": 1024 * 1024 * 400, # 400 MB uploads
# Large books can submit four form fields per chapter. Werkzeug's
# defaults reject those requests before the wizard route can process
# them, even though the encoded payload is much smaller than the upload
# limit above.
"MAX_FORM_MEMORY_SIZE": 10 * 1024 * 1024,
"MAX_FORM_PARTS": 10_000,
} }
if config: if config:
base_config.update(config) base_config.update(config)
File diff suppressed because it is too large Load Diff
+3 -2
View File
@@ -14,7 +14,8 @@ from abogen.kokoro_text_normalization import normalize_for_pipeline
from abogen.normalization_settings import build_apostrophe_config from abogen.normalization_settings import build_apostrophe_config
from abogen.text_extractor import extract_from_path from abogen.text_extractor import extract_from_path
from abogen.voice_cache import ensure_voice_assets from abogen.voice_cache import ensure_voice_assets
from abogen.webui.conversion_runner import SAMPLE_RATE, SPLIT_PATTERN, _select_device, _to_float32, _resolve_voice, _spec_to_voice_ids from abogen.webui.conversion_runner import SAMPLE_RATE, _select_device, _to_float32, _resolve_voice, _spec_to_voice_ids
from abogen.domain.split_pattern import get_split_pattern
from abogen.tts_plugin.utils import create_pipeline from abogen.tts_plugin.utils import create_pipeline
@@ -200,7 +201,7 @@ def run_debug_tts_wavs(
normalized, normalized,
voice=voice_choice, voice=voice_choice,
speed=speed, speed=speed,
split_pattern=SPLIT_PATTERN, split_pattern=get_split_pattern(language, "Disabled"),
): ):
audio = _to_float32(getattr(segment, "audio", None)) audio = _to_float32(getattr(segment, "audio", None))
if audio.size: if audio.size:
+1 -1
View File
@@ -25,7 +25,7 @@ from abogen.voice_profiles import (
normalize_profile_entry, normalize_profile_entry,
) )
from abogen.webui.routes.utils.common import split_profile_spec from abogen.webui.routes.utils.common import split_profile_spec
from abogen.webui.routes.utils.preview import synthesize_preview, generate_preview_audio from abogen.webui.routes.utils.synthesize import synthesize_preview, generate_preview_audio
from abogen.webui.routes.utils.voice import formula_from_profile from abogen.webui.routes.utils.voice import formula_from_profile
from abogen.normalization_settings import ( from abogen.normalization_settings import (
build_llm_configuration, build_llm_configuration,
+2 -9
View File
@@ -19,6 +19,7 @@ from abogen.webui.routes.utils.settings import (
_NORMALIZATION_STRING_KEYS, _NORMALIZATION_STRING_KEYS,
_DEFAULT_ANALYSIS_THRESHOLD, _DEFAULT_ANALYSIS_THRESHOLD,
) )
from abogen.webui.routes.utils.common import extract_checkbox
from abogen.webui.routes.utils.voice import template_options from abogen.webui.routes.utils.voice import template_options
from abogen.webui.debug_tts_runner import run_debug_tts_wavs from abogen.webui.debug_tts_runner import run_debug_tts_wavs
from abogen.debug_tts_samples import DEBUG_TTS_SAMPLES from abogen.debug_tts_samples import DEBUG_TTS_SAMPLES
@@ -93,17 +94,9 @@ def update_settings() -> ResponseReturnValue:
maximum=25, maximum=25,
) )
def _extract_checkbox(name: str, default: bool) -> bool:
values = form.getlist(name) if hasattr(form, "getlist") else []
if values:
return coerce_bool(values[-1], default)
if hasattr(form, "__contains__") and name in form:
return False
return default
# Normalization settings # Normalization settings
for key in _NORMALIZATION_BOOLEAN_KEYS: for key in _NORMALIZATION_BOOLEAN_KEYS:
current[key] = _extract_checkbox(key, bool(current.get(key, True))) current[key] = extract_checkbox(form, key, bool(current.get(key, True)))
for key in _NORMALIZATION_STRING_KEYS: for key in _NORMALIZATION_STRING_KEYS:
if hasattr(form, "__contains__") and key in form: if hasattr(form, "__contains__") and key in form:
current[key] = (form.get(key) or "").strip() current[key] = (form.get(key) or "").strip()
+36 -1
View File
@@ -1,6 +1,17 @@
from typing import Any, Optional, Tuple, Iterable, List from typing import Any, Optional, Tuple, Iterable, List, Mapping
from pathlib import Path from pathlib import Path
def coerce_bool(value: Any, default: bool) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() in {"true", "1", "yes", "on"}
if value is None:
return default
return bool(value)
def split_profile_spec(value: Any) -> Tuple[str, Optional[str]]: def split_profile_spec(value: Any) -> Tuple[str, Optional[str]]:
text = str(value or "").strip() text = str(value or "").strip()
if not text: if not text:
@@ -18,7 +29,31 @@ def split_speaker_spec(value: Any) -> Tuple[str, Optional[str]]:
return split_profile_spec(value) return split_profile_spec(value)
def existing_paths(paths: Optional[Iterable[Path]]) -> List[Path]: def existing_paths(paths: Optional[Iterable[Path]]) -> List[Path]:
if not paths: if not paths:
return [] return []
return [p for p in paths if p.exists()] return [p for p in paths if p.exists()]
def extract_checkbox(form: Mapping[str, Any], name: str, default: bool) -> bool:
"""Extract a boolean checkbox value from a form-like mapping.
Handles both multi-value forms (Flask's `getlist`) and simple mappings.
If the checkbox name is present but has no value, it means unchecked (False).
"""
values: List[str] = []
getter = getattr(form, "getlist", None)
if callable(getter):
raw_values = getter(name)
if raw_values:
values = list(raw_values)
else:
raw_flag = form.get(name)
if raw_flag is not None:
values = [raw_flag]
if values:
return coerce_bool(values[-1], default)
if name in form:
return False
return default
+8 -139
View File
@@ -1,10 +1,14 @@
import re
import time import time
import uuid import uuid
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
from flask import request, render_template, jsonify from flask import request, render_template, jsonify
from flask.typing import ResponseReturnValue from flask.typing import ResponseReturnValue
from abogen.domain.chapter_classification import (
supplement_score,
should_preselect_chapter,
ensure_at_least_one_chapter_enabled,
)
from abogen.webui.service import PendingJob, JobStatus from abogen.webui.service import PendingJob, JobStatus
from abogen.webui.routes.utils.service import get_service from abogen.webui.routes.utils.service import get_service
from abogen.tts_plugin.utils import is_plugin_registered from abogen.tts_plugin.utils import is_plugin_registered
@@ -29,7 +33,7 @@ from abogen.webui.routes.utils.voice import (
) )
from abogen.webui.routes.utils.entity import sync_pronunciation_overrides from abogen.webui.routes.utils.entity import sync_pronunciation_overrides
from abogen.webui.routes.utils.epub import job_download_flags from abogen.webui.routes.utils.epub import job_download_flags
from abogen.webui.routes.utils.common import split_profile_spec from abogen.webui.routes.utils.common import split_profile_spec, extract_checkbox
from abogen.utils import calculate_text_length from abogen.utils import calculate_text_length
from abogen.voice_profiles import serialize_profiles, normalize_profile_entry from abogen.voice_profiles import serialize_profiles, normalize_profile_entry
from abogen.chunking import ChunkLevel, build_chunks_for_chapters from abogen.chunking import ChunkLevel, build_chunks_for_chapters
@@ -66,109 +70,6 @@ _WIZARD_STEP_META = {
}, },
} }
_SUPPLEMENT_TITLE_PATTERNS: List[tuple[re.Pattern[str], float]] = [
(re.compile(r"\btitle\s+page\b"), 3.0),
(re.compile(r"\bcopyright\b"), 2.4),
(re.compile(r"\btable\s+of\s+contents\b"), 2.8),
(re.compile(r"\bcontents\b"), 2.0),
(re.compile(r"\backnowledg(e)?ments?\b"), 2.0),
(re.compile(r"\bdedication\b"), 2.0),
(re.compile(r"\babout\s+the\s+author(s)?\b"), 2.4),
(re.compile(r"\balso\s+by\b"), 2.0),
(re.compile(r"\bpraise\s+for\b"), 2.0),
(re.compile(r"\bcolophon\b"), 2.2),
(re.compile(r"\bpublication\s+data\b"), 2.2),
(re.compile(r"\btranscriber'?s?\s+note\b"), 2.2),
(re.compile(r"\bglossary\b"), 2.0),
(re.compile(r"\bindex\b"), 2.0),
(re.compile(r"\bbibliograph(y|ies)\b"), 2.0),
(re.compile(r"\breferences\b"), 1.8),
(re.compile(r"\bappendix\b"), 1.9),
]
_CONTENT_TITLE_PATTERNS: List[re.Pattern[str]] = [
re.compile(r"\bchapter\b"),
re.compile(r"\bbook\b"),
re.compile(r"\bpart\b"),
re.compile(r"\bsection\b"),
re.compile(r"\bscene\b"),
re.compile(r"\bprologue\b"),
re.compile(r"\bepilogue\b"),
re.compile(r"\bintroduction\b"),
re.compile(r"\bstory\b"),
]
_SUPPLEMENT_TEXT_KEYWORDS: List[tuple[str, float]] = [
("copyright", 1.2),
("all rights reserved", 1.1),
("isbn", 0.9),
("library of congress", 1.0),
("table of contents", 1.0),
("dedicated to", 0.8),
("acknowledg", 0.8),
("printed in", 0.6),
("permission", 0.6),
("publisher", 0.5),
("praise for", 0.9),
("also by", 0.9),
("glossary", 0.8),
("index", 0.8),
("newsletter", 3.2),
("mailing list", 2.6),
("sign-up", 2.2),
]
def supplement_score(title: str, text: str, index: int) -> float:
normalized_title = (title or "").lower()
score = 0.0
for pattern, weight in _SUPPLEMENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score += weight
for pattern in _CONTENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score -= 2.0
stripped_text = (text or "").strip()
length = len(stripped_text)
if length <= 150:
score += 0.9
elif length <= 400:
score += 0.6
elif length <= 800:
score += 0.35
lowercase_text = stripped_text.lower()
for keyword, weight in _SUPPLEMENT_TEXT_KEYWORDS:
if keyword in lowercase_text:
score += weight
if index == 0 and score > 0:
score += 0.25
return score
def should_preselect_chapter(
title: str,
text: str,
index: int,
total_count: int,
) -> bool:
if total_count <= 1:
return True
score = supplement_score(title, text, index)
return score < 1.9
def ensure_at_least_one_chapter_enabled(chapters: List[Dict[str, Any]]) -> None:
if not chapters:
return
if any(chapter.get("enabled") for chapter in chapters):
return
best_index = max(range(len(chapters)), key=lambda idx: chapters[idx].get("characters", 0))
chapters[best_index]["enabled"] = True
def apply_prepare_form( def apply_prepare_form(
pending: PendingJob, form: Mapping[str, Any] pending: PendingJob, form: Mapping[str, Any]
@@ -537,28 +438,11 @@ def apply_book_step_form(
else: else:
pending.normalize_chapter_opening_caps = caps_default pending.normalize_chapter_opening_caps = caps_default
def _extract_checkbox(name: str, default: bool) -> bool:
values: List[str] = []
getter = getattr(form, "getlist", None)
if callable(getter):
raw_values = getter(name)
if raw_values:
values = list(cast(Iterable[str], raw_values))
else:
raw_flag = form.get(name)
if raw_flag is not None:
values = [raw_flag]
if values:
return coerce_bool(values[-1], default)
if hasattr(form, "__contains__") and name in form:
return False
return default
overrides_existing = getattr(pending, "normalization_overrides", None) overrides_existing = getattr(pending, "normalization_overrides", None)
overrides: Dict[str, Any] = dict(overrides_existing or {}) overrides: Dict[str, Any] = dict(overrides_existing or {})
for key in _NORMALIZATION_BOOLEAN_KEYS: for key in _NORMALIZATION_BOOLEAN_KEYS:
default_toggle = overrides.get(key, bool(settings.get(key, True))) default_toggle = overrides.get(key, bool(settings.get(key, True)))
overrides[key] = _extract_checkbox(key, default_toggle) overrides[key] = extract_checkbox(form, key, default_toggle)
for key in _NORMALIZATION_STRING_KEYS: for key in _NORMALIZATION_STRING_KEYS:
default_val = overrides.get(key, str(settings.get(key, ""))) default_val = overrides.get(key, str(settings.get(key, "")))
val = form.get(key) val = form.get(key)
@@ -886,25 +770,10 @@ def build_pending_job_from_extraction(
apply_config=bool(speaker_config_payload), apply_config=bool(speaker_config_payload),
) )
def _extract_checkbox(name: str, default: bool) -> bool:
values: List[str] = []
getter = getattr(form, "getlist", None)
if callable(getter):
raw_values = getter(name)
if raw_values:
values = list(cast(Iterable[str], raw_values))
else:
raw_flag = form.get(name)
if raw_flag is not None:
values = [raw_flag]
if values:
return coerce_bool(values[-1], default)
return default
normalization_overrides = {} normalization_overrides = {}
for key in _NORMALIZATION_BOOLEAN_KEYS: for key in _NORMALIZATION_BOOLEAN_KEYS:
default_val = bool(settings.get(key, True)) default_val = bool(settings.get(key, True))
normalization_overrides[key] = _extract_checkbox(key, default_val) normalization_overrides[key] = extract_checkbox(form, key, default_val)
for key in _NORMALIZATION_STRING_KEYS: for key in _NORMALIZATION_STRING_KEYS:
default_val = str(settings.get(key, "")) default_val = str(settings.get(key, ""))
+1 -11
View File
@@ -15,7 +15,7 @@ from abogen.normalization_settings import (
from abogen.utils import load_config, save_config from abogen.utils import load_config, save_config
from abogen.integrations.calibre_opds import CalibreOPDSClient from abogen.integrations.calibre_opds import CalibreOPDSClient
from abogen.integrations.audiobookshelf import AudiobookshelfConfig from abogen.integrations.audiobookshelf import AudiobookshelfConfig
from abogen.webui.routes.utils.common import split_profile_spec from abogen.webui.routes.utils.common import split_profile_spec, coerce_bool
SAVE_MODE_LABELS = { SAVE_MODE_LABELS = {
"save_next_to_input": "Save next to input file", "save_next_to_input": "Save next to input file",
@@ -245,16 +245,6 @@ def render_prompt_template(template: str, context: Mapping[str, str]) -> str:
return _PROMPT_TOKEN_RE.sub(_replace, template) return _PROMPT_TOKEN_RE.sub(_replace, template)
def coerce_bool(value: Any, default: bool) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() in {"true", "1", "yes", "on"}
if value is None:
return default
return bool(value)
def coerce_float(value: Any, default: float) -> float: def coerce_float(value: Any, default: float) -> float:
try: try:
return max(0.0, float(value)) return max(0.0, float(value))
@@ -6,8 +6,10 @@ import soundfile as sf
from flask import current_app, send_file from flask import current_app, send_file
from flask.typing import ResponseReturnValue from flask.typing import ResponseReturnValue
from abogen.domain.device import select_device as _select_device
from abogen.domain.split_pattern import get_split_pattern
SPLIT_PATTERN = r"\n+"
SAMPLE_RATE = 24000 SAMPLE_RATE = 24000
_preview_pipelines: Dict[Tuple[str, str], Any] = {} _preview_pipelines: Dict[Tuple[str, str], Any] = {}
@@ -25,31 +27,6 @@ def clear_preview_pipelines() -> None:
_preview_pipelines.clear() _preview_pipelines.clear()
def _select_device() -> str:
import platform
try:
import torch # type: ignore[import-not-found]
except Exception:
return "cpu"
system = platform.system()
if system == "Darwin" and platform.processor() == "arm":
try:
if torch.backends.mps.is_available():
return "mps"
except Exception:
pass
return "cpu"
try:
if torch.cuda.is_available():
return "cuda"
except Exception:
pass
return "cpu"
def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]: def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]:
devices: List[str] = ["cpu"] devices: List[str] = ["cpu"]
if use_gpu: if use_gpu:
@@ -67,22 +44,6 @@ def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]:
raise RuntimeError("Preview pipeline is unavailable") from last_error raise RuntimeError("Preview pipeline is unavailable") from last_error
def _to_float32(audio_segment) -> np.ndarray:
if audio_segment is None:
return np.zeros(0, dtype="float32")
tensor = audio_segment
if hasattr(tensor, "detach"):
tensor = tensor.detach()
if hasattr(tensor, "cpu"):
try:
tensor = tensor.cpu()
except Exception:
pass
if hasattr(tensor, "numpy"):
return np.asarray(tensor.numpy(), dtype="float32").reshape(-1)
return np.asarray(tensor, dtype="float32").reshape(-1)
def get_preview_pipeline(language: str, device: str) -> Any: def get_preview_pipeline(language: str, device: str) -> Any:
key = (language, device) key = (language, device)
with _preview_pipeline_lock: with _preview_pipeline_lock:
@@ -146,6 +107,8 @@ def generate_preview_audio(
current_app.logger.exception("Preview normalization failed; using raw text") current_app.logger.exception("Preview normalization failed; using raw text")
normalized_text = source_text normalized_text = source_text
preview_split = get_split_pattern(str(language or "a"), "Disabled")
if provider == "supertonic": if provider == "supertonic":
from abogen.tts_plugin.utils import create_pipeline from abogen.tts_plugin.utils import create_pipeline
@@ -154,7 +117,7 @@ def generate_preview_audio(
normalized_text, normalized_text,
voice=voice_spec, voice=voice_spec,
speed=speed, speed=speed,
split_pattern=SPLIT_PATTERN, split_pattern=preview_split,
total_steps=supertonic_total_steps, total_steps=supertonic_total_steps,
) )
else: else:
@@ -172,7 +135,7 @@ def generate_preview_audio(
normalized_text, normalized_text,
voice=voice_choice, voice=voice_choice,
speed=speed, speed=speed,
split_pattern=SPLIT_PATTERN, split_pattern=preview_split,
) )
audio_chunks: List[np.ndarray] = [] audio_chunks: List[np.ndarray] = []
+1 -80
View File
@@ -1,6 +1,4 @@
import threading
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
import numpy as np
from abogen.speaker_configs import slugify_label from abogen.speaker_configs import slugify_label
from abogen.speaker_analysis import analyze_speakers from abogen.speaker_analysis import analyze_speakers
@@ -10,7 +8,7 @@ from abogen.voice_profiles import (
load_profiles, load_profiles,
serialize_profiles, serialize_profiles,
) )
from abogen.voice_formulas import get_new_voice, parse_formula_terms from abogen.voice_formulas import parse_formula_terms
from abogen.constants import ( from abogen.constants import (
LANGUAGE_DESCRIPTIONS, LANGUAGE_DESCRIPTIONS,
SUBTITLE_FORMATS, SUBTITLE_FORMATS,
@@ -20,11 +18,7 @@ from abogen.constants import (
) )
from abogen.tts_plugin.utils import get_voices from abogen.tts_plugin.utils import get_voices
from abogen.speaker_configs import list_configs from abogen.speaker_configs import list_configs
from abogen.tts_plugin.utils import create_pipeline
from abogen.webui.conversion_runner import _select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN
_preview_pipeline_lock = threading.RLock()
_preview_pipelines: Dict[Tuple[str, str], Any] = {}
def build_narrator_roster( def build_narrator_roster(
voice: str, voice: str,
@@ -733,76 +727,3 @@ def pairs_to_formula(pairs: Iterable[Tuple[str, float]]) -> Optional[str]:
def profiles_payload() -> Dict[str, Any]: def profiles_payload() -> Dict[str, Any]:
return {"profiles": serialize_profiles()} return {"profiles": serialize_profiles()}
def get_preview_pipeline(language: str, device: str):
key = (language, device)
with _preview_pipeline_lock:
pipeline = _preview_pipelines.get(key)
if pipeline is not None:
return pipeline
pipeline = create_pipeline("kokoro", lang_code=language, device=device)
_preview_pipelines[key] = pipeline
return pipeline
def synthesize_audio_from_normalized(
*,
normalized_text: str,
voice_spec: str,
language: str,
speed: float,
use_gpu: bool,
max_seconds: float,
) -> np.ndarray:
if not normalized_text.strip():
raise ValueError("Preview text is required")
device = "cpu"
if use_gpu:
try:
device = _select_device()
except Exception:
device = "cpu"
use_gpu = False
pipeline = get_preview_pipeline(language, device)
if pipeline is None:
raise RuntimeError("Preview pipeline is unavailable")
voice_choice: Any = voice_spec
if voice_spec and "*" in voice_spec:
voice_choice = get_new_voice(pipeline, voice_spec, use_gpu)
segments = pipeline(
normalized_text,
voice=voice_choice,
speed=speed,
split_pattern=SPLIT_PATTERN,
)
audio_chunks: List[np.ndarray] = []
accumulated = 0
max_samples = int(max(1.0, max_seconds) * SAMPLE_RATE)
for segment in segments:
graphemes = getattr(segment, "graphemes", "").strip()
if not graphemes:
continue
audio = _to_float32(getattr(segment, "audio", None))
if audio.size == 0:
continue
remaining = max_samples - accumulated
if remaining <= 0:
break
if audio.shape[0] > remaining:
audio = audio[:remaining]
audio_chunks.append(audio)
accumulated += audio.shape[0]
if accumulated >= max_samples:
break
if not audio_chunks:
raise RuntimeError("Preview could not be generated")
return np.concatenate(audio_chunks)
+1 -1
View File
@@ -9,7 +9,7 @@ from abogen.webui.routes.utils.voice import (
parse_voice_formula, parse_voice_formula,
) )
from abogen.webui.routes.utils.settings import load_settings, coerce_bool from abogen.webui.routes.utils.settings import load_settings, coerce_bool
from abogen.webui.routes.utils.preview import synthesize_preview from abogen.webui.routes.utils.synthesize import synthesize_preview
from abogen.speaker_configs import ( from abogen.speaker_configs import (
list_configs, list_configs,
get_config, get_config,
+31 -264
View File
@@ -2,9 +2,7 @@ from __future__ import annotations
import json import json
import logging import logging
import math
import os import os
import re
import shutil import shutil
import sys import sys
import threading import threading
@@ -14,7 +12,7 @@ import traceback
from dataclasses import dataclass, field from dataclasses import dataclass, field
from enum import Enum from enum import Enum
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Dict, Iterable, List, Optional, Mapping, Tuple from typing import Any, Callable, Dict, Iterable, List, Optional, Mapping
from abogen.utils import get_internal_cache_path, get_user_settings_dir, load_config from abogen.utils import get_internal_cache_path, get_user_settings_dir, load_config
from abogen.voice_cache import bootstrap_voice_cache from abogen.voice_cache import bootstrap_voice_cache
@@ -23,6 +21,17 @@ from abogen.integrations.audiobookshelf import (
AudiobookshelfConfig, AudiobookshelfConfig,
AudiobookshelfUploadError, AudiobookshelfUploadError,
) )
from abogen.domain.metadata_helpers import (
normalize_metadata_casefold as _normalize_metadata_casefold,
split_people_field as _split_people_field,
split_simple_list as _split_simple_list,
first_nonempty as _first_nonempty,
extract_year as _extract_year,
normalize_series_sequence as _normalize_series_sequence,
build_audiobookshelf_metadata as _build_abs_metadata,
load_audiobookshelf_chapters as _load_abs_chapters,
_SERIES_SEQUENCE_TAG_KEYS,
)
def _create_set_event() -> threading.Event: def _create_set_event() -> threading.Event:
@@ -53,9 +62,6 @@ _JOB_LEVEL_MAP: Dict[str, int] = {
} }
_PEOPLE_SPLIT_RE = re.compile(r"[;,/&]|\band\b", re.IGNORECASE)
def _emit_job_log(job_id: str, level: str, message: str) -> None: def _emit_job_log(job_id: str, level: str, message: str) -> None:
normalized = (level or "info").lower() normalized = (level or "info").lower()
log_level = _JOB_LEVEL_MAP.get(normalized, logging.INFO) log_level = _JOB_LEVEL_MAP.get(normalized, logging.INFO)
@@ -131,6 +137,7 @@ class Job:
progress: float = 0.0 progress: float = 0.0
total_characters: int = 0 total_characters: int = 0
processed_characters: int = 0 processed_characters: int = 0
etr_str: str = ""
logs: List[JobLog] = field(default_factory=list) logs: List[JobLog] = field(default_factory=list)
error: Optional[str] = None error: Optional[str] = None
result: JobResult = field(default_factory=JobResult) result: JobResult = field(default_factory=JobResult)
@@ -162,20 +169,25 @@ class Job:
@property @property
def estimated_time_remaining(self) -> Optional[float]: def estimated_time_remaining(self) -> Optional[float]:
""" """
Returns the estimated seconds remaining based on current progress and elapsed time. Returns the estimated seconds remaining.
Returns None if the job hasn't started, is finished, or progress is 0. Uses the same calc_etr_str from domain/progress.py as the PyQt desktop GUI.
""" """
if self.status != JobStatus.RUNNING or not self.started_at or self.progress <= 0: from abogen.domain.progress import calc_etr_str
if self.status != JobStatus.RUNNING or not self.started_at or self.total_characters <= 0:
return None return None
elapsed = time.time() - self.started_at elapsed = time.time() - self.started_at
if elapsed <= 0: if elapsed <= 0:
return None return None
# Estimate total time based on current progress etr = calc_etr_str(elapsed, self.processed_characters, self.total_characters)
total_estimated = elapsed / self.progress if etr == "Processing...":
remaining = total_estimated - elapsed return None
return max(0.0, remaining)
# Parse "HH:MM:SS" back to seconds for backward compatibility
parts = etr.split(":")
return int(parts[0]) * 3600 + int(parts[1]) * 60 + int(parts[2])
def add_log(self, message: str, level: str = "info") -> None: def add_log(self, message: str, level: str = "info") -> None:
entry = JobLog(timestamp=time.time(), message=message, level=level) entry = JobLog(timestamp=time.time(), message=message, level=level)
@@ -194,6 +206,7 @@ class Job:
"progress": self.progress, "progress": self.progress,
"total_characters": self.total_characters, "total_characters": self.total_characters,
"processed_characters": self.processed_characters, "processed_characters": self.processed_characters,
"etr_str": self.etr_str,
"error": self.error, "error": self.error,
"logs": [log.__dict__ for log in self.logs], "logs": [log.__dict__ for log in self.logs],
"result": { "result": {
@@ -252,234 +265,13 @@ class Job:
} }
def _normalize_metadata_casefold(values: Optional[Mapping[str, Any]]) -> Dict[str, Any]:
normalized: Dict[str, Any] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
key_text = str(key).strip().lower()
if not key_text:
continue
if isinstance(value, (list, tuple, set)):
normalized[key_text] = value
else:
text = str(value).strip()
if text:
normalized[key_text] = text
return normalized
def _split_people_field(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(_split_people_field(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _PEOPLE_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
_LIST_SPLIT_RE = re.compile(r"[;,\n]")
_SERIES_SEQUENCE_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?")
_SERIES_SEQUENCE_TAG_KEYS: Tuple[str, ...] = (
"series_index",
"series_position",
"series_sequence",
"series_number",
"seriesnumber",
"book_number",
"booknumber",
)
def _split_simple_list(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(_split_simple_list(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _LIST_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def _first_nonempty(*values: Any) -> Optional[str]:
for value in values:
if value is None:
continue
if isinstance(value, (list, tuple, set)):
items = list(value)
if not items:
continue
value = items[0]
text = str(value).strip()
if text:
return text
return None
def _extract_year(raw: Optional[str]) -> Optional[int]:
if not raw:
return None
text = str(raw).strip()
if not text:
return None
match = re.search(r"(19|20)\d{2}", text)
if match:
try:
return int(match.group(0))
except ValueError:
return None
try:
parsed = int(text)
except ValueError:
return None
if 0 < parsed < 3000:
return parsed
return None
def build_audiobookshelf_metadata(job: Job) -> Dict[str, Any]: def build_audiobookshelf_metadata(job: Job) -> Dict[str, Any]:
tags = _normalize_metadata_casefold(job.metadata_tags)
filename = Path(job.original_filename or "").stem or job.original_filename or "Audiobook" filename = Path(job.original_filename or "").stem or job.original_filename or "Audiobook"
title = _first_nonempty( return _build_abs_metadata(
tags.get("title"), job.metadata_tags,
tags.get("book_title"), language=job.language or "",
tags.get("name"), filename=filename,
tags.get("album"),
filename,
) )
authors = _split_people_field(
tags.get("authors")
or tags.get("author")
or tags.get("album_artist")
or tags.get("artist")
)
narrators = _split_people_field(tags.get("narrators") or tags.get("narrator"))
description = _first_nonempty(tags.get("description"), tags.get("summary"), tags.get("comment"))
genres = _split_simple_list(tags.get("genre"))
keywords = _split_simple_list(tags.get("tags") or tags.get("keywords"))
language = _first_nonempty(tags.get("language"), tags.get("lang")) or job.language or ""
series_name = _first_nonempty(
tags.get("series"),
tags.get("series_name"),
tags.get("seriesname"),
tags.get("series_title"),
tags.get("seriestitle"),
)
series_sequence = None
for key in _SERIES_SEQUENCE_TAG_KEYS:
raw_value = tags.get(key)
normalized_sequence = _normalize_series_sequence(raw_value)
if normalized_sequence:
series_sequence = normalized_sequence
break
if not series_name:
series_sequence = None
data: Dict[str, Any] = {
"title": title,
"subtitle": tags.get("subtitle"),
"authors": authors,
"narrators": narrators,
"description": description,
"publisher": tags.get("publisher"),
"genres": genres,
"tags": keywords,
"language": language,
"publishedYear": _extract_year(tags.get("published") or tags.get("publication_year") or tags.get("date") or tags.get("year")),
"seriesName": series_name,
"seriesSequence": series_sequence,
"isbn": _first_nonempty(tags.get("isbn"), tags.get("asin")),
}
published_date = _first_nonempty(tags.get("published"), tags.get("publication_date"), tags.get("date"))
if published_date:
data["publishedDate"] = published_date
rating_text = _first_nonempty(tags.get("rating"), tags.get("my_rating"))
if rating_text:
try:
data["rating"] = float(str(rating_text).strip())
except ValueError:
pass
rating_max_text = _first_nonempty(tags.get("rating_max"), tags.get("rating_scale"))
if rating_max_text:
try:
data["ratingMax"] = float(str(rating_max_text).strip())
except ValueError:
pass
# Remove empty values
cleaned: Dict[str, Any] = {}
for key, value in data.items():
if value is None:
continue
if isinstance(value, str) and not value.strip():
continue
if isinstance(value, (list, tuple)) and not value:
continue
cleaned[key] = value
return cleaned
def _normalize_series_sequence(raw: Any) -> Optional[str]:
if raw is None:
return None
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return None
text = str(raw)
else:
text = str(raw).strip()
if not text:
return None
candidate = text.replace(",", ".")
match = _SERIES_SEQUENCE_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
def load_audiobookshelf_chapters(job: Job) -> Optional[List[Dict[str, Any]]]: def load_audiobookshelf_chapters(job: Job) -> Optional[List[Dict[str, Any]]]:
@@ -487,32 +279,7 @@ def load_audiobookshelf_chapters(job: Job) -> Optional[List[Dict[str, Any]]]:
if not metadata_ref: if not metadata_ref:
return None return None
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref)) metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
if not metadata_path.exists(): return _load_abs_chapters(metadata_path)
return None
try:
payload = json.loads(metadata_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
chapters = payload.get("chapters")
if not isinstance(chapters, list):
return None
cleaned: List[Dict[str, Any]] = []
for entry in chapters:
if not isinstance(entry, Mapping):
continue
title = _first_nonempty(entry.get("title"), entry.get("original_title"))
start = entry.get("start")
end = entry.get("end")
if title is None or not isinstance(start, (int, float)):
continue
chapter_payload: Dict[str, Any] = {
"title": title,
"start": float(start),
}
if isinstance(end, (int, float)):
chapter_payload["end"] = float(end)
cleaned.append(chapter_payload)
return cleaned or None
def _existing_paths(paths: Iterable[Any]) -> List[Path]: def _existing_paths(paths: Iterable[Any]) -> List[Path]:
+2 -2
View File
@@ -28,8 +28,8 @@
</div> </div>
<div class="job-card__progress-meta"> <div class="job-card__progress-meta">
<small>{{ progress_value }}% · {{ job.processed_characters }} / {{ job.total_characters or '—' }}</small> <small>{{ progress_value }}% · {{ job.processed_characters }} / {{ job.total_characters or '—' }}</small>
{% if job.estimated_time_remaining %} {% if job.etr_str and job.etr_str != 'Processing...' %}
<small class="job-card__eta">~{{ job.estimated_time_remaining | durationformat }} remaining</small> <small class="job-card__eta">~{{ job.etr_str }} remaining</small>
{% endif %} {% endif %}
</div> </div>
</div> </div>
+1 -1
View File
@@ -44,7 +44,7 @@ dependencies = [
"python-dotenv>=1.0.1", "python-dotenv>=1.0.1",
"static_ffmpeg>=2.13", "static_ffmpeg>=2.13",
"Markdown>=3.9", "Markdown>=3.9",
"Flask>=3.0.3", "Flask>=3.1.0",
"numpy>=1.24.0", "numpy>=1.24.0",
"gpustat>=1.1.1", "gpustat>=1.1.1",
"num2words>=0.5.13", "num2words>=0.5.13",
+18
View File
@@ -12,6 +12,7 @@ from __future__ import annotations
import logging import logging
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
from unittest.mock import patch
import pytest import pytest
@@ -147,6 +148,23 @@ def engine_config() -> Any:
return EngineConfig(device="cpu") return EngineConfig(device="cpu")
@pytest.fixture(autouse=True)
def _mock_kokoro_pipeline():
"""Prevent real KPipeline initialization during generic plugin tests.
The real KPipeline requires spacy model downloads which aren't available
in externally-managed environments. Mock spacy's download and load so
the engine contract can be tested without heavy dependencies.
"""
from unittest.mock import MagicMock
mock_nlp = MagicMock()
with patch("spacy.cli.download"), \
patch("spacy.load", return_value=mock_nlp):
yield
@pytest.fixture @pytest.fixture
def create_engine(loaded_plugin, host_context, engine_config): def create_engine(loaded_plugin, host_context, engine_config):
"""Create an engine instance from a loaded plugin. """Create an engine instance from a loaded plugin.
+307
View File
@@ -0,0 +1,307 @@
"""Tests for abogen.domain.audio_buffer module."""
import numpy as np
import pytest
from abogen.domain.audio_buffer import (
create_silence,
mix_audio,
normalize_audio,
ensure_buffer_size,
concatenate_audio,
audio_duration,
samples_for_duration,
SAMPLE_RATE,
)
class TestCreateSilence:
"""Tests for create_silence function."""
def test_positive_duration(self):
"""Test creating silence with positive duration."""
duration = 1.0 # 1 second
silence = create_silence(duration)
expected_samples = int(round(duration * SAMPLE_RATE))
assert len(silence) == expected_samples
assert silence.dtype == np.float32
assert np.all(silence == 0)
def test_zero_duration(self):
"""Test creating silence with zero duration returns empty array."""
silence = create_silence(0)
assert len(silence) == 0
assert silence.dtype == np.float32
def test_negative_duration(self):
"""Test creating silence with negative duration returns empty array."""
silence = create_silence(-1.0)
assert len(silence) == 0
assert silence.dtype == np.float32
def test_very_small_duration(self):
"""Test creating silence with very small duration."""
duration = 0.001 # 1 millisecond
silence = create_silence(duration)
# Should round to at least 1 sample or 0
assert len(silence) >= 0
assert silence.dtype == np.float32
def test_half_second(self):
"""Test creating 0.5 second of silence."""
silence = create_silence(0.5)
expected_samples = int(round(0.5 * SAMPLE_RATE))
assert len(silence) == expected_samples
class TestMixAudio:
"""Tests for mix_audio function."""
def test_basic_mix(self):
"""Test basic audio mixing."""
target = np.ones(100, dtype="float32")
source = np.ones(50, dtype="float32") * 2
mix_audio(target, source, start_sample=25)
# First 25 samples should remain 1.0
assert np.all(target[:25] == 1.0)
# Middle 50 samples should be 1.0 + 2.0 = 3.0
assert np.all(target[25:75] == 3.0)
# Last 25 samples should remain 1.0
assert np.all(target[75:] == 1.0)
def test_empty_source(self):
"""Test mixing empty source does nothing."""
target = np.ones(100, dtype="float32")
original = target.copy()
mix_audio(target, np.array([], dtype="float32"), start_sample=50)
assert np.array_equal(target, original)
def test_extend_target_buffer(self):
"""Test that target buffer is extended when needed."""
target = np.ones(100, dtype="float32")
source = np.ones(50, dtype="float32") * 2
# This should extend target to 170 samples (120 + 50)
target = mix_audio(target, source, start_sample=120)
assert len(target) == 170
# Check that source was mixed correctly
assert np.all(target[120:170] == 2.0)
def test_start_at_zero(self):
"""Test mixing starting at sample 0."""
target = np.zeros(100, dtype="float32")
source = np.ones(50, dtype="float32")
mix_audio(target, source, start_sample=0)
assert np.all(target[:50] == 1.0)
assert np.all(target[50:] == 0.0)
def test_explicit_end_sample(self):
"""Test mixing with explicit end_sample."""
target = np.zeros(100, dtype="float32")
source = np.ones(50, dtype="float32")
mix_audio(target, source, start_sample=10, end_sample=60)
# Only first 10 samples of source should be mixed (60-10=50, but source is only 50)
# Actually, end_sample overrides the length
assert target[10] == 1.0
class TestNormalizeAudio:
"""Tests for normalize_audio function."""
def test_no_normalization_needed(self):
"""Test audio within range is not modified."""
audio = np.ones(100, dtype="float32") * 0.5
result = normalize_audio(audio)
assert not np.shares_memory(audio, result) # Should be a copy
assert np.array_equal(result, audio)
def test_normalization_applied(self):
"""Test audio above target peak is scaled down."""
audio = np.ones(100, dtype="float32") * 2.0
result = normalize_audio(audio)
assert np.all(result <= 1.0)
assert np.isclose(result[0], 1.0)
def test_empty_audio(self):
"""Test normalizing empty audio returns empty copy."""
audio = np.array([], dtype="float32")
result = normalize_audio(audio)
assert len(result) == 0
assert result.dtype == np.float32
def test_custom_target_peak(self):
"""Test normalization with custom target peak."""
audio = np.ones(100, dtype="float32") * 4.0
result = normalize_audio(audio, target_peak=2.0)
assert np.all(result <= 2.0)
assert np.isclose(result[0], 2.0)
def test_negative_peak(self):
"""Test normalization handles negative peaks."""
audio = np.ones(100, dtype="float32") * -2.0
result = normalize_audio(audio)
assert np.all(result >= -1.0)
assert np.isclose(result[0], -1.0)
def test_mixed_positive_negative(self):
"""Test normalization with both positive and negative peaks."""
audio = np.array([-3.0, 2.0, -1.0, 4.0], dtype="float32")
result = normalize_audio(audio)
# Should scale by 1/4 (max absolute value is 4)
assert np.isclose(result[0], -0.75)
assert np.isclose(result[1], 0.5)
assert np.isclose(result[3], 1.0)
class TestEnsureBufferSize:
"""Tests for ensure_buffer_size function."""
def test_buffer_already_large_enough(self):
"""Test buffer that is already large enough is unchanged."""
buffer = np.ones(100, dtype="float32")
result = ensure_buffer_size(buffer, 50)
assert np.array_equal(result, buffer)
def test_buffer_needs_extension(self):
"""Test buffer is extended with zeros when too small."""
buffer = np.ones(50, dtype="float32")
result = ensure_buffer_size(buffer, 100)
assert len(result) == 100
assert np.all(result[:50] == 1.0)
assert np.all(result[50:] == 0.0)
def test_exact_size(self):
"""Test buffer of exact size is unchanged."""
buffer = np.ones(100, dtype="float32")
result = ensure_buffer_size(buffer, 100)
assert len(result) == 100
assert np.array_equal(result, buffer)
class TestConcatenateAudio:
"""Tests for concatenate_audio function."""
def test_concatenate_two_buffers(self):
"""Test concatenating two audio buffers."""
a = np.ones(50, dtype="float32")
b = np.ones(50, dtype="float32") * 2
result = concatenate_audio(a, b)
assert len(result) == 100
assert np.all(result[:50] == 1.0)
assert np.all(result[50:] == 2.0)
def test_concatenate_multiple_buffers(self):
"""Test concatenating multiple audio buffers."""
a = np.ones(20, dtype="float32")
b = np.ones(30, dtype="float32") * 2
c = np.ones(40, dtype="float32") * 3
result = concatenate_audio(a, b, c)
assert len(result) == 90
assert np.all(result[:20] == 1.0)
assert np.all(result[20:50] == 2.0)
assert np.all(result[50:] == 3.0)
def test_concatenate_empty_buffers(self):
"""Test concatenating empty buffers returns empty array."""
result = concatenate_audio(
np.array([], dtype="float32"),
np.array([], dtype="float32")
)
assert len(result) == 0
def test_concatenate_with_empty(self):
"""Test concatenating with some empty buffers."""
a = np.ones(50, dtype="float32")
result = concatenate_audio(a, np.array([], dtype="float32"))
assert len(result) == 50
assert np.array_equal(result, a)
class TestAudioDuration:
"""Tests for audio_duration function."""
def test_one_second_duration(self):
"""Test duration calculation for 1 second of audio."""
audio = np.zeros(SAMPLE_RATE, dtype="float32")
duration = audio_duration(audio)
assert duration == 1.0
def test_half_second_duration(self):
"""Test duration calculation for 0.5 second of audio."""
audio = np.zeros(SAMPLE_RATE // 2, dtype="float32")
duration = audio_duration(audio)
assert duration == 0.5
def test_empty_audio_duration(self):
"""Test duration of empty audio is 0."""
duration = audio_duration(np.array([], dtype="float32"))
assert duration == 0.0
def test_custom_sample_rate(self):
"""Test duration with custom sample rate."""
audio = np.zeros(48000, dtype="float32") # 48k samples
duration = audio_duration(audio, sample_rate=48000)
assert duration == 1.0
class TestSamplesForDuration:
"""Tests for samples_for_duration function."""
def test_one_second(self):
"""Test samples for 1 second at default rate."""
samples = samples_for_duration(1.0)
assert samples == SAMPLE_RATE
def test_half_second(self):
"""Test samples for 0.5 second at default rate."""
samples = samples_for_duration(0.5)
assert samples == SAMPLE_RATE // 2
def test_zero_duration(self):
"""Test samples for 0 duration."""
samples = samples_for_duration(0)
assert samples == 0
def test_negative_duration(self):
"""Test samples for negative duration."""
samples = samples_for_duration(-1.0)
assert samples == 0
def test_custom_sample_rate(self):
"""Test samples with custom sample rate."""
samples = samples_for_duration(1.0, sample_rate=44100)
assert samples == 44100
class TestSampleRateConstant:
"""Tests for SAMPLE_RATE constant."""
def test_sample_rate_value(self):
"""Test SAMPLE_RATE is 24000."""
assert SAMPLE_RATE == 24000
+128
View File
@@ -0,0 +1,128 @@
"""Tests for audio helper utilities.
Tests import from domain/audio_helpers.py (new module).
"""
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
import numpy as np
import pytest
# ---------------------------------------------------------------------------
# build_ffmpeg_command
# ---------------------------------------------------------------------------
class TestBuildFfmpegCommand:
"""build_ffmpeg_command builds ffmpeg argument list."""
def test_base_structure(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out/audio.wav"), "wav")
assert cmd[0] == "ffmpeg"
assert "-y" in cmd
assert "pipe:0" in cmd
assert str(Path("/out/audio.wav")) in cmd
def test_mp3_codec(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out.mp3"), "mp3")
assert "libmp3lame" in cmd
def test_opus_codec(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out.opus"), "opus")
assert "libopus" in cmd
def test_m4b_codec(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out.m4b"), "m4b")
assert "aac" in cmd
assert "-q:a" in cmd
assert "+faststart+use_metadata_tags" in cmd
def test_wav_copy_codec(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out.wav"), "wav")
assert "copy" in cmd
def test_with_metadata(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out.mp3"), "mp3", metadata={"album": "Test"})
assert str(Path("/out.mp3")) in cmd
# ---------------------------------------------------------------------------
# to_float32
# ---------------------------------------------------------------------------
class TestToFloat32:
"""to_float32 converts audio to float32 numpy array."""
def test_none_returns_empty(self):
from abogen.domain.audio_helpers import to_float32
result = to_float32(None)
assert isinstance(result, np.ndarray)
assert result.dtype == np.float32
assert len(result) == 0
def test_numpy_array(self):
from abogen.domain.audio_helpers import to_float32
arr = np.array([1.0, 2.0, 3.0], dtype="float64")
result = to_float32(arr)
assert result.dtype == np.float32
assert len(result) == 3
def test_mock_tensor(self):
from abogen.domain.audio_helpers import to_float32
tensor = MagicMock()
tensor.detach.return_value = tensor
tensor.cpu.return_value = tensor
tensor.numpy.return_value = np.array([1.0, 2.0])
result = to_float32(tensor)
assert result.dtype == np.float32
assert len(result) == 2
def test_list_input(self):
from abogen.domain.audio_helpers import to_float32
result = to_float32([1.0, 2.0])
assert result.dtype == np.float32
assert len(result) == 2
# ---------------------------------------------------------------------------
# apply_m4b_chapters_with_mutagen
# ---------------------------------------------------------------------------
class TestApplyM4bChaptersWithMutagen:
"""apply_m4b_chapters_with_mutagen writes chapter atoms to MP4."""
def test_empty_chapters_returns_false(self):
from abogen.domain.audio_helpers import apply_m4b_chapters_with_mutagen
assert apply_m4b_chapters_with_mutagen(Path("/fake.m4b"), []) is False
def test_missing_mutagen_raises(self):
from abogen.domain.audio_helpers import apply_m4b_chapters_with_mutagen
with patch.dict("sys.modules", {"mutagen": None, "mutagen.mp4": None}):
with pytest.raises((ImportError, KeyError)):
apply_m4b_chapters_with_mutagen(
Path("/fake.m4b"), [{"start": 0, "title": "Ch1"}]
)
+97
View File
@@ -0,0 +1,97 @@
"""Tests for domain/chapter_classification.py."""
from abogen.domain.chapter_classification import (
supplement_score,
should_preselect_chapter,
ensure_at_least_one_chapter_enabled,
)
class TestSupplementScore:
def test_title_page_high_score(self):
score = supplement_score("Title Page", "", 0)
assert score > 3.0
def test_chapter_title_negative_score(self):
score = supplement_score("Chapter 1", "", 0)
assert score < 0
def test_copyright_high_score(self):
score = supplement_score("Copyright", "All rights reserved.", 0)
assert score > 2.0
def test_short_text_adds_score(self):
score = supplement_score("Some Title", "Short text.", 0)
assert score > 0.5
def test_long_text_low_score_contribution(self):
score = supplement_score("Some Title", "word " * 200, 0)
assert score < 0.5
def test_index_zero_bonus(self):
score_title = supplement_score("Dedication", "For my family.", 0)
score_other = supplement_score("Dedication", "For my family.", 3)
assert score_title > score_other
def test_empty_title_and_text(self):
score = supplement_score("", "", 5)
assert score == 0.9 # short text bonus only (len("") ≤ 150)
def test_newsletter_keyword_high_score(self):
score = supplement_score("Subscribe", "Join our newsletter today", 0)
assert score > 3.0
def test_acknowledgments_pattern(self):
score = supplement_score("Acknowledgements", "", 0)
assert score > 2.0
def test_glossary_in_title(self):
score = supplement_score("Glossary", "", 0)
assert score > 2.0
class TestShouldPreselectChapter:
def test_single_chapter_always_preselected(self):
assert should_preselect_chapter("Anything", "", 0, 1) is True
def test_chapter_preselected_when_low_score(self):
assert should_preselect_chapter("Chapter 1", "The story begins.", 0, 10) is True
def test_title_page_not_preselected(self):
assert should_preselect_chapter("Title Page", "", 0, 10) is False
def test_copyright_not_preselected(self):
assert should_preselect_chapter("Copyright", "All rights reserved.", 0, 10) is False
def test_toc_not_preselected(self):
assert should_preselect_chapter("Table of Contents", "", 0, 10) is False
class TestEnsureAtLeastOneChapterEnabled:
def test_empty_list(self):
chapters = []
ensure_at_least_one_chapter_enabled(chapters)
assert chapters == []
def test_already_has_enabled(self):
chapters = [
{"title": "Ch1", "enabled": False},
{"title": "Ch2", "enabled": True},
]
ensure_at_least_one_chapter_enabled(chapters)
assert chapters[1]["enabled"] is True
assert chapters[0]["enabled"] is False
def test_none_enabled_picks_longest(self):
chapters = [
{"title": "Ch1", "enabled": False, "characters": 100},
{"title": "Ch2", "enabled": False, "characters": 500},
{"title": "Ch3", "enabled": False, "characters": 200},
]
ensure_at_least_one_chapter_enabled(chapters)
assert chapters[1]["enabled"] is True
def test_single_chapter_gets_enabled(self):
chapters = [{"title": "Only", "enabled": False}]
ensure_at_least_one_chapter_enabled(chapters)
assert chapters[0]["enabled"] is True
+167
View File
@@ -0,0 +1,167 @@
"""Tests for chapter_overrides, merge_metadata, normalize_for_pipeline.
Tests import from domain modules (new location).
"""
from __future__ import annotations
import pytest
from abogen.text_extractor import ExtractedChapter
# ---------------------------------------------------------------------------
# apply_chapter_overrides
# ---------------------------------------------------------------------------
class TestApplyChapterOverrides:
"""apply_chapter_overrides applies chapter overrides to extracted chapters."""
def test_empty_overrides(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
result, updates, diags = apply_chapter_overrides([], [])
assert result == []
assert updates == {}
assert diags == []
def test_basic_override_by_index(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
chapters = [ExtractedChapter(title="Ch1", text="original")]
overrides = [{"index": 0, "title": "New Title", "text": "new text"}]
result, updates, diags = apply_chapter_overrides(chapters, overrides)
assert len(result) == 1
assert result[0].title == "New Title"
assert result[0].text == "new text"
def test_override_by_source_title(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
chapters = [ExtractedChapter(title="Ch1", text="text1")]
overrides = [{"source_title": "Ch1", "title": "Renamed"}]
result, _, _ = apply_chapter_overrides(chapters, overrides)
assert result[0].title == "Renamed"
def test_disabled_override_skipped(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
chapters = [ExtractedChapter(title="Ch1", text="text1")]
overrides = [{"index": 0, "enabled": False}]
result, _, _ = apply_chapter_overrides(chapters, overrides)
assert len(result) == 0
def test_metadata_updates_collected(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
chapters = [ExtractedChapter(title="Ch1", text="text1")]
overrides = [{"index": 0, "metadata": {"album": "New Album"}}]
_, updates, _ = apply_chapter_overrides(chapters, overrides)
assert updates["album"] == "New Album"
def test_no_matching_chapter_diagnostic(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
overrides = [{"index": 99, "title": "X"}]
_, _, diags = apply_chapter_overrides([], overrides)
assert len(diags) == 1
assert "Skipped" in diags[0]
def test_non_dict_override_skipped(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
_, _, diags = apply_chapter_overrides([], ["bad"])
assert len(diags) == 1
def test_text_from_base_when_not_provided(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
chapters = [ExtractedChapter(title="Ch1", text="original text")]
overrides = [{"index": 0, "title": "New Title"}]
result, _, _ = apply_chapter_overrides(chapters, overrides)
assert result[0].text == "original text"
def test_default_title_when_no_base(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
overrides = [{"text": "some text"}]
result, _, _ = apply_chapter_overrides([], overrides)
assert result[0].title == "Chapter 1"
# ---------------------------------------------------------------------------
# merge_metadata
# ---------------------------------------------------------------------------
class TestMergeMetadata:
"""merge_metadata merges extracted metadata with overrides."""
def test_both_empty(self):
from abogen.domain.metadata_merge import merge_metadata
assert merge_metadata({}, {}) == {}
def test_only_extracted(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata({"album": "Book"}, {})
assert result == {"album": "Book"}
def test_only_overrides(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata(None, {"album": "Override"})
assert result == {"album": "Override"}
def test_override_wins(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata({"album": "Old"}, {"album": "New"})
assert result == {"album": "New"}
def test_none_value_deletes_key(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata({"album": "Book"}, {"album": None})
assert "album" not in result
def test_none_values_in_extracted_skipped(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata({"album": None, "artist": "X"}, {})
assert result == {"artist": "X"}
def test_numeric_values_stringified(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata({"track": 1}, {})
assert result["track"] == "1"
# ---------------------------------------------------------------------------
# normalize_for_pipeline (thin wrapper)
# ---------------------------------------------------------------------------
class TestNormalizeForPipeline:
"""normalize_for_pipeline normalizes text with runtime settings."""
def test_basic_normalize(self):
from abogen.domain.normalization import normalize_text_for_pipeline
result = normalize_text_for_pipeline("Hello World")
assert isinstance(result, str)
assert len(result) > 0
def test_empty_string(self):
from abogen.domain.normalization import normalize_text_for_pipeline
result = normalize_text_for_pipeline("")
assert result == ""
def test_with_overrides(self):
from abogen.domain.normalization import normalize_text_for_pipeline
result = normalize_text_for_pipeline("test", normalization_overrides={"number_format": "words"})
assert isinstance(result, str)
+144
View File
@@ -0,0 +1,144 @@
"""Tests for domain/chapter_titles.py."""
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from abogen.domain.chapter_titles import (
simplify_heading_text,
headings_equivalent,
strip_duplicate_heading_line,
normalize_caps_word,
normalize_chapter_opening_caps,
format_spoken_chapter_title,
)
class TestSimplifyHeadingText:
def test_empty(self):
assert simplify_heading_text("") == ""
def test_none(self):
assert simplify_heading_text(None) == ""
def test_chapter_prefix_removed(self):
assert simplify_heading_text("Chapter 1") == "1"
def test_lowercase(self):
assert simplify_heading_text("Chapter 1: The Beginning") == "1thebeginning"
def test_strips_special_chars(self):
result = simplify_heading_text("Ch. 1")
assert "1" in result
assert "." not in result
def test_no_chapter_prefix(self):
result = simplify_heading_text("Part 2")
assert "part" in result
class TestHeadingsEquivalent:
def test_exact_match(self):
assert headings_equivalent("Chapter 1", "Chapter 1")
def test_prefix_match(self):
assert headings_equivalent("Chapter 2", "Chapter 2: The Return")
def test_reverse_prefix(self):
assert headings_equivalent("Chapter 2: The Return", "Chapter 2")
def test_different_numbers(self):
assert not headings_equivalent("Chapter 1", "Chapter 2")
def test_empty(self):
assert not headings_equivalent("", "Chapter 1")
def test_long_containment(self):
assert headings_equivalent("Introduction", "Introduction to Everything")
class TestStripDuplicateHeadingLine:
def test_removes_heading(self):
text = "Chapter 1\n\nSome text here"
result, removed = strip_duplicate_heading_line(text, "Chapter 1")
assert removed is True
assert "Chapter 1" not in result
assert "Some text here" in result
def test_no_heading(self):
text = "Just some text"
result, removed = strip_duplicate_heading_line(text, "Chapter 1")
assert removed is False
assert result == text
def test_empty_text(self):
result, removed = strip_duplicate_heading_line("", "Chapter 1")
assert removed is False
def test_strips_leading_empty_lines(self):
text = "Chapter 1\n\n\n\nText"
result, removed = strip_duplicate_heading_line(text, "Chapter 1")
assert removed is True
assert result.startswith("Text")
class TestNormalizeCapsWord:
def test_acronym_kept(self):
assert normalize_caps_word("TTS") == "TTS"
def test_single_letter_kept(self):
assert normalize_caps_word("A") == "A"
def test_roman_numeral_kept(self):
assert normalize_caps_word("IV") == "IV"
def test_all_caps_converted(self):
result = normalize_caps_word("HELLO")
assert result == "Hello"
def test_with_hyphen(self):
result = normalize_caps_word("WELL-KNOWN")
assert result == "Well-Known"
class TestNormalizeChapterOpeningCaps:
def test_all_caps_words(self):
text = "THIS IS A TEST"
result, changed = normalize_chapter_opening_caps(text)
assert changed is True
assert result == "This Is A Test"
def test_already_normal(self):
text = "This is normal"
result, changed = normalize_chapter_opening_caps(text)
assert changed is False
def test_empty(self):
result, changed = normalize_chapter_opening_caps("")
assert changed is False
def test_mixed(self):
text = "HELLO world"
result, changed = normalize_chapter_opening_caps(text)
assert changed is True
class TestFormatSpokenChapterTitle:
def test_empty_no_prefix(self):
assert format_spoken_chapter_title("", 1, False) == ""
def test_empty_with_prefix(self):
assert format_spoken_chapter_title("", 1, True) == "Chapter 1"
def test_no_prefix_returns_base(self):
assert format_spoken_chapter_title("My Chapter", 1, False) == "My Chapter"
def test_already_has_chapter(self):
assert format_spoken_chapter_title("Chapter 5", 1, True) == "Chapter 5"
def test_number_prefix(self):
result = format_spoken_chapter_title("3. The End", 1, True)
assert result == "Chapter 3. The End"
def test_number_only(self):
result = format_spoken_chapter_title("7", 1, True)
assert result == "Chapter 7"
+6 -5
View File
@@ -3,7 +3,8 @@ from __future__ import annotations
from types import SimpleNamespace from types import SimpleNamespace
from abogen.chunking import chunk_text from abogen.chunking import chunk_text
from abogen.webui.conversion_runner import _chunk_voice_spec, _group_chunks_by_chapter from abogen.domain.voice_resolution import chunk_voice_spec
from abogen.domain.chunk_utils import group_chunks_by_chapter
def test_group_chunks_by_chapter_orders_and_groups() -> None: def test_group_chunks_by_chapter_orders_and_groups() -> None:
@@ -13,7 +14,7 @@ def test_group_chunks_by_chapter_orders_and_groups() -> None:
{"chapter_index": 1, "chunk_index": 0, "text": "next"}, {"chapter_index": 1, "chunk_index": 0, "text": "next"},
] ]
grouped = _group_chunks_by_chapter(chunks) grouped = group_chunks_by_chapter(chunks)
assert [entry["text"] for entry in grouped[0]] == ["body", "tail"] assert [entry["text"] for entry in grouped[0]] == ["body", "tail"]
assert grouped[1][0]["text"] == "next" assert grouped[1][0]["text"] == "next"
@@ -23,7 +24,7 @@ def test_chunk_voice_spec_prefers_chunk_overrides() -> None:
job = SimpleNamespace(voice="base_voice", speakers={}) job = SimpleNamespace(voice="base_voice", speakers={})
chunk = {"voice": "override_voice", "speaker_id": "narrator"} chunk = {"voice": "override_voice", "speaker_id": "narrator"}
assert _chunk_voice_spec(job, chunk, "fallback") == "override_voice" assert chunk_voice_spec(job, chunk, "fallback") == "override_voice"
def test_chunk_voice_spec_falls_back_to_speaker_voice() -> None: def test_chunk_voice_spec_falls_back_to_speaker_voice() -> None:
@@ -32,14 +33,14 @@ def test_chunk_voice_spec_falls_back_to_speaker_voice() -> None:
) )
chunk = {"speaker_id": "narrator"} chunk = {"speaker_id": "narrator"}
assert _chunk_voice_spec(job, chunk, "fallback") == "speaker_voice" assert chunk_voice_spec(job, chunk, "fallback") == "speaker_voice"
def test_chunk_voice_spec_uses_fallback_when_no_overrides() -> None: def test_chunk_voice_spec_uses_fallback_when_no_overrides() -> None:
job = SimpleNamespace(voice="base_voice", speakers={}) job = SimpleNamespace(voice="base_voice", speakers={})
chunk = {"speaker_id": "unknown"} chunk = {"speaker_id": "unknown"}
assert _chunk_voice_spec(job, chunk, "fallback") == "fallback" assert chunk_voice_spec(job, chunk, "fallback") == "fallback"
def test_chunk_text_merges_title_abbreviations() -> None: def test_chunk_text_merges_title_abbreviations() -> None:
+4 -4
View File
@@ -1,4 +1,4 @@
from abogen.webui.conversion_runner import _chunk_text_for_tts from abogen.domain.chunk_utils import chunk_text_for_tts
def test_chunk_text_for_tts_prefers_text_over_normalized_text(): def test_chunk_text_for_tts_prefers_text_over_normalized_text():
@@ -9,7 +9,7 @@ def test_chunk_text_for_tts_prefers_text_over_normalized_text():
"text": "Unfu*k", "text": "Unfu*k",
} }
assert _chunk_text_for_tts(entry) == "Unfu*k" assert chunk_text_for_tts(entry) == "Unfu*k"
def test_chunk_text_for_tts_falls_back_to_original_text_then_normalized_text(): def test_chunk_text_for_tts_falls_back_to_original_text_then_normalized_text():
@@ -17,9 +17,9 @@ def test_chunk_text_for_tts_falls_back_to_original_text_then_normalized_text():
"original_text": "Hello * world", "original_text": "Hello * world",
"normalized_text": "Hello world", "normalized_text": "Hello world",
} }
assert _chunk_text_for_tts(entry) == "Hello * world" assert chunk_text_for_tts(entry) == "Hello * world"
entry2 = { entry2 = {
"normalized_text": "Only normalized", "normalized_text": "Only normalized",
} }
assert _chunk_text_for_tts(entry2) == "Only normalized" assert chunk_text_for_tts(entry2) == "Only normalized"
+123
View File
@@ -0,0 +1,123 @@
"""Tests for chunk processing utilities.
Tests import from domain.chunk_utils (new location).
"""
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import patch
import pytest
# ---------------------------------------------------------------------------
# safe_int
# ---------------------------------------------------------------------------
class TestSafeInt:
"""safe_int safely converts to int with a default."""
def test_int_value(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int(42) == 42
def test_string_number(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int("7") == 7
def test_float_truncated(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int(3.9) == 3
def test_none_returns_default(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int(None) == 0
def test_garbage_returns_default(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int("abc") == 0
def test_custom_default(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int(None, default=-1) == -1
# ---------------------------------------------------------------------------
# chunk_text_for_tts (supplement existing tests)
# ---------------------------------------------------------------------------
class TestChunkTextForTts:
"""chunk_text_for_tts selects the best text source."""
def test_non_mapping_returns_empty(self):
from abogen.domain.chunk_utils import chunk_text_for_tts
assert chunk_text_for_tts("not a dict") == ""
def test_none_returns_empty(self):
from abogen.domain.chunk_utils import chunk_text_for_tts
assert chunk_text_for_tts(None) == ""
def test_empty_dict_returns_empty(self):
from abogen.domain.chunk_utils import chunk_text_for_tts
assert chunk_text_for_tts({}) == ""
def test_whitespace_only_returns_empty(self):
from abogen.domain.chunk_utils import chunk_text_for_tts
assert chunk_text_for_tts({"text": " "}) == ""
# ---------------------------------------------------------------------------
# record_override_usage
# ---------------------------------------------------------------------------
class TestRecordOverrideUsage:
"""record_override_usage records pronunciation override usage."""
def test_noop_when_empty(self):
from abogen.domain.chunk_utils import record_override_usage
job = SimpleNamespace(language="en")
record_override_usage(job, {}, {})
def test_noop_when_all_zero(self):
from abogen.domain.chunk_utils import record_override_usage
job = SimpleNamespace(language="en")
record_override_usage(job, {"hello": 0}, {"hello": "hi"})
def test_records_usage(self):
from abogen.domain.chunk_utils import record_override_usage
job = SimpleNamespace(language="en", add_log=lambda *a, **kw: None)
with patch("abogen.domain.chunk_utils.increment_usage") as mock_inc:
record_override_usage(job, {"hello": 2}, {"hello": "hi"})
mock_inc.assert_called_once_with(language="en", token="hi", amount=2)
def test_fallback_token_from_normalized(self):
from abogen.domain.chunk_utils import record_override_usage
job = SimpleNamespace(language="ja", add_log=lambda *a, **kw: None)
with patch("abogen.domain.chunk_utils.increment_usage") as mock_inc:
record_override_usage(job, {"test": 1}, {})
mock_inc.assert_called_once_with(language="ja", token="test", amount=1)
def test_handles_exception_gracefully(self):
from abogen.domain.chunk_utils import record_override_usage
job = SimpleNamespace(language="en", add_log=lambda *a, **kw: None)
with patch("abogen.domain.chunk_utils.increment_usage", side_effect=RuntimeError("db error")):
record_override_usage(job, {"hello": 1}, {"hello": "hi"})
+114
View File
@@ -124,3 +124,117 @@ def test_normalize_chapter_opening_caps_keeps_mixed_case() -> None:
normalized, changed = _normalize_chapter_opening_caps("Already Mixed Case") normalized, changed = _normalize_chapter_opening_caps("Already Mixed Case")
assert normalized == "Already Mixed Case" assert normalized == "Already Mixed Case"
assert changed is False assert changed is False
class TestApplyChapterTextTransforms:
"""Tests for the combined heading-strip + opening-caps helper."""
def test_both_enabled_heading_matches(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"Chapter 1: The Beginning\nBody text here",
heading_text="Chapter 1: The Beginning",
raw_title="Chapter 1: The Beginning",
strip_heading=True,
normalize_caps=True,
)
assert heading_removed is True
assert "Body text here" in text
assert "Chapter 1" not in text
def test_heading_fallback_to_number(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"1. The Beginning\nBody text",
heading_text="Chapter 1: The Beginning",
raw_title="1: The Beginning",
strip_heading=True,
normalize_caps=False,
)
assert heading_removed is True
assert "Body text" in text
def test_only_heading_strip(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"Chapter 1: Title\nBody text",
heading_text="Chapter 1: Title",
raw_title="",
strip_heading=True,
normalize_caps=False,
)
assert heading_removed is True
assert caps_changed is False
def test_only_opening_caps(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"ALL CAPS START OF CHAPTER",
heading_text="Chapter 1",
raw_title="",
strip_heading=False,
normalize_caps=True,
)
assert heading_removed is False
assert caps_changed is True
assert text == "All Caps Start Of Chapter"
def test_both_disabled_no_change(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
original = "Some text here"
text, heading_removed, caps_changed = apply_chapter_text_transforms(
original,
heading_text="Chapter 1",
raw_title="",
strip_heading=False,
normalize_caps=False,
)
assert text == original
assert heading_removed is False
assert caps_changed is False
def test_heading_not_matching(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"Completely different text",
heading_text="Chapter 1: Title",
raw_title="",
strip_heading=True,
normalize_caps=False,
)
assert heading_removed is False
assert text == "Completely different text"
def test_empty_text(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"",
heading_text="Chapter 1",
raw_title="",
strip_heading=True,
normalize_caps=True,
)
assert text == ""
assert heading_removed is False
assert caps_changed is False
def test_both_enabled_text_only_has_caps(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"NASA MISSION LOG",
heading_text="Chapter 1",
raw_title="",
strip_heading=True,
normalize_caps=True,
)
assert heading_removed is False
assert caps_changed is True
assert text == "NASA Mission Log"
+189
View File
@@ -0,0 +1,189 @@
"""Tests for domain/audio_sink.py"""
import numpy as np
import soundfile as sf
from pathlib import Path
from unittest.mock import MagicMock, patch, call
import subprocess
from abogen.domain.audio_sink import AudioSink, open_audio_sink, _ensure_ffmpeg
class TestAudioSinkDataclass:
def test_audio_sink_is_frozen(self):
sink = AudioSink(write=lambda d: None, close=lambda: None)
try:
sink.write = lambda d: None # type: ignore
except AttributeError:
pass # Expected — frozen=True
assert hasattr(sink, "write")
assert hasattr(sink, "close")
def test_audio_sink_stores_callables(self):
write_fn = MagicMock()
close_fn = MagicMock()
sink = AudioSink(write=write_fn, close=close_fn)
assert sink.write is write_fn
assert sink.close is close_fn
def test_audio_sink_write_callable(self):
calls = []
sink = AudioSink(write=lambda d: calls.append(d), close=lambda: None)
data = np.zeros(100, dtype="float32")
sink.write(data)
assert len(calls) == 1
np.testing.assert_array_equal(calls[0], data)
def test_audio_sink_close_callable(self):
closed = []
sink = AudioSink(write=lambda d: None, close=lambda: closed.append(True))
sink.close()
assert closed == [True]
class TestOpenAudioSinkWav:
def test_wav_creates_file(self, tmp_path: Path):
out = tmp_path / "test.wav"
with open_audio_sink(out, "wav") as sink:
sink.write(np.zeros(100, dtype="float32"))
assert out.exists()
def test_flac_creates_file(self, tmp_path: Path):
out = tmp_path / "test.flac"
with open_audio_sink(out, "flac") as sink:
sink.write(np.zeros(100, dtype="float32"))
assert out.exists()
def test_wav_sink_writes_audio(self, tmp_path: Path):
out = tmp_path / "out.wav"
audio = np.random.uniform(-0.5, 0.5, 24000).astype("float32") # 1 second
with open_audio_sink(out, "wav") as sink:
sink.write(audio)
data, sr = sf.read(str(out))
assert sr == 24000
assert len(data) == 24000
np.testing.assert_allclose(data, audio, atol=1e-4)
def test_wav_sink_close_flushes(self, tmp_path: Path):
out = tmp_path / "flush.wav"
with open_audio_sink(out, "wav") as sink:
sink.write(np.ones(1000, dtype="float32"))
assert out.exists()
info = sf.info(str(out))
assert info.samplerate == 24000
assert info.channels == 1
def test_wav_sink_context_manager(self, tmp_path: Path):
out = tmp_path / "ctx.wav"
with open_audio_sink(out, "wav") as sink:
assert isinstance(sink, AudioSink)
sink.write(np.zeros(50, dtype="float32"))
assert out.exists()
def test_wav_sink_multiple_writes(self, tmp_path: Path):
out = tmp_path / "multi.wav"
with open_audio_sink(out, "wav") as sink:
sink.write(np.ones(1000, dtype="float32"))
sink.write(np.ones(500, dtype="float32"))
data, _ = sf.read(str(out))
assert len(data) == 1500
class TestCancelCheck:
def test_cancel_check_skips_wav_writes(self, tmp_path: Path):
out = tmp_path / "cancelled.wav"
with open_audio_sink(out, "wav", cancel_check=lambda: True) as sink:
sink.write(np.ones(1000, dtype="float32"))
sink.write(np.ones(500, dtype="float32"))
data, _ = sf.read(str(out))
assert len(data) == 0
def test_cancel_check_none_allows_writes(self, tmp_path: Path):
out = tmp_path / "ok.wav"
with open_audio_sink(out, "wav", cancel_check=None) as sink:
sink.write(np.ones(1000, dtype="float32"))
data, _ = sf.read(str(out))
assert len(data) == 1000
class TestUnsupportedFormat:
def test_unsupported_format_raises(self, tmp_path: Path):
out = tmp_path / "bad.xyz"
try:
with open_audio_sink(out, "xyz") as sink:
pass
assert False, "Should have raised"
except Exception:
pass # Expected
class TestOpenAudioSinkCompressed:
@patch("abogen.domain.audio_sink._ensure_ffmpeg")
@patch("abogen.domain.audio_sink.build_ffmpeg_command")
@patch("abogen.domain.audio_sink.subprocess.Popen")
def test_mp3_sink_returns_sink(self, mock_popen, mock_build, mock_ensure, tmp_path: Path):
mock_build.return_value = ["ffmpeg", "-y", "-i", "pipe:0", "out.mp3"]
proc = MagicMock()
proc.stdin = MagicMock()
proc.stdin.closed = False
proc.wait.return_value = 0
mock_popen.return_value = proc
out = tmp_path / "test.mp3"
sink = open_audio_sink(out, "mp3")
assert isinstance(sink, AudioSink)
sink.write(np.zeros(100, dtype="float32"))
assert proc.stdin.write.called
sink.close()
@patch("abogen.domain.audio_sink._ensure_ffmpeg")
@patch("abogen.domain.audio_sink.build_ffmpeg_command")
@patch("abogen.domain.audio_sink.subprocess.Popen")
def test_cancel_check_skips_compressed_writes(self, mock_popen, mock_build, mock_ensure, tmp_path: Path):
mock_build.return_value = ["ffmpeg", "-y", "-i", "pipe:0", "out.mp3"]
proc = MagicMock()
proc.stdin = MagicMock()
proc.stdin.closed = False
proc.wait.return_value = 0
mock_popen.return_value = proc
sink = open_audio_sink(tmp_path / "c.mp3", "mp3", cancel_check=lambda: True)
sink.write(np.zeros(100, dtype="float32"))
assert not proc.stdin.write.called
sink.close()
@patch("abogen.domain.audio_sink._ensure_ffmpeg")
@patch("abogen.domain.audio_sink.build_ffmpeg_command")
@patch("abogen.domain.audio_sink.subprocess.Popen")
def test_extra_ffmpeg_args_passed(self, mock_popen, mock_build, mock_ensure, tmp_path: Path):
mock_build.return_value = ["ffmpeg", "-y", "-header", "-i", "pipe:0", "out.mp3"]
proc = MagicMock()
proc.stdin = MagicMock()
proc.stdin.closed = False
proc.wait.return_value = 0
mock_popen.return_value = proc
sink = open_audio_sink(
tmp_path / "extra.mp3",
"mp3",
extra_ffmpeg_args=["-thread_queue_size", "32768"],
)
args = mock_popen.call_args[0][0]
assert "-thread_queue_size" in args
assert "32768" in args
sink.close()
@patch("abogen.domain.audio_sink._ensure_ffmpeg")
@patch("abogen.domain.audio_sink.build_ffmpeg_command")
@patch("abogen.domain.audio_sink.subprocess.Popen")
def test_metadata_passed_to_ffmpeg(self, mock_popen, mock_build, mock_ensure, tmp_path: Path):
mock_build.return_value = ["ffmpeg", "-y", "-i", "pipe:0", "out.opus"]
proc = MagicMock()
proc.stdin = MagicMock()
proc.stdin.closed = False
proc.wait.return_value = 0
mock_popen.return_value = proc
meta = {"title": "Test", "artist": "Author"}
open_audio_sink(tmp_path / "meta.opus", "opus", metadata=meta)
mock_build.assert_called_once_with(tmp_path / "meta.opus", "opus", metadata=meta)
+73
View File
@@ -0,0 +1,73 @@
"""Tests that voice.py imports from domain modules, not from conversion_runner."""
import pathlib
_MODULE_SOURCE_CACHE: dict[str, str] = {}
def _read_source(module_file: str) -> str:
if module_file not in _MODULE_SOURCE_CACHE:
_MODULE_SOURCE_CACHE[module_file] = pathlib.Path(module_file).read_text(
encoding="utf-8"
)
return _MODULE_SOURCE_CACHE[module_file]
def test_voice_module_does_not_import_conversion_runner():
"""voice.py must not import from conversion_runner (architecture rule)."""
import abogen.webui.routes.utils.voice as voice_mod
source_text = _read_source(voice_mod.__file__)
assert "from abogen.webui.conversion_runner import" not in source_text, (
"voice.py still imports from conversion_runner — must use domain modules"
)
def test_synthesize_module_does_not_import_conversion_runner():
"""synthesize.py must not import from conversion_runner."""
import abogen.webui.routes.utils.synthesize as synthesize_mod
source_text = _read_source(synthesize_mod.__file__)
assert "from abogen.webui.conversion_runner import" not in source_text
def test_synthesize_module_imports_select_device_from_domain():
"""synthesize.py must import select_device from abogen.domain.device."""
import abogen.webui.routes.utils.synthesize as synthesize_mod
source_text = _read_source(synthesize_mod.__file__)
assert "from abogen.domain.device import" in source_text
def test_voice_module_does_not_define_synthesize_audio_from_normalized():
"""Dead code: synthesize_audio_from_normalized must be removed from voice.py."""
import abogen.webui.routes.utils.voice as voice_mod
source_text = _read_source(voice_mod.__file__)
assert "def synthesize_audio_from_normalized(" not in source_text
def test_voice_module_does_not_define_get_preview_pipeline():
"""Dead code: get_preview_pipeline must be removed from voice.py."""
import abogen.webui.routes.utils.voice as voice_mod
source_text = _read_source(voice_mod.__file__)
assert "def get_preview_pipeline(" not in source_text
def test_voice_module_does_not_import_domain_audio_helpers():
"""After removing dead code, voice.py no longer needs audio_helpers imports."""
import abogen.webui.routes.utils.voice as voice_mod
source_text = _read_source(voice_mod.__file__)
assert "from abogen.domain.audio_helpers import" not in source_text
def test_voice_module_does_not_import_domain_device():
"""After removing dead code, voice.py no longer needs device imports."""
import abogen.webui.routes.utils.voice as voice_mod
source_text = _read_source(voice_mod.__file__)
assert "from abogen.domain.device import" not in source_text
+147
View File
@@ -0,0 +1,147 @@
"""Tests for domain/normalization.py — prepare_text_for_tts."""
import pytest
from unittest.mock import patch, MagicMock
from abogen.domain.normalization import prepare_text_for_tts, normalize_text_for_pipeline
class TestPrepareTextForTts:
"""Tests for the comprehensive TTS text preparation pipeline."""
def test_empty_text(self):
result = prepare_text_for_tts("")
assert result == ""
def test_none_text(self):
result = prepare_text_for_tts(None)
assert result == ""
def test_passthrough_no_rules(self):
result = prepare_text_for_tts("Hello world")
assert isinstance(result, str)
assert len(result) > 0
def test_heteronym_rules_applied(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [
{
"token": "read",
"pronunciation": "red",
"context": "past tense",
}
]
rules = compile_heteronym_sentence_rules(overrides)
if rules:
result = prepare_text_for_tts("I will read the book", heteronym_rules=rules)
assert isinstance(result, str)
def test_pronunciation_rules_applied(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [
{
"token": "epub",
"pronunciation": "ee-pub",
"normalized": "epub",
}
]
rules = compile_pronunciation_rules(overrides)
result = prepare_text_for_tts(
"This is an epub file",
pronunciation_rules=rules,
)
assert "ee-pub" in result
def test_usage_counter_tracks_pronunciation(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [
{
"token": "data",
"pronunciation": "day-ta",
"normalized": "data",
}
]
rules = compile_pronunciation_rules(overrides)
counter = {}
prepare_text_for_tts(
"The data is here and the data is there",
pronunciation_rules=rules,
usage_counter=counter,
)
assert counter.get("data", 0) >= 1
def test_combined_heteronym_and_pronunciation(self):
from abogen.domain.pronunciation import (
compile_heteronym_sentence_rules,
compile_pronunciation_rules,
)
heteronym_overrides = [
{
"token": "lead",
"pronunciation": "led",
"context": "metal",
}
]
pronunciation_overrides = [
{
"token": "gif",
"pronunciation": "jif",
"normalized": "gif",
}
]
h_rules = compile_heteronym_sentence_rules(heteronym_overrides)
p_rules = compile_pronunciation_rules(pronunciation_overrides)
result = prepare_text_for_tts(
"A lead gif",
heteronym_rules=h_rules if h_rules else None,
pronunciation_rules=p_rules,
)
assert isinstance(result, str)
@patch("abogen.domain.normalization.get_runtime_settings")
def test_normalization_overrides_passed_through(self, mock_settings):
mock_settings.return_value = {
"normalization_apostrophe_mode": "spacy",
"normalization_enabled": True,
}
result = prepare_text_for_tts(
"It's a test",
normalization_overrides={"normalization_enabled": False},
)
assert isinstance(result, str)
def test_pronunciation_rules_empty(self):
result = prepare_text_for_tts("Hello", pronunciation_rules=[])
assert isinstance(result, str)
def test_heteronym_rules_empty(self):
result = prepare_text_for_tts("Hello", heteronym_rules=[])
assert isinstance(result, str)
class TestNormalizeTextForPipeline:
"""Tests for the simpler normalization function."""
def test_basic_normalization(self):
result = normalize_text_for_pipeline("It's a test")
assert isinstance(result, str)
assert len(result) > 0
def test_empty_text(self):
result = normalize_text_for_pipeline("")
assert result == ""
@patch("abogen.domain.normalization.get_runtime_settings")
def test_with_overrides(self, mock_settings):
mock_settings.return_value = {
"normalization_apostrophe_mode": "spacy",
}
result = normalize_text_for_pipeline(
"It's a test",
normalization_overrides={"normalization_apostrophe_mode": "spacy"},
)
assert isinstance(result, str)
+175
View File
@@ -0,0 +1,175 @@
from __future__ import annotations
from unittest.mock import MagicMock, patch
from abogen.domain.pipeline_factory import (
PipelinePool,
create_pipeline_for_job,
dispose_pipelines,
resolve_device,
)
class TestResolveDevice:
@patch("abogen.utils.load_config", return_value={"use_gpu": True})
@patch("abogen.domain.pipeline_factory.select_device", return_value="cuda:0")
def test_gpu_enabled(self, _sel, _cfg):
assert resolve_device(use_gpu=True) == "cuda:0"
@patch("abogen.utils.load_config", return_value={"use_gpu": True})
def test_gpu_disabled_by_job(self, _cfg):
assert resolve_device(use_gpu=False) == "cpu"
@patch("abogen.utils.load_config", return_value={"use_gpu": False})
@patch("abogen.domain.pipeline_factory.select_device", return_value="cuda:0")
def test_gpu_disabled_by_config(self, _sel, _cfg):
assert resolve_device(use_gpu=True) == "cpu"
class TestCreatePipelineForJob:
@patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
def test_supertonic_provider(self, _reg, mock_create):
mock_create.return_value = MagicMock()
result = create_pipeline_for_job("supertonic", "en", use_gpu=True)
mock_create.assert_called_once_with("supertonic")
assert result is mock_create.return_value
@patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_kokoro_provider(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock()
result = create_pipeline_for_job("kokoro", "en", use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="en", device="cpu")
assert result is mock_create.return_value
@patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=False)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_unknown_provider_falls_back_to_kokoro(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock()
result = create_pipeline_for_job("unknown_provider", "en", use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="en", device="cpu")
@patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_empty_provider_defaults_to_kokoro(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock()
result = create_pipeline_for_job("", "en", use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="en", device="cpu")
@patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_none_provider_defaults_to_kokoro(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock()
result = create_pipeline_for_job(None, "en", use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="en", device="cpu")
class TestDisposePipelines:
def test_disposes_all_and_clears(self):
p1 = MagicMock()
p2 = MagicMock()
pipelines = {"kokoro": p1, "supertonic": p2}
dispose_pipelines(pipelines)
p1.dispose.assert_called_once()
p2.dispose.assert_called_once()
assert pipelines == {}
def test_handles_dispose_error(self):
p1 = MagicMock()
p1.dispose.side_effect = RuntimeError("boom")
pipelines = {"kokoro": p1}
dispose_pipelines(pipelines)
assert pipelines == {}
def test_empty_dict(self):
pipelines = {}
dispose_pipelines(pipelines)
assert pipelines == {}
class TestPipelinePool:
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_get_creates_and_caches(self, _cache, mock_create):
mock_pipeline = MagicMock()
mock_create.return_value = mock_pipeline
pool = PipelinePool()
result = pool.get("kokoro", "en", use_gpu=True)
assert result is mock_pipeline
mock_create.assert_called_once()
result2 = pool.get("kokoro", "en", use_gpu=True)
assert result2 is mock_pipeline
assert mock_create.call_count == 1
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_get_initializes_voice_cache_once(self, mock_cache, mock_create):
mock_create.return_value = MagicMock()
pool = PipelinePool()
job = MagicMock()
pool.get("kokoro", "en", use_gpu=True, job=job)
assert mock_cache.call_count == 1
pool.get("kokoro", "en", use_gpu=True, job=job)
assert mock_cache.call_count == 1
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
def test_get_no_job_skips_voice_cache(self, mock_create, mock_cache):
mock_create.return_value = MagicMock()
pool = PipelinePool()
pool.get("kokoro", "en", use_gpu=True)
mock_cache.assert_not_called()
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
def test_get_separately_per_provider(self, mock_create):
p1 = MagicMock(name="kokoro")
p2 = MagicMock(name="supertonic")
mock_create.side_effect = [p1, p2]
pool = PipelinePool()
r1 = pool.get("kokoro", "en", use_gpu=True)
r2 = pool.get("supertonic", "en", use_gpu=True)
assert r1 is p1
assert r2 is p2
assert mock_create.call_count == 2
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_dispose_all(self, mock_cache, mock_create):
p1 = MagicMock(name="kokoro")
p2 = MagicMock(name="supertonic")
mock_create.side_effect = [p1, p2]
pool = PipelinePool()
pool.get("kokoro", "en", use_gpu=True)
pool.get("supertonic", "en", use_gpu=True)
pool.dispose_all()
p1.dispose.assert_called_once()
p2.dispose.assert_called_once()
assert pool._pipelines == {}
assert pool._voice_cache_initialized is False
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
def test_dispose_empty_pool(self, mock_create):
pool = PipelinePool()
pool.dispose_all()
mock_create.assert_not_called()
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=False)
def test_unknown_provider_falls_back(self, _reg, _cache, mock_create):
mock_create.return_value = MagicMock()
pool = PipelinePool()
pool.get("bogus_provider", "en", use_gpu=True)
mock_create.assert_called_once_with("kokoro", "en", True)
+71
View File
@@ -0,0 +1,71 @@
from __future__ import annotations
import time
from unittest.mock import patch
from abogen.domain.progress import ProgressTracker, calc_etr_str
class TestCalcEtrStr:
def test_returns_processing_when_not_started(self):
assert calc_etr_str(0.0, 0, 100) == "Processing..."
def test_returns_processing_when_elapsed_too_short(self):
assert calc_etr_str(0.3, 50, 100) == "Processing..."
def test_returns_processing_when_done_zero(self):
assert calc_etr_str(2.0, 0, 100) == "Processing..."
def test_returns_zero_when_complete(self):
assert calc_etr_str(10.0, 100, 100) == "00:00:00"
def test_returns_zero_when_overcomplete(self):
assert calc_etr_str(10.0, 150, 100) == "00:00:00"
def test_half_done(self):
# 10s for 50 chars -> 10s remaining -> 00:00:10
assert calc_etr_str(10.0, 50, 100) == "00:00:10"
def test_one_third_done_large(self):
# 100s for 1000 chars out of 3000 -> 200s remaining
assert calc_etr_str(100.0, 1000, 3000) == "00:03:20"
def test_hours_format(self):
# 3600s for 1000 chars out of 4000 -> 3 * 3600 = 10800s remaining
assert calc_etr_str(3600.0, 1000, 4000) == "03:00:00"
def test_minutes_and_seconds(self):
# 60s for 100 chars out of 200 -> 60s remaining
assert calc_etr_str(60.0, 100, 200) == "00:01:00"
class TestProgressTracker:
def test_percent_at_zero(self):
t = ProgressTracker(total_chars=1000)
assert t.percent == 0
def test_percent_half(self):
t = ProgressTracker(total_chars=1000)
t.update(500)
assert t.percent == 50
def test_percent_capped_at_99(self):
t = ProgressTracker(total_chars=100)
t.update(100)
assert t.percent == 99 # matches original behavior
def test_etr_processing_at_start(self):
t = ProgressTracker(total_chars=1000)
t.update(0)
assert t.etr_str == "Processing..."
def test_etr_computes_correctly(self):
t = ProgressTracker(total_chars=200)
with patch("abogen.domain.progress.time") as mock_time:
mock_time.time.return_value = t._start_time + 2.0
t.update(100)
assert t.etr_str == "00:00:02"
def test_zero_total_chars(self):
t = ProgressTracker(total_chars=0)
assert t.percent == 0
+195
View File
@@ -0,0 +1,195 @@
from __future__ import annotations
from unittest.mock import patch
from abogen.domain.voice_utils import (
coerce_truthy,
formula_from_kokoro_entry,
infer_provider_from_spec,
resolve_voice_target,
split_speaker_reference,
supertonic_voice_from_spec,
)
class TestSplitSpeakerReference:
def test_speaker_prefix(self):
assert split_speaker_reference("speaker:af_sarah") == ("af_sarah", "speaker:af_sarah")
def test_profile_prefix(self):
assert split_speaker_reference("profile:custom") == ("custom", "profile:custom")
def test_no_prefix(self):
assert split_speaker_reference("af_sarah") == (None, "af_sarah")
def test_empty(self):
assert split_speaker_reference("") == (None, "")
def test_none(self):
assert split_speaker_reference(None) == (None, "")
def test_unknown_prefix(self):
assert split_speaker_reference("unknown:name") == (None, "unknown:name")
def test_empty_name_after_colon(self):
assert split_speaker_reference("speaker:") == (None, "speaker:")
class TestSupertonicVoiceFromSpec:
def test_uppercase_passthrough(self):
assert supertonic_voice_from_spec("M1", "M1") == "M1"
def test_lowercase_converted(self):
assert supertonic_voice_from_spec("m1", "M1") == "M1"
def test_empty_spec_uses_fallback(self):
assert supertonic_voice_from_spec("", "F1") == "F1"
def test_formula_spec_uses_fallback(self):
assert supertonic_voice_from_spec("af_sarah*0.5+bf_emma*0.5", "M1") == "M1"
def test_empty_both_gives_default(self):
assert supertonic_voice_from_spec("", "") == "M1"
class TestFormulaFromKokoroEntry:
def test_single_voice(self):
entry = {"voices": [["af_sarah", 1.0]]}
result = formula_from_kokoro_entry(entry)
assert "af_sarah" in result
assert "1.000000" in result
def test_weighted_mix(self):
entry = {"voices": [["af_sarah", 0.6], ["bf_emma", 0.4]]}
result = formula_from_kokoro_entry(entry)
assert "af_sarah" in result
assert "bf_emma" in result
assert "+" in result
def test_empty_voices(self):
assert formula_from_kokoro_entry({"voices": []}) == ""
def test_missing_voices_key(self):
assert formula_from_kokoro_entry({}) == ""
def test_invalid_entries_filtered(self):
entry = {"voices": [["af_sarah", "bad"], ["bf_emma", 0.5]]}
result = formula_from_kokoro_entry(entry)
assert "bf_emma" in result
assert "af_sarah" not in result
class TestCoerceTruthy:
def test_bool_passthrough(self):
assert coerce_truthy(True) is True
assert coerce_truthy(False) is False
def test_string_true(self):
assert coerce_truthy("yes") is True
assert coerce_truthy("1") is True
def test_string_false(self):
assert coerce_truthy("false") is False
assert coerce_truthy("0") is False
assert coerce_truthy("") is False
def test_none_default(self):
assert coerce_truthy(None) is True
assert coerce_truthy(None, False) is False
def test_int(self):
assert coerce_truthy(1) is True
assert coerce_truthy(0) is False
class TestInferProviderFromSpec:
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah", "bf_emma"])
def test_known_kokoro_voice(self, _mock):
assert infer_provider_from_spec("af_sarah") == "kokoro"
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_uppercase_supertonic(self, _mock):
assert infer_provider_from_spec("M1") == "supertonic"
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_formula_kokoro(self, _mock):
assert infer_provider_from_spec("af_sarah*0.5+bf_emma*0.5") == "kokoro"
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_empty_fallback(self, _mock):
assert infer_provider_from_spec("", "kokoro") == "kokoro"
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_unknown_falls_back(self, _mock):
assert infer_provider_from_spec("unknown_xyz", "supertonic") == "supertonic"
class TestResolveVoiceTarget:
def test_empty_spec_kokoro_default(self):
provider, spec, speed, steps = resolve_voice_target(
"", {}, job_voice="af_sarah", job_tts_provider="kokoro",
)
assert provider == "kokoro"
assert spec == ""
def test_speaker_profile_kokoro(self):
profiles = {
"narrator": {
"provider": "kokoro",
"voices": [["af_sarah", 0.7], ["bf_emma", 0.3]],
},
}
provider, spec, speed, steps = resolve_voice_target(
"speaker:narrator", profiles,
)
assert provider == "kokoro"
assert "af_sarah" in spec
assert speed is None
assert steps is None
def test_speaker_profile_supertonic(self):
profiles = {
"narrator": {
"provider": "supertonic",
"voice": "F1",
"speed": 1.2,
"total_steps": 10,
},
}
provider, spec, speed, steps = resolve_voice_target(
"speaker:narrator", profiles,
job_voice="M1", job_speed=1.0, job_supertonic_total_steps=5,
)
assert provider == "supertonic"
assert spec == "F1"
assert speed == 1.2
assert steps == 10
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_direct_supertonic_spec(self, _mock):
provider, spec, speed, steps = resolve_voice_target(
"M1", {},
job_voice="M1",
)
assert provider == "supertonic"
assert spec == "M1"
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_direct_kokoro_spec(self, _mock):
provider, spec, speed, steps = resolve_voice_target(
"af_sarah", {},
job_tts_provider="kokoro",
)
assert provider == "kokoro"
assert spec == "af_sarah"
def test_profile_missing_provider_defaults_kokoro(self):
profiles = {
"narrator": {
"voices": [["af_sarah", 1.0]],
},
}
provider, spec, speed, steps = resolve_voice_target(
"speaker:narrator", profiles,
)
assert provider == "kokoro"
+163
View File
@@ -0,0 +1,163 @@
"""Tests for ExportService FFmpeg metadata methods."""
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from pathlib import Path
from abogen.infrastructure.exporters import ExportService
class TestEscapeFfmetadataValue:
def setup_method(self):
self.svc = ExportService()
def test_simple_string(self):
assert self.svc._escape_ffmetadata_value("hello") == "hello"
def test_escapes_backslash(self):
assert self.svc._escape_ffmetadata_value("a\\b") == "a\\\\b"
def test_escapes_newline(self):
assert self.svc._escape_ffmetadata_value("line1\nline2") == "line1\\nline2"
def test_escapes_equals(self):
assert self.svc._escape_ffmetadata_value("key=value") == "key\\=value"
def test_escapes_semicolon(self):
assert self.svc._escape_ffmetadata_value("a;b") == "a\\;b"
def test_escapes_hash(self):
assert self.svc._escape_ffmetadata_value("#comment") == "\\#comment"
def test_escapes_all_special(self):
result = self.svc._escape_ffmetadata_value("a\\b\nc=d;e#f")
assert "\\\\" in result
assert "\\n" in result
assert "\\=" in result
assert "\\;" in result
assert "\\#" in result
def test_empty_string(self):
assert self.svc._escape_ffmetadata_value("") == ""
class TestRenderFfmetadata:
def setup_method(self):
self.svc = ExportService()
def test_renders_header(self):
result = self.svc.render_ffmetadata({"title": "My Book"}, [])
assert result.startswith(";FFMETADATA1\n")
assert "title=My Book\n" in result
def test_renders_multiple_keys(self):
result = self.svc.render_ffmetadata({"title": "T", "artist": "A"}, [])
assert "title=T\n" in result
assert "artist=A\n" in result
def test_skips_none_values(self):
result = self.svc.render_ffmetadata({"title": None}, [])
assert "title=" not in result
def test_renders_chapters(self):
chapters = [{"start": 0.0, "end": 10.0, "title": "Ch 1"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "[CHAPTER]" in result
assert "TIMEBASE=1/1000" in result
assert "START=0" in result
assert "END=10000" in result
assert "title=Ch 1" in result
def test_renders_voice_in_chapter(self):
chapters = [{"start": 0.0, "end": 5.0, "voice": "af_heart"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "voice=af_heart" in result
def test_skips_chapters_without_times(self):
chapters = [{"title": "No times"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "[CHAPTER]" not in result
def test_end_equals_start_gets_minimum_duration(self):
chapters = [{"start": 5.0, "end": 5.0, "title": "Zero"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "START=5000" in result
assert "END=5001" in result
def test_empty_metadata_and_chapters(self):
result = self.svc.render_ffmetadata({}, [])
assert result.strip() == ";FFMETADATA1"
def test_escapes_special_chars_in_title(self):
chapters = [{"start": 0.0, "end": 1.0, "title": "A=B;C#D"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "\\=" in result
assert "\\;" in result
assert "\\#" in result
def test_negative_start_clamped_to_zero(self):
chapters = [{"start": -1.0, "end": 5.0, "title": "Neg"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "START=0" in result
class TestMetadataToFfmpegArgs:
def setup_method(self):
self.svc = ExportService()
def test_simple_metadata(self):
args = self.svc._metadata_to_ffmpeg_args({"title": "My Book"})
assert args == ["-metadata", "title=My Book"]
def test_year_becomes_date(self):
args = self.svc._metadata_to_ffmpeg_args({"year": "2024"})
assert args == ["-metadata", "date=2024"]
def test_skips_none_and_empty(self):
args = self.svc._metadata_to_ffmpeg_args({"title": None, "artist": ""})
assert args == []
def test_skips_empty_key(self):
args = self.svc._metadata_to_ffmpeg_args({"": "value"})
assert args == []
def test_multiple_keys(self):
args = self.svc._metadata_to_ffmpeg_args({"title": "T", "artist": "A"})
assert "-metadata" in args
assert "title=T" in args
assert "artist=A" in args
def test_empty_metadata(self):
assert self.svc._metadata_to_ffmpeg_args({}) == []
def test_none_metadata(self):
assert self.svc._metadata_to_ffmpeg_args(None) == []
class TestWriteFfmetadataFile:
def setup_method(self):
self.svc = ExportService()
def test_writes_file(self, tmp_path):
audio = tmp_path / "test.mp3"
audio.touch()
meta = {"title": "My Book"}
chapters = [{"start": 0.0, "end": 5.0, "title": "Ch 1"}]
result = self.svc.write_ffmetadata_file(audio, meta, chapters)
assert result is not None
assert result.exists()
content = result.read_text()
assert ";FFMETADATA1" in content
assert "title=My Book" in content
def test_returns_none_for_empty(self, tmp_path):
audio = tmp_path / "test.mp3"
audio.touch()
result = self.svc.write_ffmetadata_file(audio, {}, [])
assert result is None
def test_returns_none_for_only_header(self, tmp_path):
audio = tmp_path / "test.mp3"
audio.touch()
result = self.svc.write_ffmetadata_file(audio, None, None)
assert result is None
+5 -4
View File
@@ -1,8 +1,9 @@
from __future__ import annotations from __future__ import annotations
from pathlib import Path from abogen.infrastructure.exporters import ExportService
from abogen.webui.conversion_runner import _render_ffmetadata, _write_ffmetadata_file
svc = ExportService()
def test_render_ffmetadata_includes_chapters(tmp_path): def test_render_ffmetadata_includes_chapters(tmp_path):
@@ -17,7 +18,7 @@ def test_render_ffmetadata_includes_chapters(tmp_path):
{"start": 5.0, "end": 12.345, "title": "Chapter 2"}, {"start": 5.0, "end": 12.345, "title": "Chapter 2"},
] ]
rendered = _render_ffmetadata(metadata, chapters) rendered = svc.render_ffmetadata(metadata, chapters)
assert ";FFMETADATA1" in rendered assert ";FFMETADATA1" in rendered
assert "title=Sample Book" in rendered assert "title=Sample Book" in rendered
@@ -30,7 +31,7 @@ def test_render_ffmetadata_includes_chapters(tmp_path):
assert "voice=voice_a" in rendered assert "voice=voice_a" in rendered
audio_path = tmp_path / "book.m4b" audio_path = tmp_path / "book.m4b"
metadata_path = _write_ffmetadata_file(audio_path, metadata, chapters) metadata_path = svc.write_ffmetadata_file(audio_path, metadata, chapters)
assert metadata_path is not None assert metadata_path is not None
assert metadata_path.exists() assert metadata_path.exists()
+3 -1
View File
@@ -38,8 +38,10 @@ from tests.contracts.engine_contract import EngineContractMixin
def _kokoro_available() -> bool: def _kokoro_available() -> bool:
try: try:
from kokoro import KPipeline # type: ignore[import-not-found] from kokoro import KPipeline # type: ignore[import-not-found]
import spacy
spacy.load("en_core_web_sm")
return True return True
except ImportError: except (ImportError, OSError):
return False return False
+199
View File
@@ -0,0 +1,199 @@
"""Tests for abogen.domain.metadata_extraction module."""
import pytest
from abogen.domain.metadata_extraction import (
extract_metadata_from_text,
get_filename_from_path,
build_ffmpeg_metadata_args,
extract_metadata_and_build_args,
read_text_for_metadata,
)
class TestExtractMetadataFromText:
"""Tests for extract_metadata_from_text function."""
def test_extract_all_metadata(self):
"""Test extracting all metadata tags."""
text = """
<<METADATA_TITLE:Test Book>>
<<METADATA_ARTIST:Test Author>>
<<METADATA_ALBUM:Test Album>>
<<METADATA_YEAR:2024>>
<<METADATA_ALBUM_ARTIST:Album Artist>>
<<METADATA_COMPOSER:Composer Name>>
<<METADATA_GENRE:Fiction>>
<<METADATA_COVER_PATH:/path/to/cover.jpg>>
"""
metadata = extract_metadata_from_text(text)
assert metadata["title"] == "Test Book"
assert metadata["artist"] == "Test Author"
assert metadata["album"] == "Test Album"
assert metadata["year"] == "2024"
assert metadata["album_artist"] == "Album Artist"
assert metadata["composer"] == "Composer Name"
assert metadata["genre"] == "Fiction"
assert metadata["cover_path"] == "/path/to/cover.jpg"
def test_extract_partial_metadata(self):
"""Test extracting partial metadata."""
text = "<<METADATA_TITLE:Only Title>>"
metadata = extract_metadata_from_text(text)
assert metadata["title"] == "Only Title"
assert metadata["artist"] is None
assert metadata["cover_path"] is None
def test_empty_text(self):
"""Test extracting from empty text."""
metadata = extract_metadata_from_text("")
for key in metadata:
assert metadata[key] is None
def test_no_tags(self):
"""Test text without metadata tags."""
text = "This is just regular text without any metadata tags."
metadata = extract_metadata_from_text(text)
for key in metadata:
assert metadata[key] is None
def test_strip_whitespace(self):
"""Test that values are stripped of whitespace."""
text = "<<METADATA_TITLE: Test Book >>"
metadata = extract_metadata_from_text(text)
assert metadata["title"] == "Test Book"
class TestGetFilenameFromPath:
"""Tests for get_filename_from_path function."""
def test_simple_path(self):
"""Test extracting filename from simple path."""
filename = get_filename_from_path("/path/to/file.txt")
assert filename == "file"
def test_path_with_multiple_extensions(self):
"""Test extracting filename from path with multiple extensions."""
filename = get_filename_from_path("/path/to/file.tar.gz")
assert filename == "file.tar"
def test_windows_path(self):
"""Test extracting filename from Windows path."""
# Note: This test may behave differently on Windows vs Unix
# but should work correctly on the current platform
filename = get_filename_from_path("C:\\path\\to\\file.txt")
assert "file" in filename
def test_with_display_path(self):
"""Test using display_path when not from_queue."""
filename = get_filename_from_path(
file_path="/original/path/file.txt",
display_path="/display/path/display_file.txt",
from_queue=False,
)
assert filename == "display_file"
def test_with_display_path_from_queue(self):
"""Test ignoring display_path when from_queue."""
filename = get_filename_from_path(
file_path="/original/path/file.txt",
display_path="/display/path/display_file.txt",
from_queue=True,
)
assert filename == "file"
class TestBuildFfmpegMetadataArgs:
"""Tests for build_ffmpeg_metadata_args function."""
def test_all_metadata_provided(self):
"""Test building args with all metadata provided."""
metadata = {
"title": "Test Title",
"artist": "Test Artist",
"album": "Test Album",
"year": "2024",
"album_artist": "Album Artist",
"composer": "Composer",
"genre": "Fiction",
}
args = build_ffmpeg_metadata_args(metadata, "fallback")
assert "-metadata" in args
assert "title=Test Title" in args
assert "artist=Test Artist" in args
assert "album=Test Album" in args
assert "date=2024" in args # year -> date
def test_use_defaults(self):
"""Test that defaults are used for missing metadata."""
metadata = {} # Empty metadata
args = build_ffmpeg_metadata_args(metadata, "mybook")
# Should use defaults
assert any("title=mybook" in arg for arg in args)
assert any("artist=Unknown" in arg for arg in args)
assert any("genre=Audiobook" in arg for arg in args)
def test_empty_values_skipped(self):
"""Test that empty values are skipped."""
metadata = {
"title": "Test",
"artist": "",
"album": None,
}
args = build_ffmpeg_metadata_args(metadata, "fallback")
# Should have title but not artist/album (empty)
assert any("title=Test" in arg for arg in args)
class TestExtractMetadataAndBuildArgs:
"""Tests for extract_metadata_and_build_args function."""
def test_full_workflow(self):
"""Test full metadata extraction and arg building."""
text = "<<METADATA_TITLE:My Book>>\n<<METADATA_ARTIST:My Author>>"
args, cover_path = extract_metadata_and_build_args(
text=text,
filename="mybook.txt",
)
assert any("title=My Book" in arg for arg in args)
assert any("artist=My Author" in arg for arg in args)
assert cover_path is None
def test_with_cover_path(self):
"""Test extraction with cover path."""
text = "<<METADATA_COVER_PATH:/covers/cover.jpg>>"
args, cover_path = extract_metadata_and_build_args(
text=text,
filename="mybook.txt",
)
assert cover_path == "/covers/cover.jpg"
class TestReadTextForMetadata:
"""Tests for read_text_for_metadata function."""
def test_direct_text(self):
"""Test reading direct text."""
text = read_text_for_metadata(
file_path="This is direct text",
is_direct_text=True,
)
assert text == "This is direct text"
def test_file_not_found(self):
"""Test handling of file not found."""
text = read_text_for_metadata(
file_path="/nonexistent/file.txt",
is_direct_text=False,
)
assert text == ""
+133
View File
@@ -0,0 +1,133 @@
"""Tests for domain/metadata_helpers.py."""
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from abogen.domain.metadata_helpers import (
normalize_metadata_map,
format_author_sentence,
ensure_sentence,
normalize_series_number,
extract_series_metadata,
format_series_sentence,
)
class TestNormalizeMetadataMap:
def test_empty(self):
assert normalize_metadata_map({}) == {}
def test_none(self):
assert normalize_metadata_map(None) == {}
def test_normalizes_keys(self):
result = normalize_metadata_map({"Title": "My Book", "artist": "John"})
assert "title" in result
assert "artist" in result
def test_skips_none_values(self):
result = normalize_metadata_map({"title": None, "artist": "John"})
assert "title" not in result
def test_skips_empty_values(self):
result = normalize_metadata_map({"title": "", "artist": "John"})
assert "title" not in result
class TestFormatAuthorSentence:
def test_none(self):
assert format_author_sentence(None) == ""
def test_empty(self):
assert format_author_sentence("") == ""
def test_unknown(self):
assert format_author_sentence("Unknown") == ""
def test_single(self):
assert format_author_sentence("John Doe") == "By John Doe"
def test_two(self):
assert format_author_sentence("John, Jane") == "By John and Jane"
def test_three(self):
assert format_author_sentence("John, Jane, Bob") == "By John, Jane, and Bob"
def test_ampersand(self):
assert format_author_sentence("John & Jane") == "By John and Jane"
class TestEnsureSentence:
def test_empty(self):
assert ensure_sentence("") == ""
def test_already_sentence(self):
assert ensure_sentence("Hello.") == "Hello."
def test_adds_period(self):
assert ensure_sentence("Hello") == "Hello."
def test_exclamation(self):
assert ensure_sentence("Hello!") == "Hello!"
class TestNormalizeSeriesNumber:
def test_empty(self):
assert normalize_series_number("") is None
def test_integer(self):
assert normalize_series_number("3") == "3"
def test_float(self):
assert normalize_series_number("3.5") == "3.5"
def test_float_trailing_zero(self):
assert normalize_series_number("3.10") == "3.1"
def test_comma_as_separator(self):
assert normalize_series_number("3,5") == "3.5"
def test_text_with_number(self):
assert normalize_series_number("Book 3") == "3"
def test_none(self):
assert normalize_series_number(None) is None
class TestExtractSeriesMetadata:
def test_empty(self):
name, number = extract_series_metadata({})
assert name is None
assert number is None
def test_series_name(self):
name, number = extract_series_metadata({"series": "My Series"})
assert name == "My Series"
assert number is None
def test_series_number(self):
name, number = extract_series_metadata({"series_index": "3"})
assert name is None
assert number == "3"
def test_both(self):
name, number = extract_series_metadata({"series": "My Series", "series_index": "3"})
assert name == "My Series"
assert number == "3"
class TestFormatSeriesSentence:
def test_empty(self):
assert format_series_sentence(None, None) == ""
def test_name_only(self):
assert format_series_sentence("My Series", None) == ""
def test_number_only(self):
assert format_series_sentence(None, "3") == ""
def test_both(self):
assert format_series_sentence("My Series", "3") == "Book 3 of the My Series"
def test_with_the(self):
assert format_series_sentence("The Lord of the Rings", "1") == "Book 1 of The Lord of the Rings"
+170
View File
@@ -0,0 +1,170 @@
"""Tests for domain/metadata_helpers.py Audiobookshelf helpers."""
from abogen.domain.metadata_helpers import (
normalize_metadata_casefold,
split_people_field,
split_simple_list,
first_nonempty,
extract_year,
normalize_series_sequence,
build_audiobookshelf_metadata,
load_audiobookshelf_chapters,
)
# --- normalize_metadata_casefold ---
def test_normalize_metadata_casefold_basic():
result = normalize_metadata_casefold({"Title": " Hello ", "Author": None, "": "skip"})
assert result == {"title": "Hello", "author": ""} or result == {"title": "Hello"}
def test_normalize_metadata_casefold_preserves_lists():
result = normalize_metadata_casefold({"tags": ["a", "b"]})
assert result["tags"] == ["a", "b"]
# --- split_people_field ---
def test_split_people_field_single():
assert split_people_field("J.K. Rowling") == ["J.K. Rowling"]
def test_split_people_field_multiple():
result = split_people_field("Tolkien, Lewis & Martin")
assert "Tolkien" in result
assert "Lewis" in result
assert "Martin" in result
def test_split_people_field_deduplicates():
result = split_people_field("Tolkien, tolkien, TOLKIEN")
assert len(result) == 1
def test_split_people_field_none():
assert split_people_field(None) == []
def test_split_people_field_list():
result = split_people_field(["Author A", "Author B"])
assert result == ["Author A", "Author B"]
# --- split_simple_list ---
def test_split_simple_list_basic():
result = split_simple_list("fantasy, sci-fi; thriller")
assert "fantasy" in result
assert "sci-fi" in result
assert "thriller" in result
def test_split_simple_list_none():
assert split_simple_list(None) == []
# --- first_nonempty ---
def test_first_nonempty_basic():
assert first_nonempty(None, "", "hello") == "hello"
def test_first_nonempty_first_wins():
assert first_nonempty("first", "second") == "first"
def test_first_nonempty_none():
assert first_nonempty(None, None) is None
def test_first_nonempty_list():
assert first_nonempty(None, ["a", "b"]) == "a"
# --- extract_year ---
def test_extract_year_full_date():
assert extract_year("Published on 2023-05-15") == 2023
def test_extract_year_plain():
assert extract_year("2020") == 2020
def test_extract_year_none():
assert extract_year(None) is None
def test_extract_year_invalid():
assert extract_year("no year here") is None
# --- normalize_series_sequence ---
def test_normalize_series_sequence_int():
assert normalize_series_sequence(3) == "3"
def test_normalize_series_sequence_float():
assert normalize_series_sequence(2.5) == "2.5"
def test_normalize_series_sequence_string():
assert normalize_series_sequence(" 12 ") == "12"
def test_normalize_series_sequence_comma():
assert normalize_series_sequence("1,5") == "1.5"
def test_normalize_series_sequence_none():
assert normalize_series_sequence(None) is None
def test_normalize_series_sequence_nan():
import math
assert normalize_series_sequence(float("nan")) is None
# --- build_audiobookshelf_metadata ---
def test_build_audiobookshelf_metadata_basic():
tags = {
"title": "My Book",
"author": "Author Name",
"description": "A great book",
}
result = build_audiobookshelf_metadata(tags, language="en")
assert result["title"] == "My Book"
assert result["authors"] == ["Author Name"]
assert result["language"] == "en"
def test_build_audiobookshelf_metadata_series():
tags = {
"title": "Book 2",
"series": "My Series",
"series_index": "2",
}
result = build_audiobookshelf_metadata(tags, language="en")
assert result["seriesName"] == "My Series"
assert result["seriesSequence"] == "2"
def test_build_audiobookshelf_metadata_fallback_title():
tags = {"author": "Someone"}
result = build_audiobookshelf_metadata(tags, language="en", filename="chapter1")
assert result["title"] == "chapter1"
def test_build_audiobookshelf_metadata_empty():
result = build_audiobookshelf_metadata({}, language="en")
assert result["language"] == "en"
assert "authors" not in result # empty list stripped
def test_build_audiobookshelf_metadata_strips_empty():
tags = {"title": "Book", "subtitle": "", "description": None}
result = build_audiobookshelf_metadata(tags, language="en")
assert "subtitle" not in result
assert "description" not in result
+239 -61
View File
@@ -1,79 +1,257 @@
"""Tests for output path utilities.
Tests import from domain/output_paths.py (new module).
"""
from __future__ import annotations from __future__ import annotations
import time import re
from datetime import datetime
from pathlib import Path from pathlib import Path
from types import SimpleNamespace
import pytest import pytest
from abogen.webui.conversion_runner import _build_output_path, _prepare_project_layout
from abogen.webui.service import Job # ---------------------------------------------------------------------------
# slugify
# ---------------------------------------------------------------------------
def _sample_job(tmp_path: Path) -> Job: class TestSlugify:
source = tmp_path / "sample.txt" """slugify converts title to filesystem-safe slug."""
source.write_text("example", encoding="utf-8")
return Job( def test_basic_slug(self):
id="job-1", from abogen.domain.output_paths import slugify
original_filename="Sample Title.txt",
stored_path=source, assert slugify("Hello World", 0) == "hello_world"
language="en",
voice="af_alloy", def test_strips_special_chars(self):
speed=1.0, from abogen.domain.output_paths import slugify
use_gpu=False,
subtitle_mode="Sentence", result = slugify("Chapter 1: The Beginning!", 0)
output_format="mp3", assert result == "chapter_1_the_beginning"
def test_empty_title_uses_index(self):
from abogen.domain.output_paths import slugify
assert slugify("", 5) == "chapter_05"
def test_truncated_at_80(self):
from abogen.domain.output_paths import slugify
long_title = "a" * 100
assert len(slugify(long_title, 0)) == 80
def test_preserves_hyphens(self):
from abogen.domain.output_paths import slugify
assert slugify("mid-night", 0) == "mid-night"
# ---------------------------------------------------------------------------
# sanitize_output_stem
# ---------------------------------------------------------------------------
class TestSanitizeOutputStem:
"""sanitize_output_stem cleans filename stem."""
def test_basic_sanitize(self):
from abogen.domain.output_paths import sanitize_output_stem
assert sanitize_output_stem("my file.mp3") == "my_file"
def test_empty_returns_default(self):
from abogen.domain.output_paths import sanitize_output_stem
assert sanitize_output_stem("") == "output"
def test_strips_underscores(self):
from abogen.domain.output_paths import sanitize_output_stem
assert sanitize_output_stem("__test__") == "test"
# ---------------------------------------------------------------------------
# output_timestamp_token
# ---------------------------------------------------------------------------
class TestOutputTimestampToken:
"""output_timestamp_token returns timestamp string."""
def test_format(self):
from abogen.domain.output_paths import output_timestamp_token
token = output_timestamp_token()
assert re.match(r"\d{8}-\d{6}", token)
# ---------------------------------------------------------------------------
# build_output_path
# ---------------------------------------------------------------------------
class TestBuildOutputPath:
"""build_output_path builds the output file path."""
def test_basic_path(self, tmp_path):
from abogen.domain.output_paths import build_output_path
result = build_output_path(tmp_path, "test.mp3", "mp3")
assert result.suffix == ".mp3"
assert result.parent == tmp_path
def test_stem_sanitized(self, tmp_path):
from abogen.domain.output_paths import build_output_path
result = build_output_path(tmp_path, "my file.txt", "wav")
assert "my_file" in result.name
# ---------------------------------------------------------------------------
# apply_newline_policy
# ---------------------------------------------------------------------------
class TestApplyNewlinePolicy:
"""apply_newline_policy replaces single newlines in chapter text."""
def test_noop_when_disabled(self):
from abogen.domain.output_paths import apply_newline_policy
from abogen.text_extractor import ExtractedChapter
chapters = [ExtractedChapter(title="t", text="a\nb")]
apply_newline_policy(chapters, False)
assert chapters[0].text == "a\nb"
def test_replaces_single_newlines(self):
from abogen.domain.output_paths import apply_newline_policy
from abogen.text_extractor import ExtractedChapter
chapters = [ExtractedChapter(title="t", text="a\nb\nc")]
apply_newline_policy(chapters, True)
assert chapters[0].text == "a b c"
def test_preserves_double_newlines(self):
from abogen.domain.output_paths import apply_newline_policy
from abogen.text_extractor import ExtractedChapter
chapters = [ExtractedChapter(title="t", text="a\n\nb")]
apply_newline_policy(chapters, True)
assert chapters[0].text == "a\n\nb"
# ---------------------------------------------------------------------------
# resolve_output_directory
# ---------------------------------------------------------------------------
class TestResolveOutputDirectory:
"""resolve_output_directory determines output dir from save_mode."""
def test_save_to_desktop(self, tmp_path):
from abogen.domain.output_paths import resolve_output_directory
result = resolve_output_directory(
save_mode="Save to Desktop",
stored_path=Path("/input/book.epub"),
output_folder=None,
desktop_dir=tmp_path,
user_output_path=None,
user_cache_outputs=None,
)
assert result == tmp_path
def test_save_next_to_input(self, tmp_path):
from abogen.domain.output_paths import resolve_output_directory
stored = tmp_path / "book.epub"
result = resolve_output_directory(
save_mode="Save next to input file",
stored_path=stored,
output_folder=None,
desktop_dir=None,
user_output_path=None,
user_cache_outputs=None,
)
assert result == tmp_path
def test_choose_output_folder(self, tmp_path):
from abogen.domain.output_paths import resolve_output_directory
custom = tmp_path / "custom"
result = resolve_output_directory(
save_mode="Choose output folder",
stored_path=Path("/x/y.epub"),
output_folder=str(custom),
desktop_dir=None,
user_output_path=None,
user_cache_outputs=None,
)
assert result == custom
def test_use_default_save_location(self, tmp_path):
from abogen.domain.output_paths import resolve_output_directory
result = resolve_output_directory(
save_mode="Use default save location", save_mode="Use default save location",
output_folder=tmp_path, stored_path=Path("/x/y.epub"),
replace_single_newlines=False, output_folder=None,
subtitle_format="srt", desktop_dir=None,
created_at=time.time(), user_output_path=tmp_path / "default",
user_cache_outputs=None,
) )
assert result == tmp_path / "default"
def test_fallback_to_cache(self, tmp_path):
from abogen.domain.output_paths import resolve_output_directory
def test_prepare_project_layout_uses_timestamped_folder( result = resolve_output_directory(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path save_mode="unknown",
) -> None: stored_path=Path("/x/y.epub"),
job = _sample_job(tmp_path) output_folder=None,
monkeypatch.setattr( desktop_dir=None,
"abogen.webui.conversion_runner._output_timestamp_token", user_output_path=None,
lambda: "20250101-120000", user_cache_outputs=tmp_path / "cache",
) )
assert result == tmp_path / "cache"
project_root, audio_dir, subtitle_dir, metadata_dir = _prepare_project_layout(
job, tmp_path # ---------------------------------------------------------------------------
# resolve_project_layout
# ---------------------------------------------------------------------------
class TestResolveProjectLayout:
"""resolve_project_layout computes project folder structure."""
def test_flat_layout(self, tmp_path):
from abogen.domain.output_paths import resolve_project_layout
root, audio, subs, meta = resolve_project_layout(
original_filename="book.epub",
save_as_project=False,
base_dir=tmp_path,
timestamp_fn=lambda: "20260101-000000",
sanitize_fn=lambda n, i: "book",
) )
assert audio == root
assert subs == root
assert meta is None
assert project_root.name.startswith( def test_project_layout(self, tmp_path):
"20250101-120000_Sample_Title" from abogen.domain.output_paths import resolve_project_layout
), project_root.name
assert audio_dir == project_root
assert subtitle_dir == project_root
assert metadata_dir is None
output_path = _build_output_path(audio_dir, job.original_filename, "mp3") root, audio, subs, meta = resolve_project_layout(
assert output_path == project_root / "Sample_Title.mp3" original_filename="book.epub",
save_as_project=True,
base_dir=tmp_path,
def test_prepare_project_layout_creates_project_subdirs( timestamp_fn=lambda: "20260101-000000",
monkeypatch: pytest.MonkeyPatch, tmp_path: Path sanitize_fn=lambda n, i: "book",
) -> None:
job = _sample_job(tmp_path)
job.save_as_project = True
monkeypatch.setattr(
"abogen.webui.conversion_runner._output_timestamp_token",
lambda: "20250101-120500",
) )
assert root.name == "20260101-000000_book"
project_root, audio_dir, subtitle_dir, metadata_dir = _prepare_project_layout( assert audio.name == "audio"
job, tmp_path assert subs.name == "subtitles"
) assert meta.name == "metadata"
assert audio_dir == project_root / "audio"
assert subtitle_dir == project_root / "subtitles"
assert metadata_dir == project_root / "metadata"
assert audio_dir.is_dir()
assert subtitle_dir.is_dir()
assert metadata_dir is not None and metadata_dir.is_dir()
output_path = _build_output_path(audio_dir, job.original_filename, "wav")
assert output_path == audio_dir / "Sample_Title.wav"
@@ -1,11 +1,11 @@
from abogen.webui.routes.utils import preview from abogen.webui.routes.utils import synthesize
def test_preview_applies_manual_override_before_normalization(monkeypatch): def test_preview_applies_manual_override_before_normalization(monkeypatch):
# Don't run real TTS/normalization; just exercise the override stage by # Don't run real TTS/normalization; just exercise the override stage by
# forcing provider=kokoro and then stubbing normalize_for_pipeline. # forcing provider=kokoro and then stubbing normalize_for_pipeline.
monkeypatch.setattr(preview, "get_preview_pipeline", lambda language, device: None) monkeypatch.setattr(synthesize, "get_preview_pipeline", lambda language, device: None)
# Stub normalize_for_pipeline to be identity; we only care that overrides run. # Stub normalize_for_pipeline to be identity; we only care that overrides run.
class _Norm: class _Norm:
@@ -42,7 +42,7 @@ def test_preview_applies_manual_override_before_normalization(monkeypatch):
monkeypatch.setattr(utils, "create_pipeline", _mock_create_pipeline) monkeypatch.setattr(utils, "create_pipeline", _mock_create_pipeline)
try: try:
preview.generate_preview_audio( synthesize.generate_preview_audio(
text="He said Unfu*k loudly.", text="He said Unfu*k loudly.",
voice_spec="M1", voice_spec="M1",
language="en", language="en",
+334
View File
@@ -0,0 +1,334 @@
"""Tests for abogen.domain.pronunciation — compile/apply pronunciation rules."""
from __future__ import annotations
import re
import pytest
# ---------------------------------------------------------------------------
# We import the domain functions. The module must be created first.
# For now the tests are written against the expected public API so they can
# serve as the contract during extraction.
# ---------------------------------------------------------------------------
class TestCompilePronunciationRules:
"""compile_pronunciation_rules turns override dicts into regex-based rules."""
def test_empty_input(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
assert compile_pronunciation_rules(None) == []
assert compile_pronunciation_rules([]) == []
def test_single_entry(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [{"token": "albeit", "pronunciation": "all be it"}]
rules = compile_pronunciation_rules(overrides)
assert len(rules) == 1
assert rules[0]["replacement"] == "all be it"
assert rules[0]["pattern"].search("albeit")
def test_skips_entries_without_pronunciation(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [{"token": "hello"}]
assert compile_pronunciation_rules(overrides) == []
def test_skips_entries_without_token(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [{"pronunciation": "foo"}]
assert compile_pronunciation_rules(overrides) == []
def test_deduplication_by_casefold(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [
{"token": "Albeit", "pronunciation": "all be it"},
{"token": "ALBEIT", "pronunciation": "all be it"},
]
rules = compile_pronunciation_rules(overrides)
assert len(rules) == 1
def test_longer_token_sorted_first(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [
{"token": "ice cream", "pronunciation": "ice cream"},
{"token": "ice", "pronunciation": "ais"},
]
rules = compile_pronunciation_rules(overrides)
assert len(rules) == 2
assert len(rules[0]["token"]) >= len(rules[1]["token"])
def test_normalized_fallback_to_entity_token(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [{"normalized": "USA", "pronunciation": "you ess ay"}]
rules = compile_pronunciation_rules(overrides)
assert len(rules) == 1
def test_pattern_is_case_insensitive(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [{"token": "hello", "pronunciation": "hi"}]
rules = compile_pronunciation_rules(overrides)
assert rules[0]["pattern"].search("Hello")
assert rules[0]["pattern"].search("HELLO")
def test_non_mapping_items_skipped(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = ["bad", None, 42]
assert compile_pronunciation_rules(overrides) == []
class TestCompileHeteronymSentenceRules:
"""compile_heteronym_sentence_rules builds sentence-level replacements."""
def test_empty_input(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
assert compile_heteronym_sentence_rules(None) == []
assert compile_heteronym_sentence_rules([]) == []
def test_basic_replacement(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [
{
"sentence": "I read the book",
"choice": "past",
"options": [
{"key": "present", "replacement_sentence": "I read the book"},
{"key": "past", "replacement_sentence": "I read the book"},
],
}
]
rules = compile_heteronym_sentence_rules(overrides)
assert len(rules) == 1
assert rules[0]["replacement"] == "I read the book"
def test_skips_without_sentence(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [{"choice": "a", "options": []}]
assert compile_heteronym_sentence_rules(overrides) == []
def test_skips_without_choice(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [{"sentence": "hello", "options": []}]
assert compile_heteronym_sentence_rules(overrides) == []
def test_skips_when_no_matching_option(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [
{
"sentence": "I read the book",
"choice": "past",
"options": [{"key": "present", "replacement_sentence": "I read the book"}],
}
]
assert compile_heteronym_sentence_rules(overrides) == []
def test_deduplication(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
entry = {
"sentence": "I read the book",
"choice": "past",
"options": [{"key": "past", "replacement_sentence": "I red the book"}],
}
rules = compile_heteronym_sentence_rules([entry, entry])
assert len(rules) == 1
def test_longer_sentence_sorted_first(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [
{
"sentence": "short",
"choice": "a",
"options": [{"key": "a", "replacement_sentence": "s"}],
},
{
"sentence": "a longer sentence here",
"choice": "b",
"options": [{"key": "b", "replacement_sentence": "l"}],
},
]
rules = compile_heteronym_sentence_rules(overrides)
assert len(rules[0]["pattern"].pattern) >= len(rules[1]["pattern"].pattern)
class TestApplyPronunciationRules:
"""apply_pronunciation_rules applies compiled token-level rules."""
def test_empty_text(self):
from abogen.domain.pronunciation import apply_pronunciation_rules
assert apply_pronunciation_rules("", []) == ""
def test_no_rules(self):
from abogen.domain.pronunciation import apply_pronunciation_rules
assert apply_pronunciation_rules("hello", []) == "hello"
def test_basic_replacement(self):
from abogen.domain.pronunciation import compile_pronunciation_rules, apply_pronunciation_rules
rules = compile_pronunciation_rules([{"token": "albeit", "pronunciation": "all be it"}])
result = apply_pronunciation_rules("albeit it was raining", rules)
assert result == "all be it it was raining"
def test_possessive_preserved(self):
from abogen.domain.pronunciation import compile_pronunciation_rules, apply_pronunciation_rules
rules = compile_pronunciation_rules([{"token": "dog", "pronunciation": "dawg"}])
result = apply_pronunciation_rules("the dog's bone", rules)
assert result == "the dawg's bone"
def test_usage_counter_increments(self):
from abogen.domain.pronunciation import compile_pronunciation_rules, apply_pronunciation_rules
rules = compile_pronunciation_rules([{"token": "hello", "pronunciation": "hi"}])
counter: dict[str, int] = {}
apply_pronunciation_rules("hello hello", rules, usage_counter=counter)
assert counter.get("hello", 0) == 2
def test_case_insensitive_match(self):
from abogen.domain.pronunciation import compile_pronunciation_rules, apply_pronunciation_rules
rules = compile_pronunciation_rules([{"token": "test", "pronunciation": "tst"}])
result = apply_pronunciation_rules("This is a Test", rules)
assert "tst" in result.lower()
class TestApplyHeteronymSentenceRules:
"""apply_heteronym_sentence_rules applies sentence-level replacements."""
def test_empty_text(self):
from abogen.domain.pronunciation import apply_heteronym_sentence_rules
assert apply_heteronym_sentence_rules("", []) == ""
def test_no_rules(self):
from abogen.domain.pronunciation import apply_heteronym_sentence_rules
assert apply_heteronym_sentence_rules("hello", []) == "hello"
def test_basic_replacement(self):
from abogen.domain.pronunciation import (
compile_heteronym_sentence_rules,
apply_heteronym_sentence_rules,
)
rules = compile_heteronym_sentence_rules(
[
{
"sentence": "I read the book",
"choice": "past",
"options": [{"key": "past", "replacement_sentence": "I read the book"}],
}
]
)
result = apply_heteronym_sentence_rules("I read the book.", rules)
assert result == "I read the book."
def test_no_match_left_unchanged(self):
from abogen.domain.pronunciation import (
compile_heteronym_sentence_rules,
apply_heteronym_sentence_rules,
)
rules = compile_heteronym_sentence_rules(
[
{
"sentence": "I read the book",
"choice": "past",
"options": [{"key": "past", "replacement_sentence": "I red the book"}],
}
]
)
result = apply_heteronym_sentence_rules("something else entirely", rules)
assert result == "something else entirely"
class TestMergePronunciationOverrides:
"""merge_pronunciation_overrides consolidates override sources."""
def test_empty_job(self):
from abogen.domain.pronunciation import merge_pronunciation_overrides
class FakeJob:
pronunciation_overrides = None
speakers = None
manual_overrides = None
language = "en"
result = merge_pronunciation_overrides(FakeJob())
assert result == []
def test_pronunciation_overrides_included(self):
from abogen.domain.pronunciation import merge_pronunciation_overrides
class FakeJob:
pronunciation_overrides = [
{"token": "hello", "pronunciation": "hi", "normalized": "hello"}
]
speakers = None
manual_overrides = None
language = "en"
result = merge_pronunciation_overrides(FakeJob())
assert len(result) == 1
assert result[0]["token"] == "hello"
assert result[0]["source"] == "pronunciation"
def test_manual_overrides_win(self):
from abogen.domain.pronunciation import merge_pronunciation_overrides
class FakeJob:
pronunciation_overrides = [
{"token": "hello", "pronunciation": "hi old", "normalized": "hello"}
]
speakers = None
manual_overrides = [
{"token": "hello", "pronunciation": "hi new", "normalized": "hello"}
]
language = "en"
result = merge_pronunciation_overrides(FakeJob())
assert len(result) == 1
assert result[0]["pronunciation"] == "hi new"
assert result[0]["source"] == "manual"
def test_speaker_entries_included(self):
from abogen.domain.pronunciation import merge_pronunciation_overrides
class FakeJob:
pronunciation_overrides = None
speakers = {"narrator": {"token": "war", "pronunciation": "wɔːr"}}
manual_overrides = None
language = "en"
result = merge_pronunciation_overrides(FakeJob())
assert len(result) == 1
assert result[0]["source"] == "speaker"
def test_skips_empty_tokens(self):
from abogen.domain.pronunciation import merge_pronunciation_overrides
class FakeJob:
pronunciation_overrides = [{"token": "", "pronunciation": "foo"}]
speakers = None
manual_overrides = None
language = "en"
result = merge_pronunciation_overrides(FakeJob())
assert result == []
+145
View File
@@ -0,0 +1,145 @@
"""Regression tests: domain extraction must not break webui conversion_runner.
These tests verify that the refactored WebUI code paths still call into the
correct domain functions and produce the same results as the old inline logic.
"""
from __future__ import annotations
from unittest.mock import MagicMock, patch
from abogen.domain.voice_utils import resolve_voice_target
from abogen.domain.pipeline_factory import PipelinePool
class TestResolveVoiceTargetRegression:
"""Verify that the domain resolve_voice_target produces the same results
as the old closure in conversion_runner.py."""
def test_empty_spec_returns_kokoro_default(self):
provider, spec, speed, steps = resolve_voice_target(
"", {}, job_voice="af_sarah", job_tts_provider="kokoro",
)
assert provider == "kokoro"
assert spec == ""
def test_speaker_in_profile_kokoro(self):
profiles = {
"narrator": {
"provider": "kokoro",
"voices": [["af_sarah", 0.7], ["bf_emma", 0.3]],
},
}
provider, spec, speed, steps = resolve_voice_target(
"speaker:narrator", profiles,
)
assert provider == "kokoro"
assert "af_sarah" in spec
assert speed is None
assert steps is None
def test_speaker_in_profile_supertonic(self):
profiles = {
"narrator": {
"provider": "supertonic",
"voice": "F1",
"speed": 1.2,
"total_steps": 10,
},
}
provider, spec, speed, steps = resolve_voice_target(
"speaker:narrator", profiles,
job_voice="M1", job_speed=1.0, job_supertonic_total_steps=5,
)
assert provider == "supertonic"
assert spec == "F1"
assert speed == 1.2
assert steps == 10
def test_unknown_speaker_infers_from_spec(self):
with patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"]):
provider, spec, speed, steps = resolve_voice_target(
"af_sarah", {}, job_tts_provider="kokoro",
)
assert provider == "kokoro"
assert spec == "af_sarah"
def test_uppercase_spec_infers_supertonic(self):
with patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"]):
provider, spec, speed, steps = resolve_voice_target(
"M1", {}, job_voice="M1",
)
assert provider == "supertonic"
assert spec == "M1"
class TestPipelinePoolRegression:
"""Verify that PipelinePool behaves like the old inline get_pipeline closure."""
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_same_provider_returns_cached_pipeline(self, _cache, mock_create):
mock_pipeline = MagicMock()
mock_create.return_value = mock_pipeline
pool = PipelinePool()
r1 = pool.get("kokoro", "en", use_gpu=True)
r2 = pool.get("kokoro", "en", use_gpu=True)
assert r1 is r2
assert mock_create.call_count == 1
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_different_providers_get_separate_pipelines(self, _cache, mock_create):
p1 = MagicMock(name="kokoro")
p2 = MagicMock(name="supertonic")
mock_create.side_effect = [p1, p2]
pool = PipelinePool()
r1 = pool.get("kokoro", "en", use_gpu=True)
r2 = pool.get("supertonic", "en", use_gpu=True)
assert r1 is p1
assert r2 is p2
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_dispose_all_cleans_up(self, _cache, mock_create):
p1 = MagicMock()
p2 = MagicMock()
mock_create.side_effect = [p1, p2]
pool = PipelinePool()
pool.get("kokoro", "en", use_gpu=True)
pool.get("supertonic", "en", use_gpu=True)
pool.dispose_all()
p1.dispose.assert_called_once()
p2.dispose.assert_called_once()
assert pool._pipelines == {}
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_voice_cache_initialized_only_once(self, mock_cache, mock_create):
mock_create.return_value = MagicMock()
pool = PipelinePool()
job = MagicMock()
pool.get("kokoro", "en", use_gpu=True, job=job)
pool.get("kokoro", "en", use_gpu=True, job=job)
assert mock_cache.call_count == 1
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_after_dispose_voice_cache_can_reinitialize(self, mock_cache, mock_create):
mock_create.return_value = MagicMock()
pool = PipelinePool()
job = MagicMock()
pool.get("kokoro", "en", use_gpu=True, job=job)
assert mock_cache.call_count == 1
pool.dispose_all()
assert pool._voice_cache_initialized is False
pool.get("kokoro", "en", use_gpu=True, job=job)
assert mock_cache.call_count == 2
+94
View File
@@ -0,0 +1,94 @@
import sys
from unittest.mock import patch, MagicMock
class TestSelectDevice:
"""Tests for domain.device.select_device."""
def test_returns_mps_on_apple_silicon_when_available(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Darwin"
mock_platform.processor.return_value = "arm"
mock_torch = MagicMock()
mock_torch.backends.mps.is_available.return_value = True
mock_torch.cuda.is_available.return_value = False
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": mock_torch}):
result = select_device()
assert result == "mps"
def test_returns_cpu_on_apple_silicon_when_mps_unavailable(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Darwin"
mock_platform.processor.return_value = "arm"
mock_torch = MagicMock()
mock_torch.backends.mps.is_available.return_value = False
mock_torch.cuda.is_available.return_value = False
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": mock_torch}):
result = select_device()
assert result == "cpu"
def test_returns_cuda_when_available(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Linux"
mock_platform.processor.return_value = "x86_64"
mock_torch = MagicMock()
mock_torch.backends.mps.is_available.return_value = False
mock_torch.cuda.is_available.return_value = True
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": mock_torch}):
result = select_device()
assert result == "cuda"
def test_returns_cpu_when_cuda_unavailable(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Linux"
mock_platform.processor.return_value = "x86_64"
mock_torch = MagicMock()
mock_torch.backends.mps.is_available.return_value = False
mock_torch.cuda.is_available.return_value = False
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": mock_torch}):
result = select_device()
assert result == "cpu"
def test_returns_cpu_when_torch_not_installed(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Linux"
mock_platform.processor.return_value = "x86_64"
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": None}):
result = select_device()
assert result == "cpu"
def test_handles_torch_import_error(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Windows"
mock_platform.processor.return_value = "AMD64"
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": None}):
result = select_device()
assert result == "cpu"
+101
View File
@@ -0,0 +1,101 @@
"""Tests for split pattern logic (3 identical copies in codebase)."""
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
import pytest
from abogen.domain.split_pattern import get_split_pattern
# --- English always returns \n ---
class TestEnglish:
def test_english_sentence(self):
assert get_split_pattern("a", "Sentence") == "\n"
def test_english_sentence_comma(self):
assert get_split_pattern("a", "Sentence + Comma") == "\n"
def test_english_line(self):
assert get_split_pattern("a", "Line") == "\n"
def test_english_disabled(self):
assert get_split_pattern("a", "Disabled") == "\n"
def test_english_b(self):
assert get_split_pattern("b", "Sentence") == "\n"
# --- CJK languages ---
class TestCJK:
def test_chinese_disabled(self):
pattern = get_split_pattern("z", "Disabled")
assert pattern != "\n"
assert r"\n+" in pattern
def test_chinese_line(self):
pattern = get_split_pattern("z", "Line")
assert pattern != "\n"
assert r"\n+" in pattern
def test_chinese_sentence(self):
pattern = get_split_pattern("z", "Sentence")
assert r"\n+" in pattern
def test_chinese_sentence_comma(self):
pattern = get_split_pattern("z", "Sentence + Comma")
assert r"\n+" in pattern
def test_japanese_disabled(self):
pattern = get_split_pattern("j", "Disabled")
assert pattern != "\n"
assert r"\n+" in pattern
def test_japanese_sentence(self):
pattern = get_split_pattern("j", "Sentence")
assert r"\n+" in pattern
# --- Other languages ---
class TestOtherLanguages:
def test_spanish_sentence(self):
pattern = get_split_pattern("e", "Sentence")
assert r"\n+" in pattern
def test_spanish_line(self):
assert get_split_pattern("e", "Line") == "\n"
def test_spanish_disabled(self):
# canonical: \n+ for non-CJK Disabled
assert get_split_pattern("e", "Disabled") == r"\n+"
def test_french_sentence_comma(self):
pattern = get_split_pattern("f", "Sentence + Comma")
assert r"\n+" in pattern
def test_unknown_lang(self):
pattern = get_split_pattern("x", "Sentence")
assert r"\n+" in pattern
# --- Pattern structure ---
class TestPatternStructure:
def test_sentence_has_lookbehind(self):
pattern = get_split_pattern("e", "Sentence")
assert r"(?<=" in pattern
def test_sentence_comma_has_comma_chars(self):
pattern = get_split_pattern("e", "Sentence + Comma")
assert "," in pattern
def test_cjk_spacing_uses_star(self):
pattern = get_split_pattern("z", "Sentence")
assert r"\s*" in pattern
def test_non_cjk_spacing_uses_plus(self):
pattern = get_split_pattern("e", "Sentence")
assert r"\s+" in pattern
+205
View File
@@ -0,0 +1,205 @@
"""Tests for abogen.domain.subtitle_generation module."""
import pytest
from abogen.domain.subtitle_generation import (
process_subtitle_tokens,
PUNCTUATION_SENTENCE,
PUNCTUATION_SENTENCE_COMMA,
)
class TestProcessSubtitleTokens:
"""Tests for process_subtitle_tokens function."""
def test_empty_tokens(self):
"""Test processing empty token list does nothing."""
entries = []
process_subtitle_tokens(
tokens_with_timestamps=[],
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Sentence",
lang_code="a",
)
assert entries == []
def test_disabled_mode(self):
"""Test Disabled mode still processes tokens (no special handling).
Note: In current implementation, Disabled mode doesn't skip processing
but relies on the caller to not call this function. This test documents
current behavior.
"""
tokens = [
{"start": 0.0, "end": 1.0, "text": "Hello", "whitespace": " "},
{"start": 1.0, "end": 2.0, "text": "world", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Disabled",
lang_code="a",
)
# Disabled mode doesn't have special handling in current implementation
# It processes tokens normally
assert len(entries) >= 1
def test_line_mode_basic(self):
"""Test Line mode processes tokens."""
tokens = [
{"start": 0.0, "end": 1.0, "text": "First line", "whitespace": " "},
{"start": 1.0, "end": 2.0, "text": "Second line", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Line",
lang_code="a",
)
# Line mode processes all tokens into entries
assert len(entries) >= 1
# Check that text is preserved
combined_text = " ".join(e[2] for e in entries)
assert "First line" in combined_text
assert "Second line" in combined_text
def test_sentence_mode_punctuation_split(self):
"""Test Sentence mode splits on sentence punctuation."""
tokens = [
{"start": 0.0, "end": 0.5, "text": "First sentence", "whitespace": " "},
{"start": 0.5, "end": 1.0, "text": ".", "whitespace": " "},
{"start": 1.0, "end": 1.5, "text": "Second sentence", "whitespace": " "},
{"start": 1.5, "end": 2.0, "text": ".", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Sentence",
lang_code="a",
)
assert len(entries) >= 1
# Should have at least one entry with both sentences or split
combined_text = " ".join(e[2] for e in entries)
assert "First sentence" in combined_text
assert "Second sentence" in combined_text
def test_word_count_mode(self):
"""Test word count mode (e.g., '5' for 5 words per entry)."""
tokens = [
{"start": 0.0, "end": 0.2, "text": "word1", "whitespace": " "},
{"start": 0.2, "end": 0.4, "text": "word2", "whitespace": " "},
{"start": 0.4, "end": 0.6, "text": "word3", "whitespace": " "},
{"start": 0.6, "end": 0.8, "text": "word4", "whitespace": " "},
{"start": 0.8, "end": 1.0, "text": "word5", "whitespace": " "},
{"start": 1.0, "end": 1.2, "text": "word6", "whitespace": " "},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="2", # 2 words per entry
lang_code="a",
)
assert len(entries) >= 2
# Check that entries are split roughly by word count
for entry in entries:
# Each entry should have at least one word
assert len(entry[2].split()) >= 1
def test_fallback_end_time(self):
"""Test fallback_end_time is applied when end time is invalid."""
tokens = [
{"start": 0.0, "end": None, "text": "Test", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Line",
lang_code="a",
fallback_end_time=10.0,
)
assert len(entries) == 1
assert entries[0][1] == 10.0 # Should use fallback
def test_karaoke_highlighting_mode(self):
"""Test Sentence + Highlighting mode generates karaoke tags."""
tokens = [
{"start": 0.0, "end": 0.5, "text": "Hello", "whitespace": " "},
{"start": 0.5, "end": 1.0, "text": "world", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Sentence + Highlighting",
lang_code="a",
)
assert len(entries) >= 1
# Should contain karaoke tags
text = entries[0][2]
assert "{\\kf" in text
def test_max_subtitle_words_limit(self):
"""Test that max_subtitle_words limits entry length."""
tokens = [
{"start": float(i), "end": float(i + 0.1), "text": f"word{i}", "whitespace": " "}
for i in range(10)
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=3,
subtitle_mode="Line",
lang_code="a",
)
# Should have more than 1 entry due to word limit
assert len(entries) > 1
def test_preserves_token_timing(self):
"""Test that token timing is preserved in entries."""
tokens = [
{"start": 0.0, "end": 1.0, "text": "First", "whitespace": " "},
{"start": 1.0, "end": 2.0, "text": "Second", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Sentence",
lang_code="a",
)
assert len(entries) >= 1
# Check that timing is preserved
for entry in entries:
assert entry[0] >= 0.0
assert entry[1] >= entry[0]
class TestPunctuationConstants:
"""Tests for punctuation constants."""
def test_punctuation_sentence_contains_basic(self):
"""Test PUNCTUATION_SENTENCE contains basic sentence punctuation."""
assert "." in PUNCTUATION_SENTENCE
assert "!" in PUNCTUATION_SENTENCE
assert "?" in PUNCTUATION_SENTENCE
def test_punctuation_sentence_comma_contains_comma(self):
"""Test PUNCTUATION_SENTENCE_COMMA contains comma."""
assert "," in PUNCTUATION_SENTENCE_COMMA
assert "." in PUNCTUATION_SENTENCE_COMMA
assert "!" in PUNCTUATION_SENTENCE_COMMA
assert "?" in PUNCTUATION_SENTENCE_COMMA
+264
View File
@@ -0,0 +1,264 @@
"""Tests for infrastructure/subtitle_writer.py — SrtWriter, AssWriter, VttWriter."""
from __future__ import annotations
import pytest
from pathlib import Path
from abogen.infrastructure.subtitle_writer import (
AssWriter,
SrtWriter,
SubtitleAlignment,
SubtitleConfig,
SubtitleFormat,
SubtitleMode,
VttWriter,
create_subtitle_writer,
)
# ===================================================================
# SrtWriter._format_time
# ===================================================================
class TestSrtFormatTime:
def test_zero(self):
assert SrtWriter._format_time(0.0) == "00:00:00,000"
def test_simple(self):
assert SrtWriter._format_time(61.5) == "00:01:01,500"
def test_hours(self):
assert SrtWriter._format_time(3661.123) == "01:01:01,123"
def test_large(self):
assert SrtWriter._format_time(7384.0) == "02:03:04,000"
def test_fractional_seconds(self):
assert SrtWriter._format_time(0.999) == "00:00:00,999"
def test_matches_old_format_timestamp(self):
"""Verify matches old _format_timestamp(ass=False) from conversion_runner."""
import math
for t in [0.0, 61.5, 3661.123, 7384.0, 0.999, 125.7]:
h = int(t // 3600)
m = int((t % 3600) // 60)
s = int(t % 60)
ms = int((t - math.floor(t)) * 1000)
expected = f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"
assert SrtWriter._format_time(t) == expected, f"Mismatch at t={t}"
# ===================================================================
# AssWriter._format_time
# ===================================================================
class TestAssFormatTime:
def test_zero(self):
assert AssWriter._format_time(0.0) == "0:00:00.00"
def test_simple(self):
assert AssWriter._format_time(61.5) == "0:01:01.50"
def test_hours(self):
assert AssWriter._format_time(3661.12) == "1:01:01.12"
def test_centiseconds(self):
assert AssWriter._format_time(1.55) == "0:00:01.55"
def test_matches_old_format_timestamp_ass(self):
"""Verify matches old _format_timestamp(ass=True) from conversion_runner.
Note: the old code used int(milliseconds/10) which truncates centiseconds,
while the new code uses float formatting which rounds. For most values they
match; the difference is at most 1 centisecond due to float precision.
"""
import math
for t in [0.0, 61.5, 1.55, 125.7]:
h = int(t // 3600)
m = int((t % 3600) // 60)
s = int(t % 60)
ms = int((t - math.floor(t)) * 1000)
cs = int(ms / 10)
expected = f"{h:01d}:{m:02d}:{s:02d}.{cs:02d}"
assert AssWriter._format_time(t) == expected, f"Mismatch at t={t}"
# ===================================================================
# SrtWriter full write
# ===================================================================
class TestSrtWriter:
def test_single_entry(self, tmp_path):
path = tmp_path / "test.srt"
writer = SrtWriter(path, SubtitleConfig(format=SubtitleFormat.SRT, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=2.5, text="Hello")
writer.close()
content = path.read_text()
assert "1\n" in content
assert "00:00:00,000 --> 00:00:02,500" in content
assert "Hello\n" in content
def test_multiple_entries(self, tmp_path):
path = tmp_path / "test.srt"
writer = SrtWriter(path, SubtitleConfig(format=SubtitleFormat.SRT, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=1.0, text="First")
writer.write_entry(start=1.0, end=2.0, text="Second")
writer.close()
content = path.read_text()
assert "1\n" in content
assert "2\n" in content
assert "First" in content
assert "Second" in content
def test_voice_prefix(self, tmp_path):
path = tmp_path / "test.srt"
writer = SrtWriter(path, SubtitleConfig(format=SubtitleFormat.SRT, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=1.0, text="Hello", voice="af_heart")
writer.close()
content = path.read_text()
assert "[af_heart] Hello" in content
def test_auto_index(self, tmp_path):
path = tmp_path / "test.srt"
writer = SrtWriter(path, SubtitleConfig(format=SubtitleFormat.SRT, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=1.0, text="A")
writer.write_entry(start=1.0, end=2.0, text="B")
writer.write_entry(start=2.0, end=3.0, text="C")
writer.close()
content = path.read_text()
assert "1\n" in content
assert "2\n" in content
assert "3\n" in content
# ===================================================================
# AssWriter full write
# ===================================================================
class TestAssWriter:
def test_header_structure(self, tmp_path):
path = tmp_path / "test.ass"
writer = AssWriter(path, SubtitleConfig(format=SubtitleFormat.ASS, mode=SubtitleMode.LINE))
writer.open()
writer.close()
content = path.read_text()
assert "[Script Info]" in content
assert "[V4+ Styles]" in content
assert "[Events]" in content
assert "Format: Layer, Start, End" in content
def test_single_entry(self, tmp_path):
path = tmp_path / "test.ass"
writer = AssWriter(path, SubtitleConfig(format=SubtitleFormat.ASS, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=2.5, text="Hello")
writer.close()
content = path.read_text()
assert "Dialogue:" in content
assert "Hello" in content
def test_voice_prefix(self, tmp_path):
path = tmp_path / "test.ass"
writer = AssWriter(path, SubtitleConfig(format=SubtitleFormat.ASS, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=1.0, text="Hello", voice="af_heart")
writer.close()
content = path.read_text()
assert "[af_heart] Hello" in content
def test_highlight_mode(self, tmp_path):
path = tmp_path / "test.ass"
config = SubtitleConfig(
format=SubtitleFormat.ASS,
mode=SubtitleMode.SENTENCE_HIGHLIGHT,
)
writer = AssWriter(path, config)
writer.write_entry(start=0.0, end=1.0, text="Hello world")
writer.close()
content = path.read_text()
assert "Highlight" in content
assert r"{\k100}" in content
def test_centered_alignment(self, tmp_path):
path = tmp_path / "test.ass"
config = SubtitleConfig(
format=SubtitleFormat.ASS,
mode=SubtitleMode.LINE,
alignment=SubtitleAlignment.CENTER,
)
writer = AssWriter(path, config)
writer.open()
writer.close()
content = path.read_text()
# Centered uses alignment=5
assert ",5," in content or ",5\n" in content
# ===================================================================
# VttWriter
# ===================================================================
class TestVttWriter:
def test_header(self, tmp_path):
path = tmp_path / "test.vtt"
writer = VttWriter(path, SubtitleConfig(format=SubtitleFormat.VTT, mode=SubtitleMode.LINE))
writer.open()
writer.close()
content = path.read_text()
assert content.startswith("WEBVTT")
def test_single_entry(self, tmp_path):
path = tmp_path / "test.vtt"
writer = VttWriter(path, SubtitleConfig(format=SubtitleFormat.VTT, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=2.5, text="Hello")
writer.close()
content = path.read_text()
assert "1\n" in content
assert "Hello" in content
# ===================================================================
# create_subtitle_writer factory
# ===================================================================
class TestCreateSubtitleWriter:
def test_srt(self, tmp_path):
path = tmp_path / "test.srt"
writer = create_subtitle_writer(path, "srt", "Line")
assert isinstance(writer, SrtWriter)
writer.close()
def test_ass(self, tmp_path):
path = tmp_path / "test.ass"
writer = create_subtitle_writer(path, "ass", "Line")
assert isinstance(writer, AssWriter)
writer.close()
def test_vtt(self, tmp_path):
path = tmp_path / "test.vtt"
writer = create_subtitle_writer(path, "vtt", "Line")
assert isinstance(writer, VttWriter)
writer.close()
def test_unsupported_raises(self, tmp_path):
path = tmp_path / "test.xyz"
with pytest.raises(ValueError):
create_subtitle_writer(path, "xyz", "Line")
# ===================================================================
# Context manager
# ===================================================================
class TestContextManager:
def test_srt_context_manager(self, tmp_path):
path = tmp_path / "test.srt"
with SrtWriter(path, SubtitleConfig(format=SubtitleFormat.SRT, mode=SubtitleMode.LINE)) as writer:
writer.write_entry(start=0.0, end=1.0, text="Hello")
content = path.read_text()
assert "Hello" in content
def test_ass_context_manager(self, tmp_path):
path = tmp_path / "test.ass"
with AssWriter(path, SubtitleConfig(format=SubtitleFormat.ASS, mode=SubtitleMode.LINE)) as writer:
writer.write_entry(start=0.0, end=1.0, text="Hello")
content = path.read_text()
assert "Dialogue:" in content
+70
View File
@@ -0,0 +1,70 @@
"""Tests that preview/synthesis module is correctly named and importable."""
import pathlib
def _read_source(module_file: str) -> str:
return pathlib.Path(module_file).read_text(encoding="utf-8")
def test_preview_file_renamed_to_synthesize():
"""preview.py must be renamed to synthesize.py."""
import abogen.webui.routes.utils.synthesize as synthesize_mod
synthesize_path = pathlib.Path(synthesize_mod.__file__)
assert synthesize_path.name == "synthesize.py"
assert synthesize_path.exists()
assert not synthesize_path.with_name("preview.py").exists()
def test_synthesize_module_has_generate_preview_audio():
"""synthesize.py must export generate_preview_audio."""
from abogen.webui.routes.utils.synthesize import generate_preview_audio
assert callable(generate_preview_audio)
def test_synthesize_module_has_synthesize_preview():
"""synthesize.py must export synthesize_preview."""
from abogen.webui.routes.utils.synthesize import synthesize_preview
assert callable(synthesize_preview)
def test_synthesize_module_has_get_preview_pipeline():
"""synthesize.py must export get_preview_pipeline."""
from abogen.webui.routes.utils.synthesize import get_preview_pipeline
assert callable(get_preview_pipeline)
def test_api_imports_from_synthesize():
"""api.py must import from synthesize, not preview."""
import abogen.webui.routes.api as api_mod
source = _read_source(api_mod.__file__)
assert "from abogen.webui.routes.utils.synthesize import" in source
assert "from abogen.webui.routes.utils.preview import" not in source
def test_voices_imports_from_synthesize():
"""voices.py must import from synthesize, not preview."""
import abogen.webui.routes.voices as voices_mod
source = _read_source(voices_mod.__file__)
assert "from abogen.webui.routes.utils.synthesize import" in source
assert "from abogen.webui.routes.utils.preview import" not in source
def test_no_module_imports_preview():
"""No module should import from the old preview path."""
import glob
import os
project_root = pathlib.Path(__file__).parent.parent
py_files = glob.glob(str(project_root / "abogen" / "**" / "*.py"), recursive=True)
for py_file in py_files:
content = pathlib.Path(py_file).read_text(encoding="utf-8")
assert "from abogen.webui.routes.utils.preview import" not in content, (
f"{py_file} still imports from preview"
)
+59
View File
@@ -0,0 +1,59 @@
"""Tests for domain/title_builder.py."""
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from abogen.domain.title_builder import build_title_intro_text, build_outro_text
class TestBuildTitleIntroText:
def test_empty_metadata(self):
result = build_title_intro_text({}, "book.epub")
assert result == "book."
def test_title_from_metadata(self):
result = build_title_intro_text({"title": "My Book"}, "book.epub")
assert result == "My Book."
def test_title_fallback_basename(self):
result = build_title_intro_text({}, "my_book.epub")
assert result == "my_book."
def test_with_author(self):
result = build_title_intro_text({"title": "My Book", "author": "John Doe"}, "book.epub")
assert result == "My Book. By John Doe."
def test_with_subtitle(self):
result = build_title_intro_text({"title": "My Book", "subtitle": "A Tale"}, "book.epub")
assert result == "My Book. A Tale."
def test_duplicate_title_subtitle(self):
result = build_title_intro_text({"title": "My Book", "subtitle": "My Book"}, "book.epub")
assert result == "My Book."
def test_with_series(self):
result = build_title_intro_text({"title": "My Book", "series": "Series", "series_index": "3"}, "book.epub")
assert result == "Book 3 of the Series. My Book."
class TestBuildOutroText:
def test_empty(self):
result = build_outro_text({}, "book.epub")
assert result == "The end of book."
def test_title_only(self):
result = build_outro_text({"title": "My Book"}, "book.epub")
assert result == "The end of My Book."
def test_author_only(self):
result = build_outro_text({"author": "John Doe"}, "book.epub")
assert result == "The end of book from John Doe."
def test_title_and_author(self):
result = build_outro_text({"title": "My Book", "author": "John Doe"}, "book.epub")
assert result == "The end of My Book from John Doe."
def test_with_series(self):
result = build_outro_text({"title": "My Book", "series": "Series", "series_index": "3"}, "book.epub")
assert "The end of My Book." in result
assert "Book 3 of the Series." in result
+139
View File
@@ -0,0 +1,139 @@
import sys
import types
if "soundfile" not in sys.modules:
soundfile_stub = types.ModuleType("soundfile")
class _SoundFileStub: # pragma: no cover - placeholder to satisfy imports
def __init__(self, *args: object, **kwargs: object) -> None:
raise RuntimeError("soundfile is not installed in the test environment")
soundfile_stub.SoundFile = _SoundFileStub # type: ignore[attr-defined]
sys.modules["soundfile"] = soundfile_stub
if "static_ffmpeg" not in sys.modules:
sys.modules["static_ffmpeg"] = types.ModuleType("static_ffmpeg")
if "ebooklib" not in sys.modules:
ebooklib_stub = types.ModuleType("ebooklib")
ebooklib_epub_stub = types.ModuleType("ebooklib.epub")
ebooklib_stub.epub = ebooklib_epub_stub # type: ignore[attr-defined]
sys.modules["ebooklib"] = ebooklib_stub
sys.modules["ebooklib.epub"] = ebooklib_epub_stub
if "fitz" not in sys.modules:
sys.modules["fitz"] = types.ModuleType("fitz")
if "markdown" not in sys.modules:
markdown_stub = types.ModuleType("markdown")
class _MarkdownStub:
def __init__(self, *args: object, **kwargs: object) -> None:
self.toc_tokens = []
def convert(self, text: str) -> str:
return text
markdown_stub.Markdown = _MarkdownStub # type: ignore[attr-defined]
sys.modules["markdown"] = markdown_stub
if "bs4" not in sys.modules:
bs4_stub = types.ModuleType("bs4")
class _BeautifulSoupStub:
def __init__(self, *args: object, **kwargs: object) -> None:
self._text = ""
def find(self, *args: object, **kwargs: object) -> None:
return None
def get_text(self) -> str:
return self._text
def decompose(self) -> None: # pragma: no cover - compatibility shim
return None
class _NavigableStringStub(str):
pass
bs4_stub.BeautifulSoup = _BeautifulSoupStub # type: ignore[attr-defined]
bs4_stub.NavigableString = _NavigableStringStub # type: ignore[attr-defined]
sys.modules["bs4"] = bs4_stub
from unittest.mock import patch, MagicMock
class TestResolveFallbackVoiceSpec:
"""Tests for the voice fallback resolution helper."""
def test_uses_base_voice_spec(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="af_heart",
job_voice="af_bella",
voice_cache_keys=[],
)
assert result == "af_heart"
def test_falls_back_to_job_voice(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="",
job_voice="af_bella",
voice_cache_keys=[],
)
assert result == "af_bella"
def test_skips_custom_mix_uses_job_voice(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="__custom_mix",
job_voice="af_bella",
voice_cache_keys=[],
)
assert result == "af_bella"
def test_falls_back_to_voice_cache(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="",
job_voice="",
voice_cache_keys=["kokoro:af_heart"],
)
assert result == "af_heart"
def test_skips_custom_mix_in_cache(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="",
job_voice="",
voice_cache_keys=["__custom_mix", "kokoro:af_heart"],
)
assert result == "af_heart"
def test_falls_back_to_default_voice(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
with patch("abogen.domain.voice_resolution.get_default_voice", return_value="af_heart"):
result = resolve_fallback_voice_spec(
base_spec="",
job_voice="",
voice_cache_keys=[],
)
assert result == "af_heart"
def test_empty_base_and_job_with_cache(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="",
job_voice="",
voice_cache_keys=["kokoro:af_bella", "kokoro:af_heart"],
)
assert result == "af_bella"
+137
View File
@@ -0,0 +1,137 @@
"""Tests for abogen.domain.voice_loader module."""
import pytest
from abogen.domain.voice_loader import (
VoiceCache,
resolve_voice,
load_voice_cached,
)
class TestVoiceCache:
"""Tests for VoiceCache class."""
def test_get_set(self):
"""Test basic get/set operations."""
cache = VoiceCache()
cache.set("test_voice", "loaded_voice")
assert cache.get("test_voice") == "loaded_voice"
def test_get_missing(self):
"""Test get returns None for missing voice."""
cache = VoiceCache()
assert cache.get("missing_voice") is None
def test_contains(self):
"""Test contains method."""
cache = VoiceCache()
cache.set("test_voice", "loaded_voice")
assert cache.contains("test_voice")
assert not cache.contains("missing_voice")
def test_in_operator(self):
"""Test __contains__ (in operator)."""
cache = VoiceCache()
cache.set("test_voice", "loaded_voice")
assert "test_voice" in cache
assert "missing_voice" not in cache
def test_clear(self):
"""Test clear method."""
cache = VoiceCache()
cache.set("voice1", "loaded1")
cache.set("voice2", "loaded2")
cache.clear()
assert not cache.contains("voice1")
assert not cache.contains("voice2")
class TestResolveVoice:
"""Tests for resolve_voice function."""
def test_simple_voice_name(self):
"""Test that simple voice names are returned as-is."""
result = resolve_voice(
voice_spec="test_voice",
pipeline=None,
use_gpu=False,
)
assert result == "test_voice"
def test_formula_voice_without_pipeline(self):
"""Test formula voice returns spec when no pipeline."""
result = resolve_voice(
voice_spec="model*0.5+0.3*other",
pipeline=None,
use_gpu=False,
)
assert result == "model*0.5+0.3*other"
def test_caching(self):
"""Test that voices are cached."""
cache = VoiceCache()
# First call should load (we'll mock with simple name)
result1 = resolve_voice(
voice_spec="test_voice",
pipeline=None,
use_gpu=False,
cache=cache,
)
assert result1 == "test_voice"
assert cache.contains("test_voice")
# Second call should use cache
result2 = resolve_voice(
voice_spec="test_voice",
pipeline=None,
use_gpu=False,
cache=cache,
)
assert result2 == "test_voice"
class TestLoadVoiceCached:
"""Tests for load_voice_cached function."""
def test_simple_voice_name(self):
"""Test that simple voice names are returned as-is."""
result = load_voice_cached(
voice_name="test_voice",
pipeline=None,
use_gpu=False,
)
assert result == "test_voice"
def test_dict_cache(self):
"""Test caching with dict."""
cache = {}
result1 = load_voice_cached(
voice_name="test_voice",
pipeline=None,
use_gpu=False,
cache=cache,
)
assert result1 == "test_voice"
assert "test_voice" in cache
result2 = load_voice_cached(
voice_name="test_voice",
pipeline=None,
use_gpu=False,
cache=cache,
)
assert result2 == "test_voice"
assert cache["test_voice"] == "test_voice"
def test_no_cache(self):
"""Test without cache parameter."""
result = load_voice_cached(
voice_name="test_voice",
pipeline=None,
use_gpu=False,
cache=None,
)
assert result == "test_voice"
+274
View File
@@ -0,0 +1,274 @@
"""Tests for voice resolution helpers.
Tests import from domain.voice_resolution (new location).
"""
from __future__ import annotations
from types import SimpleNamespace
from typing import Any, Dict
from unittest.mock import patch
import pytest
# ---------------------------------------------------------------------------
# spec_to_voice_ids
# ---------------------------------------------------------------------------
class TestSpecToVoiceIds:
"""spec_to_voice_ids extracts voice identifiers from a spec string."""
def test_empty_string(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
assert spec_to_voice_ids("") == set()
def test_none(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
assert spec_to_voice_ids(None) == set()
def test_custom_mix_returns_empty(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
assert spec_to_voice_ids("__custom_mix") == set()
def test_single_known_voice(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
with patch("abogen.domain.voice_resolution.get_voices", return_value={"af_heart"}):
assert spec_to_voice_ids("af_heart") == {"af_heart"}
def test_unknown_single_voice_returns_empty(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
with patch("abogen.domain.voice_resolution.get_voices", return_value=set()):
assert spec_to_voice_ids("nonexistent") == set()
def test_formula_with_star(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
with patch("abogen.domain.voice_resolution.extract_voice_ids", return_value=["v1", "v2"]):
result = spec_to_voice_ids("v1*v2")
assert result == {"v1", "v2"}
def test_formula_value_error_returns_empty(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
with patch("abogen.domain.voice_resolution.extract_voice_ids", side_effect=ValueError("bad")):
assert spec_to_voice_ids("bad*spec") == set()
def test_whitespace_stripped(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
assert spec_to_voice_ids(" ") == set()
# ---------------------------------------------------------------------------
# job_voice_fallback
# ---------------------------------------------------------------------------
class TestJobVoiceFallback:
"""job_voice_fallback resolves a fallback voice from job attributes."""
def test_direct_voice(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(voice="af_heart", speakers=None, chapters=[])
assert job_voice_fallback(job) == "af_heart"
def test_custom_mix_ignored(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(voice="__custom_mix", speakers=None, chapters=[])
assert job_voice_fallback(job) == ""
def test_narrator_speaker(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(
voice="__custom_mix",
speakers={"narrator": {"resolved_voice": "af_heart"}},
chapters=[],
)
assert job_voice_fallback(job) == "af_heart"
def test_speaker_voice_formula(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(
voice="",
speakers={"speaker1": {"voice_formula": "v1*v2"}},
chapters=[],
)
assert job_voice_fallback(job) == "v1*v2"
def test_chapter_voice(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(
voice="",
speakers=None,
chapters=[{"resolved_voice": "af_bella"}],
)
assert job_voice_fallback(job) == "af_bella"
def test_empty_job(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(voice="", speakers=None, chapters=[])
assert job_voice_fallback(job) == ""
def test_narrator_custom_mix_falls_through(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(
voice="",
speakers={"narrator": {"voice": "__custom_mix"}},
chapters=[{"voice": "af_heart"}],
)
assert job_voice_fallback(job) == "af_heart"
# ---------------------------------------------------------------------------
# chapter_voice_spec
# ---------------------------------------------------------------------------
class TestChapterVoiceSpec:
"""chapter_voice_spec resolves voice for a chapter override."""
def test_no_override_uses_fallback(self):
from abogen.domain.voice_resolution import chapter_voice_spec
job = SimpleNamespace(voice="af_heart", speakers=None, chapters=[])
assert chapter_voice_spec(job, None) == "af_heart"
def test_resolved_voice_wins(self):
from abogen.domain.voice_resolution import chapter_voice_spec
job = SimpleNamespace(voice="af_heart", speakers=None, chapters=[])
override = {"resolved_voice": "af_bella", "voice_formula": "x", "voice": "y"}
assert chapter_voice_spec(job, override) == "af_bella"
def test_formula_second(self):
from abogen.domain.voice_resolution import chapter_voice_spec
job = SimpleNamespace(voice="", speakers=None, chapters=[])
override = {"voice_formula": "v1*v2", "voice": "y"}
assert chapter_voice_spec(job, override) == "v1*v2"
def test_voice_third(self):
from abogen.domain.voice_resolution import chapter_voice_spec
job = SimpleNamespace(voice="", speakers=None, chapters=[])
override = {"voice": "af_nicole"}
assert chapter_voice_spec(job, override) == "af_nicole"
def test_empty_override_falls_to_fallback(self):
from abogen.domain.voice_resolution import chapter_voice_spec
job = SimpleNamespace(voice="af_heart", speakers=None, chapters=[])
assert chapter_voice_spec(job, {}) == "af_heart"
# ---------------------------------------------------------------------------
# chunk_voice_spec
# ---------------------------------------------------------------------------
class TestChunkVoiceSpec:
"""chunk_voice_spec resolves voice for a TTS chunk."""
def test_chunk_direct_voice(self):
from abogen.domain.voice_resolution import chunk_voice_spec
job = SimpleNamespace(speakers=None)
chunk = {"resolved_voice": "af_heart"}
assert chunk_voice_spec(job, chunk, "fallback") == "af_heart"
def test_chunk_speaker_lookup(self):
from abogen.domain.voice_resolution import chunk_voice_spec
job = SimpleNamespace(speakers={"narrator": {"resolved_voice": "af_bella"}})
chunk = {"speaker_id": "narrator"}
assert chunk_voice_spec(job, chunk, "") == "af_bella"
def test_chunk_voice_profile_lookup(self):
from abogen.domain.voice_resolution import chunk_voice_spec
job = SimpleNamespace(speakers={"角色A": {"voice": "af_nicole"}})
chunk = {"voice_profile": "角色A"}
assert chunk_voice_spec(job, chunk, "") == "af_nicole"
def test_uses_fallback_string(self):
from abogen.domain.voice_resolution import chunk_voice_spec
job = SimpleNamespace(speakers=None)
chunk = {}
assert chunk_voice_spec(job, chunk, "my_fallback") == "my_fallback"
def test_fallback_to_job(self):
from abogen.domain.voice_resolution import chunk_voice_spec
job = SimpleNamespace(voice="af_heart", speakers=None, chapters=[])
chunk = {}
assert chunk_voice_spec(job, chunk, "") == "af_heart"
# ---------------------------------------------------------------------------
# collect_required_voice_ids
# ---------------------------------------------------------------------------
class TestCollectRequiredVoiceIds:
"""collect_required_voice_ids gathers all voice IDs from a job."""
def test_includes_job_voice(self):
from abogen.domain.voice_resolution import collect_required_voice_ids
job = SimpleNamespace(voice="af_heart", chapters=[], chunks=[], speakers={})
with patch("abogen.domain.voice_resolution.get_voices", return_value={"af_heart"}), \
patch("abogen.domain.voice_resolution.job_voice_fallback", return_value=""):
result = collect_required_voice_ids(job)
assert "af_heart" in result
def test_includes_chapter_voices(self):
from abogen.domain.voice_resolution import collect_required_voice_ids
job = SimpleNamespace(
voice="",
chapters=[{"resolved_voice": "af_bella"}],
chunks=[],
speakers={},
)
with patch("abogen.domain.voice_resolution.get_voices", return_value={"af_bella"}), \
patch("abogen.domain.voice_resolution.job_voice_fallback", return_value=""):
result = collect_required_voice_ids(job)
assert "af_bella" in result
def test_includes_chunk_voices(self):
from abogen.domain.voice_resolution import collect_required_voice_ids
job = SimpleNamespace(
voice="",
chapters=[],
chunks=[{"voice": "af_nicole"}],
speakers={},
)
with patch("abogen.domain.voice_resolution.get_voices", return_value={"af_nicole"}), \
patch("abogen.domain.voice_resolution.job_voice_fallback", return_value=""):
result = collect_required_voice_ids(job)
assert "af_nicole" in result
def test_always_includes_kokoro_voices(self):
from abogen.domain.voice_resolution import collect_required_voice_ids
job = SimpleNamespace(voice="", chapters=[], chunks=[], speakers={})
with patch("abogen.domain.voice_resolution.get_voices", return_value={"af_heart", "af_bella"}), \
patch("abogen.domain.voice_resolution.job_voice_fallback", return_value=""):
result = collect_required_voice_ids(job)
assert {"af_heart", "af_bella"}.issubset(result)
+118
View File
@@ -0,0 +1,118 @@
"""Tests for domain/voice_utils.py."""
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from abogen.domain.voice_utils import (
infer_provider_from_spec,
supertonic_voice_from_spec,
split_speaker_reference,
formula_from_kokoro_entry,
coerce_truthy,
)
class TestInferProviderFromSpec:
def test_empty_returns_fallback(self):
assert infer_provider_from_spec("", "kokoro") == "kokoro"
def test_supertonic_uppercase(self):
assert infer_provider_from_spec("M1", "kokoro") == "supertonic"
def test_kokoro_voice(self):
assert infer_provider_from_spec("af_bella", "kokoro") == "kokoro"
def test_custom_mix(self):
assert infer_provider_from_spec("__custom_mix", "kokoro") == "kokoro"
def test_formula(self):
assert infer_provider_from_spec("af_bella*0.5+am_adam*0.5", "kokoro") == "kokoro"
class TestSupertonicVoiceFromSpec:
def test_normal(self):
assert supertonic_voice_from_spec("m1", "m2") == "M1"
def test_empty_uses_fallback(self):
assert supertonic_voice_from_spec("", "m2") == "M2"
def test_formula_uses_fallback(self):
assert supertonic_voice_from_spec("m1*0.5", "m2") == "M2"
def test_both_empty_uses_default(self):
assert supertonic_voice_from_spec("", "") == "M1"
class TestSplitSpeakerReference:
def test_speaker(self):
name, original = split_speaker_reference("speaker:John")
assert name == "John"
assert original == "speaker:John"
def test_profile(self):
name, original = split_speaker_reference("profile:Main")
assert name == "Main"
assert original == "profile:Main"
def test_invalid_prefix(self):
name, original = split_speaker_reference("voice:John")
assert name is None
assert original == "voice:John"
def test_no_colon(self):
name, original = split_speaker_reference("John")
assert name is None
assert original == "John"
def test_empty(self):
name, original = split_speaker_reference("")
assert name is None
assert original == ""
class TestFormulaFromKokoroEntry:
def test_normal(self):
entry = {"voices": [["af_bella", 0.5], ["am_adam", 0.5]]}
result = formula_from_kokoro_entry(entry)
assert "af_bella" in result
assert "am_adam" in result
def test_empty(self):
assert formula_from_kokoro_entry({}) == ""
def test_invalid_items(self):
entry = {"voices": [["af_bella", "invalid"], ["am_adam", 0.5]]}
result = formula_from_kokoro_entry(entry)
assert "am_adam" in result
assert "af_bella" not in result
class TestCoerceTruthy:
def test_bool_true(self):
assert coerce_truthy(True) is True
def test_bool_false(self):
assert coerce_truthy(False) is False
def test_string_true(self):
assert coerce_truthy("true") is True
assert coerce_truthy("yes") is True
assert coerce_truthy("1") is True
assert coerce_truthy("on") is True
def test_string_false(self):
assert coerce_truthy("false") is False
assert coerce_truthy("no") is False
assert coerce_truthy("0") is False
assert coerce_truthy("off") is False
assert coerce_truthy("") is False
def test_none_default_true(self):
assert coerce_truthy(None, True) is True
def test_none_default_false(self):
assert coerce_truthy(None, False) is False
def test_int(self):
assert coerce_truthy(1) is True
assert coerce_truthy(0) is False
+53
View File
@@ -0,0 +1,53 @@
from abogen.webui.app import create_app
def _large_chapter_form() -> dict[str, str]:
data = {"step": "chapters"}
for index in range(370):
prefix = f"chapter-{index}"
data[f"{prefix}-enabled"] = "on"
data[f"{prefix}-title"] = f"Chapter {index} " + ("x" * 1400)
data[f"{prefix}-voice"] = "af_heart"
data[f"{prefix}-formula"] = "default"
return data
def test_large_chapter_form_reaches_wizard_route(tmp_path):
app = create_app(
{
"TESTING": True,
"SECRET_KEY": "test",
"OUTPUT_FOLDER": str(tmp_path / "output"),
"UPLOAD_FOLDER": str(tmp_path / "uploads"),
}
)
with app.test_client() as client:
response = client.post(
"/wizard/update?format=json",
data=_large_chapter_form(),
)
assert response.status_code == 400
assert response.get_json()["error"] == "Missing job ID"
def test_large_multipart_chapter_form_reaches_wizard_route(tmp_path):
app = create_app(
{
"TESTING": True,
"SECRET_KEY": "test",
"OUTPUT_FOLDER": str(tmp_path / "output"),
"UPLOAD_FOLDER": str(tmp_path / "uploads"),
}
)
with app.test_client() as client:
response = client.post(
"/wizard/update?format=json",
data=_large_chapter_form(),
content_type="multipart/form-data",
)
assert response.status_code == 400
assert response.get_json()["error"] == "Missing job ID"