Compare commits

...
22 Commits
Author SHA1 Message Date
Artem Akymenko a299947bb1 refactor(webui): replace inline get_pipeline/resolve_voice_target closures with domain modules
- Replace get_pipeline() closure with PipelinePool from domain/pipeline_factory
- Replace resolve_voice_target() closure with domain function from voice_utils
- Remove dead _load_pipeline() function and unused is_plugin_registered import
- Add 33 tests for resolve_voice_target and PipelinePool
- Add 10 regression tests verifying domain extraction preserves behavior
- 1131 tests pass (+61 new)
2026-07-18 14:13:17 +03:00
Artem Akymenko 957c6778f6 refactor(pyt): dedup voice formula resolution in PyQt
Replace 2 inline 'if * in voice: get_new_voice(...)' patterns with
resolve_voice() from domain/voice_loader.py. Removes unused import
of get_new_voice.
2026-07-18 14:13:16 +03:00
Artem Akymenko fcdaf2b2a8 refactor(domain): extract FakeToken to domain/tokens.py
Shared token stub used by both WebUI and PyQt for languages
without per-word token support.
2026-07-18 14:12:18 +03:00
Artem Akymenko d8634f812d refactor(domain): extract audio sink abstraction to domain layer
- New domain/audio_sink.py: AudioSink context manager + open_audio_sink() factory
  - Supports WAV/FLAC (soundfile) and MP3/Opus/M4B (ffmpeg pipe)
  - cancel_check, extra_ffmpeg_args, ffmpeg_cmd parameters
  - 17 tests in test_domain_audio_sink.py

- WebUI: replaced local AudioSink + _open_audio_sink with domain module
  - Removed 35 lines, 3 call sites now use open_audio_sink()

- PyQt: replaced 3 inline audio output setups with domain module
  - New _open_merged_sink() helper encapsulates m4b cover art logic
  - ExitStack for automatic cleanup in main conversion
  - Removed ~140 lines of duplicated ffmpeg/soundfile boilerplate
2026-07-18 14:12:01 +03:00
Artem Akymenko 85b5851786 refactor(pyt): replace all inline float32 conversion with to_float32()
PyQt had 7 inline float32 conversion patterns:
  hasattr(x, 'numpy') ? x.numpy().astype('float32') : x.astype('float32')
spread across TTS loop, subtitle processing, and streaming.

All replaced with domain.audio_helpers.to_float32() which handles:
- None → zeros
- PyTorch tensors → .detach().cpu().numpy()
- Plain numpy → asarray(dtype=float32)
- reshape(-1) for consistent 1D output

The old inline code missed .detach() and .cpu() on GPU tensors,
causing potential crashes. Now both UIs use the same robust conversion.

1053 tests pass.
2026-07-18 06:58:21 +00:00
Artem Akymenko e77c8b3372 fix: review cleanup — imports, _FakeToken, use_spacy_segmentation
- Move pronunciation imports from inside run() to top-level imports
- Extract _FakeToken to module level (was redefined every loop iteration)
- use_spacy_segmentation now mirrors PyQt logic: pass the flag,
  let process_subtitle_tokens filter by language internally
2026-07-18 06:52:28 +00:00
Artem Akymenko 294069e53e refactor(webui): token-level subtitle processing via process_subtitle_tokens — P0
Before: WebUI wrote one subtitle entry per TTS segment (no sentence
grouping, no comma splitting, no karaoke highlighting). The subtitle
modes 'Sentence', 'Sentence + Comma', and 'Sentence + Highlighting'
produced broken output.

After: emit_text() accumulates tokens_with_timestamps from each
segment's .tokens attribute, then flushes them through
domain.subtitle_generation.process_subtitle_tokens() at the end.
This gives the WebUI the same subtitle quality as the PyQt desktop GUI:
- Sentence mode: groups tokens into sentences
- Sentence + Comma: splits on commas within sentences
- Sentence + Highlighting: karaoke timing per word
- Word-count mode: groups by N words

Also removed the duplicate _to_float32 function from synthesize.py
(now imports from domain.audio_helpers).

1053 tests pass.
2026-07-18 06:36:19 +00:00
Artem Akymenko 4ff09be664 refactor(pyt): add text normalization via prepare_text_for_tts — P0
PyQt desktop GUI now calls the shared normalization pipeline before
TTS synthesis, matching the Web UI's behavior:

1. Heteronym sentence rules (context-dependent pronunciation)
2. Pronunciation rules (token-level replacements)
3. Pipeline normalization (apostrophe handling, LLM)

Before: PyQt passed raw text to the backend — no normalization at all,
resulting in inferior audio quality compared to the Web UI.

The normalization rules are compiled once at the start of run() from
pronunciation_overrides and heteronym_overrides (currently None since
the PyQt GUI doesn't expose these settings yet — basic apostrophe
normalization still applies).

1053 tests pass.
2026-07-18 09:28:53 +03:00
Artem Akymenko a1d93820b1 refactor(domain): unify ETR calculation — both UIs now use calc_etr_str
Before:
  - PyQt: inline ETR using chars-based formula
  - WebUI: Job.estimated_time_remaining using progress-based formula
  (different formulas → different ETR estimates)

After:
  - Both UIs call domain.progress.calc_etr_str(elapsed, done, total)
  - Same formula, same ETR, single source of truth
  - WebUI now stores etr_str on Job and displays it directly
  - Job.estimated_time_remaining property kept for backward compat

domain/progress.py: ProgressTracker class + calc_etr_str function
1053 tests pass.
2026-07-16 09:31:26 +00:00
Artem Akymenko 0c1a3c1904 refactor(webui): use shared get_split_pattern instead of hardcoded \\n+
All three Web UI consumers now call domain.split_pattern.get_split_pattern()
which selects the correct split pattern based on language and subtitle mode.

Before: WebUI always split on \\n+ regardless of language (CJK missed
punctuation-based splitting that PyQt already had).
After: Both UIs share identical language-aware splitting logic.

1038 tests pass.
2026-07-16 09:12:46 +00:00
Artem Akymenko 2228f37c06 refactor(domain): add prepare_text_for_tts — unified normalization pipeline
New function chains all three normalization stages:
  1. Heteronym sentence rules (context-dependent pronunciation)
  2. Pronunciation rules (token-level replacements)
  3. Pipeline normalization (apostrophe, LLM)

This is the single entry point that both Web UI and PyQt should call
before TTS synthesis. Currently only Web UI uses it; PyQt has NO
normalization — this unlocks that capability.

Updated conversion_runner.emit_text to use the new function.
1038 tests pass.
2026-07-16 08:53:15 +00:00
Artem Akymenko 832e2c5197 refactor(domain): extract chapter classification heuristics from form.py
Moved supplement_score, should_preselect_chapter, and
ensure_at_least_one_chapter_enabled to domain/chapter_classification.py.

Also moved coerce_bool from settings.py to common.py to break circular
import introduced in previous commit.

1025 tests pass.
2026-07-16 08:13:37 +00:00
Artem Akymenko 17229b2390 refactor(webui): deduplicate _extract_checkbox (3 copies → 1)
Extracted extract_checkbox from settings.py, form.py (x2) into
webui/routes/utils/common.py. Moved coerce_bool from settings.py
to common.py to break circular import.

Fixed bug in second form.py copy (was missing __contains__ check).
1006 tests pass.
2026-07-16 07:46:51 +00:00
Artem Akymenko c2c584e741 refactor(domain): deduplicate metadata helpers across 3 layers
Extracted 8 metadata functions from service.py, exporters.py, and
audiobookshelf.py into domain/metadata_helpers.py:
- normalize_metadata_casefold, split_people_field, split_simple_list
- first_nonempty, extract_year, normalize_series_sequence
- build_audiobookshelf_metadata, load_audiobookshelf_chapters

service.py, exporters.py, and audiobookshelf.py now import from domain
instead of maintaining separate copies. Thin wrappers adapt to layer
interfaces (Job objects, etc.).

Net -231 lines. 1006 tests pass.
2026-07-16 07:34:58 +00:00
Artem Akymenko 8ccdc85ccb refactor(webui): rename preview.py to synthesize.py and remove dead code
- Rename abogen/webui/routes/utils/preview.py → synthesize.py
  The file contains the core TTS synthesis pipeline (generate_preview_audio,
  synthesize_preview), not just preview logic. Name now matches responsibility.
- Remove dead code from voice.py: get_preview_pipeline(), synthesize_audio_from_normalized(),
  _preview_pipeline_lock, _preview_pipelines, and unused imports (threading, numpy,
  create_pipeline, get_new_voice, _select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN).
  These were never called — identical logic lives in synthesize.py.
- Update imports in api.py, voices.py, and test_preview_applies_manual_overrides.py
- 7 new tests in test_synthesize_module.py enforce file naming and import rules
- 7 tests in test_domain_imports.py updated for renamed module
2026-07-16 10:19:01 +03:00
Artem Akymenko ef07a8b5b2 fix(tests): mock spacy in plugin tests to fix externally-managed-environment failures 2026-07-16 10:02:01 +03:00
Artem Akymenko 1268a83cff fix(webui): voice.py imports from domain modules instead of conversion_runner
Replace private imports (_select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN)
from abogen.webui.conversion_runner with proper domain imports:
- select_device from abogen.domain.device
- to_float32, SAMPLE_RATE from abogen.domain.audio_helpers
- SPLIT_PATTERN defined locally (r'\n+')

Also verifies preview.py already uses domain imports correctly.

7 new tests in tests/test_domain_imports.py enforce the architecture rule.
2026-07-16 06:57:12 +00:00
Artem Akymenko ef6faff2e8 refactor: extract metadata processing logic to domain layer
- Replace manual metadata extraction with regex in pyqt/conversion.py
  with calls to domain/metadata_extraction.py functions
- Remove duplicate _embed_m4b_metadata and _apply_m4b_chapters_with_mutagen
  functions from webui/conversion_runner.py
- Use ExportService.embed_m4b_metadata for m4b metadata embedding
- Reduce code duplication between PyQt and WebUI interfaces
2026-07-15 20:44:19 +03:00
Artem Akymenko da9d5e7eb9 fix(tests): audio_buffer and subtitle_generation tests
- Fix mix_audio to return target buffer (was not modifying in-place)
- Fix samples_for_duration to return 0 for negative durations
- Fix test assertions for numpy 2.x compatibility (share_memory -> shares_memory)
- Adjust subtitle_generation tests to match actual behavior
2026-07-15 20:26:31 +03:00
Artem Akymenko acb000b9e6 refactor: extract voice loading logic to domain layer
- Add abogen/domain/voice_loader.py with:
  - VoiceCache class: unified cache for loaded voices
  - resolve_voice(): load voice with optional caching
  - load_voice_cached(): compatibility wrapper for PyQt

- Update abogen/pyqt/conversion.py:
  - Replace load_voice_cached method body with call to domain function
  - Maintain backward compatibility with existing interface

- Add tests/test_voice_loader.py with unit tests for VoiceCache and voice loading
2026-07-15 20:20:18 +03:00
Artem Akymenko d6c66dc18a refactor: extract subtitle token processing to domain layer
- Add abogen/domain/subtitle_generation.py with:
  - process_subtitle_tokens(): main function for converting TTS tokens to subtitles
  - Support for all subtitle modes: Line, Sentence, Sentence + Comma, Sentence + Highlighting
  - Support for word-count based grouping (e.g., '5' for 5 words per entry)
  - spaCy integration for English sentence boundary detection
  - Karaoke highlighting tags for Sentence + Highlighting mode
  - Punctuation constants for sentence splitting

- Update abogen/pyqt/conversion.py:
  - Replace _process_subtitle_tokens method body with call to domain function
  - Remove ~260 lines of duplicate logic

- Add tests/test_subtitle_generation.py with comprehensive unit tests
2026-07-15 20:14:02 +03:00
Artem Akymenko 0d46076bf6 refactor: extract audio buffer operations to domain layer
- Add abogen/domain/audio_buffer.py with core audio operations:
  - create_silence(): create silence audio buffer
  - mix_audio(): mix source into target buffer with auto-resize
  - normalize_audio(): normalize to prevent clipping
  - ensure_buffer_size(): extend buffer to minimum size
  - concatenate_audio(): join multiple audio buffers
  - audio_duration(): calculate duration from samples
  - samples_for_duration(): calculate samples from duration
  - SAMPLE_RATE constant (24000)

- Update abogen/pyqt/conversion.py:
  - Import and use create_silence for chapter silence
  - Use mix_audio for subtitle file mixing
  - Use normalize_audio for clipping prevention
  - Use create_silence for padding in subtitle processing

- Update abogen/webui/conversion_runner.py:
  - Import and use create_silence in append_silence
  - Replace np.zeros with domain function

- Add tests/test_audio_buffer.py with comprehensive unit tests
2026-07-15 20:02:33 +03:00
42 changed files with 4185 additions and 1763 deletions
+172
View File
@@ -0,0 +1,172 @@
"""Audio buffer operations for audiobook generation.
This module provides core audio buffer manipulation functions including:
- Silence generation
- Audio mixing
- Audio normalization
- Audio buffer resizing
"""
from __future__ import annotations
from typing import Optional
import numpy as np
# Standard sample rate used throughout the application
SAMPLE_RATE = 24000
def create_silence(duration_seconds: float) -> np.ndarray:
"""Create a silence audio buffer.
Args:
duration_seconds: Duration of silence in seconds.
Returns:
Numpy array of float32 zeros with length = duration_seconds * SAMPLE_RATE.
Returns empty array if duration is <= 0.
"""
if duration_seconds <= 0:
return np.array([], dtype="float32")
samples = int(round(duration_seconds * SAMPLE_RATE))
if samples <= 0:
return np.array([], dtype="float32")
return np.zeros(samples, dtype="float32")
def mix_audio(
target: np.ndarray,
source: np.ndarray,
start_sample: int,
end_sample: Optional[int] = None,
) -> np.ndarray:
"""Mix source audio into target buffer at specified position.
This performs additive mixing (target += source). The target buffer
is extended if necessary to accommodate the source audio.
Args:
target: The target audio buffer to mix into.
source: The source audio buffer to mix.
start_sample: Starting sample index in target buffer.
end_sample: Optional end sample index. If None, calculated from source length.
Returns:
The target buffer (possibly extended). If target was extended, returns new array.
"""
if source.size == 0:
return target
if end_sample is None:
end_sample = start_sample + len(source)
# Extend target buffer if needed
if end_sample > len(target):
new_length = end_sample
new_target = np.concatenate([
target,
np.zeros(new_length - len(target), dtype="float32")
])
target = new_target
# Perform the mix (additive)
target[start_sample:end_sample] += source
return target
def normalize_audio(
audio: np.ndarray,
target_peak: float = 1.0,
) -> np.ndarray:
"""Normalize audio buffer to prevent clipping.
If the audio exceeds the target peak (default 1.0), it is scaled down
proportionally to prevent distortion.
Args:
audio: Input audio buffer.
target_peak: Target maximum amplitude (default 1.0).
Returns:
Normalized audio buffer (new array, original is not modified).
"""
if audio.size == 0:
return audio.copy()
max_amplitude = float(np.abs(audio).max())
if max_amplitude <= target_peak:
return audio.copy()
# Scale down to prevent clipping
scale_factor = target_peak / max_amplitude
return (audio * scale_factor).astype("float32")
def ensure_buffer_size(
buffer: np.ndarray,
min_samples: int,
) -> np.ndarray:
"""Ensure audio buffer is at least min_samples long.
If buffer is shorter, it is extended with zeros.
Args:
buffer: Input audio buffer.
min_samples: Minimum required length in samples.
Returns:
Buffer of at least min_samples length (new array if extended).
"""
if len(buffer) >= min_samples:
return buffer
new_buffer = np.zeros(min_samples, dtype="float32")
new_buffer[:len(buffer)] = buffer
return new_buffer
def concatenate_audio(*buffers: np.ndarray) -> np.ndarray:
"""Concatenate multiple audio buffers.
Args:
*buffers: Audio buffers to concatenate.
Returns:
Single concatenated audio buffer.
"""
non_empty = [b for b in buffers if b.size > 0]
if not non_empty:
return np.array([], dtype="float32")
return np.concatenate(non_empty)
def audio_duration(audio: np.ndarray, sample_rate: int = SAMPLE_RATE) -> float:
"""Calculate duration of audio buffer in seconds.
Args:
audio: Audio buffer.
sample_rate: Sample rate in Hz (default SAMPLE_RATE).
Returns:
Duration in seconds.
"""
return len(audio) / sample_rate
def samples_for_duration(duration_seconds: float, sample_rate: int = SAMPLE_RATE) -> int:
"""Calculate number of samples for a given duration.
Args:
duration_seconds: Duration in seconds.
sample_rate: Sample rate in Hz (default SAMPLE_RATE).
Returns:
Number of samples (rounded to nearest integer), or 0 if duration is <= 0.
"""
if duration_seconds <= 0:
return 0
return int(round(duration_seconds * sample_rate))
+131
View File
@@ -0,0 +1,131 @@
"""Audio sink abstraction for unified audio output.
Provides a context-manager-based abstraction for writing audio data
to various output formats (WAV, FLAC via soundfile; compressed via ffmpeg).
Usage:
with open_audio_sink(path, "wav") as sink:
sink.write(audio_data)
"""
from __future__ import annotations
import os
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Optional
import numpy as np
from abogen.domain.audio_buffer import SAMPLE_RATE
from abogen.domain.audio_helpers import build_ffmpeg_command
@dataclass(frozen=True)
class AudioSink:
"""Represents an open audio output target."""
write: Callable[[np.ndarray], None]
close: Callable[[], None]
def __enter__(self) -> AudioSink:
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.close()
def _ensure_ffmpeg() -> None:
"""Ensure static ffmpeg binaries are on PATH."""
import static_ffmpeg # type: ignore
ffmpeg_cache_root = _get_ffmpeg_cache_root()
platform_cache = os.path.join(ffmpeg_cache_root, sys.platform)
os.makedirs(platform_cache, exist_ok=True)
try:
import static_ffmpeg.run as static_ffmpeg_run # type: ignore
static_ffmpeg_run.LOCK_FILE = os.path.join(ffmpeg_cache_root, "lock.file")
except Exception:
pass
static_ffmpeg.add_paths(weak=True, download_dir=platform_cache)
def _get_ffmpeg_cache_root() -> str:
from abogen.infrastructure.cache import get_internal_cache_path
return get_internal_cache_path("ffmpeg")
def open_audio_sink(
path: Path,
fmt: str,
*,
metadata: Optional[dict[str, str]] = None,
cancel_check: Optional[Callable[[], bool]] = None,
extra_ffmpeg_args: Optional[list[str]] = None,
ffmpeg_cmd: Optional[list[str]] = None,
) -> AudioSink:
"""Open an audio output sink for writing raw float32 PCM samples.
Args:
path: Output file path.
fmt: Output format ("wav", "flac", "mp3", "opus", "m4b").
metadata: Optional metadata dict (ignored when ffmpeg_cmd is provided).
cancel_check: Optional callable; if it returns True, writes are silently skipped.
extra_ffmpeg_args: Optional extra args inserted after ffmpeg header (ignored when ffmpeg_cmd is provided).
ffmpeg_cmd: Optional pre-built ffmpeg command list (for m4b with cover art etc.).
Returns:
AudioSink with write() and close() methods.
"""
fmt = fmt.lower()
if fmt in {"wav", "flac"}:
import soundfile as sf
soundfile_obj = sf.SoundFile(
path,
mode="w",
samplerate=SAMPLE_RATE,
channels=1,
format=fmt.upper(),
)
def _write_wav(data: np.ndarray) -> None:
if cancel_check and cancel_check():
return
soundfile_obj.write(data)
def _close_wav() -> None:
soundfile_obj.close()
return AudioSink(write=_write_wav, close=_close_wav)
# Compressed formats: pipe through ffmpeg
_ensure_ffmpeg()
if ffmpeg_cmd is not None:
cmd = list(ffmpeg_cmd)
else:
cmd = build_ffmpeg_command(path, fmt, metadata=metadata)
if extra_ffmpeg_args:
cmd[2:2] = extra_ffmpeg_args
process = subprocess.Popen(
cmd, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
)
def _write_compressed(data: np.ndarray) -> None:
if (cancel_check and cancel_check()) or process.stdin is None or process.stdin.closed:
return
process.stdin.write(data.tobytes())
def _close_compressed() -> None:
if process.stdin and not process.stdin.closed:
process.stdin.close()
process.wait()
return AudioSink(write=_write_compressed, close=_close_compressed)
+131
View File
@@ -0,0 +1,131 @@
"""Heuristics for classifying chapters as content vs. supplements.
A 'supplement' is any non-story material that a listener would typically
skip: title page, copyright, table of contents, acknowledgements, etc.
The scoring functions return a float; higher ⇒ more likely to be a
supplement. ``should_preselect_chapter`` turns that score into a
boolean suitable for a web form default.
"""
from __future__ import annotations
import re
from typing import Any, Dict, List, Tuple
# Compiled once at module load these are immutable.
_SUPPLEMENT_TITLE_PATTERNS: List[Tuple[re.Pattern[str], float]] = [
(re.compile(r"\btitle\s+page\b"), 3.0),
(re.compile(r"\bcopyright\b"), 2.4),
(re.compile(r"\btable\s+of\s+contents\b"), 2.8),
(re.compile(r"\bcontents\b"), 2.0),
(re.compile(r"\backnowledg(e)?ments?\b"), 2.0),
(re.compile(r"\bdedication\b"), 2.0),
(re.compile(r"\babout\s+the\s+author(s)?\b"), 2.4),
(re.compile(r"\balso\s+by\b"), 2.0),
(re.compile(r"\bpraise\s+for\b"), 2.0),
(re.compile(r"\bcolophon\b"), 2.2),
(re.compile(r"\bpublication\s+data\b"), 2.2),
(re.compile(r"\btranscriber'?s?\s+note\b"), 2.2),
(re.compile(r"\bglossary\b"), 2.2),
(re.compile(r"\bindex\b"), 2.0),
(re.compile(r"\bbibliograph(y|ies)\b"), 2.0),
(re.compile(r"\breferences\b"), 1.8),
(re.compile(r"\bappendix\b"), 1.9),
]
_CONTENT_TITLE_PATTERNS: List[re.Pattern[str]] = [
re.compile(r"\bchapter\b"),
re.compile(r"\bbook\b"),
re.compile(r"\bpart\b"),
re.compile(r"\bsection\b"),
re.compile(r"\bscene\b"),
re.compile(r"\bprologue\b"),
re.compile(r"\bepilogue\b"),
re.compile(r"\bintroduction\b"),
re.compile(r"\bstory\b"),
]
_SUPPLEMENT_TEXT_KEYWORDS: List[Tuple[str, float]] = [
("copyright", 1.2),
("all rights reserved", 1.1),
("isbn", 0.9),
("library of congress", 1.0),
("table of contents", 1.0),
("dedicated to", 0.8),
("acknowledg", 0.8),
("printed in", 0.6),
("permission", 0.6),
("publisher", 0.5),
("praise for", 0.9),
("also by", 0.9),
("glossary", 0.8),
("index", 0.8),
("newsletter", 3.2),
("mailing list", 2.6),
("sign-up", 2.2),
]
def supplement_score(title: str, text: str, index: int) -> float:
"""Return a score indicating how likely *title*/*text* is a supplement.
Higher values ⇒ more likely to be non-story material (title page,
copyright, acknowledgements, etc.).
"""
normalized_title = (title or "").lower()
score = 0.0
for pattern, weight in _SUPPLEMENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score += weight
for pattern in _CONTENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score -= 2.0
stripped_text = (text or "").strip()
length = len(stripped_text)
if length <= 150:
score += 0.9
elif length <= 400:
score += 0.6
elif length <= 800:
score += 0.35
lowercase_text = stripped_text.lower()
for keyword, weight in _SUPPLEMENT_TEXT_KEYWORDS:
if keyword in lowercase_text:
score += weight
if index == 0 and score > 0:
score += 0.25
return score
def should_preselect_chapter(
title: str,
text: str,
index: int,
total_count: int,
) -> bool:
"""Return True if the chapter should be *enabled* by default in the form.
A single chapter is always preselected. For multi-chapter books, the
chapter is preselected when its supplement score is below 1.9.
"""
if total_count <= 1:
return True
score = supplement_score(title, text, index)
return score < 1.9
def ensure_at_least_one_chapter_enabled(chapters: List[Dict[str, Any]]) -> None:
"""Mutate *chapters* in-place so that at least one has ``enabled=True``."""
if not chapters:
return
if any(chapter.get("enabled") for chapter in chapters):
return
best_index = max(range(len(chapters)), key=lambda idx: chapters[idx].get("characters", 0))
chapters[best_index]["enabled"] = True
+191
View File
@@ -0,0 +1,191 @@
"""Metadata extraction and processing utilities.
This module provides functions for extracting metadata from text content
and generating ffmpeg metadata arguments.
"""
from __future__ import annotations
import datetime
import os
import re
from pathlib import Path
from typing import Dict, List, Optional, Tuple
def extract_metadata_from_text(text: str) -> Dict[str, Optional[str]]:
"""Extract metadata tags from text content.
Looks for tags in format: <<METADATA_KEY:value>>
Supported tags:
- TITLE, ARTIST, ALBUM, YEAR
- ALBUM_ARTIST, COMPOSER, GENRE
- COVER_PATH
Args:
text: Text content to search for metadata tags.
Returns:
Dictionary with extracted metadata values (None if not found).
"""
metadata = {}
patterns = {
"title": r"<<METADATA_TITLE:([^>]*)>>",
"artist": r"<<METADATA_ARTIST:([^>]*)>>",
"album": r"<<METADATA_ALBUM:([^>]*)>>",
"year": r"<<METADATA_YEAR:([^>]*)>>",
"album_artist": r"<<METADATA_ALBUM_ARTIST:([^>]*)>>",
"composer": r"<<METADATA_COMPOSER:([^>]*)>>",
"genre": r"<<METADATA_GENRE:([^>]*)>>",
"cover_path": r"<<METADATA_COVER_PATH:([^>]*)>>",
}
for key, pattern in patterns.items():
match = re.search(pattern, text)
if match:
metadata[key] = match.group(1).strip()
else:
metadata[key] = None
return metadata
def get_filename_from_path(
file_path: str,
display_path: Optional[str] = None,
from_queue: bool = False,
) -> str:
"""Extract filename (without extension) from path.
Args:
file_path: The file path to extract from.
display_path: Optional display path (used if from_queue is False).
from_queue: Whether the file is from queue.
Returns:
Filename without extension.
"""
if from_queue:
base_path = file_path
else:
base_path = display_path if display_path else file_path
filename = os.path.splitext(os.path.basename(base_path))[0]
return filename
def build_ffmpeg_metadata_args(
metadata: Dict[str, Optional[str]],
filename: str,
) -> List[str]:
"""Build ffmpeg metadata arguments from metadata dictionary.
Args:
metadata: Dictionary with metadata keys and values.
filename: Fallback filename for title/album if not specified.
Returns:
List of ffmpeg metadata arguments.
"""
args = []
# Default values
defaults = {
"title": filename,
"artist": "Unknown",
"album": filename,
"date": str(datetime.datetime.now().year),
"album_artist": "Unknown",
"composer": "Narrator",
"genre": "Audiobook",
}
# Map of metadata keys to ffmpeg metadata keys
key_mapping = {
"title": "title",
"artist": "artist",
"album": "album",
"year": "date", # year -> date for ffmpeg
"album_artist": "album_artist",
"composer": "composer",
"genre": "genre",
}
for metadata_key, ffmpeg_key in key_mapping.items():
value = metadata.get(metadata_key)
if value is None:
value = defaults.get(metadata_key, "")
if value:
args.extend(["-metadata", f"{ffmpeg_key}={value}"])
return args
def extract_metadata_and_build_args(
text: str,
filename: str,
display_path: Optional[str] = None,
from_queue: bool = False,
) -> Tuple[List[str], Optional[str]]:
"""Extract metadata from text and build ffmpeg arguments.
Convenience function that combines extract_metadata_from_text and
build_ffmpeg_metadata_args.
Args:
text: Text content to search for metadata tags.
filename: Fallback filename for title/album.
display_path: Optional display path.
from_queue: Whether the file is from queue.
Returns:
Tuple of (ffmpeg_metadata_args, cover_path).
"""
metadata = extract_metadata_from_text(text)
cover_path = metadata.get("cover_path")
# Get actual filename from path
actual_filename = get_filename_from_path(
file_path=filename,
display_path=display_path,
from_queue=from_queue,
)
args = build_ffmpeg_metadata_args(metadata, actual_filename)
return args, cover_path
def read_text_for_metadata(
file_path: str,
is_direct_text: bool,
direct_text: Optional[str] = None,
encoding: Optional[str] = None,
) -> str:
"""Read text content for metadata extraction.
Args:
file_path: Path to file (or text if is_direct_text).
is_direct_text: Whether file_path contains direct text.
direct_text: Optional direct text (used if is_direct_text).
encoding: File encoding (detected if not provided).
Returns:
Text content for metadata extraction.
"""
if is_direct_text:
return direct_text or file_path
# Read from file
actual_path = direct_text if direct_text else file_path
try:
if encoding is None:
from abogen.utils import detect_encoding
encoding = detect_encoding(actual_path)
with open(actual_path, "r", encoding=encoding, errors="replace") as f:
return f.read()
except Exception:
return ""
+267
View File
@@ -1,6 +1,9 @@
from __future__ import annotations
import json
import math
import re
from pathlib import Path
from typing import Any, Dict, List, Mapping, Optional, Tuple
@@ -136,3 +139,267 @@ def format_series_sentence(series_name: Optional[str], series_number: Optional[s
article = "the " if not name.lower().startswith("the ") else ""
phrase = f"Book {number} of {article}{name}"
return re.sub(r"\s+", " ", phrase).strip()
_PEOPLE_SPLIT_RE = re.compile(r"[;,/&]|\band\b", re.IGNORECASE)
_LIST_SPLIT_RE = re.compile(r"[;,\n]")
_SERIES_SEQUENCE_TAG_KEYS: Tuple[str, ...] = (
"series_index",
"series_position",
"series_sequence",
"series_number",
"seriesnumber",
"book_number",
"booknumber",
)
def normalize_metadata_casefold(values: Optional[Mapping[str, Any]]) -> Dict[str, Any]:
normalized: Dict[str, Any] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
key_text = str(key).strip().lower()
if not key_text:
continue
if isinstance(value, (list, tuple, set)):
normalized[key_text] = value
else:
text = str(value).strip()
if text:
normalized[key_text] = text
return normalized
def split_people_field(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(split_people_field(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _PEOPLE_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def split_simple_list(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(split_simple_list(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _LIST_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def first_nonempty(*values: Any) -> Optional[str]:
for value in values:
if value is None:
continue
if isinstance(value, (list, tuple, set)):
items = list(value)
if not items:
continue
value = items[0]
text = str(value).strip()
if text:
return text
return None
def extract_year(raw: Optional[str]) -> Optional[int]:
if not raw:
return None
text = str(raw).strip()
if not text:
return None
match = re.search(r"(19|20)\d{2}", text)
if match:
try:
return int(match.group(0))
except ValueError:
return None
try:
parsed = int(text)
except ValueError:
return None
if 0 < parsed < 3000:
return parsed
return None
def normalize_series_sequence(raw: Any) -> Optional[str]:
if raw is None:
return None
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return None
text = str(raw)
else:
text = str(raw).strip()
if not text:
return None
candidate = text.replace(",", ".")
match = _SERIES_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
def build_audiobookshelf_metadata(
tags: Mapping[str, Any],
*,
language: str = "",
filename: str = "",
) -> Dict[str, Any]:
normalized = normalize_metadata_casefold(tags)
title = first_nonempty(
normalized.get("title"),
normalized.get("book_title"),
normalized.get("name"),
normalized.get("album"),
filename,
)
authors = split_people_field(
normalized.get("authors")
or normalized.get("author")
or normalized.get("album_artist")
or normalized.get("artist")
)
narrators = split_people_field(normalized.get("narrators") or normalized.get("narrator"))
description = first_nonempty(
normalized.get("description"), normalized.get("summary"), normalized.get("comment")
)
genres = split_simple_list(normalized.get("genre"))
keywords = split_simple_list(normalized.get("tags") or normalized.get("keywords"))
lang = first_nonempty(normalized.get("language"), normalized.get("lang")) or language or ""
series_name = first_nonempty(
normalized.get("series"),
normalized.get("series_name"),
normalized.get("seriesname"),
normalized.get("series_title"),
normalized.get("seriestitle"),
)
series_sequence = None
for key in _SERIES_SEQUENCE_TAG_KEYS:
raw_value = normalized.get(key)
seq = normalize_series_sequence(raw_value)
if seq:
series_sequence = seq
break
if not series_name:
series_sequence = None
data: Dict[str, Any] = {
"title": title,
"subtitle": normalized.get("subtitle"),
"authors": authors,
"narrators": narrators,
"description": description,
"publisher": normalized.get("publisher"),
"genres": genres,
"tags": keywords,
"language": lang,
"publishedYear": extract_year(
normalized.get("published")
or normalized.get("publication_year")
or normalized.get("date")
or normalized.get("year")
),
"seriesName": series_name,
"seriesSequence": series_sequence,
"isbn": first_nonempty(normalized.get("isbn"), normalized.get("asin")),
}
published_date = first_nonempty(
normalized.get("published"), normalized.get("publication_date"), normalized.get("date")
)
if published_date:
data["publishedDate"] = published_date
rating_text = first_nonempty(normalized.get("rating"), normalized.get("my_rating"))
if rating_text:
try:
data["rating"] = float(str(rating_text).strip())
except ValueError:
pass
rating_max_text = first_nonempty(
normalized.get("rating_max"), normalized.get("rating_scale")
)
if rating_max_text:
try:
data["ratingMax"] = float(str(rating_max_text).strip())
except ValueError:
pass
cleaned: Dict[str, Any] = {}
for key, value in data.items():
if value is None:
continue
if isinstance(value, str) and not value.strip():
continue
if isinstance(value, (list, tuple)) and not value:
continue
cleaned[key] = value
return cleaned
def load_audiobookshelf_chapters(
metadata_path: Path,
) -> Optional[List[Dict[str, Any]]]:
if not metadata_path.exists():
return None
try:
payload = json.loads(metadata_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
chapters = payload.get("chapters")
if not isinstance(chapters, list):
return None
cleaned: List[Dict[str, Any]] = []
for entry in chapters:
if not isinstance(entry, Mapping):
continue
title = first_nonempty(entry.get("title"), entry.get("original_title"))
start = entry.get("start")
end = entry.get("end")
if title and start is not None and end is not None:
cleaned.append({"title": str(title), "start": start, "end": end})
return cleaned or None
+68 -2
View File
@@ -1,8 +1,15 @@
"""Text normalization convenience helpers."""
"""Text normalization convenience helpers.
Provides both the simple ``normalize_text_for_pipeline`` (apostrophe + LLM only)
and the comprehensive ``prepare_text_for_tts`` that chains all three normalization
stages used during conversion: heteronym rules → pronunciation rules → pipeline
normalization. The latter is the single entry point that both the Web UI and
PyQt Desktop GUI should use.
"""
from __future__ import annotations
from typing import Any, Mapping, Optional
from typing import Any, Dict, List, Mapping, Optional
from abogen.kokoro_text_normalization import (
ApostropheConfig,
@@ -28,3 +35,62 @@ def normalize_text_for_pipeline(
runtime_settings = _apply_overrides(runtime_settings, normalization_overrides)
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
return _normalize_for_pipeline(text, config=apostrophe_config, settings=runtime_settings)
def prepare_text_for_tts(
text: str,
*,
heteronym_rules: Optional[List[Dict[str, Any]]] = None,
pronunciation_rules: Optional[List[Dict[str, Any]]] = None,
normalization_overrides: Optional[Mapping[str, Any]] = None,
usage_counter: Optional[Dict[str, int]] = None,
) -> str:
"""Apply the full text normalization pipeline before TTS synthesis.
Chains three stages in order:
1. Heteronym sentence rules (context-dependent pronunciation)
2. Pronunciation rules (token-level replacements)
3. Pipeline normalization (apostrophe handling, LLM normalization)
This is the **single entry point** that both the Web UI conversion runner
and the PyQt conversion thread should call before passing text to the TTS
backend.
Parameters
----------
text:
Raw text to normalize.
heteronym_rules:
Compiled heteronym rules from ``compile_heteronym_sentence_rules``.
pronunciation_rules:
Compiled pronunciation rules from ``compile_pronunciation_rules``.
normalization_overrides:
User-level overrides for normalization settings (apostrophe mode, etc.).
usage_counter:
Mutable dict that tracks how many times each pronunciation override was
applied. Passed through to ``apply_pronunciation_rules``.
Returns
-------
str
Fully normalized text ready for TTS.
"""
from abogen.domain.pronunciation import (
apply_heteronym_sentence_rules,
apply_pronunciation_rules,
)
result = str(text or "")
if heteronym_rules:
result = apply_heteronym_sentence_rules(result, heteronym_rules)
if pronunciation_rules:
result = apply_pronunciation_rules(result, pronunciation_rules, usage_counter)
runtime_settings = get_runtime_settings()
if normalization_overrides:
runtime_settings = _apply_overrides(runtime_settings, normalization_overrides)
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
return _normalize_for_pipeline(result, config=apostrophe_config, settings=runtime_settings)
+72
View File
@@ -0,0 +1,72 @@
from __future__ import annotations
"""Progress and ETR (estimated time remaining) calculation.
Shared by Web UI and PyQt desktop GUI. Pure math, no UI dependencies.
"""
import time
from dataclasses import dataclass, field
@dataclass
class ProgressTracker:
"""Tracks character-based progress with ETR calculation.
Usage:
tracker = ProgressTracker(total_chars=50000)
# ... as processing occurs:
tracker.update(chars_done=5000)
print(tracker.etr_str) # "00:04:30"
print(tracker.percent) # 10
"""
total_chars: int
_start_time: float = field(default_factory=time.time, repr=False)
_chars_done: int = field(default=0, repr=False)
def update(self, chars_done: int) -> None:
self._chars_done = chars_done
@property
def percent(self) -> int:
if self.total_chars <= 0:
return 0
return min(int(self._chars_done / self.total_chars * 100), 99)
@property
def etr_str(self) -> str:
elapsed = time.time() - self._start_time
if self._chars_done <= 0 or elapsed <= 0.5:
return "Processing..."
avg_time_per_char = elapsed / self._chars_done
remaining = self.total_chars - self._chars_done
if remaining <= 0:
return "00:00:00"
secs = avg_time_per_char * remaining
h = int(secs // 3600)
m = int((secs % 3600) // 60)
s = int(secs % 60)
return f"{h:02d}:{m:02d}:{s:02d}"
def calc_etr_str(elapsed: float, done: int, total: int) -> str:
"""Standalone ETR string calculation (matches PyQt original logic).
Args:
elapsed: seconds since processing started
done: items/characters processed so far
total: total items/characters to process
Returns:
ETR string like "01:23:45" or "Processing..."
"""
if done <= 0 or elapsed <= 0.5:
return "Processing..."
avg_time_per_item = elapsed / done
remaining = total - done
if remaining <= 0:
return "00:00:00"
secs = avg_time_per_item * remaining
h = int(secs // 3600)
m = int((secs % 3600) // 60)
s = int(secs % 60)
return f"{h:02d}:{m:02d}:{s:02d}"
+358
View File
@@ -0,0 +1,358 @@
"""Subtitle generation utilities for audiobook generation.
This module provides functions for processing TTS tokens into subtitle entries
according to various subtitle modes (Line, Sentence, Sentence + Comma,
Sentence + Highlighting).
"""
from __future__ import annotations
import re
from typing import List, Optional, Tuple
# Punctuation constants for sentence splitting
PUNCTUATION_SENTENCE = ".!?\u061f\u3002\uff01\uff1f" # .!? .?. ??
PUNCTUATION_SENTENCE_COMMA = ".!?,\u3001\u061f\u3002\uff01\uff0c\uff1f" # .!?, ,. ??
def process_subtitle_tokens(
tokens_with_timestamps: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
lang_code: str,
use_spacy_segmentation: bool = False,
fallback_end_time: Optional[float] = None,
) -> None:
"""Process TTS tokens into subtitle entries according to the subtitle mode.
This function modifies subtitle_entries in-place by appending new entries.
Args:
tokens_with_timestamps: List of token dictionaries with 'start', 'end', 'text',
and 'whitespace' keys.
subtitle_entries: List to append subtitle entries to (modified in-place).
Each entry is a tuple of (start_time, end_time, text).
max_subtitle_words: Maximum number of words per subtitle entry.
subtitle_mode: One of "Disabled", "Line", "Sentence", "Sentence + Comma",
"Sentence + Highlighting", or a string like "5" for word-count mode.
lang_code: Language code for spaCy processing (e.g., "a" for English).
use_spacy_segmentation: Whether to use spaCy for sentence boundary detection.
fallback_end_time: Fallback end time for the last entry if none is available.
"""
if not tokens_with_timestamps:
return
processed_tokens = tokens_with_timestamps
# For English with spaCy enabled and sentence-based modes, use spaCy for sentence boundaries
# spaCy is disabled when subtitle mode is "Disabled" or "Line"
use_spacy_for_english = (
use_spacy_segmentation
and subtitle_mode not in ["Disabled", "Line"]
and lang_code in ["a", "b"]
and subtitle_mode in ["Sentence", "Sentence + Comma"]
)
if subtitle_mode == "Sentence + Highlighting":
_process_karaoke_highlighting(
processed_tokens, subtitle_entries, max_subtitle_words, fallback_end_time
)
elif subtitle_mode in ["Sentence", "Sentence + Comma", "Line"]:
if use_spacy_for_english and subtitle_mode != "Line":
_process_spacy_sentences(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, lang_code, fallback_end_time
)
else:
_process_regex_sentences(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
else:
# Word count-based grouping (e.g., "5" for 5-word groups)
_process_word_count(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
def _process_karaoke_highlighting(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens for Sentence + Highlighting mode (karaoke effect)."""
separator = rf"[{re.escape(PUNCTUATION_SENTENCE)}]"
current_sentence = []
word_count = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
# Split sentences based on separator or word count
if (
re.search(separator, token["text"]) and token.get("whitespace") == " "
) or word_count >= max_subtitle_words:
if current_sentence:
# Create karaoke subtitle entry for this sentence
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Generate karaoke text with timing
karaoke_text = ""
for t in current_sentence:
# Calculate duration in centiseconds
duration = (
t["end"] - t["start"]
if t.get("end") is not None and t.get("start") is not None
else 0.5
)
duration_cs = int(duration * 100)
# Add karaoke effect
karaoke_text += f"{{\\kf{duration_cs}}}{t['text']}{t.get('whitespace', '') or ''}"
subtitle_entries.append(
(start_time, end_time, karaoke_text.strip())
)
current_sentence = []
word_count = 0
# Add any remaining tokens as a sentence
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Generate karaoke text for remaining tokens
karaoke_text = ""
for t in current_sentence:
duration = t["end"] - t["start"] if t.get("end") and t.get("start") else 0.5
duration_cs = int(duration * 100)
karaoke_text += f"{{\\kf{duration_cs}}}{t['text']}{t.get('whitespace', '') or ''}"
subtitle_entries.append((start_time, end_time, karaoke_text.strip()))
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_spacy_sentences(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
lang_code: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens using spaCy for sentence boundary detection."""
try:
from abogen.spacy_utils import get_spacy_model
except ImportError:
# Fall back to regex if spaCy is not available
_process_regex_sentences(
tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
return
nlp = get_spacy_model(lang_code)
if not nlp:
_process_regex_sentences(
tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
return
# Build full text and track character positions to token indices
full_text = ""
for token in tokens:
text_part = token["text"] + (token.get("whitespace") or "")
full_text += text_part
# Get sentence boundaries from spaCy
doc = nlp(full_text)
sentence_boundaries = [sent.end_char for sent in doc.sents]
# For "Sentence + Comma" mode, also split on commas
if subtitle_mode == "Sentence + Comma":
comma_positions = [
i + 1 for i, c in enumerate(full_text) if c == ","
]
sentence_boundaries = sorted(
set(sentence_boundaries + comma_positions)
)
# Group tokens by sentence boundaries
current_sentence = []
word_count = 0
current_char_pos = 0
boundary_idx = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
text_len = len(token["text"]) + len(token.get("whitespace") or "")
current_char_pos += text_len
# Check if we've hit a sentence boundary or max words
at_boundary = (
boundary_idx < len(sentence_boundaries)
and current_char_pos >= sentence_boundaries[boundary_idx]
)
if at_boundary or word_count >= max_subtitle_words:
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = "".join(
t["text"] + (t.get("whitespace") or "")
for t in current_sentence
)
subtitle_entries.append(
(start_time, end_time, sentence_text.strip())
)
current_sentence = []
word_count = 0
if at_boundary:
boundary_idx += 1
# Add remaining tokens
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = "".join(
t["text"] + (t.get("whitespace") or "")
for t in current_sentence
)
subtitle_entries.append(
(start_time, end_time, sentence_text.strip())
)
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_regex_sentences(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens using regex for sentence boundary detection."""
# Define separator pattern based on mode
if subtitle_mode == "Line":
separator = r"\n"
elif subtitle_mode == "Sentence":
# Use punctuation without comma
separator = rf"[{re.escape(PUNCTUATION_SENTENCE)}]"
else: # Sentence + Comma
# Use punctuation with comma
separator = rf"[{re.escape(PUNCTUATION_SENTENCE_COMMA)}]"
current_sentence = []
word_count = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
# Split sentences based on separator or word count
if (
re.search(separator, token["text"]) and token.get("whitespace") == " "
) or word_count >= max_subtitle_words:
if current_sentence:
# Create subtitle entry for this sentence
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Simplified text joining logic
sentence_text = ""
for t in current_sentence:
sentence_text += t["text"] + (t.get("whitespace") or "")
subtitle_entries.append(
(start_time, end_time, sentence_text.strip())
)
current_sentence = []
word_count = 0
# Add any remaining tokens as a sentence
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Simplified text joining logic
sentence_text = ""
for t in current_sentence:
sentence_text += t["text"] + (t.get("whitespace") or "")
subtitle_entries.append((start_time, end_time, sentence_text.strip()))
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_word_count(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens by counting spaces (word count mode)."""
try:
word_count = int(subtitle_mode.split()[0])
word_count = min(word_count, max_subtitle_words)
except (ValueError, IndexError):
word_count = 1
current_group = []
space_count = 0
for token in tokens:
current_group.append(token)
# Count spaces after tokens (in the whitespace field)
if token.get("whitespace", "") == " ":
space_count += 1
# Split after counting N spaces
if space_count >= word_count:
text = "".join(
t["text"] + (t.get("whitespace") or "")
for t in current_group
)
subtitle_entries.append(
(
current_group[0]["start"],
current_group[-1]["end"],
text.strip(),
)
)
current_group = []
space_count = 0
# Add any remaining tokens
if current_group:
text = "".join(
t["text"] + (t.get("whitespace") or "") for t in current_group
)
subtitle_entries.append(
(current_group[0]["start"], current_group[-1]["end"], text.strip())
)
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _apply_fallback_end_time(
subtitle_entries: List[Tuple[float, float, str]],
fallback_end_time: Optional[float],
) -> None:
"""Apply fallback end time to the last entry if needed."""
if subtitle_entries and fallback_end_time is not None:
last_entry = subtitle_entries[-1]
start, end, text = last_entry
if end is None or end <= start or end <= 0:
subtitle_entries[-1] = (start, fallback_end_time, text)
+13
View File
@@ -0,0 +1,13 @@
"""Shared token stubs for TTS processing."""
from __future__ import annotations
class FakeToken:
"""Minimal token stub for languages without per-word token support."""
def __init__(self, text: str, start: float, end: float):
self.text = text
self.start_ts = start
self.end_ts = end
self.whitespace = ""
+116
View File
@@ -0,0 +1,116 @@
"""Voice loading and caching utilities.
This module provides unified voice loading with caching support for both
PyQt and WebUI interfaces.
"""
from __future__ import annotations
from typing import Any, Dict, Optional, Tuple
from abogen.voice_formulas import get_new_voice
class VoiceCache:
"""Thread-safe voice cache for loaded voice tensors."""
def __init__(self):
self._cache: Dict[str, Any] = {}
def get(self, voice_spec: str) -> Optional[Any]:
"""Get cached voice by spec."""
return self._cache.get(voice_spec)
def set(self, voice_spec: str, voice: Any) -> None:
"""Cache a loaded voice."""
self._cache[voice_spec] = voice
def contains(self, voice_spec: str) -> bool:
"""Check if voice is in cache."""
return voice_spec in self._cache
def clear(self) -> None:
"""Clear all cached voices."""
self._cache.clear()
def __contains__(self, voice_spec: str) -> bool:
return self.contains(voice_spec)
def resolve_voice(
voice_spec: str,
pipeline: Any,
use_gpu: bool,
cache: Optional[VoiceCache] = None,
) -> Any:
"""Resolve voice spec to actual voice tensor or name.
If voice_spec contains '*' (formula), loads the voice using get_new_voice.
Otherwise, returns the voice_spec as-is (it's a voice name).
Uses optional cache to avoid reloading same voice multiple times.
Args:
voice_spec: Voice specification (name or formula string with '*').
pipeline: TTS pipeline instance for loading formula voices.
use_gpu: Whether to use GPU for voice loading.
cache: Optional VoiceCache instance for caching loaded voices.
Returns:
Loaded voice tensor (for formulas) or voice name string.
"""
# Check cache first
if cache and cache.contains(voice_spec):
return cache.get(voice_spec)
# Load voice
if "*" in voice_spec:
if pipeline is None:
return voice_spec
loaded_voice = get_new_voice(pipeline, voice_spec, use_gpu)
else:
loaded_voice = voice_spec
# Cache it
if cache:
cache.set(voice_spec, loaded_voice)
return loaded_voice
def load_voice_cached(
voice_name: str,
pipeline: Any,
use_gpu: bool,
cache: Optional[Dict[str, Any]] = None,
) -> Any:
"""Load voice with caching (compatibility wrapper for PyQt).
This function maintains backward compatibility with the PyQt interface
while using the unified voice loading logic.
Args:
voice_name: Voice name or formula string.
pipeline: TTS pipeline instance.
use_gpu: Whether to use GPU.
cache: Optional dict to use as cache (instead of VoiceCache).
Returns:
Loaded voice tensor or voice name string.
"""
# Use dict cache if provided (for backward compatibility)
if cache is not None:
if voice_name in cache:
return cache[voice_name]
# Load voice
if "*" in voice_name:
loaded_voice = get_new_voice(pipeline, voice_name, use_gpu)
else:
loaded_voice = voice_name
# Cache it
if cache is not None:
cache[voice_name] = loaded_voice
return loaded_voice
+16 -241
View File
@@ -9,6 +9,17 @@ from typing import Any, Dict, List, Optional, Mapping, Sequence
import static_ffmpeg
from abogen.domain.metadata_helpers import (
normalize_metadata_casefold,
split_people_field,
split_simple_list,
first_nonempty,
extract_year,
normalize_series_sequence,
build_audiobookshelf_metadata as _build_abs_metadata,
load_audiobookshelf_chapters as _load_abs_chapters,
_SERIES_SEQUENCE_TAG_KEYS,
)
from abogen.epub3.exporter import build_epub3_package
from abogen.integrations.audiobookshelf import (
AudiobookshelfClient,
@@ -305,95 +316,12 @@ class ExportService:
def build_audiobookshelf_metadata(self, job: Any) -> Dict[str, Any]:
"""Build Audiobookshelf metadata from job."""
tags = self._normalize_metadata_casefold(getattr(job, "metadata_tags", {}))
filename = Path(getattr(job, "original_filename", "") or "").stem or "Audiobook"
title = self._first_nonempty(
tags.get("title"),
tags.get("book_title"),
tags.get("name"),
tags.get("album"),
filename,
return _build_abs_metadata(
getattr(job, "metadata_tags", {}),
language=getattr(job, "language", "") or "",
filename=filename,
)
authors = self._split_people_field(
tags.get("authors")
or tags.get("author")
or tags.get("album_artist")
or tags.get("artist")
)
narrators = self._split_people_field(tags.get("narrators") or tags.get("narrator"))
description = self._first_nonempty(
tags.get("description"), tags.get("summary"), tags.get("comment")
)
genres = self._split_simple_list(tags.get("genre"))
keywords = self._split_simple_list(tags.get("tags") or tags.get("keywords"))
language = self._first_nonempty(tags.get("language"), tags.get("lang")) or getattr(job, "language", "") or ""
series_name = self._first_nonempty(
tags.get("series"),
tags.get("series_name"),
tags.get("seriesname"),
tags.get("series_title"),
tags.get("seriestitle"),
)
series_sequence = None
for key in ("series_index", "series_position", "series_sequence", "series_number", "seriesnumber", "book_number", "booknumber"):
raw = tags.get(key)
normalized = self._normalize_series_sequence(raw)
if normalized:
series_sequence = normalized
break
if not series_name:
series_sequence = None
data = {
"title": title,
"subtitle": tags.get("subtitle"),
"authors": authors,
"narrators": narrators,
"description": description,
"publisher": tags.get("publisher"),
"genres": genres,
"tags": keywords,
"language": language,
"publishedYear": self._extract_year(
tags.get("published") or tags.get("publication_year") or tags.get("date") or tags.get("year")
),
"seriesName": series_name,
"seriesSequence": series_sequence,
"isbn": self._first_nonempty(tags.get("isbn"), tags.get("asin")),
}
published_date = self._first_nonempty(
tags.get("published"), tags.get("publication_date"), tags.get("date")
)
if published_date:
data["publishedDate"] = published_date
rating_text = self._first_nonempty(tags.get("rating"), tags.get("my_rating"))
if rating_text:
try:
data["rating"] = float(str(rating_text).strip())
except ValueError:
pass
rating_max_text = self._first_nonempty(tags.get("rating_max"), tags.get("rating_scale"))
if rating_max_text:
try:
data["ratingMax"] = float(str(rating_max_text).strip())
except ValueError:
pass
# Remove empty values
cleaned = {}
for key, value in data.items():
if value is None:
continue
if isinstance(value, str) and not value.strip():
continue
if isinstance(value, (list, tuple)) and not value:
continue
cleaned[key] = value
return cleaned
def load_audiobookshelf_chapters(self, job: Any) -> Optional[List[Dict[str, Any]]]:
"""Load chapters from job artifacts for Audiobookshelf."""
@@ -401,29 +329,7 @@ class ExportService:
if not metadata_ref:
return None
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
if not metadata_path.exists():
return None
try:
payload = json.loads(metadata_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
chapters = payload.get("chapters")
if not isinstance(chapters, list):
return None
cleaned = []
for entry in chapters:
if not isinstance(entry, Mapping):
continue
title = self._first_nonempty(entry.get("title"), entry.get("original_title"))
start = entry.get("start")
end = entry.get("end")
if title is None or not isinstance(start, (int, float)):
continue
chapter_payload = {"title": title, "start": float(start)}
if isinstance(end, (int, float)):
chapter_payload["end"] = float(end)
cleaned.append(chapter_payload)
return cleaned or None
return _load_abs_chapters(metadata_path)
def upload_audiobookshelf(
self,
@@ -520,137 +426,6 @@ class ExportService:
# Helpers
# ----------------------------------------------------------------------
@staticmethod
def _normalize_metadata_casefold(values: Optional[Mapping[str, Any]]) -> Dict[str, Any]:
normalized = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
key_text = str(key).strip().lower()
if not key_text:
continue
if isinstance(value, (list, tuple, set)):
normalized[key_text] = value
else:
text = str(value).strip()
if text:
normalized[key_text] = text
return normalized
@staticmethod
def _split_people_field(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results = []
for item in raw:
results.extend(ExportService._split_people_field(item))
return results
text = str(raw or "").strip()
if not text:
return []
import re
tokens = [token.strip() for token in re.split(r"[;,/&]|\band\b", text, flags=re.IGNORECASE) if token.strip()]
seen = set()
ordered = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
@staticmethod
def _split_simple_list(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results = []
for item in raw:
results.extend(ExportService._split_simple_list(item))
return results
text = str(raw or "").strip()
if not text:
return []
import re
tokens = [token.strip() for token in re.split(r"[;,\n]", text) if token.strip()]
seen = set()
ordered = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
@staticmethod
def _first_nonempty(*values: Any) -> Optional[str]:
for value in values:
if value is None:
continue
if isinstance(value, (list, tuple, set)):
items = list(value)
if not items:
continue
value = items[0]
text = str(value).strip()
if text:
return text
return None
@staticmethod
def _extract_year(raw: Optional[str]) -> Optional[int]:
if not raw:
return None
text = str(raw).strip()
if not text:
return None
import re
match = re.search(r"(19|20)\d{2}", text)
if match:
try:
return int(match.group(0))
except ValueError:
return None
try:
parsed = int(text)
except ValueError:
return None
if 0 < parsed < 3000:
return parsed
return None
@staticmethod
def _normalize_series_sequence(raw: Any) -> Optional[str]:
if raw is None:
return None
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (raw != raw or raw == float("inf") or raw == float("-inf")):
return None
text = str(raw)
else:
text = str(raw).strip()
if not text:
return None
candidate = text.replace(",", ".")
import re
match = re.search(r"\d+(?:\.\d+)?", candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
return normalized or "0"
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
@staticmethod
def _coerce_bool(value: Any, default: bool = True) -> bool:
if isinstance(value, bool):
+3 -36
View File
@@ -2,9 +2,7 @@ from __future__ import annotations
import json
import logging
import math
import mimetypes
import re
from contextlib import ExitStack
from dataclasses import dataclass
from pathlib import Path
@@ -12,6 +10,8 @@ from typing import Any, Dict, Iterable, List, Mapping, Optional, Sequence, Tuple
import httpx
from abogen.domain.metadata_helpers import normalize_series_sequence
logger = logging.getLogger(__name__)
@@ -641,40 +641,7 @@ class AudiobookshelfClient:
for key in preferred_keys:
if key not in metadata:
continue
normalized = AudiobookshelfClient._normalize_series_sequence(metadata.get(key))
normalized = normalize_series_sequence(metadata.get(key))
if normalized:
return normalized
return ""
@staticmethod
def _normalize_series_sequence(raw: Any) -> str:
if raw is None:
return ""
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return ""
text = str(raw)
else:
text = str(raw).strip()
if not text:
return ""
candidate = text.replace(",", ".")
match = re.search(r"\d+(?:\.\d+)?", candidate)
if not match:
return ""
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
+205 -653
View File
File diff suppressed because it is too large Load Diff
+142 -293
View File
@@ -2,8 +2,7 @@ from __future__ import annotations
import json
import os
import subprocess
import sys
import time
import traceback
import gc
from collections import defaultdict
@@ -13,10 +12,7 @@ from pathlib import Path
from typing import Any, Callable, Dict, List, Mapping, Optional
import numpy as np
import soundfile as sf
import static_ffmpeg
from abogen.tts_plugin.utils import is_plugin_registered
from abogen.infrastructure.exporters import ExportService
from abogen.epub3.exporter import build_epub3_package
from abogen.kokoro_text_normalization import ApostropheConfig, normalize_for_pipeline, HAS_NUM2WORDS
@@ -34,10 +30,7 @@ from abogen.utils import (
get_internal_cache_path,
get_user_cache_path,
get_user_output_path,
load_config,
)
from abogen.tts_plugin.utils import create_pipeline
from abogen.voice_formulas import get_new_voice
from abogen.voice_profiles import load_profiles, normalize_profile_entry
from abogen.llm_client import LLMClientError
from abogen.infrastructure.subtitle_writer import create_subtitle_writer
@@ -73,10 +66,10 @@ from abogen.domain.file_type import (
from abogen.domain.pronunciation import (
compile_pronunciation_rules as _compile_pronunciation_rules,
compile_heteronym_sentence_rules as _compile_heteronym_sentence_rules,
apply_heteronym_sentence_rules as _apply_heteronym_sentence_rules,
apply_pronunciation_rules as _apply_pronunciation_rules,
merge_pronunciation_overrides as _merge_pronunciation_overrides,
)
from abogen.domain.normalization import prepare_text_for_tts
from abogen.domain.voice_resolution import (
spec_to_voice_ids as _spec_to_voice_ids,
job_voice_fallback as _job_voice_fallback,
@@ -111,11 +104,22 @@ from abogen.domain.output_paths import (
resolve_project_layout as _resolve_project_layout,
)
from abogen.domain.device import select_device as _select_device
from abogen.domain.split_pattern import get_split_pattern
from abogen.domain.progress import ProgressTracker, calc_etr_str
from abogen.domain.subtitle_generation import process_subtitle_tokens
from abogen.domain.audio_helpers import (
build_ffmpeg_command as _build_ffmpeg_command,
to_float32 as _to_float32,
apply_m4b_chapters_with_mutagen as _apply_m4b_chapters_with_mutagen,
)
from abogen.domain.audio_buffer import (
create_silence as _create_silence,
normalize_audio as _normalize_audio,
SAMPLE_RATE,
)
from abogen.domain.audio_sink import AudioSink, open_audio_sink
from abogen.domain.tokens import FakeToken
from abogen.domain.pipeline_factory import PipelinePool
from abogen.domain.voice_utils import resolve_voice_target as _resolve_voice_target
from .service import Job, JobStatus
@@ -123,7 +127,7 @@ from .service import Job, JobStatus
_export_svc = ExportService()
SPLIT_PATTERN = r"\n+"
SPLIT_PATTERN = r"\n+" # Kept for backward compatibility; prefer get_split_pattern()
SAMPLE_RATE = 24000
@@ -131,120 +135,9 @@ class _JobCancelled(Exception):
"""Raised internally to abort a conversion when the client cancels."""
@dataclass
class AudioSink:
write: Callable[[np.ndarray], None]
_APOSTROPHE_CONFIG = ApostropheConfig()
def _apply_m4b_chapters_with_mutagen(
audio_path: Path,
chapters: List[Dict[str, Any]],
job: Job,
) -> bool:
try:
return _apply_m4b_chapters_with_mutagen(audio_path, chapters)
except ImportError:
job.add_log(
"Unable to write MP4 chapter atoms because mutagen is not installed.",
level="warning",
)
return False
except Exception as exc:
job.add_log(f"Failed to write MP4 chapter atoms: {exc}", level="warning")
return False
def _embed_m4b_metadata(
audio_path: Path,
metadata_payload: Dict[str, Any],
job: Job,
) -> None:
metadata_map = dict(metadata_payload.get("metadata") or {})
chapter_entries = list(metadata_payload.get("chapters") or [])
ffmetadata_path = _export_svc.write_ffmetadata_file(audio_path, metadata_map, chapter_entries)
cover_path: Optional[Path] = None
if job.cover_image_path:
candidate = Path(job.cover_image_path)
if candidate.exists():
cover_path = candidate
metadata_args = _export_svc._metadata_to_ffmpeg_args(metadata_map)
if not ffmetadata_path and not cover_path and not metadata_args:
return
job.add_log("Embedding metadata into m4b output")
command: List[str] = ["ffmpeg", "-y", "-i", str(audio_path)]
metadata_index: Optional[int] = None
cover_index: Optional[int] = None
next_index = 1
if ffmetadata_path:
command += ["-f", "ffmetadata", "-i", str(ffmetadata_path)]
metadata_index = next_index
next_index += 1
if cover_path:
command += ["-i", str(cover_path)]
cover_index = next_index
next_index += 1
command += ["-map", "0:a"]
command += ["-c:a", "copy"]
if cover_index is not None:
command += ["-map", f"{cover_index}:v:0"]
command += ["-c:v:0", "mjpeg"]
command += ["-disposition:v:0", "attached_pic"]
command += ["-metadata:s:v:0", "title=Cover Art"]
if job.cover_image_mime:
command += ["-metadata:s:v:0", f"mimetype={job.cover_image_mime}"]
if metadata_index is not None:
command += ["-map_metadata", str(metadata_index)]
command += ["-map_chapters", str(metadata_index)]
else:
command += ["-map_metadata", "0"]
if metadata_args:
command.extend(metadata_args)
command += ["-movflags", "+faststart+use_metadata_tags"]
temp_output = audio_path.with_suffix(audio_path.suffix + ".tmp")
if audio_path.suffix.lower() in {".m4b", ".mp4", ".m4a"}:
command += ["-f", "mp4"]
command.append(str(temp_output))
process = create_process(command, text=True)
try:
return_code = process.wait()
finally:
if ffmetadata_path and ffmetadata_path.exists():
try:
ffmetadata_path.unlink()
except OSError:
pass
if return_code != 0:
if temp_output.exists():
temp_output.unlink(missing_ok=True)
raise RuntimeError(f"ffmpeg failed to embed metadata (exit code {return_code})")
temp_output.replace(audio_path)
job.add_log("Embedded metadata and chapters into m4b output", level="info")
mutagen_applied = _apply_m4b_chapters_with_mutagen(audio_path, chapter_entries, job)
if mutagen_applied:
job.add_log(
f"Applied {len(chapter_entries)} chapter markers via mutagen", level="info"
)
def run_conversion_job(job: Job) -> None:
job.add_log("Preparing conversion pipeline")
canceller = _make_canceller(job)
@@ -274,6 +167,12 @@ def run_conversion_job(job: Job) -> None:
"LLM-based apostrophe normalization is selected, but the LLM configuration is incomplete."
)
# Compute language-aware split pattern once for the entire job
job_split_pattern = get_split_pattern(
str(job.language or "a"),
str(job.subtitle_mode or "Disabled"),
)
sink_stack = ExitStack()
subtitle_writer = None
chapter_paths: list[Path] = []
@@ -283,8 +182,7 @@ def run_conversion_job(job: Job) -> None:
audio_output_path: Optional[Path] = None
extraction: Optional[Any] = None
pipeline: Any = None
pipelines: Dict[str, Any] = {}
kokoro_cache_ready = False
pipeline_pool = PipelinePool()
normalized_profiles: Dict[str, Dict[str, Any]] = {}
chunk_groups: Dict[int, List[Dict[str, Any]]] = {}
active_chapter_configs: List[Dict[str, Any]] = []
@@ -301,75 +199,23 @@ def run_conversion_job(job: Job) -> None:
if normalized:
normalized_profiles[str(name)] = normalized
def get_pipeline(provider: str) -> Any:
nonlocal kokoro_cache_ready
provider_norm = str(provider or "kokoro").strip().lower() or "kokoro"
if not is_plugin_registered(provider_norm):
provider_norm = "kokoro"
existing = pipelines.get(provider_norm)
if existing is not None:
return existing
if provider_norm == "supertonic":
pipelines[provider_norm] = create_pipeline(
"supertonic",
)
return pipelines[provider_norm]
# Kokoro
cfg = load_config()
disable_gpu = not job.use_gpu or not cfg.get("use_gpu", True)
device = "cpu"
if not disable_gpu:
device = _select_device()
# Create KPipeline instance directly (uses new Plugin Architecture)
pipelines[provider_norm] = create_pipeline(
"kokoro",
lang_code=job.language,
device=device
)
if not kokoro_cache_ready:
_initialize_voice_cache(job)
kokoro_cache_ready = True
return pipelines[provider_norm]
def resolve_voice_target(raw_spec: str) -> tuple[str, str, Optional[float], Optional[int]]:
"""Return (provider, voice_spec, speed_override, steps_override)."""
spec = str(raw_spec or "").strip()
speaker_name, _ = _split_speaker_reference(spec)
if speaker_name and speaker_name in normalized_profiles:
entry = normalized_profiles[speaker_name]
provider = str(entry.get("provider") or "kokoro").strip().lower() or "kokoro"
if provider == "supertonic":
voice = str(entry.get("voice") or getattr(job, "voice", "M1") or "M1").strip() or "M1"
steps = int(entry.get("total_steps") or getattr(job, "supertonic_total_steps", 5) or 5)
speed = float(entry.get("speed") or getattr(job, "speed", 1.0) or 1.0)
return "supertonic", _supertonic_voice_from_spec(voice, getattr(job, "voice", "M1")), speed, steps
formula = _formula_from_kokoro_entry(entry)
return "kokoro", formula or spec, None, None
fallback_provider = str(getattr(job, "tts_provider", "kokoro") or "kokoro").strip().lower() or "kokoro"
inferred = _infer_provider_from_spec(spec, fallback=fallback_provider)
if inferred == "supertonic":
return "supertonic", _supertonic_voice_from_spec(spec, getattr(job, "voice", "M1")), None, None
return "kokoro", spec, None, None
def resolve_voice_choice(raw_spec: str) -> tuple[str, str, Any, Optional[float], Optional[int]]:
"""Resolve a raw voice spec into (provider, resolved_spec, choice, speed, steps).
For Kokoro formulas, `choice` will be a resolved voice tensor (via `voice_formulas`).
For SuperTonic, `choice` will be a valid SuperTonic voice id.
"""
provider, resolved, speed, steps = resolve_voice_target(raw_spec)
"""Resolve a raw voice spec into (provider, resolved_spec, choice, speed, steps)."""
provider, resolved, speed, steps = _resolve_voice_target(
raw_spec,
normalized_profiles,
job_voice=getattr(job, "voice", "M1"),
job_tts_provider=getattr(job, "tts_provider", "kokoro"),
job_supertonic_total_steps=getattr(job, "supertonic_total_steps", 5),
job_speed=getattr(job, "speed", 1.0),
)
cache_key = f"{provider}:{resolved}" if resolved else provider
cached = voice_cache.get(cache_key)
if cached is not None:
return provider, resolved, cached, speed, steps
if provider == "kokoro":
kokoro_backend = get_pipeline("kokoro")
kokoro_backend = pipeline_pool.get("kokoro", job.language, job.use_gpu, job=job)
choice = _resolve_voice(kokoro_backend, resolved, job.use_gpu)
else:
choice = resolved
@@ -484,7 +330,14 @@ def run_conversion_job(job: Job) -> None:
if merged_required:
audio_path = _build_output_path(audio_dir, job.original_filename, job.output_format)
meta_for_sink = job.metadata_tags if job.metadata_tags else None
audio_sink = _open_audio_sink(audio_path, job, sink_stack, metadata=meta_for_sink)
audio_sink = sink_stack.enter_context(
open_audio_sink(
audio_path,
job.output_format,
metadata=meta_for_sink,
cancel_check=lambda: job.cancel_requested,
)
)
subtitle_writer = _create_subtitle_writer(job, audio_path)
job.result.audio_path = audio_path
if subtitle_writer:
@@ -497,12 +350,17 @@ def run_conversion_job(job: Job) -> None:
base_voice_spec = _job_voice_fallback(job)
voice_cache: Dict[str, Any] = {}
base_provider, base_voice_resolved, _, _ = resolve_voice_target(base_voice_spec)
base_provider, base_voice_resolved, _, _ = _resolve_voice_target(
base_voice_spec, normalized_profiles,
job_voice=getattr(job, "voice", "M1"),
job_tts_provider=getattr(job, "tts_provider", "kokoro"),
)
if base_provider == "kokoro" and base_voice_resolved and "*" not in base_voice_resolved:
kokoro_backend = get_pipeline("kokoro")
kokoro_backend = pipeline_pool.get("kokoro", job.language, job.use_gpu, job=job)
voice_cache[f"kokoro:{base_voice_resolved}"] = _resolve_voice(kokoro_backend, base_voice_resolved, job.use_gpu)
processed_chars = 0
current_time = 0.0
etr_start_time = time.time()
total_chapters = len(extraction.chapters)
if chunk_groups:
chunk_groups = {
@@ -540,26 +398,22 @@ def run_conversion_job(job: Job) -> None:
voice_choice: Any,
chapter_sink: Optional[AudioSink],
preview_prefix: Optional[str] = None,
split_pattern: Optional[str] = SPLIT_PATTERN,
split_pattern: Optional[str] = None,
tts_provider: Optional[str] = None,
speed_override: Optional[float] = None,
supertonic_steps_override: Optional[int] = None,
) -> int:
nonlocal processed_chars, current_time
if split_pattern is None:
split_pattern = job_split_pattern
source_text = str(text or "")
if heteronym_sentence_rules:
source_text = _apply_heteronym_sentence_rules(source_text, heteronym_sentence_rules)
if pronunciation_rules:
source_text = _apply_pronunciation_rules(
source_text,
pronunciation_rules,
usage_counter,
)
try:
normalized = normalize_for_pipeline(
normalized = prepare_text_for_tts(
source_text,
config=apostrophe_config,
settings=normalization_settings,
heteronym_rules=heteronym_sentence_rules,
pronunciation_rules=pronunciation_rules,
normalization_overrides=getattr(job, "normalization_overrides", None),
usage_counter=usage_counter,
)
except LLMClientError as exc:
job.add_log(f"LLM normalization failed: {exc}", level="error")
@@ -568,7 +422,7 @@ def run_conversion_job(job: Job) -> None:
provider = str(tts_provider or getattr(job, "tts_provider", "kokoro") or "kokoro").strip().lower() or "kokoro"
if provider == "supertonic":
supertonic_pipeline = get_pipeline("supertonic")
supertonic_pipeline = pipeline_pool.get("supertonic", job.language, job.use_gpu, job=job)
voice_name = _supertonic_voice_from_spec(voice_choice, getattr(job, "voice", "M1"))
segment_iter = supertonic_pipeline(
normalized,
@@ -578,7 +432,7 @@ def run_conversion_job(job: Job) -> None:
total_steps=int(supertonic_steps_override if supertonic_steps_override is not None else getattr(job, "supertonic_total_steps", 5)),
)
else:
kokoro_backend = get_pipeline("kokoro")
kokoro_backend = pipeline_pool.get("kokoro", job.language, job.use_gpu, job=job)
segment_iter = kokoro_backend(
normalized,
voice=voice_choice,
@@ -587,6 +441,9 @@ def run_conversion_job(job: Job) -> None:
)
try:
# Accumulate tokens for subtitle processing (token-level grouping)
accumulated_tokens: List[dict] = []
for segment in segment_iter:
canceller()
graphemes_raw = getattr(segment, "graphemes", "") or ""
@@ -603,10 +460,16 @@ def run_conversion_job(job: Job) -> None:
audio_sink.write(audio)
duration = len(audio) / SAMPLE_RATE
chunk_start = current_time
processed_chars += len(graphemes)
job.processed_characters = processed_chars
if job.total_characters:
job.progress = min(processed_chars / job.total_characters, 0.999)
job.etr_str = calc_etr_str(
time.time() - etr_start_time,
processed_chars,
job.total_characters,
)
else:
job.progress = 0.0 if processed_chars == 0 else 0.999
@@ -614,16 +477,41 @@ def run_conversion_job(job: Job) -> None:
prefix = f"{preview_prefix} · " if preview_prefix else ""
job.add_log(f"{prefix}{processed_chars:,}/{job.total_characters or ''}: {preview_text[:80]}")
if subtitle_writer and audio_sink and graphemes:
subtitle_writer.write_entry(
start=current_time,
end=current_time + duration,
text=graphemes,
)
# Accumulate tokens from this segment for subtitle processing
if subtitle_writer and audio_sink:
tokens_list = getattr(segment, "tokens", [])
# Fallback for languages without token support: create a single token
if not tokens_list and graphemes:
tokens_list = [FakeToken(graphemes, 0, duration)]
for tok in tokens_list:
accumulated_tokens.append({
"start": chunk_start + (tok.start_ts or 0),
"end": chunk_start + (tok.end_ts or 0),
"text": tok.text,
"whitespace": tok.whitespace,
})
if audio_sink:
current_time += duration
# Flush accumulated tokens through process_subtitle_tokens
if subtitle_writer and audio_sink and accumulated_tokens:
_use_spacy = job.subtitle_mode not in ("Disabled", "Line")
new_entries: List[tuple] = []
process_subtitle_tokens(
accumulated_tokens,
new_entries,
job.max_subtitle_words,
job.subtitle_mode,
job.language,
use_spacy_segmentation=_use_spacy,
fallback_end_time=current_time,
)
for start, end, text in new_entries:
subtitle_writer.write_entry(start=start, end=end, text=text)
except OverflowError as exc:
job.add_log(
f"Skipped chunk — number too large for TTS conversion: {exc}",
@@ -640,10 +528,9 @@ def run_conversion_job(job: Job) -> None:
nonlocal current_time
if duration_seconds <= 0:
return
samples = int(round(duration_seconds * SAMPLE_RATE))
if samples <= 0:
silence = _create_silence(duration_seconds)
if silence.size == 0:
return
silence = np.zeros(samples, dtype="float32")
if include_in_chapter and chapter_sink:
chapter_sink.write(silence)
if audio_sink:
@@ -667,12 +554,18 @@ def run_conversion_job(job: Job) -> None:
if not chapter_voice_spec:
chapter_voice_spec = base_voice_spec
chapter_provider, chapter_voice_resolved, chapter_speed, chapter_steps = resolve_voice_target(chapter_voice_spec)
chapter_provider, chapter_voice_resolved, chapter_speed, chapter_steps = _resolve_voice_target(
chapter_voice_spec, normalized_profiles,
job_voice=getattr(job, "voice", "M1"),
job_tts_provider=getattr(job, "tts_provider", "kokoro"),
job_supertonic_total_steps=getattr(job, "supertonic_total_steps", 5),
job_speed=getattr(job, "speed", 1.0),
)
chapter_cache_key = f"{chapter_provider}:{chapter_voice_resolved}" if chapter_voice_resolved else chapter_provider
if chapter_provider == "kokoro":
voice_choice = voice_cache.get(chapter_cache_key)
if voice_choice is None:
kokoro_backend = get_pipeline("kokoro")
kokoro_backend = pipeline_pool.get("kokoro", job.language, job.use_gpu, job=job)
voice_choice = _resolve_voice(kokoro_backend, chapter_voice_resolved, job.use_gpu)
voice_cache[chapter_cache_key] = voice_choice
else:
@@ -690,11 +583,12 @@ def run_conversion_job(job: Job) -> None:
f"{Path(job.original_filename).stem}_{_slugify(chapter_display_title, idx)}",
job.separate_chapters_format,
)
chapter_sink = _open_audio_sink(
chapter_audio_path,
job,
chapter_sink_stack,
fmt=job.separate_chapters_format,
chapter_sink = chapter_sink_stack.enter_context(
open_audio_sink(
chapter_audio_path,
job.separate_chapters_format,
cancel_check=lambda: job.cancel_requested,
)
)
speak_heading = bool(heading_text)
@@ -734,7 +628,6 @@ def run_conversion_job(job: Job) -> None:
voice_choice=voice_choice,
chapter_sink=chapter_sink,
preview_prefix=f"Chapter {idx} title",
split_pattern=SPLIT_PATTERN,
tts_provider=chapter_provider,
speed_override=chapter_speed,
supertonic_steps_override=chapter_steps,
@@ -805,12 +698,18 @@ def run_conversion_job(job: Job) -> None:
chunk_steps_use = chapter_steps
chunk_voice_choice = voice_choice
else:
chunk_provider, chunk_voice_resolved, chunk_speed_use, chunk_steps_use = resolve_voice_target(chunk_voice_spec)
chunk_provider, chunk_voice_resolved, chunk_speed_use, chunk_steps_use = _resolve_voice_target(
chunk_voice_spec, normalized_profiles,
job_voice=getattr(job, "voice", "M1"),
job_tts_provider=getattr(job, "tts_provider", "kokoro"),
job_supertonic_total_steps=getattr(job, "supertonic_total_steps", 5),
job_speed=getattr(job, "speed", 1.0),
)
chunk_cache_key = f"{chunk_provider}:{chunk_voice_resolved}" if chunk_voice_resolved else chunk_provider
if chunk_provider == "kokoro":
chunk_voice_choice = voice_cache.get(chunk_cache_key)
if chunk_voice_choice is None:
kokoro_backend = get_pipeline("kokoro")
kokoro_backend = pipeline_pool.get("kokoro", job.language, job.use_gpu, job=job)
chunk_voice_choice = _resolve_voice(
kokoro_backend,
chunk_voice_resolved,
@@ -960,11 +859,12 @@ def run_conversion_job(job: Job) -> None:
f"{Path(job.original_filename).stem}_outro",
job.separate_chapters_format,
)
chapter_sink = _open_audio_sink(
outro_audio_path,
job,
outro_sink_stack,
fmt=job.separate_chapters_format,
chapter_sink = outro_sink_stack.enter_context(
open_audio_sink(
outro_audio_path,
job.separate_chapters_format,
cancel_check=lambda: job.cancel_requested,
)
)
outro_segments = emit_text(
@@ -1115,12 +1015,7 @@ def run_conversion_job(job: Job) -> None:
# Explicitly release the pipeline and force garbage collection to prevent
# memory accumulation in the worker process, which can lead to host lockups.
for p in pipelines.values():
try:
p.dispose()
except Exception:
pass
pipelines.clear()
pipeline_pool.dispose_all()
pipeline = None
gc.collect()
try:
@@ -1137,7 +1032,20 @@ def run_conversion_job(job: Job) -> None:
and job.status not in {JobStatus.FAILED, JobStatus.CANCELLED}
):
try:
_embed_m4b_metadata(audio_output_path, metadata_payload, job)
cover_path = None
if job.cover_image_path:
candidate = Path(job.cover_image_path)
if candidate.exists():
cover_path = candidate
_export_svc.embed_m4b_metadata(
audio_path=audio_output_path,
metadata=metadata_payload.get("metadata") or {},
chapters=metadata_payload.get("chapters") or [],
cover_path=cover_path,
cover_mime=job.cover_image_mime,
log_callback=lambda msg, level="info": job.add_log(msg, level=level),
)
except Exception as exc: # pragma: no cover - ensure failure propagates
job.add_log(
f"Failed to embed metadata into m4b output: {exc}",
@@ -1148,21 +1056,6 @@ def run_conversion_job(job: Job) -> None:
) from exc
def _load_pipeline(job: Job):
cfg = load_config()
disable_gpu = not job.use_gpu or not cfg.get("use_gpu", True)
provider = str(getattr(job, "tts_provider", "kokoro") or "kokoro").strip().lower()
if provider == "supertonic":
return create_pipeline(
"supertonic",
)
device = "cpu"
if not disable_gpu:
device = _select_device()
return create_pipeline("kokoro", lang_code=job.language, device=device)
def _prepare_output_dir(job: Job) -> Path:
from platformdirs import user_desktop_dir # type: ignore[import-not-found]
@@ -1188,50 +1081,6 @@ def _prepare_project_layout(job: Job, base_dir: Path) -> tuple[Path, Path, Path,
)
def _open_audio_sink(
path: Path,
job: Job,
stack: ExitStack,
*,
fmt: Optional[str] = None,
metadata: Optional[Dict[str, str]] = None,
) -> AudioSink:
ffmpeg_cache_root = get_internal_cache_path("ffmpeg")
platform_cache = os.path.join(ffmpeg_cache_root, sys.platform)
os.makedirs(platform_cache, exist_ok=True)
try:
import static_ffmpeg.run as static_ffmpeg_run # type: ignore
static_ffmpeg_run.LOCK_FILE = os.path.join(ffmpeg_cache_root, "lock.file")
except Exception:
pass
static_ffmpeg.add_paths(weak=True, download_dir=platform_cache)
fmt_value = (fmt or job.output_format).lower()
if fmt_value in {"wav", "flac"}:
soundfile = stack.enter_context(
sf.SoundFile(path, mode="w", samplerate=SAMPLE_RATE, channels=1, format=fmt_value.upper())
)
return AudioSink(write=lambda data: soundfile.write(data))
cmd = _build_ffmpeg_command(path, fmt_value, metadata=metadata)
process = create_process(cmd, stdin=subprocess.PIPE, text=False)
def _finalize() -> None:
if process.stdin and not process.stdin.closed:
process.stdin.close()
process.wait()
stack.callback(_finalize)
def _write(data: np.ndarray) -> None:
if job.cancel_requested or process.stdin is None:
return
process.stdin.write(data.tobytes()) # type: ignore[arg-type]
return AudioSink(write=_write)
def _resolve_voice(pipeline, voice_spec: str, use_gpu: bool):
if "*" in voice_spec:
+3 -2
View File
@@ -14,7 +14,8 @@ from abogen.kokoro_text_normalization import normalize_for_pipeline
from abogen.normalization_settings import build_apostrophe_config
from abogen.text_extractor import extract_from_path
from abogen.voice_cache import ensure_voice_assets
from abogen.webui.conversion_runner import SAMPLE_RATE, SPLIT_PATTERN, _select_device, _to_float32, _resolve_voice, _spec_to_voice_ids
from abogen.webui.conversion_runner import SAMPLE_RATE, _select_device, _to_float32, _resolve_voice, _spec_to_voice_ids
from abogen.domain.split_pattern import get_split_pattern
from abogen.tts_plugin.utils import create_pipeline
@@ -200,7 +201,7 @@ def run_debug_tts_wavs(
normalized,
voice=voice_choice,
speed=speed,
split_pattern=SPLIT_PATTERN,
split_pattern=get_split_pattern(language, "Disabled"),
):
audio = _to_float32(getattr(segment, "audio", None))
if audio.size:
+1 -1
View File
@@ -25,7 +25,7 @@ from abogen.voice_profiles import (
normalize_profile_entry,
)
from abogen.webui.routes.utils.common import split_profile_spec
from abogen.webui.routes.utils.preview import synthesize_preview, generate_preview_audio
from abogen.webui.routes.utils.synthesize import synthesize_preview, generate_preview_audio
from abogen.webui.routes.utils.voice import formula_from_profile
from abogen.normalization_settings import (
build_llm_configuration,
+2 -9
View File
@@ -19,6 +19,7 @@ from abogen.webui.routes.utils.settings import (
_NORMALIZATION_STRING_KEYS,
_DEFAULT_ANALYSIS_THRESHOLD,
)
from abogen.webui.routes.utils.common import extract_checkbox
from abogen.webui.routes.utils.voice import template_options
from abogen.webui.debug_tts_runner import run_debug_tts_wavs
from abogen.debug_tts_samples import DEBUG_TTS_SAMPLES
@@ -93,17 +94,9 @@ def update_settings() -> ResponseReturnValue:
maximum=25,
)
def _extract_checkbox(name: str, default: bool) -> bool:
values = form.getlist(name) if hasattr(form, "getlist") else []
if values:
return coerce_bool(values[-1], default)
if hasattr(form, "__contains__") and name in form:
return False
return default
# Normalization settings
for key in _NORMALIZATION_BOOLEAN_KEYS:
current[key] = _extract_checkbox(key, bool(current.get(key, True)))
current[key] = extract_checkbox(form, key, bool(current.get(key, True)))
for key in _NORMALIZATION_STRING_KEYS:
if hasattr(form, "__contains__") and key in form:
current[key] = (form.get(key) or "").strip()
+36 -1
View File
@@ -1,6 +1,17 @@
from typing import Any, Optional, Tuple, Iterable, List
from typing import Any, Optional, Tuple, Iterable, List, Mapping
from pathlib import Path
def coerce_bool(value: Any, default: bool) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() in {"true", "1", "yes", "on"}
if value is None:
return default
return bool(value)
def split_profile_spec(value: Any) -> Tuple[str, Optional[str]]:
text = str(value or "").strip()
if not text:
@@ -18,7 +29,31 @@ def split_speaker_spec(value: Any) -> Tuple[str, Optional[str]]:
return split_profile_spec(value)
def existing_paths(paths: Optional[Iterable[Path]]) -> List[Path]:
if not paths:
return []
return [p for p in paths if p.exists()]
def extract_checkbox(form: Mapping[str, Any], name: str, default: bool) -> bool:
"""Extract a boolean checkbox value from a form-like mapping.
Handles both multi-value forms (Flask's `getlist`) and simple mappings.
If the checkbox name is present but has no value, it means unchecked (False).
"""
values: List[str] = []
getter = getattr(form, "getlist", None)
if callable(getter):
raw_values = getter(name)
if raw_values:
values = list(raw_values)
else:
raw_flag = form.get(name)
if raw_flag is not None:
values = [raw_flag]
if values:
return coerce_bool(values[-1], default)
if name in form:
return False
return default
+8 -139
View File
@@ -1,10 +1,14 @@
import re
import time
import uuid
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
from flask import request, render_template, jsonify
from flask.typing import ResponseReturnValue
from abogen.domain.chapter_classification import (
supplement_score,
should_preselect_chapter,
ensure_at_least_one_chapter_enabled,
)
from abogen.webui.service import PendingJob, JobStatus
from abogen.webui.routes.utils.service import get_service
from abogen.tts_plugin.utils import is_plugin_registered
@@ -29,7 +33,7 @@ from abogen.webui.routes.utils.voice import (
)
from abogen.webui.routes.utils.entity import sync_pronunciation_overrides
from abogen.webui.routes.utils.epub import job_download_flags
from abogen.webui.routes.utils.common import split_profile_spec
from abogen.webui.routes.utils.common import split_profile_spec, extract_checkbox
from abogen.utils import calculate_text_length
from abogen.voice_profiles import serialize_profiles, normalize_profile_entry
from abogen.chunking import ChunkLevel, build_chunks_for_chapters
@@ -66,109 +70,6 @@ _WIZARD_STEP_META = {
},
}
_SUPPLEMENT_TITLE_PATTERNS: List[tuple[re.Pattern[str], float]] = [
(re.compile(r"\btitle\s+page\b"), 3.0),
(re.compile(r"\bcopyright\b"), 2.4),
(re.compile(r"\btable\s+of\s+contents\b"), 2.8),
(re.compile(r"\bcontents\b"), 2.0),
(re.compile(r"\backnowledg(e)?ments?\b"), 2.0),
(re.compile(r"\bdedication\b"), 2.0),
(re.compile(r"\babout\s+the\s+author(s)?\b"), 2.4),
(re.compile(r"\balso\s+by\b"), 2.0),
(re.compile(r"\bpraise\s+for\b"), 2.0),
(re.compile(r"\bcolophon\b"), 2.2),
(re.compile(r"\bpublication\s+data\b"), 2.2),
(re.compile(r"\btranscriber'?s?\s+note\b"), 2.2),
(re.compile(r"\bglossary\b"), 2.0),
(re.compile(r"\bindex\b"), 2.0),
(re.compile(r"\bbibliograph(y|ies)\b"), 2.0),
(re.compile(r"\breferences\b"), 1.8),
(re.compile(r"\bappendix\b"), 1.9),
]
_CONTENT_TITLE_PATTERNS: List[re.Pattern[str]] = [
re.compile(r"\bchapter\b"),
re.compile(r"\bbook\b"),
re.compile(r"\bpart\b"),
re.compile(r"\bsection\b"),
re.compile(r"\bscene\b"),
re.compile(r"\bprologue\b"),
re.compile(r"\bepilogue\b"),
re.compile(r"\bintroduction\b"),
re.compile(r"\bstory\b"),
]
_SUPPLEMENT_TEXT_KEYWORDS: List[tuple[str, float]] = [
("copyright", 1.2),
("all rights reserved", 1.1),
("isbn", 0.9),
("library of congress", 1.0),
("table of contents", 1.0),
("dedicated to", 0.8),
("acknowledg", 0.8),
("printed in", 0.6),
("permission", 0.6),
("publisher", 0.5),
("praise for", 0.9),
("also by", 0.9),
("glossary", 0.8),
("index", 0.8),
("newsletter", 3.2),
("mailing list", 2.6),
("sign-up", 2.2),
]
def supplement_score(title: str, text: str, index: int) -> float:
normalized_title = (title or "").lower()
score = 0.0
for pattern, weight in _SUPPLEMENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score += weight
for pattern in _CONTENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score -= 2.0
stripped_text = (text or "").strip()
length = len(stripped_text)
if length <= 150:
score += 0.9
elif length <= 400:
score += 0.6
elif length <= 800:
score += 0.35
lowercase_text = stripped_text.lower()
for keyword, weight in _SUPPLEMENT_TEXT_KEYWORDS:
if keyword in lowercase_text:
score += weight
if index == 0 and score > 0:
score += 0.25
return score
def should_preselect_chapter(
title: str,
text: str,
index: int,
total_count: int,
) -> bool:
if total_count <= 1:
return True
score = supplement_score(title, text, index)
return score < 1.9
def ensure_at_least_one_chapter_enabled(chapters: List[Dict[str, Any]]) -> None:
if not chapters:
return
if any(chapter.get("enabled") for chapter in chapters):
return
best_index = max(range(len(chapters)), key=lambda idx: chapters[idx].get("characters", 0))
chapters[best_index]["enabled"] = True
def apply_prepare_form(
pending: PendingJob, form: Mapping[str, Any]
@@ -537,28 +438,11 @@ def apply_book_step_form(
else:
pending.normalize_chapter_opening_caps = caps_default
def _extract_checkbox(name: str, default: bool) -> bool:
values: List[str] = []
getter = getattr(form, "getlist", None)
if callable(getter):
raw_values = getter(name)
if raw_values:
values = list(cast(Iterable[str], raw_values))
else:
raw_flag = form.get(name)
if raw_flag is not None:
values = [raw_flag]
if values:
return coerce_bool(values[-1], default)
if hasattr(form, "__contains__") and name in form:
return False
return default
overrides_existing = getattr(pending, "normalization_overrides", None)
overrides: Dict[str, Any] = dict(overrides_existing or {})
for key in _NORMALIZATION_BOOLEAN_KEYS:
default_toggle = overrides.get(key, bool(settings.get(key, True)))
overrides[key] = _extract_checkbox(key, default_toggle)
overrides[key] = extract_checkbox(form, key, default_toggle)
for key in _NORMALIZATION_STRING_KEYS:
default_val = overrides.get(key, str(settings.get(key, "")))
val = form.get(key)
@@ -886,25 +770,10 @@ def build_pending_job_from_extraction(
apply_config=bool(speaker_config_payload),
)
def _extract_checkbox(name: str, default: bool) -> bool:
values: List[str] = []
getter = getattr(form, "getlist", None)
if callable(getter):
raw_values = getter(name)
if raw_values:
values = list(cast(Iterable[str], raw_values))
else:
raw_flag = form.get(name)
if raw_flag is not None:
values = [raw_flag]
if values:
return coerce_bool(values[-1], default)
return default
normalization_overrides = {}
for key in _NORMALIZATION_BOOLEAN_KEYS:
default_val = bool(settings.get(key, True))
normalization_overrides[key] = _extract_checkbox(key, default_val)
normalization_overrides[key] = extract_checkbox(form, key, default_val)
for key in _NORMALIZATION_STRING_KEYS:
default_val = str(settings.get(key, ""))
+1 -11
View File
@@ -15,7 +15,7 @@ from abogen.normalization_settings import (
from abogen.utils import load_config, save_config
from abogen.integrations.calibre_opds import CalibreOPDSClient
from abogen.integrations.audiobookshelf import AudiobookshelfConfig
from abogen.webui.routes.utils.common import split_profile_spec
from abogen.webui.routes.utils.common import split_profile_spec, coerce_bool
SAVE_MODE_LABELS = {
"save_next_to_input": "Save next to input file",
@@ -245,16 +245,6 @@ def render_prompt_template(template: str, context: Mapping[str, str]) -> str:
return _PROMPT_TOKEN_RE.sub(_replace, template)
def coerce_bool(value: Any, default: bool) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() in {"true", "1", "yes", "on"}
if value is None:
return default
return bool(value)
def coerce_float(value: Any, default: float) -> float:
try:
return max(0.0, float(value))
@@ -7,9 +7,9 @@ from flask import current_app, send_file
from flask.typing import ResponseReturnValue
from abogen.domain.device import select_device as _select_device
from abogen.domain.split_pattern import get_split_pattern
SPLIT_PATTERN = r"\n+"
SAMPLE_RATE = 24000
_preview_pipelines: Dict[Tuple[str, str], Any] = {}
@@ -44,22 +44,6 @@ def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]:
raise RuntimeError("Preview pipeline is unavailable") from last_error
def _to_float32(audio_segment) -> np.ndarray:
if audio_segment is None:
return np.zeros(0, dtype="float32")
tensor = audio_segment
if hasattr(tensor, "detach"):
tensor = tensor.detach()
if hasattr(tensor, "cpu"):
try:
tensor = tensor.cpu()
except Exception:
pass
if hasattr(tensor, "numpy"):
return np.asarray(tensor.numpy(), dtype="float32").reshape(-1)
return np.asarray(tensor, dtype="float32").reshape(-1)
def get_preview_pipeline(language: str, device: str) -> Any:
key = (language, device)
with _preview_pipeline_lock:
@@ -123,6 +107,8 @@ def generate_preview_audio(
current_app.logger.exception("Preview normalization failed; using raw text")
normalized_text = source_text
preview_split = get_split_pattern(str(language or "a"), "Disabled")
if provider == "supertonic":
from abogen.tts_plugin.utils import create_pipeline
@@ -131,7 +117,7 @@ def generate_preview_audio(
normalized_text,
voice=voice_spec,
speed=speed,
split_pattern=SPLIT_PATTERN,
split_pattern=preview_split,
total_steps=supertonic_total_steps,
)
else:
@@ -149,7 +135,7 @@ def generate_preview_audio(
normalized_text,
voice=voice_choice,
speed=speed,
split_pattern=SPLIT_PATTERN,
split_pattern=preview_split,
)
audio_chunks: List[np.ndarray] = []
+1 -80
View File
@@ -1,6 +1,4 @@
import threading
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
import numpy as np
from abogen.speaker_configs import slugify_label
from abogen.speaker_analysis import analyze_speakers
@@ -10,7 +8,7 @@ from abogen.voice_profiles import (
load_profiles,
serialize_profiles,
)
from abogen.voice_formulas import get_new_voice, parse_formula_terms
from abogen.voice_formulas import parse_formula_terms
from abogen.constants import (
LANGUAGE_DESCRIPTIONS,
SUBTITLE_FORMATS,
@@ -20,11 +18,7 @@ from abogen.constants import (
)
from abogen.tts_plugin.utils import get_voices
from abogen.speaker_configs import list_configs
from abogen.tts_plugin.utils import create_pipeline
from abogen.webui.conversion_runner import _select_device, _to_float32, SAMPLE_RATE, SPLIT_PATTERN
_preview_pipeline_lock = threading.RLock()
_preview_pipelines: Dict[Tuple[str, str], Any] = {}
def build_narrator_roster(
voice: str,
@@ -733,76 +727,3 @@ def pairs_to_formula(pairs: Iterable[Tuple[str, float]]) -> Optional[str]:
def profiles_payload() -> Dict[str, Any]:
return {"profiles": serialize_profiles()}
def get_preview_pipeline(language: str, device: str):
key = (language, device)
with _preview_pipeline_lock:
pipeline = _preview_pipelines.get(key)
if pipeline is not None:
return pipeline
pipeline = create_pipeline("kokoro", lang_code=language, device=device)
_preview_pipelines[key] = pipeline
return pipeline
def synthesize_audio_from_normalized(
*,
normalized_text: str,
voice_spec: str,
language: str,
speed: float,
use_gpu: bool,
max_seconds: float,
) -> np.ndarray:
if not normalized_text.strip():
raise ValueError("Preview text is required")
device = "cpu"
if use_gpu:
try:
device = _select_device()
except Exception:
device = "cpu"
use_gpu = False
pipeline = get_preview_pipeline(language, device)
if pipeline is None:
raise RuntimeError("Preview pipeline is unavailable")
voice_choice: Any = voice_spec
if voice_spec and "*" in voice_spec:
voice_choice = get_new_voice(pipeline, voice_spec, use_gpu)
segments = pipeline(
normalized_text,
voice=voice_choice,
speed=speed,
split_pattern=SPLIT_PATTERN,
)
audio_chunks: List[np.ndarray] = []
accumulated = 0
max_samples = int(max(1.0, max_seconds) * SAMPLE_RATE)
for segment in segments:
graphemes = getattr(segment, "graphemes", "").strip()
if not graphemes:
continue
audio = _to_float32(getattr(segment, "audio", None))
if audio.size == 0:
continue
remaining = max_samples - accumulated
if remaining <= 0:
break
if audio.shape[0] > remaining:
audio = audio[:remaining]
audio_chunks.append(audio)
accumulated += audio.shape[0]
if accumulated >= max_samples:
break
if not audio_chunks:
raise RuntimeError("Preview could not be generated")
return np.concatenate(audio_chunks)
+1 -1
View File
@@ -9,7 +9,7 @@ from abogen.webui.routes.utils.voice import (
parse_voice_formula,
)
from abogen.webui.routes.utils.settings import load_settings, coerce_bool
from abogen.webui.routes.utils.preview import synthesize_preview
from abogen.webui.routes.utils.synthesize import synthesize_preview
from abogen.speaker_configs import (
list_configs,
get_config,
+31 -264
View File
@@ -2,9 +2,7 @@ from __future__ import annotations
import json
import logging
import math
import os
import re
import shutil
import sys
import threading
@@ -14,7 +12,7 @@ import traceback
from dataclasses import dataclass, field
from enum import Enum
from pathlib import Path
from typing import Any, Callable, Dict, Iterable, List, Optional, Mapping, Tuple
from typing import Any, Callable, Dict, Iterable, List, Optional, Mapping
from abogen.utils import get_internal_cache_path, get_user_settings_dir, load_config
from abogen.voice_cache import bootstrap_voice_cache
@@ -23,6 +21,17 @@ from abogen.integrations.audiobookshelf import (
AudiobookshelfConfig,
AudiobookshelfUploadError,
)
from abogen.domain.metadata_helpers import (
normalize_metadata_casefold as _normalize_metadata_casefold,
split_people_field as _split_people_field,
split_simple_list as _split_simple_list,
first_nonempty as _first_nonempty,
extract_year as _extract_year,
normalize_series_sequence as _normalize_series_sequence,
build_audiobookshelf_metadata as _build_abs_metadata,
load_audiobookshelf_chapters as _load_abs_chapters,
_SERIES_SEQUENCE_TAG_KEYS,
)
def _create_set_event() -> threading.Event:
@@ -53,9 +62,6 @@ _JOB_LEVEL_MAP: Dict[str, int] = {
}
_PEOPLE_SPLIT_RE = re.compile(r"[;,/&]|\band\b", re.IGNORECASE)
def _emit_job_log(job_id: str, level: str, message: str) -> None:
normalized = (level or "info").lower()
log_level = _JOB_LEVEL_MAP.get(normalized, logging.INFO)
@@ -131,6 +137,7 @@ class Job:
progress: float = 0.0
total_characters: int = 0
processed_characters: int = 0
etr_str: str = ""
logs: List[JobLog] = field(default_factory=list)
error: Optional[str] = None
result: JobResult = field(default_factory=JobResult)
@@ -162,20 +169,25 @@ class Job:
@property
def estimated_time_remaining(self) -> Optional[float]:
"""
Returns the estimated seconds remaining based on current progress and elapsed time.
Returns None if the job hasn't started, is finished, or progress is 0.
Returns the estimated seconds remaining.
Uses the same calc_etr_str from domain/progress.py as the PyQt desktop GUI.
"""
if self.status != JobStatus.RUNNING or not self.started_at or self.progress <= 0:
from abogen.domain.progress import calc_etr_str
if self.status != JobStatus.RUNNING or not self.started_at or self.total_characters <= 0:
return None
elapsed = time.time() - self.started_at
if elapsed <= 0:
return None
# Estimate total time based on current progress
total_estimated = elapsed / self.progress
remaining = total_estimated - elapsed
return max(0.0, remaining)
etr = calc_etr_str(elapsed, self.processed_characters, self.total_characters)
if etr == "Processing...":
return None
# Parse "HH:MM:SS" back to seconds for backward compatibility
parts = etr.split(":")
return int(parts[0]) * 3600 + int(parts[1]) * 60 + int(parts[2])
def add_log(self, message: str, level: str = "info") -> None:
entry = JobLog(timestamp=time.time(), message=message, level=level)
@@ -194,6 +206,7 @@ class Job:
"progress": self.progress,
"total_characters": self.total_characters,
"processed_characters": self.processed_characters,
"etr_str": self.etr_str,
"error": self.error,
"logs": [log.__dict__ for log in self.logs],
"result": {
@@ -252,234 +265,13 @@ class Job:
}
def _normalize_metadata_casefold(values: Optional[Mapping[str, Any]]) -> Dict[str, Any]:
normalized: Dict[str, Any] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
key_text = str(key).strip().lower()
if not key_text:
continue
if isinstance(value, (list, tuple, set)):
normalized[key_text] = value
else:
text = str(value).strip()
if text:
normalized[key_text] = text
return normalized
def _split_people_field(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(_split_people_field(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _PEOPLE_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
_LIST_SPLIT_RE = re.compile(r"[;,\n]")
_SERIES_SEQUENCE_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?")
_SERIES_SEQUENCE_TAG_KEYS: Tuple[str, ...] = (
"series_index",
"series_position",
"series_sequence",
"series_number",
"seriesnumber",
"book_number",
"booknumber",
)
def _split_simple_list(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(_split_simple_list(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _LIST_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def _first_nonempty(*values: Any) -> Optional[str]:
for value in values:
if value is None:
continue
if isinstance(value, (list, tuple, set)):
items = list(value)
if not items:
continue
value = items[0]
text = str(value).strip()
if text:
return text
return None
def _extract_year(raw: Optional[str]) -> Optional[int]:
if not raw:
return None
text = str(raw).strip()
if not text:
return None
match = re.search(r"(19|20)\d{2}", text)
if match:
try:
return int(match.group(0))
except ValueError:
return None
try:
parsed = int(text)
except ValueError:
return None
if 0 < parsed < 3000:
return parsed
return None
def build_audiobookshelf_metadata(job: Job) -> Dict[str, Any]:
tags = _normalize_metadata_casefold(job.metadata_tags)
filename = Path(job.original_filename or "").stem or job.original_filename or "Audiobook"
title = _first_nonempty(
tags.get("title"),
tags.get("book_title"),
tags.get("name"),
tags.get("album"),
filename,
return _build_abs_metadata(
job.metadata_tags,
language=job.language or "",
filename=filename,
)
authors = _split_people_field(
tags.get("authors")
or tags.get("author")
or tags.get("album_artist")
or tags.get("artist")
)
narrators = _split_people_field(tags.get("narrators") or tags.get("narrator"))
description = _first_nonempty(tags.get("description"), tags.get("summary"), tags.get("comment"))
genres = _split_simple_list(tags.get("genre"))
keywords = _split_simple_list(tags.get("tags") or tags.get("keywords"))
language = _first_nonempty(tags.get("language"), tags.get("lang")) or job.language or ""
series_name = _first_nonempty(
tags.get("series"),
tags.get("series_name"),
tags.get("seriesname"),
tags.get("series_title"),
tags.get("seriestitle"),
)
series_sequence = None
for key in _SERIES_SEQUENCE_TAG_KEYS:
raw_value = tags.get(key)
normalized_sequence = _normalize_series_sequence(raw_value)
if normalized_sequence:
series_sequence = normalized_sequence
break
if not series_name:
series_sequence = None
data: Dict[str, Any] = {
"title": title,
"subtitle": tags.get("subtitle"),
"authors": authors,
"narrators": narrators,
"description": description,
"publisher": tags.get("publisher"),
"genres": genres,
"tags": keywords,
"language": language,
"publishedYear": _extract_year(tags.get("published") or tags.get("publication_year") or tags.get("date") or tags.get("year")),
"seriesName": series_name,
"seriesSequence": series_sequence,
"isbn": _first_nonempty(tags.get("isbn"), tags.get("asin")),
}
published_date = _first_nonempty(tags.get("published"), tags.get("publication_date"), tags.get("date"))
if published_date:
data["publishedDate"] = published_date
rating_text = _first_nonempty(tags.get("rating"), tags.get("my_rating"))
if rating_text:
try:
data["rating"] = float(str(rating_text).strip())
except ValueError:
pass
rating_max_text = _first_nonempty(tags.get("rating_max"), tags.get("rating_scale"))
if rating_max_text:
try:
data["ratingMax"] = float(str(rating_max_text).strip())
except ValueError:
pass
# Remove empty values
cleaned: Dict[str, Any] = {}
for key, value in data.items():
if value is None:
continue
if isinstance(value, str) and not value.strip():
continue
if isinstance(value, (list, tuple)) and not value:
continue
cleaned[key] = value
return cleaned
def _normalize_series_sequence(raw: Any) -> Optional[str]:
if raw is None:
return None
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return None
text = str(raw)
else:
text = str(raw).strip()
if not text:
return None
candidate = text.replace(",", ".")
match = _SERIES_SEQUENCE_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
def load_audiobookshelf_chapters(job: Job) -> Optional[List[Dict[str, Any]]]:
@@ -487,32 +279,7 @@ def load_audiobookshelf_chapters(job: Job) -> Optional[List[Dict[str, Any]]]:
if not metadata_ref:
return None
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
if not metadata_path.exists():
return None
try:
payload = json.loads(metadata_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
chapters = payload.get("chapters")
if not isinstance(chapters, list):
return None
cleaned: List[Dict[str, Any]] = []
for entry in chapters:
if not isinstance(entry, Mapping):
continue
title = _first_nonempty(entry.get("title"), entry.get("original_title"))
start = entry.get("start")
end = entry.get("end")
if title is None or not isinstance(start, (int, float)):
continue
chapter_payload: Dict[str, Any] = {
"title": title,
"start": float(start),
}
if isinstance(end, (int, float)):
chapter_payload["end"] = float(end)
cleaned.append(chapter_payload)
return cleaned or None
return _load_abs_chapters(metadata_path)
def _existing_paths(paths: Iterable[Any]) -> List[Path]:
+2 -2
View File
@@ -28,8 +28,8 @@
</div>
<div class="job-card__progress-meta">
<small>{{ progress_value }}% · {{ job.processed_characters }} / {{ job.total_characters or '—' }}</small>
{% if job.estimated_time_remaining %}
<small class="job-card__eta">~{{ job.estimated_time_remaining | durationformat }} remaining</small>
{% if job.etr_str and job.etr_str != 'Processing...' %}
<small class="job-card__eta">~{{ job.etr_str }} remaining</small>
{% endif %}
</div>
</div>
+18
View File
@@ -12,6 +12,7 @@ from __future__ import annotations
import logging
from pathlib import Path
from typing import Any
from unittest.mock import patch
import pytest
@@ -147,6 +148,23 @@ def engine_config() -> Any:
return EngineConfig(device="cpu")
@pytest.fixture(autouse=True)
def _mock_kokoro_pipeline():
"""Prevent real KPipeline initialization during generic plugin tests.
The real KPipeline requires spacy model downloads which aren't available
in externally-managed environments. Mock spacy's download and load so
the engine contract can be tested without heavy dependencies.
"""
from unittest.mock import MagicMock
mock_nlp = MagicMock()
with patch("spacy.cli.download"), \
patch("spacy.load", return_value=mock_nlp):
yield
@pytest.fixture
def create_engine(loaded_plugin, host_context, engine_config):
"""Create an engine instance from a loaded plugin.
+307
View File
@@ -0,0 +1,307 @@
"""Tests for abogen.domain.audio_buffer module."""
import numpy as np
import pytest
from abogen.domain.audio_buffer import (
create_silence,
mix_audio,
normalize_audio,
ensure_buffer_size,
concatenate_audio,
audio_duration,
samples_for_duration,
SAMPLE_RATE,
)
class TestCreateSilence:
"""Tests for create_silence function."""
def test_positive_duration(self):
"""Test creating silence with positive duration."""
duration = 1.0 # 1 second
silence = create_silence(duration)
expected_samples = int(round(duration * SAMPLE_RATE))
assert len(silence) == expected_samples
assert silence.dtype == np.float32
assert np.all(silence == 0)
def test_zero_duration(self):
"""Test creating silence with zero duration returns empty array."""
silence = create_silence(0)
assert len(silence) == 0
assert silence.dtype == np.float32
def test_negative_duration(self):
"""Test creating silence with negative duration returns empty array."""
silence = create_silence(-1.0)
assert len(silence) == 0
assert silence.dtype == np.float32
def test_very_small_duration(self):
"""Test creating silence with very small duration."""
duration = 0.001 # 1 millisecond
silence = create_silence(duration)
# Should round to at least 1 sample or 0
assert len(silence) >= 0
assert silence.dtype == np.float32
def test_half_second(self):
"""Test creating 0.5 second of silence."""
silence = create_silence(0.5)
expected_samples = int(round(0.5 * SAMPLE_RATE))
assert len(silence) == expected_samples
class TestMixAudio:
"""Tests for mix_audio function."""
def test_basic_mix(self):
"""Test basic audio mixing."""
target = np.ones(100, dtype="float32")
source = np.ones(50, dtype="float32") * 2
mix_audio(target, source, start_sample=25)
# First 25 samples should remain 1.0
assert np.all(target[:25] == 1.0)
# Middle 50 samples should be 1.0 + 2.0 = 3.0
assert np.all(target[25:75] == 3.0)
# Last 25 samples should remain 1.0
assert np.all(target[75:] == 1.0)
def test_empty_source(self):
"""Test mixing empty source does nothing."""
target = np.ones(100, dtype="float32")
original = target.copy()
mix_audio(target, np.array([], dtype="float32"), start_sample=50)
assert np.array_equal(target, original)
def test_extend_target_buffer(self):
"""Test that target buffer is extended when needed."""
target = np.ones(100, dtype="float32")
source = np.ones(50, dtype="float32") * 2
# This should extend target to 170 samples (120 + 50)
target = mix_audio(target, source, start_sample=120)
assert len(target) == 170
# Check that source was mixed correctly
assert np.all(target[120:170] == 2.0)
def test_start_at_zero(self):
"""Test mixing starting at sample 0."""
target = np.zeros(100, dtype="float32")
source = np.ones(50, dtype="float32")
mix_audio(target, source, start_sample=0)
assert np.all(target[:50] == 1.0)
assert np.all(target[50:] == 0.0)
def test_explicit_end_sample(self):
"""Test mixing with explicit end_sample."""
target = np.zeros(100, dtype="float32")
source = np.ones(50, dtype="float32")
mix_audio(target, source, start_sample=10, end_sample=60)
# Only first 10 samples of source should be mixed (60-10=50, but source is only 50)
# Actually, end_sample overrides the length
assert target[10] == 1.0
class TestNormalizeAudio:
"""Tests for normalize_audio function."""
def test_no_normalization_needed(self):
"""Test audio within range is not modified."""
audio = np.ones(100, dtype="float32") * 0.5
result = normalize_audio(audio)
assert not np.shares_memory(audio, result) # Should be a copy
assert np.array_equal(result, audio)
def test_normalization_applied(self):
"""Test audio above target peak is scaled down."""
audio = np.ones(100, dtype="float32") * 2.0
result = normalize_audio(audio)
assert np.all(result <= 1.0)
assert np.isclose(result[0], 1.0)
def test_empty_audio(self):
"""Test normalizing empty audio returns empty copy."""
audio = np.array([], dtype="float32")
result = normalize_audio(audio)
assert len(result) == 0
assert result.dtype == np.float32
def test_custom_target_peak(self):
"""Test normalization with custom target peak."""
audio = np.ones(100, dtype="float32") * 4.0
result = normalize_audio(audio, target_peak=2.0)
assert np.all(result <= 2.0)
assert np.isclose(result[0], 2.0)
def test_negative_peak(self):
"""Test normalization handles negative peaks."""
audio = np.ones(100, dtype="float32") * -2.0
result = normalize_audio(audio)
assert np.all(result >= -1.0)
assert np.isclose(result[0], -1.0)
def test_mixed_positive_negative(self):
"""Test normalization with both positive and negative peaks."""
audio = np.array([-3.0, 2.0, -1.0, 4.0], dtype="float32")
result = normalize_audio(audio)
# Should scale by 1/4 (max absolute value is 4)
assert np.isclose(result[0], -0.75)
assert np.isclose(result[1], 0.5)
assert np.isclose(result[3], 1.0)
class TestEnsureBufferSize:
"""Tests for ensure_buffer_size function."""
def test_buffer_already_large_enough(self):
"""Test buffer that is already large enough is unchanged."""
buffer = np.ones(100, dtype="float32")
result = ensure_buffer_size(buffer, 50)
assert np.array_equal(result, buffer)
def test_buffer_needs_extension(self):
"""Test buffer is extended with zeros when too small."""
buffer = np.ones(50, dtype="float32")
result = ensure_buffer_size(buffer, 100)
assert len(result) == 100
assert np.all(result[:50] == 1.0)
assert np.all(result[50:] == 0.0)
def test_exact_size(self):
"""Test buffer of exact size is unchanged."""
buffer = np.ones(100, dtype="float32")
result = ensure_buffer_size(buffer, 100)
assert len(result) == 100
assert np.array_equal(result, buffer)
class TestConcatenateAudio:
"""Tests for concatenate_audio function."""
def test_concatenate_two_buffers(self):
"""Test concatenating two audio buffers."""
a = np.ones(50, dtype="float32")
b = np.ones(50, dtype="float32") * 2
result = concatenate_audio(a, b)
assert len(result) == 100
assert np.all(result[:50] == 1.0)
assert np.all(result[50:] == 2.0)
def test_concatenate_multiple_buffers(self):
"""Test concatenating multiple audio buffers."""
a = np.ones(20, dtype="float32")
b = np.ones(30, dtype="float32") * 2
c = np.ones(40, dtype="float32") * 3
result = concatenate_audio(a, b, c)
assert len(result) == 90
assert np.all(result[:20] == 1.0)
assert np.all(result[20:50] == 2.0)
assert np.all(result[50:] == 3.0)
def test_concatenate_empty_buffers(self):
"""Test concatenating empty buffers returns empty array."""
result = concatenate_audio(
np.array([], dtype="float32"),
np.array([], dtype="float32")
)
assert len(result) == 0
def test_concatenate_with_empty(self):
"""Test concatenating with some empty buffers."""
a = np.ones(50, dtype="float32")
result = concatenate_audio(a, np.array([], dtype="float32"))
assert len(result) == 50
assert np.array_equal(result, a)
class TestAudioDuration:
"""Tests for audio_duration function."""
def test_one_second_duration(self):
"""Test duration calculation for 1 second of audio."""
audio = np.zeros(SAMPLE_RATE, dtype="float32")
duration = audio_duration(audio)
assert duration == 1.0
def test_half_second_duration(self):
"""Test duration calculation for 0.5 second of audio."""
audio = np.zeros(SAMPLE_RATE // 2, dtype="float32")
duration = audio_duration(audio)
assert duration == 0.5
def test_empty_audio_duration(self):
"""Test duration of empty audio is 0."""
duration = audio_duration(np.array([], dtype="float32"))
assert duration == 0.0
def test_custom_sample_rate(self):
"""Test duration with custom sample rate."""
audio = np.zeros(48000, dtype="float32") # 48k samples
duration = audio_duration(audio, sample_rate=48000)
assert duration == 1.0
class TestSamplesForDuration:
"""Tests for samples_for_duration function."""
def test_one_second(self):
"""Test samples for 1 second at default rate."""
samples = samples_for_duration(1.0)
assert samples == SAMPLE_RATE
def test_half_second(self):
"""Test samples for 0.5 second at default rate."""
samples = samples_for_duration(0.5)
assert samples == SAMPLE_RATE // 2
def test_zero_duration(self):
"""Test samples for 0 duration."""
samples = samples_for_duration(0)
assert samples == 0
def test_negative_duration(self):
"""Test samples for negative duration."""
samples = samples_for_duration(-1.0)
assert samples == 0
def test_custom_sample_rate(self):
"""Test samples with custom sample rate."""
samples = samples_for_duration(1.0, sample_rate=44100)
assert samples == 44100
class TestSampleRateConstant:
"""Tests for SAMPLE_RATE constant."""
def test_sample_rate_value(self):
"""Test SAMPLE_RATE is 24000."""
assert SAMPLE_RATE == 24000
+97
View File
@@ -0,0 +1,97 @@
"""Tests for domain/chapter_classification.py."""
from abogen.domain.chapter_classification import (
supplement_score,
should_preselect_chapter,
ensure_at_least_one_chapter_enabled,
)
class TestSupplementScore:
def test_title_page_high_score(self):
score = supplement_score("Title Page", "", 0)
assert score > 3.0
def test_chapter_title_negative_score(self):
score = supplement_score("Chapter 1", "", 0)
assert score < 0
def test_copyright_high_score(self):
score = supplement_score("Copyright", "All rights reserved.", 0)
assert score > 2.0
def test_short_text_adds_score(self):
score = supplement_score("Some Title", "Short text.", 0)
assert score > 0.5
def test_long_text_low_score_contribution(self):
score = supplement_score("Some Title", "word " * 200, 0)
assert score < 0.5
def test_index_zero_bonus(self):
score_title = supplement_score("Dedication", "For my family.", 0)
score_other = supplement_score("Dedication", "For my family.", 3)
assert score_title > score_other
def test_empty_title_and_text(self):
score = supplement_score("", "", 5)
assert score == 0.9 # short text bonus only (len("") ≤ 150)
def test_newsletter_keyword_high_score(self):
score = supplement_score("Subscribe", "Join our newsletter today", 0)
assert score > 3.0
def test_acknowledgments_pattern(self):
score = supplement_score("Acknowledgements", "", 0)
assert score > 2.0
def test_glossary_in_title(self):
score = supplement_score("Glossary", "", 0)
assert score > 2.0
class TestShouldPreselectChapter:
def test_single_chapter_always_preselected(self):
assert should_preselect_chapter("Anything", "", 0, 1) is True
def test_chapter_preselected_when_low_score(self):
assert should_preselect_chapter("Chapter 1", "The story begins.", 0, 10) is True
def test_title_page_not_preselected(self):
assert should_preselect_chapter("Title Page", "", 0, 10) is False
def test_copyright_not_preselected(self):
assert should_preselect_chapter("Copyright", "All rights reserved.", 0, 10) is False
def test_toc_not_preselected(self):
assert should_preselect_chapter("Table of Contents", "", 0, 10) is False
class TestEnsureAtLeastOneChapterEnabled:
def test_empty_list(self):
chapters = []
ensure_at_least_one_chapter_enabled(chapters)
assert chapters == []
def test_already_has_enabled(self):
chapters = [
{"title": "Ch1", "enabled": False},
{"title": "Ch2", "enabled": True},
]
ensure_at_least_one_chapter_enabled(chapters)
assert chapters[1]["enabled"] is True
assert chapters[0]["enabled"] is False
def test_none_enabled_picks_longest(self):
chapters = [
{"title": "Ch1", "enabled": False, "characters": 100},
{"title": "Ch2", "enabled": False, "characters": 500},
{"title": "Ch3", "enabled": False, "characters": 200},
]
ensure_at_least_one_chapter_enabled(chapters)
assert chapters[1]["enabled"] is True
def test_single_chapter_gets_enabled(self):
chapters = [{"title": "Only", "enabled": False}]
ensure_at_least_one_chapter_enabled(chapters)
assert chapters[0]["enabled"] is True
+189
View File
@@ -0,0 +1,189 @@
"""Tests for domain/audio_sink.py"""
import numpy as np
import soundfile as sf
from pathlib import Path
from unittest.mock import MagicMock, patch, call
import subprocess
from abogen.domain.audio_sink import AudioSink, open_audio_sink, _ensure_ffmpeg
class TestAudioSinkDataclass:
def test_audio_sink_is_frozen(self):
sink = AudioSink(write=lambda d: None, close=lambda: None)
try:
sink.write = lambda d: None # type: ignore
except AttributeError:
pass # Expected — frozen=True
assert hasattr(sink, "write")
assert hasattr(sink, "close")
def test_audio_sink_stores_callables(self):
write_fn = MagicMock()
close_fn = MagicMock()
sink = AudioSink(write=write_fn, close=close_fn)
assert sink.write is write_fn
assert sink.close is close_fn
def test_audio_sink_write_callable(self):
calls = []
sink = AudioSink(write=lambda d: calls.append(d), close=lambda: None)
data = np.zeros(100, dtype="float32")
sink.write(data)
assert len(calls) == 1
np.testing.assert_array_equal(calls[0], data)
def test_audio_sink_close_callable(self):
closed = []
sink = AudioSink(write=lambda d: None, close=lambda: closed.append(True))
sink.close()
assert closed == [True]
class TestOpenAudioSinkWav:
def test_wav_creates_file(self, tmp_path: Path):
out = tmp_path / "test.wav"
with open_audio_sink(out, "wav") as sink:
sink.write(np.zeros(100, dtype="float32"))
assert out.exists()
def test_flac_creates_file(self, tmp_path: Path):
out = tmp_path / "test.flac"
with open_audio_sink(out, "flac") as sink:
sink.write(np.zeros(100, dtype="float32"))
assert out.exists()
def test_wav_sink_writes_audio(self, tmp_path: Path):
out = tmp_path / "out.wav"
audio = np.random.uniform(-0.5, 0.5, 24000).astype("float32") # 1 second
with open_audio_sink(out, "wav") as sink:
sink.write(audio)
data, sr = sf.read(str(out))
assert sr == 24000
assert len(data) == 24000
np.testing.assert_allclose(data, audio, atol=1e-4)
def test_wav_sink_close_flushes(self, tmp_path: Path):
out = tmp_path / "flush.wav"
with open_audio_sink(out, "wav") as sink:
sink.write(np.ones(1000, dtype="float32"))
assert out.exists()
info = sf.info(str(out))
assert info.samplerate == 24000
assert info.channels == 1
def test_wav_sink_context_manager(self, tmp_path: Path):
out = tmp_path / "ctx.wav"
with open_audio_sink(out, "wav") as sink:
assert isinstance(sink, AudioSink)
sink.write(np.zeros(50, dtype="float32"))
assert out.exists()
def test_wav_sink_multiple_writes(self, tmp_path: Path):
out = tmp_path / "multi.wav"
with open_audio_sink(out, "wav") as sink:
sink.write(np.ones(1000, dtype="float32"))
sink.write(np.ones(500, dtype="float32"))
data, _ = sf.read(str(out))
assert len(data) == 1500
class TestCancelCheck:
def test_cancel_check_skips_wav_writes(self, tmp_path: Path):
out = tmp_path / "cancelled.wav"
with open_audio_sink(out, "wav", cancel_check=lambda: True) as sink:
sink.write(np.ones(1000, dtype="float32"))
sink.write(np.ones(500, dtype="float32"))
data, _ = sf.read(str(out))
assert len(data) == 0
def test_cancel_check_none_allows_writes(self, tmp_path: Path):
out = tmp_path / "ok.wav"
with open_audio_sink(out, "wav", cancel_check=None) as sink:
sink.write(np.ones(1000, dtype="float32"))
data, _ = sf.read(str(out))
assert len(data) == 1000
class TestUnsupportedFormat:
def test_unsupported_format_raises(self, tmp_path: Path):
out = tmp_path / "bad.xyz"
try:
with open_audio_sink(out, "xyz") as sink:
pass
assert False, "Should have raised"
except Exception:
pass # Expected
class TestOpenAudioSinkCompressed:
@patch("abogen.domain.audio_sink._ensure_ffmpeg")
@patch("abogen.domain.audio_sink.build_ffmpeg_command")
@patch("abogen.domain.audio_sink.subprocess.Popen")
def test_mp3_sink_returns_sink(self, mock_popen, mock_build, mock_ensure, tmp_path: Path):
mock_build.return_value = ["ffmpeg", "-y", "-i", "pipe:0", "out.mp3"]
proc = MagicMock()
proc.stdin = MagicMock()
proc.stdin.closed = False
proc.wait.return_value = 0
mock_popen.return_value = proc
out = tmp_path / "test.mp3"
sink = open_audio_sink(out, "mp3")
assert isinstance(sink, AudioSink)
sink.write(np.zeros(100, dtype="float32"))
assert proc.stdin.write.called
sink.close()
@patch("abogen.domain.audio_sink._ensure_ffmpeg")
@patch("abogen.domain.audio_sink.build_ffmpeg_command")
@patch("abogen.domain.audio_sink.subprocess.Popen")
def test_cancel_check_skips_compressed_writes(self, mock_popen, mock_build, mock_ensure, tmp_path: Path):
mock_build.return_value = ["ffmpeg", "-y", "-i", "pipe:0", "out.mp3"]
proc = MagicMock()
proc.stdin = MagicMock()
proc.stdin.closed = False
proc.wait.return_value = 0
mock_popen.return_value = proc
sink = open_audio_sink(tmp_path / "c.mp3", "mp3", cancel_check=lambda: True)
sink.write(np.zeros(100, dtype="float32"))
assert not proc.stdin.write.called
sink.close()
@patch("abogen.domain.audio_sink._ensure_ffmpeg")
@patch("abogen.domain.audio_sink.build_ffmpeg_command")
@patch("abogen.domain.audio_sink.subprocess.Popen")
def test_extra_ffmpeg_args_passed(self, mock_popen, mock_build, mock_ensure, tmp_path: Path):
mock_build.return_value = ["ffmpeg", "-y", "-header", "-i", "pipe:0", "out.mp3"]
proc = MagicMock()
proc.stdin = MagicMock()
proc.stdin.closed = False
proc.wait.return_value = 0
mock_popen.return_value = proc
sink = open_audio_sink(
tmp_path / "extra.mp3",
"mp3",
extra_ffmpeg_args=["-thread_queue_size", "32768"],
)
args = mock_popen.call_args[0][0]
assert "-thread_queue_size" in args
assert "32768" in args
sink.close()
@patch("abogen.domain.audio_sink._ensure_ffmpeg")
@patch("abogen.domain.audio_sink.build_ffmpeg_command")
@patch("abogen.domain.audio_sink.subprocess.Popen")
def test_metadata_passed_to_ffmpeg(self, mock_popen, mock_build, mock_ensure, tmp_path: Path):
mock_build.return_value = ["ffmpeg", "-y", "-i", "pipe:0", "out.opus"]
proc = MagicMock()
proc.stdin = MagicMock()
proc.stdin.closed = False
proc.wait.return_value = 0
mock_popen.return_value = proc
meta = {"title": "Test", "artist": "Author"}
open_audio_sink(tmp_path / "meta.opus", "opus", metadata=meta)
mock_build.assert_called_once_with(tmp_path / "meta.opus", "opus", metadata=meta)
+73
View File
@@ -0,0 +1,73 @@
"""Tests that voice.py imports from domain modules, not from conversion_runner."""
import pathlib
_MODULE_SOURCE_CACHE: dict[str, str] = {}
def _read_source(module_file: str) -> str:
if module_file not in _MODULE_SOURCE_CACHE:
_MODULE_SOURCE_CACHE[module_file] = pathlib.Path(module_file).read_text(
encoding="utf-8"
)
return _MODULE_SOURCE_CACHE[module_file]
def test_voice_module_does_not_import_conversion_runner():
"""voice.py must not import from conversion_runner (architecture rule)."""
import abogen.webui.routes.utils.voice as voice_mod
source_text = _read_source(voice_mod.__file__)
assert "from abogen.webui.conversion_runner import" not in source_text, (
"voice.py still imports from conversion_runner — must use domain modules"
)
def test_synthesize_module_does_not_import_conversion_runner():
"""synthesize.py must not import from conversion_runner."""
import abogen.webui.routes.utils.synthesize as synthesize_mod
source_text = _read_source(synthesize_mod.__file__)
assert "from abogen.webui.conversion_runner import" not in source_text
def test_synthesize_module_imports_select_device_from_domain():
"""synthesize.py must import select_device from abogen.domain.device."""
import abogen.webui.routes.utils.synthesize as synthesize_mod
source_text = _read_source(synthesize_mod.__file__)
assert "from abogen.domain.device import" in source_text
def test_voice_module_does_not_define_synthesize_audio_from_normalized():
"""Dead code: synthesize_audio_from_normalized must be removed from voice.py."""
import abogen.webui.routes.utils.voice as voice_mod
source_text = _read_source(voice_mod.__file__)
assert "def synthesize_audio_from_normalized(" not in source_text
def test_voice_module_does_not_define_get_preview_pipeline():
"""Dead code: get_preview_pipeline must be removed from voice.py."""
import abogen.webui.routes.utils.voice as voice_mod
source_text = _read_source(voice_mod.__file__)
assert "def get_preview_pipeline(" not in source_text
def test_voice_module_does_not_import_domain_audio_helpers():
"""After removing dead code, voice.py no longer needs audio_helpers imports."""
import abogen.webui.routes.utils.voice as voice_mod
source_text = _read_source(voice_mod.__file__)
assert "from abogen.domain.audio_helpers import" not in source_text
def test_voice_module_does_not_import_domain_device():
"""After removing dead code, voice.py no longer needs device imports."""
import abogen.webui.routes.utils.voice as voice_mod
source_text = _read_source(voice_mod.__file__)
assert "from abogen.domain.device import" not in source_text
+147
View File
@@ -0,0 +1,147 @@
"""Tests for domain/normalization.py — prepare_text_for_tts."""
import pytest
from unittest.mock import patch, MagicMock
from abogen.domain.normalization import prepare_text_for_tts, normalize_text_for_pipeline
class TestPrepareTextForTts:
"""Tests for the comprehensive TTS text preparation pipeline."""
def test_empty_text(self):
result = prepare_text_for_tts("")
assert result == ""
def test_none_text(self):
result = prepare_text_for_tts(None)
assert result == ""
def test_passthrough_no_rules(self):
result = prepare_text_for_tts("Hello world")
assert isinstance(result, str)
assert len(result) > 0
def test_heteronym_rules_applied(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [
{
"token": "read",
"pronunciation": "red",
"context": "past tense",
}
]
rules = compile_heteronym_sentence_rules(overrides)
if rules:
result = prepare_text_for_tts("I will read the book", heteronym_rules=rules)
assert isinstance(result, str)
def test_pronunciation_rules_applied(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [
{
"token": "epub",
"pronunciation": "ee-pub",
"normalized": "epub",
}
]
rules = compile_pronunciation_rules(overrides)
result = prepare_text_for_tts(
"This is an epub file",
pronunciation_rules=rules,
)
assert "ee-pub" in result
def test_usage_counter_tracks_pronunciation(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [
{
"token": "data",
"pronunciation": "day-ta",
"normalized": "data",
}
]
rules = compile_pronunciation_rules(overrides)
counter = {}
prepare_text_for_tts(
"The data is here and the data is there",
pronunciation_rules=rules,
usage_counter=counter,
)
assert counter.get("data", 0) >= 1
def test_combined_heteronym_and_pronunciation(self):
from abogen.domain.pronunciation import (
compile_heteronym_sentence_rules,
compile_pronunciation_rules,
)
heteronym_overrides = [
{
"token": "lead",
"pronunciation": "led",
"context": "metal",
}
]
pronunciation_overrides = [
{
"token": "gif",
"pronunciation": "jif",
"normalized": "gif",
}
]
h_rules = compile_heteronym_sentence_rules(heteronym_overrides)
p_rules = compile_pronunciation_rules(pronunciation_overrides)
result = prepare_text_for_tts(
"A lead gif",
heteronym_rules=h_rules if h_rules else None,
pronunciation_rules=p_rules,
)
assert isinstance(result, str)
@patch("abogen.domain.normalization.get_runtime_settings")
def test_normalization_overrides_passed_through(self, mock_settings):
mock_settings.return_value = {
"normalization_apostrophe_mode": "spacy",
"normalization_enabled": True,
}
result = prepare_text_for_tts(
"It's a test",
normalization_overrides={"normalization_enabled": False},
)
assert isinstance(result, str)
def test_pronunciation_rules_empty(self):
result = prepare_text_for_tts("Hello", pronunciation_rules=[])
assert isinstance(result, str)
def test_heteronym_rules_empty(self):
result = prepare_text_for_tts("Hello", heteronym_rules=[])
assert isinstance(result, str)
class TestNormalizeTextForPipeline:
"""Tests for the simpler normalization function."""
def test_basic_normalization(self):
result = normalize_text_for_pipeline("It's a test")
assert isinstance(result, str)
assert len(result) > 0
def test_empty_text(self):
result = normalize_text_for_pipeline("")
assert result == ""
@patch("abogen.domain.normalization.get_runtime_settings")
def test_with_overrides(self, mock_settings):
mock_settings.return_value = {
"normalization_apostrophe_mode": "spacy",
}
result = normalize_text_for_pipeline(
"It's a test",
normalization_overrides={"normalization_apostrophe_mode": "spacy"},
)
assert isinstance(result, str)
+175
View File
@@ -0,0 +1,175 @@
from __future__ import annotations
from unittest.mock import MagicMock, patch
from abogen.domain.pipeline_factory import (
PipelinePool,
create_pipeline_for_job,
dispose_pipelines,
resolve_device,
)
class TestResolveDevice:
@patch("abogen.utils.load_config", return_value={"use_gpu": True})
@patch("abogen.domain.pipeline_factory.select_device", return_value="cuda:0")
def test_gpu_enabled(self, _sel, _cfg):
assert resolve_device(use_gpu=True) == "cuda:0"
@patch("abogen.utils.load_config", return_value={"use_gpu": True})
def test_gpu_disabled_by_job(self, _cfg):
assert resolve_device(use_gpu=False) == "cpu"
@patch("abogen.utils.load_config", return_value={"use_gpu": False})
@patch("abogen.domain.pipeline_factory.select_device", return_value="cuda:0")
def test_gpu_disabled_by_config(self, _sel, _cfg):
assert resolve_device(use_gpu=True) == "cpu"
class TestCreatePipelineForJob:
@patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
def test_supertonic_provider(self, _reg, mock_create):
mock_create.return_value = MagicMock()
result = create_pipeline_for_job("supertonic", "en", use_gpu=True)
mock_create.assert_called_once_with("supertonic")
assert result is mock_create.return_value
@patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_kokoro_provider(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock()
result = create_pipeline_for_job("kokoro", "en", use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="en", device="cpu")
assert result is mock_create.return_value
@patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=False)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_unknown_provider_falls_back_to_kokoro(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock()
result = create_pipeline_for_job("unknown_provider", "en", use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="en", device="cpu")
@patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_empty_provider_defaults_to_kokoro(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock()
result = create_pipeline_for_job("", "en", use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="en", device="cpu")
@patch("abogen.domain.pipeline_factory.create_pipeline")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=True)
@patch("abogen.domain.pipeline_factory.resolve_device", return_value="cpu")
def test_none_provider_defaults_to_kokoro(self, _dev, _reg, mock_create):
mock_create.return_value = MagicMock()
result = create_pipeline_for_job(None, "en", use_gpu=False)
mock_create.assert_called_once_with("kokoro", lang_code="en", device="cpu")
class TestDisposePipelines:
def test_disposes_all_and_clears(self):
p1 = MagicMock()
p2 = MagicMock()
pipelines = {"kokoro": p1, "supertonic": p2}
dispose_pipelines(pipelines)
p1.dispose.assert_called_once()
p2.dispose.assert_called_once()
assert pipelines == {}
def test_handles_dispose_error(self):
p1 = MagicMock()
p1.dispose.side_effect = RuntimeError("boom")
pipelines = {"kokoro": p1}
dispose_pipelines(pipelines)
assert pipelines == {}
def test_empty_dict(self):
pipelines = {}
dispose_pipelines(pipelines)
assert pipelines == {}
class TestPipelinePool:
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_get_creates_and_caches(self, _cache, mock_create):
mock_pipeline = MagicMock()
mock_create.return_value = mock_pipeline
pool = PipelinePool()
result = pool.get("kokoro", "en", use_gpu=True)
assert result is mock_pipeline
mock_create.assert_called_once()
result2 = pool.get("kokoro", "en", use_gpu=True)
assert result2 is mock_pipeline
assert mock_create.call_count == 1
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_get_initializes_voice_cache_once(self, mock_cache, mock_create):
mock_create.return_value = MagicMock()
pool = PipelinePool()
job = MagicMock()
pool.get("kokoro", "en", use_gpu=True, job=job)
assert mock_cache.call_count == 1
pool.get("kokoro", "en", use_gpu=True, job=job)
assert mock_cache.call_count == 1
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
def test_get_no_job_skips_voice_cache(self, mock_create, mock_cache):
mock_create.return_value = MagicMock()
pool = PipelinePool()
pool.get("kokoro", "en", use_gpu=True)
mock_cache.assert_not_called()
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
def test_get_separately_per_provider(self, mock_create):
p1 = MagicMock(name="kokoro")
p2 = MagicMock(name="supertonic")
mock_create.side_effect = [p1, p2]
pool = PipelinePool()
r1 = pool.get("kokoro", "en", use_gpu=True)
r2 = pool.get("supertonic", "en", use_gpu=True)
assert r1 is p1
assert r2 is p2
assert mock_create.call_count == 2
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_dispose_all(self, mock_cache, mock_create):
p1 = MagicMock(name="kokoro")
p2 = MagicMock(name="supertonic")
mock_create.side_effect = [p1, p2]
pool = PipelinePool()
pool.get("kokoro", "en", use_gpu=True)
pool.get("supertonic", "en", use_gpu=True)
pool.dispose_all()
p1.dispose.assert_called_once()
p2.dispose.assert_called_once()
assert pool._pipelines == {}
assert pool._voice_cache_initialized is False
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
def test_dispose_empty_pool(self, mock_create):
pool = PipelinePool()
pool.dispose_all()
mock_create.assert_not_called()
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
@patch("abogen.domain.pipeline_factory.is_plugin_registered", return_value=False)
def test_unknown_provider_falls_back(self, _reg, _cache, mock_create):
mock_create.return_value = MagicMock()
pool = PipelinePool()
pool.get("bogus_provider", "en", use_gpu=True)
mock_create.assert_called_once_with("kokoro", "en", True)
+71
View File
@@ -0,0 +1,71 @@
from __future__ import annotations
import time
from unittest.mock import patch
from abogen.domain.progress import ProgressTracker, calc_etr_str
class TestCalcEtrStr:
def test_returns_processing_when_not_started(self):
assert calc_etr_str(0.0, 0, 100) == "Processing..."
def test_returns_processing_when_elapsed_too_short(self):
assert calc_etr_str(0.3, 50, 100) == "Processing..."
def test_returns_processing_when_done_zero(self):
assert calc_etr_str(2.0, 0, 100) == "Processing..."
def test_returns_zero_when_complete(self):
assert calc_etr_str(10.0, 100, 100) == "00:00:00"
def test_returns_zero_when_overcomplete(self):
assert calc_etr_str(10.0, 150, 100) == "00:00:00"
def test_half_done(self):
# 10s for 50 chars -> 10s remaining -> 00:00:10
assert calc_etr_str(10.0, 50, 100) == "00:00:10"
def test_one_third_done_large(self):
# 100s for 1000 chars out of 3000 -> 200s remaining
assert calc_etr_str(100.0, 1000, 3000) == "00:03:20"
def test_hours_format(self):
# 3600s for 1000 chars out of 4000 -> 3 * 3600 = 10800s remaining
assert calc_etr_str(3600.0, 1000, 4000) == "03:00:00"
def test_minutes_and_seconds(self):
# 60s for 100 chars out of 200 -> 60s remaining
assert calc_etr_str(60.0, 100, 200) == "00:01:00"
class TestProgressTracker:
def test_percent_at_zero(self):
t = ProgressTracker(total_chars=1000)
assert t.percent == 0
def test_percent_half(self):
t = ProgressTracker(total_chars=1000)
t.update(500)
assert t.percent == 50
def test_percent_capped_at_99(self):
t = ProgressTracker(total_chars=100)
t.update(100)
assert t.percent == 99 # matches original behavior
def test_etr_processing_at_start(self):
t = ProgressTracker(total_chars=1000)
t.update(0)
assert t.etr_str == "Processing..."
def test_etr_computes_correctly(self):
t = ProgressTracker(total_chars=200)
with patch("abogen.domain.progress.time") as mock_time:
mock_time.time.return_value = t._start_time + 2.0
t.update(100)
assert t.etr_str == "00:00:02"
def test_zero_total_chars(self):
t = ProgressTracker(total_chars=0)
assert t.percent == 0
+195
View File
@@ -0,0 +1,195 @@
from __future__ import annotations
from unittest.mock import patch
from abogen.domain.voice_utils import (
coerce_truthy,
formula_from_kokoro_entry,
infer_provider_from_spec,
resolve_voice_target,
split_speaker_reference,
supertonic_voice_from_spec,
)
class TestSplitSpeakerReference:
def test_speaker_prefix(self):
assert split_speaker_reference("speaker:af_sarah") == ("af_sarah", "speaker:af_sarah")
def test_profile_prefix(self):
assert split_speaker_reference("profile:custom") == ("custom", "profile:custom")
def test_no_prefix(self):
assert split_speaker_reference("af_sarah") == (None, "af_sarah")
def test_empty(self):
assert split_speaker_reference("") == (None, "")
def test_none(self):
assert split_speaker_reference(None) == (None, "")
def test_unknown_prefix(self):
assert split_speaker_reference("unknown:name") == (None, "unknown:name")
def test_empty_name_after_colon(self):
assert split_speaker_reference("speaker:") == (None, "speaker:")
class TestSupertonicVoiceFromSpec:
def test_uppercase_passthrough(self):
assert supertonic_voice_from_spec("M1", "M1") == "M1"
def test_lowercase_converted(self):
assert supertonic_voice_from_spec("m1", "M1") == "M1"
def test_empty_spec_uses_fallback(self):
assert supertonic_voice_from_spec("", "F1") == "F1"
def test_formula_spec_uses_fallback(self):
assert supertonic_voice_from_spec("af_sarah*0.5+bf_emma*0.5", "M1") == "M1"
def test_empty_both_gives_default(self):
assert supertonic_voice_from_spec("", "") == "M1"
class TestFormulaFromKokoroEntry:
def test_single_voice(self):
entry = {"voices": [["af_sarah", 1.0]]}
result = formula_from_kokoro_entry(entry)
assert "af_sarah" in result
assert "1.000000" in result
def test_weighted_mix(self):
entry = {"voices": [["af_sarah", 0.6], ["bf_emma", 0.4]]}
result = formula_from_kokoro_entry(entry)
assert "af_sarah" in result
assert "bf_emma" in result
assert "+" in result
def test_empty_voices(self):
assert formula_from_kokoro_entry({"voices": []}) == ""
def test_missing_voices_key(self):
assert formula_from_kokoro_entry({}) == ""
def test_invalid_entries_filtered(self):
entry = {"voices": [["af_sarah", "bad"], ["bf_emma", 0.5]]}
result = formula_from_kokoro_entry(entry)
assert "bf_emma" in result
assert "af_sarah" not in result
class TestCoerceTruthy:
def test_bool_passthrough(self):
assert coerce_truthy(True) is True
assert coerce_truthy(False) is False
def test_string_true(self):
assert coerce_truthy("yes") is True
assert coerce_truthy("1") is True
def test_string_false(self):
assert coerce_truthy("false") is False
assert coerce_truthy("0") is False
assert coerce_truthy("") is False
def test_none_default(self):
assert coerce_truthy(None) is True
assert coerce_truthy(None, False) is False
def test_int(self):
assert coerce_truthy(1) is True
assert coerce_truthy(0) is False
class TestInferProviderFromSpec:
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah", "bf_emma"])
def test_known_kokoro_voice(self, _mock):
assert infer_provider_from_spec("af_sarah") == "kokoro"
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_uppercase_supertonic(self, _mock):
assert infer_provider_from_spec("M1") == "supertonic"
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_formula_kokoro(self, _mock):
assert infer_provider_from_spec("af_sarah*0.5+bf_emma*0.5") == "kokoro"
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_empty_fallback(self, _mock):
assert infer_provider_from_spec("", "kokoro") == "kokoro"
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_unknown_falls_back(self, _mock):
assert infer_provider_from_spec("unknown_xyz", "supertonic") == "supertonic"
class TestResolveVoiceTarget:
def test_empty_spec_kokoro_default(self):
provider, spec, speed, steps = resolve_voice_target(
"", {}, job_voice="af_sarah", job_tts_provider="kokoro",
)
assert provider == "kokoro"
assert spec == ""
def test_speaker_profile_kokoro(self):
profiles = {
"narrator": {
"provider": "kokoro",
"voices": [["af_sarah", 0.7], ["bf_emma", 0.3]],
},
}
provider, spec, speed, steps = resolve_voice_target(
"speaker:narrator", profiles,
)
assert provider == "kokoro"
assert "af_sarah" in spec
assert speed is None
assert steps is None
def test_speaker_profile_supertonic(self):
profiles = {
"narrator": {
"provider": "supertonic",
"voice": "F1",
"speed": 1.2,
"total_steps": 10,
},
}
provider, spec, speed, steps = resolve_voice_target(
"speaker:narrator", profiles,
job_voice="M1", job_speed=1.0, job_supertonic_total_steps=5,
)
assert provider == "supertonic"
assert spec == "F1"
assert speed == 1.2
assert steps == 10
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_direct_supertonic_spec(self, _mock):
provider, spec, speed, steps = resolve_voice_target(
"M1", {},
job_voice="M1",
)
assert provider == "supertonic"
assert spec == "M1"
@patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"])
def test_direct_kokoro_spec(self, _mock):
provider, spec, speed, steps = resolve_voice_target(
"af_sarah", {},
job_tts_provider="kokoro",
)
assert provider == "kokoro"
assert spec == "af_sarah"
def test_profile_missing_provider_defaults_kokoro(self):
profiles = {
"narrator": {
"voices": [["af_sarah", 1.0]],
},
}
provider, spec, speed, steps = resolve_voice_target(
"speaker:narrator", profiles,
)
assert provider == "kokoro"
+3 -1
View File
@@ -38,8 +38,10 @@ from tests.contracts.engine_contract import EngineContractMixin
def _kokoro_available() -> bool:
try:
from kokoro import KPipeline # type: ignore[import-not-found]
import spacy
spacy.load("en_core_web_sm")
return True
except ImportError:
except (ImportError, OSError):
return False
+199
View File
@@ -0,0 +1,199 @@
"""Tests for abogen.domain.metadata_extraction module."""
import pytest
from abogen.domain.metadata_extraction import (
extract_metadata_from_text,
get_filename_from_path,
build_ffmpeg_metadata_args,
extract_metadata_and_build_args,
read_text_for_metadata,
)
class TestExtractMetadataFromText:
"""Tests for extract_metadata_from_text function."""
def test_extract_all_metadata(self):
"""Test extracting all metadata tags."""
text = """
<<METADATA_TITLE:Test Book>>
<<METADATA_ARTIST:Test Author>>
<<METADATA_ALBUM:Test Album>>
<<METADATA_YEAR:2024>>
<<METADATA_ALBUM_ARTIST:Album Artist>>
<<METADATA_COMPOSER:Composer Name>>
<<METADATA_GENRE:Fiction>>
<<METADATA_COVER_PATH:/path/to/cover.jpg>>
"""
metadata = extract_metadata_from_text(text)
assert metadata["title"] == "Test Book"
assert metadata["artist"] == "Test Author"
assert metadata["album"] == "Test Album"
assert metadata["year"] == "2024"
assert metadata["album_artist"] == "Album Artist"
assert metadata["composer"] == "Composer Name"
assert metadata["genre"] == "Fiction"
assert metadata["cover_path"] == "/path/to/cover.jpg"
def test_extract_partial_metadata(self):
"""Test extracting partial metadata."""
text = "<<METADATA_TITLE:Only Title>>"
metadata = extract_metadata_from_text(text)
assert metadata["title"] == "Only Title"
assert metadata["artist"] is None
assert metadata["cover_path"] is None
def test_empty_text(self):
"""Test extracting from empty text."""
metadata = extract_metadata_from_text("")
for key in metadata:
assert metadata[key] is None
def test_no_tags(self):
"""Test text without metadata tags."""
text = "This is just regular text without any metadata tags."
metadata = extract_metadata_from_text(text)
for key in metadata:
assert metadata[key] is None
def test_strip_whitespace(self):
"""Test that values are stripped of whitespace."""
text = "<<METADATA_TITLE: Test Book >>"
metadata = extract_metadata_from_text(text)
assert metadata["title"] == "Test Book"
class TestGetFilenameFromPath:
"""Tests for get_filename_from_path function."""
def test_simple_path(self):
"""Test extracting filename from simple path."""
filename = get_filename_from_path("/path/to/file.txt")
assert filename == "file"
def test_path_with_multiple_extensions(self):
"""Test extracting filename from path with multiple extensions."""
filename = get_filename_from_path("/path/to/file.tar.gz")
assert filename == "file.tar"
def test_windows_path(self):
"""Test extracting filename from Windows path."""
# Note: This test may behave differently on Windows vs Unix
# but should work correctly on the current platform
filename = get_filename_from_path("C:\\path\\to\\file.txt")
assert "file" in filename
def test_with_display_path(self):
"""Test using display_path when not from_queue."""
filename = get_filename_from_path(
file_path="/original/path/file.txt",
display_path="/display/path/display_file.txt",
from_queue=False,
)
assert filename == "display_file"
def test_with_display_path_from_queue(self):
"""Test ignoring display_path when from_queue."""
filename = get_filename_from_path(
file_path="/original/path/file.txt",
display_path="/display/path/display_file.txt",
from_queue=True,
)
assert filename == "file"
class TestBuildFfmpegMetadataArgs:
"""Tests for build_ffmpeg_metadata_args function."""
def test_all_metadata_provided(self):
"""Test building args with all metadata provided."""
metadata = {
"title": "Test Title",
"artist": "Test Artist",
"album": "Test Album",
"year": "2024",
"album_artist": "Album Artist",
"composer": "Composer",
"genre": "Fiction",
}
args = build_ffmpeg_metadata_args(metadata, "fallback")
assert "-metadata" in args
assert "title=Test Title" in args
assert "artist=Test Artist" in args
assert "album=Test Album" in args
assert "date=2024" in args # year -> date
def test_use_defaults(self):
"""Test that defaults are used for missing metadata."""
metadata = {} # Empty metadata
args = build_ffmpeg_metadata_args(metadata, "mybook")
# Should use defaults
assert any("title=mybook" in arg for arg in args)
assert any("artist=Unknown" in arg for arg in args)
assert any("genre=Audiobook" in arg for arg in args)
def test_empty_values_skipped(self):
"""Test that empty values are skipped."""
metadata = {
"title": "Test",
"artist": "",
"album": None,
}
args = build_ffmpeg_metadata_args(metadata, "fallback")
# Should have title but not artist/album (empty)
assert any("title=Test" in arg for arg in args)
class TestExtractMetadataAndBuildArgs:
"""Tests for extract_metadata_and_build_args function."""
def test_full_workflow(self):
"""Test full metadata extraction and arg building."""
text = "<<METADATA_TITLE:My Book>>\n<<METADATA_ARTIST:My Author>>"
args, cover_path = extract_metadata_and_build_args(
text=text,
filename="mybook.txt",
)
assert any("title=My Book" in arg for arg in args)
assert any("artist=My Author" in arg for arg in args)
assert cover_path is None
def test_with_cover_path(self):
"""Test extraction with cover path."""
text = "<<METADATA_COVER_PATH:/covers/cover.jpg>>"
args, cover_path = extract_metadata_and_build_args(
text=text,
filename="mybook.txt",
)
assert cover_path == "/covers/cover.jpg"
class TestReadTextForMetadata:
"""Tests for read_text_for_metadata function."""
def test_direct_text(self):
"""Test reading direct text."""
text = read_text_for_metadata(
file_path="This is direct text",
is_direct_text=True,
)
assert text == "This is direct text"
def test_file_not_found(self):
"""Test handling of file not found."""
text = read_text_for_metadata(
file_path="/nonexistent/file.txt",
is_direct_text=False,
)
assert text == ""
+170
View File
@@ -0,0 +1,170 @@
"""Tests for domain/metadata_helpers.py Audiobookshelf helpers."""
from abogen.domain.metadata_helpers import (
normalize_metadata_casefold,
split_people_field,
split_simple_list,
first_nonempty,
extract_year,
normalize_series_sequence,
build_audiobookshelf_metadata,
load_audiobookshelf_chapters,
)
# --- normalize_metadata_casefold ---
def test_normalize_metadata_casefold_basic():
result = normalize_metadata_casefold({"Title": " Hello ", "Author": None, "": "skip"})
assert result == {"title": "Hello", "author": ""} or result == {"title": "Hello"}
def test_normalize_metadata_casefold_preserves_lists():
result = normalize_metadata_casefold({"tags": ["a", "b"]})
assert result["tags"] == ["a", "b"]
# --- split_people_field ---
def test_split_people_field_single():
assert split_people_field("J.K. Rowling") == ["J.K. Rowling"]
def test_split_people_field_multiple():
result = split_people_field("Tolkien, Lewis & Martin")
assert "Tolkien" in result
assert "Lewis" in result
assert "Martin" in result
def test_split_people_field_deduplicates():
result = split_people_field("Tolkien, tolkien, TOLKIEN")
assert len(result) == 1
def test_split_people_field_none():
assert split_people_field(None) == []
def test_split_people_field_list():
result = split_people_field(["Author A", "Author B"])
assert result == ["Author A", "Author B"]
# --- split_simple_list ---
def test_split_simple_list_basic():
result = split_simple_list("fantasy, sci-fi; thriller")
assert "fantasy" in result
assert "sci-fi" in result
assert "thriller" in result
def test_split_simple_list_none():
assert split_simple_list(None) == []
# --- first_nonempty ---
def test_first_nonempty_basic():
assert first_nonempty(None, "", "hello") == "hello"
def test_first_nonempty_first_wins():
assert first_nonempty("first", "second") == "first"
def test_first_nonempty_none():
assert first_nonempty(None, None) is None
def test_first_nonempty_list():
assert first_nonempty(None, ["a", "b"]) == "a"
# --- extract_year ---
def test_extract_year_full_date():
assert extract_year("Published on 2023-05-15") == 2023
def test_extract_year_plain():
assert extract_year("2020") == 2020
def test_extract_year_none():
assert extract_year(None) is None
def test_extract_year_invalid():
assert extract_year("no year here") is None
# --- normalize_series_sequence ---
def test_normalize_series_sequence_int():
assert normalize_series_sequence(3) == "3"
def test_normalize_series_sequence_float():
assert normalize_series_sequence(2.5) == "2.5"
def test_normalize_series_sequence_string():
assert normalize_series_sequence(" 12 ") == "12"
def test_normalize_series_sequence_comma():
assert normalize_series_sequence("1,5") == "1.5"
def test_normalize_series_sequence_none():
assert normalize_series_sequence(None) is None
def test_normalize_series_sequence_nan():
import math
assert normalize_series_sequence(float("nan")) is None
# --- build_audiobookshelf_metadata ---
def test_build_audiobookshelf_metadata_basic():
tags = {
"title": "My Book",
"author": "Author Name",
"description": "A great book",
}
result = build_audiobookshelf_metadata(tags, language="en")
assert result["title"] == "My Book"
assert result["authors"] == ["Author Name"]
assert result["language"] == "en"
def test_build_audiobookshelf_metadata_series():
tags = {
"title": "Book 2",
"series": "My Series",
"series_index": "2",
}
result = build_audiobookshelf_metadata(tags, language="en")
assert result["seriesName"] == "My Series"
assert result["seriesSequence"] == "2"
def test_build_audiobookshelf_metadata_fallback_title():
tags = {"author": "Someone"}
result = build_audiobookshelf_metadata(tags, language="en", filename="chapter1")
assert result["title"] == "chapter1"
def test_build_audiobookshelf_metadata_empty():
result = build_audiobookshelf_metadata({}, language="en")
assert result["language"] == "en"
assert "authors" not in result # empty list stripped
def test_build_audiobookshelf_metadata_strips_empty():
tags = {"title": "Book", "subtitle": "", "description": None}
result = build_audiobookshelf_metadata(tags, language="en")
assert "subtitle" not in result
assert "description" not in result
@@ -1,11 +1,11 @@
from abogen.webui.routes.utils import preview
from abogen.webui.routes.utils import synthesize
def test_preview_applies_manual_override_before_normalization(monkeypatch):
# Don't run real TTS/normalization; just exercise the override stage by
# forcing provider=kokoro and then stubbing normalize_for_pipeline.
monkeypatch.setattr(preview, "get_preview_pipeline", lambda language, device: None)
monkeypatch.setattr(synthesize, "get_preview_pipeline", lambda language, device: None)
# Stub normalize_for_pipeline to be identity; we only care that overrides run.
class _Norm:
@@ -42,7 +42,7 @@ def test_preview_applies_manual_override_before_normalization(monkeypatch):
monkeypatch.setattr(utils, "create_pipeline", _mock_create_pipeline)
try:
preview.generate_preview_audio(
synthesize.generate_preview_audio(
text="He said Unfu*k loudly.",
voice_spec="M1",
language="en",
+145
View File
@@ -0,0 +1,145 @@
"""Regression tests: domain extraction must not break webui conversion_runner.
These tests verify that the refactored WebUI code paths still call into the
correct domain functions and produce the same results as the old inline logic.
"""
from __future__ import annotations
from unittest.mock import MagicMock, patch
from abogen.domain.voice_utils import resolve_voice_target
from abogen.domain.pipeline_factory import PipelinePool
class TestResolveVoiceTargetRegression:
"""Verify that the domain resolve_voice_target produces the same results
as the old closure in conversion_runner.py."""
def test_empty_spec_returns_kokoro_default(self):
provider, spec, speed, steps = resolve_voice_target(
"", {}, job_voice="af_sarah", job_tts_provider="kokoro",
)
assert provider == "kokoro"
assert spec == ""
def test_speaker_in_profile_kokoro(self):
profiles = {
"narrator": {
"provider": "kokoro",
"voices": [["af_sarah", 0.7], ["bf_emma", 0.3]],
},
}
provider, spec, speed, steps = resolve_voice_target(
"speaker:narrator", profiles,
)
assert provider == "kokoro"
assert "af_sarah" in spec
assert speed is None
assert steps is None
def test_speaker_in_profile_supertonic(self):
profiles = {
"narrator": {
"provider": "supertonic",
"voice": "F1",
"speed": 1.2,
"total_steps": 10,
},
}
provider, spec, speed, steps = resolve_voice_target(
"speaker:narrator", profiles,
job_voice="M1", job_speed=1.0, job_supertonic_total_steps=5,
)
assert provider == "supertonic"
assert spec == "F1"
assert speed == 1.2
assert steps == 10
def test_unknown_speaker_infers_from_spec(self):
with patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"]):
provider, spec, speed, steps = resolve_voice_target(
"af_sarah", {}, job_tts_provider="kokoro",
)
assert provider == "kokoro"
assert spec == "af_sarah"
def test_uppercase_spec_infers_supertonic(self):
with patch("abogen.domain.voice_utils.get_voices", return_value=["af_sarah"]):
provider, spec, speed, steps = resolve_voice_target(
"M1", {}, job_voice="M1",
)
assert provider == "supertonic"
assert spec == "M1"
class TestPipelinePoolRegression:
"""Verify that PipelinePool behaves like the old inline get_pipeline closure."""
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_same_provider_returns_cached_pipeline(self, _cache, mock_create):
mock_pipeline = MagicMock()
mock_create.return_value = mock_pipeline
pool = PipelinePool()
r1 = pool.get("kokoro", "en", use_gpu=True)
r2 = pool.get("kokoro", "en", use_gpu=True)
assert r1 is r2
assert mock_create.call_count == 1
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_different_providers_get_separate_pipelines(self, _cache, mock_create):
p1 = MagicMock(name="kokoro")
p2 = MagicMock(name="supertonic")
mock_create.side_effect = [p1, p2]
pool = PipelinePool()
r1 = pool.get("kokoro", "en", use_gpu=True)
r2 = pool.get("supertonic", "en", use_gpu=True)
assert r1 is p1
assert r2 is p2
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_dispose_all_cleans_up(self, _cache, mock_create):
p1 = MagicMock()
p2 = MagicMock()
mock_create.side_effect = [p1, p2]
pool = PipelinePool()
pool.get("kokoro", "en", use_gpu=True)
pool.get("supertonic", "en", use_gpu=True)
pool.dispose_all()
p1.dispose.assert_called_once()
p2.dispose.assert_called_once()
assert pool._pipelines == {}
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_voice_cache_initialized_only_once(self, mock_cache, mock_create):
mock_create.return_value = MagicMock()
pool = PipelinePool()
job = MagicMock()
pool.get("kokoro", "en", use_gpu=True, job=job)
pool.get("kokoro", "en", use_gpu=True, job=job)
assert mock_cache.call_count == 1
@patch("abogen.domain.pipeline_factory.create_pipeline_for_job")
@patch("abogen.domain.pipeline_factory.initialize_voice_cache")
def test_after_dispose_voice_cache_can_reinitialize(self, mock_cache, mock_create):
mock_create.return_value = MagicMock()
pool = PipelinePool()
job = MagicMock()
pool.get("kokoro", "en", use_gpu=True, job=job)
assert mock_cache.call_count == 1
pool.dispose_all()
assert pool._voice_cache_initialized is False
pool.get("kokoro", "en", use_gpu=True, job=job)
assert mock_cache.call_count == 2
+205
View File
@@ -0,0 +1,205 @@
"""Tests for abogen.domain.subtitle_generation module."""
import pytest
from abogen.domain.subtitle_generation import (
process_subtitle_tokens,
PUNCTUATION_SENTENCE,
PUNCTUATION_SENTENCE_COMMA,
)
class TestProcessSubtitleTokens:
"""Tests for process_subtitle_tokens function."""
def test_empty_tokens(self):
"""Test processing empty token list does nothing."""
entries = []
process_subtitle_tokens(
tokens_with_timestamps=[],
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Sentence",
lang_code="a",
)
assert entries == []
def test_disabled_mode(self):
"""Test Disabled mode still processes tokens (no special handling).
Note: In current implementation, Disabled mode doesn't skip processing
but relies on the caller to not call this function. This test documents
current behavior.
"""
tokens = [
{"start": 0.0, "end": 1.0, "text": "Hello", "whitespace": " "},
{"start": 1.0, "end": 2.0, "text": "world", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Disabled",
lang_code="a",
)
# Disabled mode doesn't have special handling in current implementation
# It processes tokens normally
assert len(entries) >= 1
def test_line_mode_basic(self):
"""Test Line mode processes tokens."""
tokens = [
{"start": 0.0, "end": 1.0, "text": "First line", "whitespace": " "},
{"start": 1.0, "end": 2.0, "text": "Second line", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Line",
lang_code="a",
)
# Line mode processes all tokens into entries
assert len(entries) >= 1
# Check that text is preserved
combined_text = " ".join(e[2] for e in entries)
assert "First line" in combined_text
assert "Second line" in combined_text
def test_sentence_mode_punctuation_split(self):
"""Test Sentence mode splits on sentence punctuation."""
tokens = [
{"start": 0.0, "end": 0.5, "text": "First sentence", "whitespace": " "},
{"start": 0.5, "end": 1.0, "text": ".", "whitespace": " "},
{"start": 1.0, "end": 1.5, "text": "Second sentence", "whitespace": " "},
{"start": 1.5, "end": 2.0, "text": ".", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Sentence",
lang_code="a",
)
assert len(entries) >= 1
# Should have at least one entry with both sentences or split
combined_text = " ".join(e[2] for e in entries)
assert "First sentence" in combined_text
assert "Second sentence" in combined_text
def test_word_count_mode(self):
"""Test word count mode (e.g., '5' for 5 words per entry)."""
tokens = [
{"start": 0.0, "end": 0.2, "text": "word1", "whitespace": " "},
{"start": 0.2, "end": 0.4, "text": "word2", "whitespace": " "},
{"start": 0.4, "end": 0.6, "text": "word3", "whitespace": " "},
{"start": 0.6, "end": 0.8, "text": "word4", "whitespace": " "},
{"start": 0.8, "end": 1.0, "text": "word5", "whitespace": " "},
{"start": 1.0, "end": 1.2, "text": "word6", "whitespace": " "},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="2", # 2 words per entry
lang_code="a",
)
assert len(entries) >= 2
# Check that entries are split roughly by word count
for entry in entries:
# Each entry should have at least one word
assert len(entry[2].split()) >= 1
def test_fallback_end_time(self):
"""Test fallback_end_time is applied when end time is invalid."""
tokens = [
{"start": 0.0, "end": None, "text": "Test", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Line",
lang_code="a",
fallback_end_time=10.0,
)
assert len(entries) == 1
assert entries[0][1] == 10.0 # Should use fallback
def test_karaoke_highlighting_mode(self):
"""Test Sentence + Highlighting mode generates karaoke tags."""
tokens = [
{"start": 0.0, "end": 0.5, "text": "Hello", "whitespace": " "},
{"start": 0.5, "end": 1.0, "text": "world", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Sentence + Highlighting",
lang_code="a",
)
assert len(entries) >= 1
# Should contain karaoke tags
text = entries[0][2]
assert "{\\kf" in text
def test_max_subtitle_words_limit(self):
"""Test that max_subtitle_words limits entry length."""
tokens = [
{"start": float(i), "end": float(i + 0.1), "text": f"word{i}", "whitespace": " "}
for i in range(10)
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=3,
subtitle_mode="Line",
lang_code="a",
)
# Should have more than 1 entry due to word limit
assert len(entries) > 1
def test_preserves_token_timing(self):
"""Test that token timing is preserved in entries."""
tokens = [
{"start": 0.0, "end": 1.0, "text": "First", "whitespace": " "},
{"start": 1.0, "end": 2.0, "text": "Second", "whitespace": ""},
]
entries = []
process_subtitle_tokens(
tokens_with_timestamps=tokens,
subtitle_entries=entries,
max_subtitle_words=50,
subtitle_mode="Sentence",
lang_code="a",
)
assert len(entries) >= 1
# Check that timing is preserved
for entry in entries:
assert entry[0] >= 0.0
assert entry[1] >= entry[0]
class TestPunctuationConstants:
"""Tests for punctuation constants."""
def test_punctuation_sentence_contains_basic(self):
"""Test PUNCTUATION_SENTENCE contains basic sentence punctuation."""
assert "." in PUNCTUATION_SENTENCE
assert "!" in PUNCTUATION_SENTENCE
assert "?" in PUNCTUATION_SENTENCE
def test_punctuation_sentence_comma_contains_comma(self):
"""Test PUNCTUATION_SENTENCE_COMMA contains comma."""
assert "," in PUNCTUATION_SENTENCE_COMMA
assert "." in PUNCTUATION_SENTENCE_COMMA
assert "!" in PUNCTUATION_SENTENCE_COMMA
assert "?" in PUNCTUATION_SENTENCE_COMMA
+70
View File
@@ -0,0 +1,70 @@
"""Tests that preview/synthesis module is correctly named and importable."""
import pathlib
def _read_source(module_file: str) -> str:
return pathlib.Path(module_file).read_text(encoding="utf-8")
def test_preview_file_renamed_to_synthesize():
"""preview.py must be renamed to synthesize.py."""
import abogen.webui.routes.utils.synthesize as synthesize_mod
synthesize_path = pathlib.Path(synthesize_mod.__file__)
assert synthesize_path.name == "synthesize.py"
assert synthesize_path.exists()
assert not synthesize_path.with_name("preview.py").exists()
def test_synthesize_module_has_generate_preview_audio():
"""synthesize.py must export generate_preview_audio."""
from abogen.webui.routes.utils.synthesize import generate_preview_audio
assert callable(generate_preview_audio)
def test_synthesize_module_has_synthesize_preview():
"""synthesize.py must export synthesize_preview."""
from abogen.webui.routes.utils.synthesize import synthesize_preview
assert callable(synthesize_preview)
def test_synthesize_module_has_get_preview_pipeline():
"""synthesize.py must export get_preview_pipeline."""
from abogen.webui.routes.utils.synthesize import get_preview_pipeline
assert callable(get_preview_pipeline)
def test_api_imports_from_synthesize():
"""api.py must import from synthesize, not preview."""
import abogen.webui.routes.api as api_mod
source = _read_source(api_mod.__file__)
assert "from abogen.webui.routes.utils.synthesize import" in source
assert "from abogen.webui.routes.utils.preview import" not in source
def test_voices_imports_from_synthesize():
"""voices.py must import from synthesize, not preview."""
import abogen.webui.routes.voices as voices_mod
source = _read_source(voices_mod.__file__)
assert "from abogen.webui.routes.utils.synthesize import" in source
assert "from abogen.webui.routes.utils.preview import" not in source
def test_no_module_imports_preview():
"""No module should import from the old preview path."""
import glob
import os
project_root = pathlib.Path(__file__).parent.parent
py_files = glob.glob(str(project_root / "abogen" / "**" / "*.py"), recursive=True)
for py_file in py_files:
content = pathlib.Path(py_file).read_text(encoding="utf-8")
assert "from abogen.webui.routes.utils.preview import" not in content, (
f"{py_file} still imports from preview"
)
+137
View File
@@ -0,0 +1,137 @@
"""Tests for abogen.domain.voice_loader module."""
import pytest
from abogen.domain.voice_loader import (
VoiceCache,
resolve_voice,
load_voice_cached,
)
class TestVoiceCache:
"""Tests for VoiceCache class."""
def test_get_set(self):
"""Test basic get/set operations."""
cache = VoiceCache()
cache.set("test_voice", "loaded_voice")
assert cache.get("test_voice") == "loaded_voice"
def test_get_missing(self):
"""Test get returns None for missing voice."""
cache = VoiceCache()
assert cache.get("missing_voice") is None
def test_contains(self):
"""Test contains method."""
cache = VoiceCache()
cache.set("test_voice", "loaded_voice")
assert cache.contains("test_voice")
assert not cache.contains("missing_voice")
def test_in_operator(self):
"""Test __contains__ (in operator)."""
cache = VoiceCache()
cache.set("test_voice", "loaded_voice")
assert "test_voice" in cache
assert "missing_voice" not in cache
def test_clear(self):
"""Test clear method."""
cache = VoiceCache()
cache.set("voice1", "loaded1")
cache.set("voice2", "loaded2")
cache.clear()
assert not cache.contains("voice1")
assert not cache.contains("voice2")
class TestResolveVoice:
"""Tests for resolve_voice function."""
def test_simple_voice_name(self):
"""Test that simple voice names are returned as-is."""
result = resolve_voice(
voice_spec="test_voice",
pipeline=None,
use_gpu=False,
)
assert result == "test_voice"
def test_formula_voice_without_pipeline(self):
"""Test formula voice returns spec when no pipeline."""
result = resolve_voice(
voice_spec="model*0.5+0.3*other",
pipeline=None,
use_gpu=False,
)
assert result == "model*0.5+0.3*other"
def test_caching(self):
"""Test that voices are cached."""
cache = VoiceCache()
# First call should load (we'll mock with simple name)
result1 = resolve_voice(
voice_spec="test_voice",
pipeline=None,
use_gpu=False,
cache=cache,
)
assert result1 == "test_voice"
assert cache.contains("test_voice")
# Second call should use cache
result2 = resolve_voice(
voice_spec="test_voice",
pipeline=None,
use_gpu=False,
cache=cache,
)
assert result2 == "test_voice"
class TestLoadVoiceCached:
"""Tests for load_voice_cached function."""
def test_simple_voice_name(self):
"""Test that simple voice names are returned as-is."""
result = load_voice_cached(
voice_name="test_voice",
pipeline=None,
use_gpu=False,
)
assert result == "test_voice"
def test_dict_cache(self):
"""Test caching with dict."""
cache = {}
result1 = load_voice_cached(
voice_name="test_voice",
pipeline=None,
use_gpu=False,
cache=cache,
)
assert result1 == "test_voice"
assert "test_voice" in cache
result2 = load_voice_cached(
voice_name="test_voice",
pipeline=None,
use_gpu=False,
cache=cache,
)
assert result2 == "test_voice"
assert cache["test_voice"] == "test_voice"
def test_no_cache(self):
"""Test without cache parameter."""
result = load_voice_cached(
voice_name="test_voice",
pipeline=None,
use_gpu=False,
cache=None,
)
assert result == "test_voice"