Compare commits

...
26 Commits
Author SHA1 Message Date
Artem Akymenko 7fef9c1d93 extract normalize_text_for_pipeline to domain/normalization.py 2026-07-15 15:19:01 +00:00
Artem Akymenko 56cfd0810d extract resolve_fallback_voice_spec to domain/voice_resolution.py; fix missing get_default_voice import and __custom_mix reset bug 2026-07-15 15:01:17 +00:00
Artem Akymenko 7bd3177241 extract select_device to domain/device.py; fix bug where conversion_runner didn't check torch availability 2026-07-15 14:45:38 +00:00
Artem AkymenkoandGitHub d5c2a81733 Merge pull request #191 from hydraxman/fix/large-chapter-form-limits
fix(webui): allow large chapter forms
2026-07-15 17:35:52 +03:00
Artem Akymenko 514e29a761 extract apply_chapter_text_transforms to domain/chapter_titles.py 2026-07-15 14:29:29 +00:00
Artem Akymenko 86042a3315 fix bugs, remove dead code and unused imports in conversion_runner.py 2026-07-15 13:50:18 +00:00
Artem Akymenko 50d75eb2fc standardize m4b encoding to VBR -q:a 2; replace remaining ffmpeg blocks in desktop GUI with domain modules 2026-07-15 13:28:21 +00:00
Artem Akymenko ae9ab70421 refactor: extract audio helpers to domain/audio_helpers.py
- Extract build_ffmpeg_command, to_float32, apply_m4b_chapters_with_mutagen
- _apply_m4b_chapters_with_mutagen becomes thin wrapper with error handling
- Add tests/test_audio_helpers.py (12 tests)
- conversion_runner.py: 1410 → 1320 lines
- All tests pass
2026-07-15 12:15:10 +00:00
Artem Akymenko 4364276a5b refactor: extract output path utilities to domain/output_paths.py
- Extract slugify, sanitize_output_stem, output_timestamp_token, build_output_path
- Extract apply_newline_policy, resolve_output_directory, resolve_project_layout
- _prepare_output_dir and _prepare_project_layout become thin wrappers with mkdir
- Add tests/test_output_paths.py (21 tests)
- conversion_runner.py: 1443 → 1410 lines
- All tests pass
2026-07-15 11:56:06 +00:00
Artem Akymenko 914e77de46 refactor: wire up domain/voice_utils.py and remove duplicates
- Import supertonic_voice_from_spec, split_speaker_reference, formula_from_kokoro_entry
- Import infer_provider_from_spec, coerce_truthy from domain/voice_utils.py
- Remove duplicate function bodies from conversion_runner.py
- conversion_runner.py: 1518 → 1443 lines
- All tests pass
2026-07-15 11:06:29 +00:00
Artem Akymenko 1d7a2aeed6 refactor: extract chunk utils to domain/chunk_utils.py
- Extract safe_int, group_chunks_by_chapter, record_override_usage, chunk_text_for_tts
- Add tests/test_chunk_utils.py (15 tests)
- Update test_chunk_helpers.py and test_chunk_text_for_tts_prefers_raw.py imports
- conversion_runner.py: 1574 → 1518 lines
- All tests pass
2026-07-15 11:00:18 +00:00
Artem Akymenko a26e02b017 refactor: wire up domain/chapter_overrides.py and domain/metadata_merge.py
- Update chapter_overrides.py to return tuple matching original signature
- Import apply_chapter_overrides and merge_metadata from domain modules
- Remove old function bodies from conversion_runner.py
- Add tests/test_chapter_merge_normalize.py (19 tests)
- conversion_runner.py: 1677 → 1574 lines
- All tests pass
2026-07-15 10:36:31 +00:00
Artem Akymenko c94347b33b refactor: extract voice resolution to domain/voice_resolution.py
- Extract spec_to_voice_ids, job_voice_fallback, collect_required_voice_ids
- Extract initialize_voice_cache, chapter_voice_spec, chunk_voice_spec
- Add tests/test_voice_resolution.py (29 tests)
- conversion_runner.py: 1822 → 1677 lines
- All tests pass
2026-07-15 10:20:58 +00:00
Artem Akymenko b7a48e3204 refactor: extract pronunciation rules to domain/pronunciation.py
- Extract compile_pronunciation_rules, compile_heteronym_sentence_rules
- Extract apply_pronunciation_rules, apply_heteronym_sentence_rules
- Extract merge_pronunciation_overrides
- Add tests/test_pronunciation.py (31 tests)
- All tests pass
2026-07-14 18:27:22 +00:00
Artem Akymenko feb38a24ec fix: create missing domain/file_type.py from previous incomplete refactoring 2026-07-14 18:27:13 +00:00
Artem Akymenko f63590932d refactor: extract voice utils to domain/voice_utils.py
- Extract infer_provider_from_spec, supertonic_voice_from_spec, split_speaker_reference, formula_from_kokoro_entry, coerce_truthy to domain/voice_utils.py
- Add tests/test_voice_utils.py with 24 tests
- All tests match old behavior
2026-07-14 11:02:34 +00:00
Artem Akymenko 7777e58f1d refactor: extract title/outro builders into domain/title_builder.py
- Extract build_title_intro_text and build_outro_text into domain/title_builder.py
- Uses metadata_helpers for metadata processing
- Remove _build_title_intro_text and _build_outro_text from conversion_runner.py
- Add tests/test_title_builder.py with 12 tests
- All tests match old behavior
2026-07-14 10:27:48 +00:00
Artem Akymenko 364c179bd6 refactor: extract metadata helpers into domain/metadata_helpers.py
- Extract normalize_metadata_map, format_author_sentence, ensure_sentence
- Extract normalize_series_number, extract_series_metadata, format_series_sentence
- Remove _SERIES_NAME_KEYS, _SERIES_NUMBER_KEYS, _SERIES_NUMBER_RE from conversion_runner.py
- Add tests/test_metadata_helpers.py with 32 tests
- All tests match old behavior
2026-07-14 10:26:05 +00:00
Artem Akymenko 60ba01557e refactor: extract chapter title processing into domain/chapter_titles.py
- Extract simplify_heading_text, headings_equivalent, strip_duplicate_heading_line
- Extract normalize_caps_word, normalize_chapter_opening_caps
- Extract format_spoken_chapter_title
- Remove _HEADING_SANITIZE_RE, _HEADING_NUMBER_PREFIX_RE, _ACRONYM_ALLOWLIST, _ROMAN_NUMERAL_CHARS, _CAPS_WORD_RE from conversion_runner.py
- Add tests/test_chapter_titles.py with 31 tests
- All tests match old behavior
2026-07-14 10:17:20 +00:00
Artem Akymenko 39eac9b032 refactor: replace _srt_time/_ass_time with _format_timestamp from infrastructure/subtitle_writer.py
- Remove _srt_time() and _ass_time() methods from ConversionThread
- Use _format_timestamp() from infrastructure/subtitle_writer.py instead
- Supports both SRT (ass=False) and ASS (ass=True) formats
- All existing tests pass
2026-07-14 10:14:58 +00:00
Artem Akymenko 1499a3b426 refactor: extract _get_split_pattern into domain/split_pattern.py
- Extract unified split pattern logic to domain/split_pattern.py
- Add get_split_pattern() function with language and subtitle_mode support
- Remove duplicated logic from pyqt/conversion.py
- Update pyqt/conversion.py to use domain.split_pattern.get_split_pattern
- Add tests/test_split_pattern.py with 20 tests covering English, CJK, Spanish, French, and pattern structure
2026-07-14 10:11:52 +00:00
Artem Akymenko 013c80b92c refactor: migrate FFmpeg metadata functions to infrastructure/exporters.py
- Extract FFmpeg metadata functions to infrastructure/exporters.py as ExportService
- _escape_ffmetadata_value → _escape_ffmetadata_value
- _render_ffmetadata → render_ffmetadata
- _write_ffmetadata_file → write_ffmetadata_file
- _metadata_to_ffmpeg_args → _metadata_to_ffmpeg_args
- _apply_m4b_chapters_with_mutagen → _apply_m4b_chapters_mutagen
- _embed_m4b_metadata → embed_m4b_metadata
- Add tests/test_exporters.py with 28 tests for ExportService
- Update tests/test_ffmetadata.py to use ExportService
- Update conversion_runner.py to use ExportService
- All tests pass with new implementation matching old behavior
2026-07-14 10:09:27 +00:00
Artem Akymenko 62f42a9f79 refactor: migrate SubtitleWriter to infrastructure/subtitle_writer.py
- Extract SubtitleWriter classes (SrtWriter, AssWriter, VttWriter) to infrastructure/subtitle_writer.py
- Add create_subtitle_writer() factory function
- Remove old SubtitleWriter class and _format_timestamp from conversion_runner.py
- Use create_subtitle_writer() factory from infrastructure layer
- Add tests/test_subtitle_writer.py with 28 tests covering SrtWriter, AssWriter, VttWriter
- All tests match old _format_timestamp behavior
2026-07-14 10:06:51 +00:00
Bryan Nathan 2c4d13bf56 fix(webui): allow large chapter forms 2026-07-14 08:51:40 +08:00
Artem Akymenko b7026a666d refactor(shutdown): move shutdown logic in one place 2026-07-12 20:19:33 +03:00
Artem Akymenko c380a58496 tts: fix kokoro AlbertModel import for transformers 5.x (plugin architecture)
- Monkey-patch transformers.AlbertModel in plugins/kokoro/__init__.py before kokoro imports it
- Works with transformers 4.x (no-op) and 5.x (adds moved symbol)
- No pinning, forks, or extra files needed
2026-07-12 20:17:25 +03:00
49 changed files with 5706 additions and 1941 deletions
+118
View File
@@ -0,0 +1,118 @@
"""Audio helper utilities.
Functions for building ffmpeg commands, converting audio formats,
and applying chapter metadata to MP4 files.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Dict, List, Optional
import numpy as np
SAMPLE_RATE = 24000
def build_ffmpeg_command(path: Path, fmt: str, metadata: Optional[Dict[str, str]] = None) -> list[str]:
from abogen.infrastructure.exporters import ExportService
base = [
"ffmpeg",
"-y",
"-f",
"f32le",
"-ar",
str(SAMPLE_RATE),
"-ac",
"1",
"-i",
"pipe:0",
]
if fmt == "mp3":
base += ["-c:a", "libmp3lame", "-qscale:a", "2"]
elif fmt == "opus":
base += ["-c:a", "libopus", "-b:a", "24000"]
elif fmt == "m4b":
base += ["-c:a", "aac", "-q:a", "2", "-movflags", "+faststart+use_metadata_tags"]
else:
base += ["-c:a", "copy"]
if metadata:
svc = ExportService()
base.extend(svc._metadata_to_ffmpeg_args(metadata))
base.append(str(path))
return base
def to_float32(audio_segment) -> np.ndarray:
if audio_segment is None:
return np.zeros(0, dtype="float32")
tensor = audio_segment
if hasattr(tensor, "detach"):
tensor = tensor.detach()
if hasattr(tensor, "cpu"):
try:
tensor = tensor.cpu()
except Exception:
pass
if hasattr(tensor, "numpy"):
return np.asarray(tensor.numpy(), dtype="float32").reshape(-1)
return np.asarray(tensor, dtype="float32").reshape(-1)
def apply_m4b_chapters_with_mutagen(
audio_path: Path,
chapters: List[Dict[str, Any]],
) -> bool:
"""Apply chapter atoms to an MP4/M4B file using mutagen.
Returns True if chapters were written, False otherwise.
Raises ImportError if mutagen is not installed.
"""
if not chapters:
return False
from fractions import Fraction
from mutagen.mp4 import MP4, MP4Chapter # type: ignore[import]
mp4 = MP4(str(audio_path))
chapter_objects: List[MP4Chapter] = []
for index, entry in enumerate(sorted(chapters, key=lambda item: float(item.get("start") or 0.0))):
start_raw = entry.get("start")
if start_raw is None:
continue
try:
start_seconds = max(0.0, float(start_raw))
except (TypeError, ValueError):
continue
title_value = entry.get("title")
title_text = str(title_value) if title_value else f"Chapter {index + 1}"
start_fraction = Fraction(int(round(start_seconds * 1000)), 1000)
chapter_atom = MP4Chapter(start_fraction, title_text)
end_raw = entry.get("end")
if end_raw is not None:
try:
end_seconds = float(end_raw)
except (TypeError, ValueError):
end_seconds = None
if end_seconds is not None and end_seconds > start_seconds:
chapter_atom.end = Fraction(int(round(end_seconds * 1000)), 1000)
chapter_objects.append(chapter_atom)
if not chapter_objects:
return False
from typing import cast
mp4.chapters = cast(Any, chapter_objects)
mp4.save()
return True
+92
View File
@@ -0,0 +1,92 @@
from __future__ import annotations
from typing import Any, Dict, List, Optional, Tuple
from abogen.text_extractor import ExtractedChapter
from abogen.domain.voice_utils import coerce_truthy
def apply_chapter_overrides(
extracted: List[ExtractedChapter],
overrides: List[Dict[str, Any]],
) -> Tuple[List[ExtractedChapter], Dict[str, str], List[str]]:
if not overrides:
return [], {}, []
selected: List[ExtractedChapter] = []
metadata_updates: Dict[str, str] = {}
diagnostics: List[str] = []
for position, payload in enumerate(overrides):
if not isinstance(payload, dict):
diagnostics.append(
f"Skipped chapter override at position {position + 1}: unsupported payload type {type(payload).__name__}."
)
continue
enabled = coerce_truthy(payload.get("enabled", True))
payload["enabled"] = enabled
if not enabled:
continue
metadata_payload = payload.get("metadata") or {}
if isinstance(metadata_payload, dict):
for key, value in metadata_payload.items():
if value is None:
continue
metadata_updates[str(key)] = str(value)
base: Optional[ExtractedChapter] = None
idx_candidate = payload.get("index")
idx_normalized: Optional[int] = None
if isinstance(idx_candidate, int):
idx_normalized = idx_candidate
elif isinstance(idx_candidate, str):
try:
idx_normalized = int(idx_candidate)
except ValueError:
idx_normalized = None
if idx_normalized is not None and 0 <= idx_normalized < len(extracted):
base = extracted[idx_normalized]
payload["index"] = idx_normalized
if base is None:
source_title = payload.get("source_title")
if isinstance(source_title, str):
base = next((chapter for chapter in extracted if chapter.title == source_title), None)
if base is None:
candidate_title = payload.get("title")
if isinstance(candidate_title, str):
base = next((chapter for chapter in extracted if chapter.title == candidate_title), None)
text_override = payload.get("text")
if text_override is not None:
text_value = str(text_override)
elif base is not None:
text_value = base.text
else:
diagnostics.append(
f"Skipped chapter override at position {position + 1}: no text provided and no matching source chapter found."
)
continue
title_override = payload.get("title")
if title_override is not None:
title_value = str(title_override)
elif base is not None:
title_value = base.title
else:
title_value = f"Chapter {position + 1}"
if base and not payload.get("source_title"):
payload["source_title"] = base.title
payload["title"] = title_value
payload["text"] = text_value
payload["characters"] = len(text_value)
payload.setdefault("order", payload.get("order", position))
selected.append(ExtractedChapter(title=title_value, text=text_value))
return selected, metadata_updates, diagnostics
+204
View File
@@ -0,0 +1,204 @@
from __future__ import annotations
import re
from typing import List, Tuple
_HEADING_SANITIZE_RE = re.compile(r"[^a-z0-9]+")
_HEADING_NUMBER_PREFIX_RE = re.compile(
r"^\s*(?P<number>(?:\d+|[ivxlcdm]+))(?P<suffix>(?:[\s.:;-].*)?)$",
re.IGNORECASE,
)
_ACRONYM_ALLOWLIST = {
"AI", "API", "CPU", "DIY", "GPU", "HTML", "HTTP", "HTTPS", "ID",
"JSON", "MP3", "MP4", "M4B", "NASA", "OCR", "PDF", "SQL", "TV",
"TTS", "UK", "UN", "UFO", "OK", "URL", "USA", "US", "VR",
}
_ROMAN_NUMERAL_CHARS = frozenset("IVXLCDM")
_CAPS_WORD_RE = re.compile(r"[A-Z][A-Z0-9'\u2019-]*")
def simplify_heading_text(text: str) -> str:
raw = str(text or "").strip().lower()
if not raw:
return ""
simplified = _HEADING_SANITIZE_RE.sub("", raw)
if simplified.startswith("chapter"):
simplified = simplified[7:]
return simplified
def headings_equivalent(left: str, right: str) -> bool:
simple_left = simplify_heading_text(left)
simple_right = simplify_heading_text(right)
if not simple_left or not simple_right:
return False
if simple_left == simple_right:
return True
if simple_right.startswith(simple_left):
return True
if simple_left.startswith(simple_right):
return True
if len(simple_left) > 5 and simple_left in simple_right:
return True
return False
def strip_duplicate_heading_line(text: str, heading: str) -> Tuple[str, bool]:
source_text = str(text or "")
if not source_text:
return source_text, False
normalized_heading = simplify_heading_text(heading)
if not normalized_heading:
return source_text, False
lines = source_text.splitlines()
new_lines: List[str] = []
removed = False
for line in lines:
stripped = line.strip()
if not removed and stripped:
if headings_equivalent(stripped, heading):
removed = True
continue
new_lines.append(line)
if not removed:
return source_text, False
while new_lines and not new_lines[0].strip():
new_lines.pop(0)
return "\n".join(new_lines), True
def normalize_caps_word(word: str) -> str:
upper = word.upper()
letters = [char for char in upper if char.isalpha()]
if not letters:
return word
if upper in _ACRONYM_ALLOWLIST:
return word
if len(letters) <= 1:
return word
if all(char in _ROMAN_NUMERAL_CHARS for char in letters) and len(letters) <= 7:
return word
parts = re.split(r"(['\-\u2019])", word)
normalized_parts: List[str] = []
for part in parts:
if part in {"'", "-", "\u2019"}:
normalized_parts.append(part)
continue
if not part:
continue
normalized_parts.append(part[0].upper() + part[1:].lower())
return "".join(normalized_parts) or word
def normalize_chapter_opening_caps(text: str) -> Tuple[str, bool]:
if not text:
return text, False
leading_len = len(text) - len(text.lstrip())
leading = text[:leading_len]
working = text[leading_len:]
if not working:
return text, False
builder: List[str] = []
pos = 0
changed = False
while pos < len(working):
char = working[pos]
if char in "\r\n":
builder.append(working[pos:])
pos = len(working)
break
if char.isspace():
builder.append(char)
pos += 1
continue
if char.islower():
builder.append(working[pos:])
pos = len(working)
break
if not char.isalpha():
builder.append(char)
pos += 1
continue
match = _CAPS_WORD_RE.match(working, pos)
if not match:
builder.append(char)
pos += 1
continue
word = match.group(0)
if any(ch.islower() for ch in word):
builder.append(working[pos:])
pos = len(working)
break
normalized = normalize_caps_word(word)
if normalized != word:
changed = True
builder.append(normalized)
pos = match.end()
if pos < len(working):
builder.append(working[pos:])
if not changed:
return text, False
return leading + "".join(builder), True
def format_spoken_chapter_title(title: str, index: int, apply_prefix: bool) -> str:
base = str(title or "").strip()
if not base:
return f"Chapter {index}" if apply_prefix else ""
if not apply_prefix:
return base
lowered = base.lower()
if lowered.startswith("chapter") and (len(lowered) == 7 or not lowered[7].isalpha()):
return base
match = _HEADING_NUMBER_PREFIX_RE.match(base)
if match:
number = match.group("number") or ""
suffix = match.group("suffix") or ""
cleaned_suffix = suffix.lstrip(" .,:;-_ \t\u2013\u2014\u00b7\u2022")
if cleaned_suffix:
return f"Chapter {number}. {cleaned_suffix}"
return f"Chapter {number}"
return base
def apply_chapter_text_transforms(
text: str,
*,
heading_text: str,
raw_title: str,
strip_heading: bool,
normalize_caps: bool,
) -> Tuple[str, bool, bool]:
"""Strip duplicate heading and normalize opening caps.
Returns ``(text, heading_removed, caps_changed)``.
The caller is responsible for state updates (pending flags, logging,
dict mutation, ``continue``).
"""
heading_removed = False
caps_changed = False
if strip_heading and heading_text:
text, heading_removed = strip_duplicate_heading_line(text, heading_text)
if not heading_removed and raw_title:
match = _HEADING_NUMBER_PREFIX_RE.match(raw_title)
if match:
number = match.group("number")
if number:
text, heading_removed = strip_duplicate_heading_line(text, number)
if normalize_caps and text:
text, caps_changed = normalize_chapter_opening_caps(text)
return text, heading_removed, caps_changed
+75
View File
@@ -0,0 +1,75 @@
"""Chunk processing utilities.
Functions for grouping chunks, recording override usage, and selecting
text for TTS synthesis.
"""
from __future__ import annotations
from collections import defaultdict
from typing import Any, Dict, Iterable, Mapping, Optional
from abogen.pronunciation_store import increment_usage
def safe_int(value: Any, default: int = 0) -> int:
try:
return int(value)
except (TypeError, ValueError):
return default
def group_chunks_by_chapter(chunks: Iterable[Dict[str, Any]]) -> Dict[int, List[Dict[str, Any]]]:
grouped: Dict[int, List[Dict[str, Any]]] = defaultdict(list)
for entry in chunks or []:
if not isinstance(entry, dict):
continue
try:
chapter_index = int(entry.get("chapter_index", 0))
except (TypeError, ValueError):
chapter_index = 0
grouped[chapter_index].append(dict(entry))
for chapter_index, items in grouped.items():
items.sort(key=lambda payload: safe_int(payload.get("chunk_index")))
return grouped
def record_override_usage(
job: Any,
usage_counter: Mapping[str, int],
token_map: Mapping[str, str],
) -> None:
if not usage_counter:
return
language = getattr(job, "language", "") or "a"
for normalized, amount in usage_counter.items():
if amount <= 0:
continue
token_value = token_map.get(normalized, normalized)
try:
increment_usage(language=language, token=token_value, amount=int(amount))
except Exception: # pragma: no cover - defensive logging
job.add_log(f"Failed to record usage for override {token_value}", level="warning")
def chunk_text_for_tts(entry: Mapping[str, Any]) -> str:
"""Choose the best source text for synthesis.
We must prefer the raw chunk text (``text`` / ``original_text``) so
manual/pronunciation overrides can match against the original tokens
(e.g. censored words like ``Unfu*k``). ``normalized_text`` may have
already been run through ``normalize_for_pipeline``, which can remove
punctuation and prevent overrides from triggering.
"""
if not isinstance(entry, Mapping):
return ""
return str(
entry.get("text")
or entry.get("original_text")
or entry.get("normalized_text")
or ""
).strip()
+31
View File
@@ -0,0 +1,31 @@
from __future__ import annotations
import platform as _platform
def select_device() -> str:
"""Return the best available compute device (``"mps"``, ``"cuda"``, or ``"cpu"``).
Checks ``torch`` availability at runtime so this can be called from
any context without requiring torch at import time.
"""
try:
import torch # type: ignore[import-not-found]
except Exception:
return "cpu"
system = _platform.system()
if system == "Darwin" and _platform.processor() == "arm":
try:
if torch.backends.mps.is_available(): # type: ignore[union-attr]
return "mps"
except Exception:
pass
return "cpu"
try:
if torch.cuda.is_available(): # type: ignore[union-attr]
return "cuda"
except Exception:
pass
return "cpu"
+136
View File
@@ -0,0 +1,136 @@
from __future__ import annotations
import re
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Tuple
from abogen.text_extractor import ExtractedChapter
_SIGNIFICANT_LENGTH_THRESHOLDS: Dict[str, int] = {"epub": 1000, "markdown": 500}
_MIN_SHORT_CONTENT: Dict[str, int] = {"epub": 240, "markdown": 160}
_STRUCTURAL_KEYWORDS = (
"preface",
"prologue",
"introduction",
"foreword",
"epilogue",
"afterword",
"appendix",
"acknowledgment",
"acknowledgement",
)
_STRUCTURAL_MIN_LENGTH = 120
_MAX_SHORT_CHAPTERS = 2
@dataclass
class ChapterFilterResult:
kept: List[ExtractedChapter]
skipped: List[Tuple[str, int]]
def infer_file_type(path: Path) -> str:
suffix = path.suffix.lower()
if suffix == ".epub":
return "epub"
if suffix in {".md", ".markdown"}:
return "markdown"
if suffix == ".pdf":
return "pdf"
if suffix == ".txt":
return "text"
return suffix.lstrip(".") or "text"
def looks_structural(title: str) -> bool:
lowered = title.strip().lower()
if not lowered:
return False
return any(keyword in lowered for keyword in _STRUCTURAL_KEYWORDS)
def chapter_label(file_type: str) -> str:
return "chapters" if file_type.lower() in {"epub", "markdown"} else "pages"
def auto_select_relevant_chapters(
chapters: List[ExtractedChapter],
file_type: str,
) -> ChapterFilterResult:
if not chapters:
return ChapterFilterResult(kept=[], skipped=[])
normalized = file_type.lower()
threshold = _SIGNIFICANT_LENGTH_THRESHOLDS.get(normalized, 0)
min_short = _MIN_SHORT_CONTENT.get(normalized, 0)
kept: List[ExtractedChapter] = []
skipped: List[Tuple[str, int]] = []
short_kept = 0
for chapter in chapters:
stripped = chapter.text.strip()
length = len(stripped)
if length == 0:
skipped.append((chapter.title, length))
continue
keep = False
if threshold == 0:
keep = True
elif length >= threshold:
keep = True
elif not kept:
keep = True
elif min_short and length >= min_short and short_kept < _MAX_SHORT_CHAPTERS:
keep = True
short_kept += 1
elif looks_structural(chapter.title) and length >= _STRUCTURAL_MIN_LENGTH:
keep = True
if keep:
kept.append(chapter)
else:
skipped.append((chapter.title, length))
if kept:
return ChapterFilterResult(kept=kept, skipped=skipped)
longest_idx = None
longest_length = 0
for idx, chapter in enumerate(chapters):
stripped = chapter.text.strip()
if stripped and len(stripped) > longest_length:
longest_length = len(stripped)
longest_idx = idx
if longest_idx is not None:
longest = chapters[longest_idx]
fallback_skipped = [
(chapter.title, len(chapter.text.strip()))
for idx, chapter in enumerate(chapters)
if idx != longest_idx and chapter.text.strip()
]
return ChapterFilterResult(kept=[longest], skipped=fallback_skipped)
return ChapterFilterResult(kept=[], skipped=skipped)
def update_metadata_for_chapter_count(
metadata: Dict[str, Any], count: int, file_type: str
) -> None:
if not metadata or count <= 0:
return
label = "Chapters" if file_type.lower() in {"epub", "markdown"} else "Pages"
metadata["chapter_count"] = str(count)
pattern = re.compile(r"\(\d+\s+(Chapters?|Pages?)\)")
replacement = f"({count} {label})"
for key in ("album", "ALBUM"):
value = metadata.get(key)
if not isinstance(value, str):
continue
metadata[key] = pattern.sub(replacement, value)
+138
View File
@@ -0,0 +1,138 @@
from __future__ import annotations
import re
from typing import Any, Dict, List, Mapping, Optional, Tuple
_SERIES_NAME_KEYS = (
"series",
"series_name",
"series_title",
)
_SERIES_NUMBER_KEYS = (
"series_index",
"series_position",
"series_sequence",
"book_number",
"series_number",
)
_SERIES_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?")
def normalize_metadata_map(values: Optional[Mapping[str, Any]]) -> Dict[str, str]:
normalized: Dict[str, str] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
text = str(value).strip()
if not text:
continue
normalized[str(key).casefold()] = text
return normalized
def format_author_sentence(raw: Optional[str]) -> str:
if raw is None:
return ""
normalized = str(raw).strip()
if not normalized:
return ""
lowered = normalized.casefold()
if lowered in {"unknown", "various"}:
return ""
working = normalized.replace("&", " and ")
segments = [segment.strip() for segment in working.split(",") if segment.strip()]
tokens: List[str] = []
if segments:
for segment in segments:
parts = [part.strip() for part in re.split(r"\band\b", segment, flags=re.IGNORECASE) if part.strip()]
if parts:
tokens.extend(parts)
else:
tokens.append(segment)
else:
parts = [part.strip() for part in re.split(r"\band\b", working, flags=re.IGNORECASE) if part.strip()]
tokens.extend(parts or [normalized])
cleaned = [token for token in tokens if token and token.casefold() not in {"unknown", "various"}]
if not cleaned:
return ""
if len(cleaned) == 1:
return f"By {cleaned[0]}"
if len(cleaned) == 2:
return f"By {cleaned[0]} and {cleaned[1]}"
return f"By {', '.join(cleaned[:-1])}, and {cleaned[-1]}"
def ensure_sentence(text: str) -> str:
cleaned = text.strip()
if not cleaned:
return ""
if cleaned[-1] in ".!?":
return cleaned
return f"{cleaned}."
def normalize_series_number(value: Any) -> Optional[str]:
text = str(value or "").strip()
if not text:
return None
candidate = text.replace(",", ".")
if candidate.replace(".", "", 1).isdigit():
if "." in candidate:
normalized = candidate.rstrip("0").rstrip(".")
return normalized or "0"
try:
return str(int(candidate))
except ValueError:
pass
match = _SERIES_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
return normalized or "0"
try:
return str(int(normalized))
except ValueError:
return normalized
def extract_series_metadata(values: Mapping[str, str]) -> Tuple[Optional[str], Optional[str]]:
series_name: Optional[str] = None
for key in _SERIES_NAME_KEYS:
raw = values.get(key)
if raw:
cleaned = str(raw).strip()
if cleaned:
series_name = cleaned
break
series_number: Optional[str] = None
for key in _SERIES_NUMBER_KEYS:
raw = values.get(key)
if raw is None:
continue
normalized = normalize_series_number(raw)
if normalized:
series_number = normalized
break
return series_name, series_number
def format_series_sentence(series_name: Optional[str], series_number: Optional[str]) -> str:
if not series_name or not series_number:
return ""
name = series_name.strip()
number = series_number.strip()
if not name or not number:
return ""
article = "the " if not name.lower().startswith("the ") else ""
phrase = f"Book {number} of {article}{name}"
return re.sub(r"\s+", " ", phrase).strip()
+23
View File
@@ -0,0 +1,23 @@
from __future__ import annotations
from typing import Any, Dict, Optional
def merge_metadata(
extracted: Optional[Dict[str, Any]],
overrides: Optional[Dict[str, Any]],
) -> Dict[str, str]:
merged: Dict[str, str] = {}
if extracted:
for key, value in extracted.items():
if value is None:
continue
merged[str(key)] = str(value)
if overrides:
for key, value in overrides.items():
key_str = str(key)
if value is None:
merged.pop(key_str, None)
else:
merged[key_str] = str(value)
return merged
+30
View File
@@ -0,0 +1,30 @@
"""Text normalization convenience helpers."""
from __future__ import annotations
from typing import Any, Mapping, Optional
from abogen.kokoro_text_normalization import (
ApostropheConfig,
normalize_for_pipeline as _normalize_for_pipeline,
)
from abogen.normalization_settings import (
build_apostrophe_config,
get_runtime_settings,
apply_overrides as _apply_overrides,
)
_BASE_APOSTROPHE_CONFIG = ApostropheConfig()
def normalize_text_for_pipeline(
text: str,
*,
normalization_overrides: Optional[Mapping[str, Any]] = None,
) -> str:
"""Normalize text using runtime settings with optional overrides."""
runtime_settings = get_runtime_settings()
if normalization_overrides:
runtime_settings = _apply_overrides(runtime_settings, normalization_overrides)
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
return _normalize_for_pipeline(text, config=apostrophe_config, settings=runtime_settings)
+91
View File
@@ -0,0 +1,91 @@
"""Output path resolution utilities.
Pure functions for resolving output directories, building file paths,
and computing project folder layouts.
"""
from __future__ import annotations
import re
from datetime import datetime
from pathlib import Path
from typing import Any, Callable, List, Optional, Tuple
from abogen.text_extractor import ExtractedChapter
_OUTPUT_SANITIZE_RE = re.compile(r"[^\w\-_.]+")
def slugify(title: str, index: int) -> str:
sanitized = re.sub(r"[^\w\-]+", "_", title.lower()).strip("_")
if not sanitized:
sanitized = f"chapter_{index:02d}"
return sanitized[:80]
def sanitize_output_stem(name: str) -> str:
base = Path(name or "").stem
sanitized = _OUTPUT_SANITIZE_RE.sub("_", base).strip("_")
return sanitized or "output"
def output_timestamp_token() -> str:
return datetime.now().strftime("%Y%m%d-%H%M%S")
def build_output_path(directory: Path, original_name: str, extension: str) -> Path:
sanitized = sanitize_output_stem(original_name)
return directory / f"{sanitized}.{extension}"
def apply_newline_policy(chapters: List[ExtractedChapter], replace_single_newlines: bool) -> None:
if not replace_single_newlines:
return
newline_regex = re.compile(r"(?<!\n)\n(?!\n)")
for chapter in chapters:
chapter.text = newline_regex.sub(" ", chapter.text)
def resolve_output_directory(
*,
save_mode: str,
stored_path: Path,
output_folder: Optional[str],
desktop_dir: Optional[Path],
user_output_path: Optional[Path],
user_cache_outputs: Optional[Path],
) -> Path:
if save_mode == "Save to Desktop" and desktop_dir:
return desktop_dir
if save_mode == "Save next to input file":
return stored_path.parent
if save_mode == "Choose output folder" and output_folder:
return Path(output_folder)
if save_mode == "Use default save location" and user_output_path:
return user_output_path
return user_cache_outputs or Path(".")
def resolve_project_layout(
*,
original_filename: str,
save_as_project: bool,
base_dir: Path,
timestamp_fn: Callable[[], str] = output_timestamp_token,
sanitize_fn: Callable[[str, int], str] = sanitize_output_stem,
) -> Tuple[Path, Path, Path, Optional[Path]]:
sanitized = sanitize_fn(original_filename, 0)
folder_name = f"{timestamp_fn()}_{sanitized}"
project_root = base_dir / folder_name
project_root.mkdir(parents=True, exist_ok=True)
if save_as_project:
audio_dir = project_root / "audio"
subtitle_dir = project_root / "subtitles"
metadata_dir = project_root / "metadata"
for directory in (audio_dir, subtitle_dir, metadata_dir):
directory.mkdir(parents=True, exist_ok=True)
return project_root, audio_dir, subtitle_dir, metadata_dir
return project_root, project_root, project_root, None
+261
View File
@@ -0,0 +1,261 @@
"""Pronunciation rule compilation and application.
Pure functions for compiling token-level and sentence-level pronunciation
overrides into regex patterns, applying them to text, and merging multiple
override sources with precedence rules.
"""
from __future__ import annotations
import re
from typing import Any, Dict, Iterable, List, Mapping, Optional
from abogen.entity_analysis import normalize_token as normalize_entity_token
from abogen.entity_analysis import normalize_manual_override_token
def compile_pronunciation_rules(
overrides: Optional[Iterable[Mapping[str, Any]]],
) -> List[Dict[str, Any]]:
if not overrides:
return []
candidates: List[Dict[str, Any]] = []
seen: set[str] = set()
for entry in overrides:
if not isinstance(entry, Mapping):
continue
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not pronunciation_value:
continue
token_values: List[str] = []
token_raw = entry.get("token")
if token_raw:
token_value = str(token_raw).strip()
if token_value:
token_values.append(token_value)
normalized_raw = entry.get("normalized")
if normalized_raw:
normalized_value = str(normalized_raw).strip()
if normalized_value:
token_values.append(normalized_value)
if token_raw and not token_values:
fallback = normalize_entity_token(str(token_raw))
if fallback:
token_values.append(fallback)
if not token_values:
continue
usage_normalized = str(entry.get("normalized") or "").strip()
if not usage_normalized and token_values:
usage_normalized = normalize_entity_token(token_values[0]) or token_values[0]
usage_token = str(entry.get("token") or token_values[0])
for token_value in token_values:
key = token_value.casefold()
if key in seen:
continue
seen.add(key)
candidates.append(
{
"token": token_value,
"normalized": usage_normalized,
"replacement": pronunciation_value,
}
)
if not candidates:
return []
candidates.sort(key=lambda item: len(item["token"]), reverse=True)
compiled: List[Dict[str, Any]] = []
for candidate in candidates:
token_value = candidate["token"]
pronunciation_value = candidate["replacement"]
escaped = re.escape(token_value)
pattern = re.compile(rf"(?i)(?<!\w){escaped}(?P<possessive>'s|\u2019s|\u2019)?(?!\w)")
compiled.append(
{
"pattern": pattern,
"replacement": pronunciation_value,
"normalized": candidate.get("normalized") or token_value,
"token": candidate.get("token") or token_value,
}
)
return compiled
def compile_heteronym_sentence_rules(
overrides: Optional[Iterable[Mapping[str, Any]]],
) -> List[Dict[str, Any]]:
if not overrides:
return []
compiled: List[Dict[str, Any]] = []
seen: set[str] = set()
for entry in overrides:
if not isinstance(entry, Mapping):
continue
sentence = str(entry.get("sentence") or "").strip()
if not sentence:
continue
choice = str(entry.get("choice") or "").strip()
if not choice:
continue
replacement_sentence = ""
options = entry.get("options")
if isinstance(options, list):
for opt in options:
if not isinstance(opt, Mapping):
continue
if str(opt.get("key") or "").strip() == choice:
replacement_sentence = str(opt.get("replacement_sentence") or "").strip()
break
if not replacement_sentence:
continue
rule_key = f"{sentence}\n{choice}".casefold()
if rule_key in seen:
continue
seen.add(rule_key)
parts = [p for p in re.split(r"\s+", sentence) if p]
if not parts:
continue
pattern_text = r"\s+".join(re.escape(p) for p in parts)
pattern = re.compile(pattern_text)
compiled.append({"pattern": pattern, "replacement": replacement_sentence})
compiled.sort(key=lambda item: len(item["pattern"].pattern), reverse=True)
return compiled
def apply_heteronym_sentence_rules(text: str, rules: List[Dict[str, Any]]) -> str:
if not text or not rules:
return text
result = text
for rule in rules:
pattern = rule["pattern"]
replacement = rule["replacement"]
result = pattern.sub(replacement, result)
return result
def apply_pronunciation_rules(
text: str,
rules: List[Dict[str, Any]],
usage_counter: Optional[Dict[str, int]] = None,
) -> str:
if not text or not rules:
return text
result = text
for rule in rules:
pattern = rule["pattern"]
pronunciation_value = rule["replacement"]
usage_key = str(rule.get("normalized") or "").strip()
def _replacement(match: re.Match[str]) -> str:
suffix = match.group("possessive") or ""
if usage_counter is not None and usage_key:
usage_counter[usage_key] = usage_counter.get(usage_key, 0) + 1
return pronunciation_value + suffix
result = pattern.sub(_replacement, result)
return result
def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
"""Return pronunciation override entries, ensuring manual overrides are included.
Pending jobs keep both ``manual_overrides`` and ``pronunciation_overrides``, but the
latter can be stale if the UI didn't resync before enqueue. During conversion,
we must merge manual overrides so they always apply (before TTS).
Precedence: manual overrides win over existing entries for the same normalized key.
"""
collected: Dict[str, Dict[str, Any]] = {}
existing = getattr(job, "pronunciation_overrides", None)
if isinstance(existing, list):
for entry in existing:
if not isinstance(entry, Mapping):
continue
token_value = str(entry.get("token") or "").strip()
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = str(entry.get("normalized") or "").strip() or normalize_entity_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(entry.get("voice") or "").strip() or None,
"notes": str(entry.get("notes") or "").strip() or None,
"context": str(entry.get("context") or "").strip() or None,
"source": str(entry.get("source") or "pronunciation"),
"language": getattr(job, "language", None),
}
speakers = getattr(job, "speakers", None)
if isinstance(speakers, dict):
for payload in speakers.values():
if not isinstance(payload, Mapping):
continue
token_value = str(payload.get("token") or "").strip()
pronunciation_value = str(payload.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = normalize_entity_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(
payload.get("resolved_voice")
or payload.get("voice")
or getattr(job, "voice", "")
).strip()
or None,
"notes": None,
"context": None,
"source": "speaker",
"language": getattr(job, "language", None),
}
manual = getattr(job, "manual_overrides", None)
if isinstance(manual, list):
for entry in manual:
if not isinstance(entry, Mapping):
continue
token_value = str(entry.get("token") or "").strip()
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = str(entry.get("normalized") or "").strip() or normalize_manual_override_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(entry.get("voice") or "").strip() or None,
"notes": str(entry.get("notes") or "").strip() or None,
"context": str(entry.get("context") or "").strip() or None,
"source": str(entry.get("source") or "manual"),
"language": getattr(job, "language", None),
}
return list(collected.values())
+40
View File
@@ -0,0 +1,40 @@
from __future__ import annotations
"""Unified split pattern logic extracted from 3 copies."""
import re
PUNCTUATION_SENTENCE = r".!?。!?"
PUNCTUATION_SENTENCE_COMMA = r".!?,。!?、,"
def get_split_pattern(language: str, subtitle_mode: str) -> str:
"""Get the appropriate split pattern based on language and subtitle mode.
Args:
language: Language code (a, b, e, f, etc.)
subtitle_mode: Subtitle mode ("Sentence", "Sentence + Comma", "Line", etc.)
Returns:
Split pattern string
"""
# For English, always use newline splitting only
if language in ("a", "b"):
return "\n"
# Determine spacing pattern based on language
spacing = r"\s*" if language in ("z", "j") else r"\s+"
# For CJK languages, when subtitle mode is Disabled or Line, prefer
# punctuation-based splitting instead of plain newline splitting.
if subtitle_mode in ("Disabled", "Line") and language in ("z", "j"):
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
if subtitle_mode == "Line":
return "\n"
elif subtitle_mode == "Sentence":
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
elif subtitle_mode == "Sentence + Comma":
return rf"(?<=[{PUNCTUATION_SENTENCE_COMMA}]){spacing}|\n+"
else:
return r"\n+"
+97
View File
@@ -0,0 +1,97 @@
from __future__ import annotations
from pathlib import Path
from typing import Any, Dict, List, Mapping, Optional
from .metadata_helpers import (
ensure_sentence,
extract_series_metadata,
format_author_sentence,
format_series_sentence,
normalize_metadata_map,
)
def build_title_intro_text(
metadata: Optional[Mapping[str, Any]],
fallback_basename: str,
) -> str:
"""Build the title introduction text from metadata."""
normalized = normalize_metadata_map(metadata)
fallback_title = Path(fallback_basename).stem if fallback_basename else ""
title = (
normalized.get("title")
or normalized.get("book_title")
or normalized.get("album")
or fallback_title
)
if not title:
title = fallback_title
subtitle = normalized.get("subtitle") or normalized.get("sub_title")
if subtitle and title and subtitle.casefold() == title.casefold():
subtitle = ""
author_value = ""
for candidate in ("artist", "album_artist", "author", "authors", "writer", "composer"):
value = normalized.get(candidate)
if value:
author_value = value
break
series_name, series_number = extract_series_metadata(normalized)
series_sentence = format_series_sentence(series_name, series_number)
sentences: List[str] = []
if series_sentence:
sentences.append(ensure_sentence(series_sentence))
if title:
sentences.append(ensure_sentence(title))
if subtitle:
sentences.append(ensure_sentence(subtitle))
author_sentence = format_author_sentence(author_value)
if author_sentence:
sentences.append(ensure_sentence(author_sentence))
return " ".join(sentences).strip()
def build_outro_text(
metadata: Optional[Mapping[str, Any]],
fallback_basename: str,
) -> str:
"""Build the outro/closing text from metadata."""
normalized = normalize_metadata_map(metadata)
fallback_title = Path(fallback_basename).stem if fallback_basename else ""
title = (
normalized.get("title")
or normalized.get("book_title")
or normalized.get("album")
or fallback_title
)
author_value = ""
for candidate in ("authors", "author", "album_artist", "artist", "writer", "composer"):
value = normalized.get(candidate)
if value:
author_value = value
break
author_sentence = format_author_sentence(author_value)
authors_fragment = (
author_sentence[3:].strip() if author_sentence.lower().startswith("by ") else author_sentence.strip()
)
if title and authors_fragment:
closing_line = f"The end of {title} from {authors_fragment}"
elif title:
closing_line = f"The end of {title}"
elif authors_fragment:
closing_line = f"The end from {authors_fragment}"
else:
closing_line = "The end"
series_name, series_number = extract_series_metadata(normalized)
series_sentence = format_series_sentence(series_name, series_number)
sentences: List[str] = [ensure_sentence(closing_line)]
if series_sentence:
sentences.append(ensure_sentence(series_sentence))
return " ".join(sentence for sentence in sentences if sentence).strip()
+190
View File
@@ -0,0 +1,190 @@
"""Voice resolution helpers.
Functions for resolving voice specifications, collecting required voice IDs,
and determining the voice to use for chapters and chunks.
"""
from __future__ import annotations
from typing import Any, Dict, Optional, Set
from abogen.tts_plugin.utils import get_voices, get_default_voice
from abogen.voice_formulas import extract_voice_ids
from abogen.voice_cache import ensure_voice_assets
def spec_to_voice_ids(spec: Any) -> Set[str]:
text = str(spec or "").strip()
if not text:
return set()
if text == "__custom_mix":
return set()
if "*" in text:
try:
return set(extract_voice_ids(text))
except ValueError:
return set()
if text in get_voices("kokoro"):
return {text}
return set()
def job_voice_fallback(job: Any) -> str:
base = str(getattr(job, "voice", "") or "").strip()
if base and base != "__custom_mix":
return base
speakers = getattr(job, "speakers", None)
if isinstance(speakers, dict):
narrator = speakers.get("narrator")
if isinstance(narrator, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = narrator.get(key)
candidate = str(value or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
for payload in speakers.values() or []:
if not isinstance(payload, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
value = payload.get(key)
candidate = str(value or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
for chapter in getattr(job, "chapters", []) or []:
if not isinstance(chapter, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
candidate = str(chapter.get(key) or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
return ""
def collect_required_voice_ids(job: Any) -> Set[str]:
voices: Set[str] = set()
voices.update(spec_to_voice_ids(job.voice))
voices.update(spec_to_voice_ids(job_voice_fallback(job)))
for chapter in getattr(job, "chapters", []) or []:
if not isinstance(chapter, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(chapter.get(key)))
for chunk in getattr(job, "chunks", []) or []:
if not isinstance(chunk, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(chunk.get(key)))
speakers = getattr(job, "speakers", {})
if isinstance(speakers, dict):
for payload in speakers.values() or []:
if not isinstance(payload, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(payload.get(key)))
voices.update(get_voices("kokoro"))
return voices
def initialize_voice_cache(job: Any) -> None:
try:
targets = collect_required_voice_ids(job)
downloaded, errors = ensure_voice_assets(
targets,
on_progress=lambda message: job.add_log(message, level="debug"),
)
except RuntimeError as exc:
job.add_log(f"Voice cache unavailable: {exc}", level="warning")
return
if downloaded:
job.add_log(
f"Cached {len(downloaded)} voice asset{'s' if len(downloaded) != 1 else ''} locally.",
level="info",
)
for voice_id, error in errors.items():
job.add_log(f"Failed to cache voice '{voice_id}': {error}", level="warning")
def chapter_voice_spec(job: Any, override: Optional[Dict[str, Any]]) -> str:
if not override:
return job_voice_fallback(job)
resolved = str(override.get("resolved_voice", "")).strip()
if resolved:
return resolved
formula = str(override.get("voice_formula", "")).strip()
if formula:
return formula
voice = str(override.get("voice", "")).strip()
if voice:
return voice
return job_voice_fallback(job)
def chunk_voice_spec(job: Any, chunk: Dict[str, Any], fallback: str) -> str:
for key in ("resolved_voice", "voice_formula", "voice"):
value = chunk.get(key)
if value:
return str(value)
speaker_id = chunk.get("speaker_id")
speakers = getattr(job, "speakers", None)
if isinstance(speakers, dict) and speaker_id in speakers:
speaker_entry = speakers.get(speaker_id) or {}
if isinstance(speaker_entry, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = speaker_entry.get(key)
if value:
return str(value)
profile_formula = speaker_entry.get("voice_formula")
if profile_formula:
return str(profile_formula)
profile_name = chunk.get("voice_profile")
if profile_name:
if isinstance(speakers, dict):
speaker_entry = speakers.get(profile_name)
if isinstance(speaker_entry, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = speaker_entry.get(key)
if value:
return str(value)
if fallback:
return fallback
return job_voice_fallback(job)
def resolve_fallback_voice_spec(
base_spec: str,
job_voice: str,
voice_cache_keys: list[str],
provider: str = "kokoro",
) -> str:
"""Resolve the voice spec for intro/outro with a priority fallback chain.
Priority: base_spec → job_voice → first voice_cache key → default voice.
``"__custom_mix"`` is treated as empty (it is not a usable voice spec).
"""
spec = base_spec or job_voice
if spec == "__custom_mix":
spec = job_voice or ""
if not spec:
for key in voice_cache_keys:
if key and key != "__custom_mix":
spec = key.split(":", 1)[-1]
break
if not spec:
spec = get_default_voice(provider)
return spec
+97
View File
@@ -0,0 +1,97 @@
from __future__ import annotations
from typing import Any, Mapping, Optional, Tuple, Set
from abogen.voice_formulas import extract_voice_ids, get_new_voice
from abogen.tts_plugin.utils import get_voices
def infer_provider_from_spec(value: Any, fallback: str = "kokoro") -> str:
"""Infer TTS provider from voice specification."""
raw = str(value or "").strip()
if not raw:
return fallback
if raw.upper() == raw and raw.replace("_", "").isalnum():
return "supertonic"
if raw == "__custom_mix" or "*" in raw or "+" in raw:
return "kokoro"
if raw in get_voices("kokoro"):
return "kokoro"
return fallback
def supertonic_voice_from_spec(spec: Any, fallback: str) -> str:
"""Normalize a voice specification for Supertonic.
This function only performs Supertonic-specific normalization (uppercase conversion
and fallback handling). Backend resolution is handled by the registry.
"""
raw = str(spec or "").strip()
fallback_raw = str(fallback or "").strip()
# Normalize to uppercase for Supertonic voice IDs
upper = raw.upper() if raw else ""
# If empty or contains formula characters, use fallback
if not upper or "*" in upper or "+" in upper:
upper = fallback_raw.upper() if fallback_raw else ""
# If still empty, use default Supertonic voice
if not upper or "*" in upper or "+" in upper:
upper = "M1"
return upper
def split_speaker_reference(value: Any) -> Tuple[Optional[str], str]:
"""Parse speaker/profile reference from string.
Expected format: "speaker:name" or "profile:name"
Returns (name, original) or (None, original) if not a valid reference.
"""
raw = str(value or "").strip()
if not raw or ":" not in raw:
return None, raw
prefix, remainder = raw.split(":", 1)
prefix = prefix.strip().lower()
if prefix not in {"speaker", "profile"}:
return None, raw
name = remainder.strip()
return (name or None), raw
def formula_from_kokoro_entry(entry: Mapping[str, Any]) -> str:
"""Build voice formula string from kokoro entry."""
voices = entry.get("voices") or []
if not voices:
return ""
total = 0.0
parts: list[tuple[str, float]] = []
for item in voices:
if not isinstance(item, (list, tuple)) or len(item) < 2:
continue
name = str(item[0] or "").strip()
try:
weight = float(item[1])
except (TypeError, ValueError):
continue
if name and weight > 0:
parts.append((name, weight))
total += weight
if not parts:
return ""
normalized = [(name, weight / total) for name, weight in parts]
return " + ".join(f"{name}*{weight:.6f}" for name, weight in normalized)
def coerce_truthy(value: Any, default: bool = True) -> bool:
"""Coerce a value to boolean with default."""
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() not in {"false", "0", "no", "off", ""}
if value is None:
return default
return bool(value)
+673
View File
@@ -0,0 +1,673 @@
from __future__ import annotations
import json
import logging
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Optional, Mapping, Sequence
import static_ffmpeg
from abogen.epub3.exporter import build_epub3_package
from abogen.integrations.audiobookshelf import (
AudiobookshelfClient,
AudiobookshelfConfig,
AudiobookshelfUploadError,
)
from abogen.utils import create_process
logger = logging.getLogger(__name__)
@dataclass
class ExportConfig:
"""Configuration for export operations."""
ffmpeg_path: str = "ffmpeg"
verify_ssl: bool = True
class ExportService:
"""Unified service for audiobook exports (M4B, FFMETADATA, EPUB3, Audiobookshelf)."""
def __init__(self, config: Optional[ExportConfig] = None):
self.config = config or ExportConfig()
static_ffmpeg.add_paths()
# ----------------------------------------------------------------------
# FFMETADATA
# ----------------------------------------------------------------------
def render_ffmetadata(
self,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
) -> str:
"""Render FFMETADATA content."""
lines = [";FFMETADATA1"]
for key, value in (metadata or {}).items():
if value is None:
continue
key_str = str(key).strip()
if not key_str:
continue
lines.append(f"{key_str}={self._escape_ffmetadata_value(value)}")
for chapter in chapters or []:
start = chapter.get("start")
end = chapter.get("end")
if start is None or end is None:
continue
try:
start_ms = max(0, int(round(float(start) * 1000)))
end_ms = int(round(float(end) * 1000))
except (TypeError, ValueError):
continue
if end_ms <= start_ms:
end_ms = start_ms + 1
lines.append("[CHAPTER]")
lines.append("TIMEBASE=1/1000")
lines.append(f"START={start_ms}")
lines.append(f"END={end_ms}")
title = chapter.get("title")
if title:
lines.append(f"title={self._escape_ffmetadata_value(title)}")
voice = chapter.get("voice")
if voice:
lines.append(f"voice={self._escape_ffmetadata_value(voice)}")
return "\n".join(lines) + "\n"
@staticmethod
def _escape_ffmetadata_value(value: Any) -> str:
escaped = str(value).replace("\\", "\\\\").replace("\n", "\\n")
escaped = escaped.replace("=", "\\=").replace(";", "\\;").replace("#", "\\#")
return escaped
def write_ffmetadata_file(
self,
audio_path: Path,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
) -> Optional[Path]:
"""Write FFMETADATA file to temp location."""
content = self.render_ffmetadata(metadata, chapters)
if content.strip() == ";FFMETADATA1":
return None
directory = audio_path.parent if audio_path.parent.exists() else Path(tempfile.gettempdir())
with tempfile.NamedTemporaryFile(
mode="w",
encoding="utf-8",
suffix=".ffmeta",
delete=False,
dir=str(directory),
) as handle:
handle.write(content)
return Path(handle.name)
# ----------------------------------------------------------------------
# M4B Export
# ----------------------------------------------------------------------
def embed_m4b_metadata(
self,
audio_path: Path,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
cover_path: Optional[Path] = None,
cover_mime: Optional[str] = None,
log_callback: Optional[callable] = None,
) -> None:
"""Embed metadata and chapters into M4B file using FFmpeg + Mutagen."""
ffmetadata_path = self.write_ffmetadata_file(audio_path, metadata, chapters)
metadata_args = self._metadata_to_ffmpeg_args(metadata)
cmd = ["ffmpeg", "-y", "-i", str(audio_path)]
if ffmetadata_path:
cmd.extend(["-f", "ffmetadata", "-i", str(ffmetadata_path)])
if cover_path and cover_path.exists():
cmd.extend(["-i", str(cover_path)])
cmd.extend(["-map", "0:a"])
cmd.extend(["-map", "1:v:0", "-c:v:0", "mjpeg", "-disposition:v:0", "attached_pic"])
if cover_mime:
cmd.extend(["-metadata:s:v:0", f"mimetype={cover_mime}"])
cmd.extend(["-metadata:s:v:0", "title=Cover Art"])
else:
cmd.extend(["-map", "0:a"])
cmd.extend(["-c:a", "copy"])
if ffmetadata_path:
cmd.extend(["-map_metadata", "1", "-map_chapters", "1"])
else:
cmd.extend(["-map_metadata", "0"])
if metadata_args:
cmd.extend(metadata_args)
cmd.extend(["-movflags", "+faststart+use_metadata_tags"])
temp_output = audio_path.with_suffix(audio_path.suffix + ".tmp")
if audio_path.suffix.lower() in {".m4b", ".mp4", ".m4a"}:
cmd.extend(["-f", "mp4"])
cmd.append(str(temp_output))
if log_callback:
log_callback("Embedding metadata into M4B output")
process = create_process(cmd, text=True)
return_code = process.wait()
if ffmetadata_path and ffmetadata_path.exists():
try:
ffmetadata_path.unlink()
except OSError:
pass
if return_code != 0:
if temp_output.exists():
temp_output.unlink(missing_ok=True)
raise RuntimeError(f"ffmpeg failed to embed metadata (exit code {return_code})")
temp_output.replace(audio_path)
if log_callback:
log_callback("Embedded metadata and chapters into M4B output", "info")
# Apply chapters via Mutagen for better compatibility
self._apply_m4b_chapters_mutagen(audio_path, chapters, log_callback)
@staticmethod
def _metadata_to_ffmpeg_args(metadata: Dict[str, Any]) -> List[str]:
args = []
for key, value in (metadata or {}).items():
if value in (None, ""):
continue
key_str = str(key).strip()
if not key_str:
continue
normalized_key = key_str.lower()
if normalized_key == "year":
ffmpeg_key = "date"
else:
ffmpeg_key = key_str
args.extend(["-metadata", f"{ffmpeg_key}={value}"])
return args
def _apply_m4b_chapters_mutagen(
self,
audio_path: Path,
chapters: List[Dict[str, Any]],
log_callback: Optional[callable] = None,
) -> bool:
"""Apply chapter atoms using Mutagen."""
if not chapters:
return False
try:
from fractions import Fraction
from mutagen.mp4 import MP4, MP4Chapter
except ImportError:
if log_callback:
log_callback("Unable to write MP4 chapter atoms because mutagen is not installed.", "warning")
return False
try:
mp4 = MP4(str(audio_path))
except Exception as exc:
if log_callback:
log_callback(f"Failed to open m4b for chapter embedding: {exc}", "warning")
return False
chapter_objects = []
for index, entry in enumerate(sorted(chapters, key=lambda item: float(item.get("start") or 0.0))):
start_raw = entry.get("start")
if start_raw is None:
continue
try:
start_seconds = max(0.0, float(start_raw))
except (TypeError, ValueError):
continue
title_value = entry.get("title")
title_text = str(title_value) if title_value else f"Chapter {index + 1}"
start_fraction = Fraction(int(round(start_seconds * 1000)), 1000)
chapter_atom = MP4Chapter(start_fraction, title_text)
end_raw = entry.get("end")
if end_raw is not None:
try:
end_seconds = float(end_raw)
except (TypeError, ValueError):
end_seconds = None
if end_seconds is not None and end_seconds > start_seconds:
chapter_atom.end = Fraction(int(round(end_seconds * 1000)), 1000)
chapter_objects.append(chapter_atom)
if not chapter_objects:
return False
try:
mp4.chapters = chapter_objects
mp4.save()
except Exception as exc:
if log_callback:
log_callback(f"Failed to persist MP4 chapter atoms: {exc}", "warning")
return False
if log_callback:
log_callback(f"Applied {len(chapter_objects)} chapter markers via mutagen", "info")
return True
# ----------------------------------------------------------------------
# EPUB3 Export
# ----------------------------------------------------------------------
def export_epub3(
self,
output_path: Path,
book_id: str,
extraction: Any, # ExtractionResult
metadata_tags: Dict[str, Any],
chapter_markers: Sequence[Dict[str, Any]],
chunk_markers: Sequence[Dict[str, Any]],
chunks: Iterable[Dict[str, Any]],
audio_path: Path,
speaker_mode: str = "single",
cover_path: Optional[Path] = None,
cover_mime: Optional[str] = None,
) -> Path:
"""Export EPUB3 with media overlays."""
return build_epub3_package(
output_path=output_path,
book_id=book_id,
extraction=extraction,
metadata_tags=metadata_tags,
chapter_markers=chapter_markers,
chunk_markers=chunk_markers,
chunks=chunks,
audio_path=audio_path,
speaker_mode=speaker_mode,
cover_image_path=cover_path,
cover_image_mime=cover_mime,
)
# ----------------------------------------------------------------------
# Audiobookshelf Integration
# ----------------------------------------------------------------------
def build_audiobookshelf_metadata(self, job: Any) -> Dict[str, Any]:
"""Build Audiobookshelf metadata from job."""
tags = self._normalize_metadata_casefold(getattr(job, "metadata_tags", {}))
filename = Path(getattr(job, "original_filename", "") or "").stem or "Audiobook"
title = self._first_nonempty(
tags.get("title"),
tags.get("book_title"),
tags.get("name"),
tags.get("album"),
filename,
)
authors = self._split_people_field(
tags.get("authors")
or tags.get("author")
or tags.get("album_artist")
or tags.get("artist")
)
narrators = self._split_people_field(tags.get("narrators") or tags.get("narrator"))
description = self._first_nonempty(
tags.get("description"), tags.get("summary"), tags.get("comment")
)
genres = self._split_simple_list(tags.get("genre"))
keywords = self._split_simple_list(tags.get("tags") or tags.get("keywords"))
language = self._first_nonempty(tags.get("language"), tags.get("lang")) or getattr(job, "language", "") or ""
series_name = self._first_nonempty(
tags.get("series"),
tags.get("series_name"),
tags.get("seriesname"),
tags.get("series_title"),
tags.get("seriestitle"),
)
series_sequence = None
for key in ("series_index", "series_position", "series_sequence", "series_number", "seriesnumber", "book_number", "booknumber"):
raw = tags.get(key)
normalized = self._normalize_series_sequence(raw)
if normalized:
series_sequence = normalized
break
if not series_name:
series_sequence = None
data = {
"title": title,
"subtitle": tags.get("subtitle"),
"authors": authors,
"narrators": narrators,
"description": description,
"publisher": tags.get("publisher"),
"genres": genres,
"tags": keywords,
"language": language,
"publishedYear": self._extract_year(
tags.get("published") or tags.get("publication_year") or tags.get("date") or tags.get("year")
),
"seriesName": series_name,
"seriesSequence": series_sequence,
"isbn": self._first_nonempty(tags.get("isbn"), tags.get("asin")),
}
published_date = self._first_nonempty(
tags.get("published"), tags.get("publication_date"), tags.get("date")
)
if published_date:
data["publishedDate"] = published_date
rating_text = self._first_nonempty(tags.get("rating"), tags.get("my_rating"))
if rating_text:
try:
data["rating"] = float(str(rating_text).strip())
except ValueError:
pass
rating_max_text = self._first_nonempty(tags.get("rating_max"), tags.get("rating_scale"))
if rating_max_text:
try:
data["ratingMax"] = float(str(rating_max_text).strip())
except ValueError:
pass
# Remove empty values
cleaned = {}
for key, value in data.items():
if value is None:
continue
if isinstance(value, str) and not value.strip():
continue
if isinstance(value, (list, tuple)) and not value:
continue
cleaned[key] = value
return cleaned
def load_audiobookshelf_chapters(self, job: Any) -> Optional[List[Dict[str, Any]]]:
"""Load chapters from job artifacts for Audiobookshelf."""
metadata_ref = job.result.artifacts.get("metadata") if getattr(job, "result", None) else None
if not metadata_ref:
return None
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
if not metadata_path.exists():
return None
try:
payload = json.loads(metadata_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
chapters = payload.get("chapters")
if not isinstance(chapters, list):
return None
cleaned = []
for entry in chapters:
if not isinstance(entry, Mapping):
continue
title = self._first_nonempty(entry.get("title"), entry.get("original_title"))
start = entry.get("start")
end = entry.get("end")
if title is None or not isinstance(start, (int, float)):
continue
chapter_payload = {"title": title, "start": float(start)}
if isinstance(end, (int, float)):
chapter_payload["end"] = float(end)
cleaned.append(chapter_payload)
return cleaned or None
def upload_audiobookshelf(
self,
job: Any,
audio_path: Path,
subtitle_paths: List[Path],
chapters: List[Dict[str, Any]],
metadata: Dict[str, Any],
cover_path: Optional[Path] = None,
config: Optional[AudiobookshelfConfig] = None,
log_callback: Optional[callable] = None,
) -> None:
"""Upload to Audiobookshelf."""
if config is None:
# Load from job or global config
cfg = getattr(job, "_abs_config", None)
if cfg is None:
from abogen.utils import load_config
global_cfg = load_config() or {}
abs_cfg = global_cfg.get("audiobookshelf")
if isinstance(abs_cfg, Mapping):
config = AudiobookshelfConfig(
base_url=str(abs_cfg.get("base_url") or "").strip(),
api_token=str(abs_cfg.get("api_token") or "").strip(),
library_id=str(abs_cfg.get("library_id") or "").strip(),
collection_id=(str(abs_cfg.get("collection_id") or "").strip() or None),
folder_id=str(abs_cfg.get("folder_id") or "").strip(),
verify_ssl=self._coerce_bool(abs_cfg.get("verify_ssl"), True),
send_cover=self._coerce_bool(abs_cfg.get("send_cover"), True),
send_chapters=self._coerce_bool(abs_cfg.get("send_chapters"), True),
send_subtitles=self._coerce_bool(abs_cfg.get("send_subtitles"), False),
timeout=float(abs_cfg.get("timeout", 3600.0)),
)
else:
if log_callback:
log_callback("Audiobookshelf upload skipped: not configured", "warning")
return
if not config.base_url or not config.api_token or not config.library_id:
if log_callback:
log_callback("Audiobookshelf upload skipped: configure base URL, API token, and library ID first", "warning")
return
if not config.folder_id:
if log_callback:
log_callback("Audiobookshelf upload skipped: enter folder name or ID in settings", "warning")
return
if not audio_path.exists():
if log_callback:
log_callback("Audiobookshelf upload skipped: audio output not found", "warning")
return
existing_subtitles = [p for p in subtitle_paths if p.exists()] if config.send_subtitles else None
chapters_to_send = chapters if config.send_chapters else None
client = AudiobookshelfClient(config)
display_title = metadata.get("title") or audio_path.stem
try:
existing_items = client.find_existing_items(display_title, folder_id=config.folder_id)
except AudiobookshelfUploadError as exc:
if log_callback:
log_callback(f"Audiobookshelf lookup failed: {exc}", "error")
return
if existing_items:
if log_callback:
log_callback(f"Removing existing Audiobookshelf item(s) for '{display_title}' before upload.", "info")
try:
client.delete_items(existing_items)
except Exception as exc:
if log_callback:
log_callback(f"Failed to remove existing item(s): {exc}", "warning")
cover_to_send = cover_path
if config.send_cover and cover_to_send:
if isinstance(cover_to_send, str):
cover_to_send = Path(cover_to_send)
if not cover_to_send.exists():
cover_to_send = None
client.upload_audiobook(
audio_path,
metadata=metadata,
cover_path=cover_to_send,
chapters=chapters_to_send,
subtitles=existing_subtitles,
)
if log_callback:
log_callback("Audiobookshelf upload queued.", "info")
# ----------------------------------------------------------------------
# Helpers
# ----------------------------------------------------------------------
@staticmethod
def _normalize_metadata_casefold(values: Optional[Mapping[str, Any]]) -> Dict[str, Any]:
normalized = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
key_text = str(key).strip().lower()
if not key_text:
continue
if isinstance(value, (list, tuple, set)):
normalized[key_text] = value
else:
text = str(value).strip()
if text:
normalized[key_text] = text
return normalized
@staticmethod
def _split_people_field(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results = []
for item in raw:
results.extend(ExportService._split_people_field(item))
return results
text = str(raw or "").strip()
if not text:
return []
import re
tokens = [token.strip() for token in re.split(r"[;,/&]|\band\b", text, flags=re.IGNORECASE) if token.strip()]
seen = set()
ordered = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
@staticmethod
def _split_simple_list(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results = []
for item in raw:
results.extend(ExportService._split_simple_list(item))
return results
text = str(raw or "").strip()
if not text:
return []
import re
tokens = [token.strip() for token in re.split(r"[;,\n]", text) if token.strip()]
seen = set()
ordered = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
@staticmethod
def _first_nonempty(*values: Any) -> Optional[str]:
for value in values:
if value is None:
continue
if isinstance(value, (list, tuple, set)):
items = list(value)
if not items:
continue
value = items[0]
text = str(value).strip()
if text:
return text
return None
@staticmethod
def _extract_year(raw: Optional[str]) -> Optional[int]:
if not raw:
return None
text = str(raw).strip()
if not text:
return None
import re
match = re.search(r"(19|20)\d{2}", text)
if match:
try:
return int(match.group(0))
except ValueError:
return None
try:
parsed = int(text)
except ValueError:
return None
if 0 < parsed < 3000:
return parsed
return None
@staticmethod
def _normalize_series_sequence(raw: Any) -> Optional[str]:
if raw is None:
return None
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (raw != raw or raw == float("inf") or raw == float("-inf")):
return None
text = str(raw)
else:
text = str(raw).strip()
if not text:
return None
candidate = text.replace(",", ".")
import re
match = re.search(r"\d+(?:\.\d+)?", candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
return normalized or "0"
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
@staticmethod
def _coerce_bool(value: Any, default: bool = True) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
lowered = value.strip().lower()
if lowered in {"true", "1", "yes", "on"}:
return True
if lowered in {"false", "0", "no", "off"}:
return False
return default
if value is None:
return default
return bool(value)
__all__ = [
"ExportConfig",
"ExportService",
]
+303
View File
@@ -0,0 +1,303 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass
from enum import Enum
from pathlib import Path
from typing import List, Optional, TextIO
from abogen.subtitle_utils import clean_subtitle_text
class SubtitleFormat(Enum):
SRT = "srt"
ASS = "ass"
VTT = "vtt"
class SubtitleMode(Enum):
DISABLED = "Disabled"
LINE = "Line"
SENTENCE = "Sentence"
SENTENCE_COMMA = "Sentence + Comma"
SENTENCE_HIGHLIGHT = "Sentence + Highlighting"
class SubtitleAlignment(Enum):
LEFT = "left"
CENTER = "center"
NARROW = "narrow"
CENTER_NARROW = "center_narrow"
@dataclass
class SubtitleConfig:
"""Configuration for subtitle writer."""
format: SubtitleFormat
mode: SubtitleMode
alignment: SubtitleAlignment = SubtitleAlignment.LEFT
max_words: int = 50
highlight_color: str = "&H00FFFF00" # ASS highlight color
class SubtitleWriter(ABC):
"""Abstract base class for subtitle writers."""
def __init__(self, path: Path, config: SubtitleConfig):
self.path = path
self.config = config
self._file: Optional[TextIO] = None
self._index = 0
self._opened = False
def open(self) -> None:
"""Open the subtitle file and write header."""
if self._opened:
return
self._file = open(self.path, "w", encoding="utf-8", errors="replace")
self._write_header()
self._opened = True
@abstractmethod
def _write_header(self) -> None:
pass
def write_entry(
self,
start: float,
end: float,
text: str,
voice: Optional[str] = None,
) -> None:
"""Write a subtitle entry."""
if not self._opened:
self.open()
text = clean_subtitle_text(text)
if not text:
return
self._index += 1
self._write_entry(self._index, start, end, text, voice)
@abstractmethod
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
pass
def close(self) -> None:
"""Close the subtitle file."""
if self._file:
self._file.close()
self._file = None
self._opened = False
def __enter__(self) -> "SubtitleWriter":
self.open()
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.close()
class SrtWriter(SubtitleWriter):
"""SRT subtitle writer."""
def _write_header(self) -> None:
pass # SRT has no header
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
self._file.write(f"{index}\n")
self._file.write(f"{start_str} --> {end_str}\n")
self._file.write(f"{text}\n\n")
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = int(seconds % 60)
millis = int((seconds - int(seconds)) * 1000)
return f"{hours:02d}:{minutes:02d}:{secs:02d},{millis:03d}"
class VttWriter(SubtitleWriter):
"""WebVTT subtitle writer."""
def _write_header(self) -> None:
self._file.write("WEBVTT\n\n")
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
self._file.write(f"{index}\n")
self._file.write(f"{start_str} --> {end_str}\n")
self._file.write(f"{text}\n\n")
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = seconds % 60
return f"{hours:02d}:{minutes:02d}:{secs:06.3f}".replace(".", ".")
class AssWriter(SubtitleWriter):
"""ASS subtitle writer with karaoke highlighting support."""
def __init__(self, path: Path, config: SubtitleConfig):
super().__init__(path, config)
self._is_centered = config.alignment in (SubtitleAlignment.CENTER, SubtitleAlignment.CENTER_NARROW)
self._is_narrow = config.alignment in (SubtitleAlignment.NARROW, SubtitleAlignment.CENTER_NARROW)
def _write_header(self) -> None:
margin = "90" if self._is_narrow else "10"
alignment = "5" if self._is_centered else "2"
self._file.write("[Script Info]\n")
self._file.write("Title: Generated by Abogen\n")
self._file.write("ScriptType: v4.00+\n\n")
# Styles
self._file.write("[V4+ Styles]\n")
self._file.write(
"Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, "
"OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, "
"ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, "
"Alignment, MarginL, MarginR, MarginV, Encoding\n"
)
if self.config.mode == SubtitleMode.SENTENCE_HIGHLIGHT:
# Karaoke style with highlighting
self._file.write(
f"Style: Default,Arial,24,&H00FFFFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n"
)
self._file.write(
f"Style: Highlight,Arial,24,&H0000FFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n\n"
)
else:
self._file.write(
f"Style: Default,Arial,24,&H00FFFFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n\n"
)
self._file.write("[Events]\n")
self._file.write(
"Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text\n"
)
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
style = "Default"
if self.config.mode == SubtitleMode.SENTENCE_HIGHLIGHT:
# Add karaoke tags for highlighting
text = self._add_karaoke_tags(text)
style = "Highlight"
alignment_tag = r"{\an5}" if self._is_centered else ""
self._file.write(
f"Dialogue: 0,{start_str},{end_str},{style},,0,0,0,,{alignment_tag}{text}\n"
)
def _add_karaoke_tags(self, text: str) -> str:
"""Add karaoke highlighting tags to text."""
# Simple word-level karaoke timing
words = text.split()
if not words:
return text
# This is a simplified version - real karaoke needs per-word timing
# For now, just return the text with the highlight color
return r"{\k100}" + r"{\k100}".join(words) + r"{\k0}"
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = seconds % 60
return f"{hours}:{minutes:02d}:{secs:05.2f}"
def create_subtitle_writer(
path: Path,
format: str,
mode: str,
alignment: str = "left",
max_words: int = 50,
) -> SubtitleWriter:
"""Factory function to create subtitle writer."""
fmt = SubtitleFormat(format.lower())
mode = SubtitleMode(mode)
align = SubtitleAlignment(alignment.lower())
config = SubtitleConfig(
format=fmt,
mode=mode,
alignment=align,
max_words=max_words,
)
if fmt == SubtitleFormat.SRT:
return SrtWriter(path, config)
elif fmt == SubtitleFormat.VTT:
return VttWriter(path, config)
elif fmt == SubtitleFormat.ASS:
return AssWriter(path, config)
else:
raise ValueError(f"Unsupported subtitle format: {format}")
__all__ = [
"SubtitleFormat",
"SubtitleMode",
"SubtitleAlignment",
"SubtitleConfig",
"SubtitleWriter",
"SrtWriter",
"VttWriter",
"AssWriter",
"create_subtitle_writer",
]
+5 -15
View File
@@ -2,13 +2,14 @@
from __future__ import annotations
import atexit
import os
import platform
import signal
import sys
from abogen.utils import load_config, prevent_sleep_end
# Initialise global shutdown handling (atexit, signals, Qt) as early as possible.
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
from abogen.utils import load_config
from abogen.webui.app import main as _run_web_ui
# Configure Hugging Face Hub behaviour (mirrors legacy GUI defaults).
@@ -27,17 +28,6 @@ os.environ.setdefault("MIOPEN_CONV_PRECISE_ROCM_TUNING", "0")
if platform.system() == "Darwin" and platform.processor() == "arm":
os.environ.setdefault("PYTORCH_ENABLE_MPS_FALLBACK", "1")
atexit.register(prevent_sleep_end)
def _cleanup_sleep(signum, _frame):
prevent_sleep_end()
sys.exit(0)
signal.signal(signal.SIGINT, _cleanup_sleep)
signal.signal(signal.SIGTERM, _cleanup_sleep)
def main() -> None:
"""Launch the Flask-based web UI."""
+72 -181
View File
@@ -14,7 +14,6 @@ from abogen.utils import (
)
from abogen.constants import (
LANGUAGE_DESCRIPTIONS,
SAMPLE_VOICE_TEXTS,
COLORS,
CHAPTER_OPTIONS_COUNTDOWN,
SUBTITLE_FORMATS,
@@ -22,11 +21,18 @@ from abogen.constants import (
SUPPORTED_SUBTITLE_FORMATS,
)
from abogen.voice_formulas import get_new_voice
from abogen.infrastructure.subtitle_writer import _format_timestamp
from abogen.domain.split_pattern import get_split_pattern
from abogen.domain.output_paths import (
resolve_output_directory,
build_output_path,
sanitize_output_stem,
)
from abogen.domain.audio_helpers import build_ffmpeg_command, to_float32
import abogen.hf_tracker as hf_tracker
import static_ffmpeg
import threading # for efficient waiting
import subprocess
import platform
# Configuration constants
_USER_RESPONSE_TIMEOUT = (
@@ -43,10 +49,7 @@ from abogen.subtitle_utils import (
get_sample_voice_text,
sanitize_name_for_os,
_CHAPTER_MARKER_SEARCH_PATTERN,
_VOICE_MARKER_PATTERN,
_VOICE_MARKER_SEARCH_PATTERN,
split_text_by_voice_markers,
validate_voice_name,
split_text_by_voice_markers
)
class CountdownDialog(QDialog):
@@ -216,40 +219,6 @@ class ConversionThread(QThread):
PUNCTUATION_SENTENCE_COMMA = ".!?,।。!?、,"
PUNCTUATION_COMMAS = ",,、"
def _get_split_pattern(self, lang_code, subtitle_mode):
"""
Get the appropriate split pattern based on language and subtitle mode.
Args:
lang_code: Language code (a, b, e, f, etc.)
subtitle_mode: Subtitle mode ("Sentence", "Sentence + Comma", "Line", etc.)
Returns:
Split pattern string
"""
# For English, always use newline splitting only
if lang_code in ["a", "b"]:
return "\n"
# Determine spacing pattern based on language
spacing_pattern = r"\s*" if lang_code in ["z", "j"] else r"\s+"
# For Chinese/Japanese, when subtitle mode is Disabled or Line, prefer
# punctuation-based splitting instead of plain newline splitting.
if subtitle_mode in ("Disabled", "Line") and lang_code in ["z", "j"]:
return r"(?<=[{}]){}|\n+".format(self.PUNCTUATION_SENTENCE, spacing_pattern)
if subtitle_mode == "Line":
return "\n"
elif subtitle_mode == "Sentence":
return r"(?<=[{}]){}|\n+".format(self.PUNCTUATION_SENTENCE, spacing_pattern)
elif subtitle_mode == "Sentence + Comma":
return r"(?<=[{}]){}|\n+".format(
self.PUNCTUATION_SENTENCE_COMMA, spacing_pattern
)
else:
return r"\n+" # Default to line breaks
def __init__(
self,
file_name,
@@ -298,7 +267,7 @@ class ConversionThread(QThread):
self.silence_duration = 2.0 # Default value, will be overridden from GUI
self.use_spacy_segmentation = True # Default, will be overridden from GUI
# Set split pattern based on language and subtitle mode
self.split_pattern = self._get_split_pattern(lang_code, subtitle_mode)
self.split_pattern = get_split_pattern(lang_code, subtitle_mode)
self.voice_cache = {} # Cache for loaded voices
def load_voice_cached(self, voice_name, tts):
@@ -676,15 +645,17 @@ class ConversionThread(QThread):
base_path = self.display_path if self.display_path else self.file_name
base_name = os.path.splitext(os.path.basename(base_path))[0]
# Sanitize base_name for folder/file creation based on OS
sanitized_base_name = sanitize_name_for_os(base_name, is_folder=True)
if self.save_option == "Save to Desktop":
parent_dir = user_desktop_dir()
elif self.save_option == "Save next to input file":
parent_dir = os.path.dirname(base_path)
else:
parent_dir = self.output_folder or os.getcwd()
parent_dir = resolve_output_directory(
save_mode=self.save_option,
stored_path=Path(base_path),
output_folder=getattr(self, "output_folder", None),
desktop_dir=Path(user_desktop_dir()),
user_output_path=None,
user_cache_outputs=Path(os.getcwd()),
)
parent_dir = str(parent_dir)
# Ensure the output folder exists, error if it doesn't
if not os.path.exists(parent_dir):
self.log_updated.emit(
@@ -754,77 +725,40 @@ class ConversionThread(QThread):
format=self.output_format,
)
ffmpeg_proc = None
elif self.output_format == "m4b":
# Real-time M4B generation using FFmpeg pipe
elif self.output_format in ("m4b", "opus"):
# Real-time generation using FFmpeg pipe
static_ffmpeg.add_paths()
merged_out_file = None
ffmpeg_proc = None
metadata_options, cover_path = (
self._extract_and_add_metadata_tags_to_ffmpeg_cmd()
if self.output_format == "m4b"
else ([], None)
)
# Prepare ffmpeg command for m4b output
cmd = [
"ffmpeg",
"-y",
"-thread_queue_size",
"32768",
"-f",
"f32le",
"-ar",
"24000",
"-ac",
"1",
"-i",
"pipe:0",
]
if cover_path and os.path.exists(cover_path):
cmd.extend(
[
"-i",
cover_path,
"-map",
"0:a",
"-map",
"1",
"-c:v",
"copy",
"-disposition:v",
"attached_pic",
]
cmd = build_ffmpeg_command(
Path(merged_out_path),
self.output_format,
)
cmd.extend(
[
"-c:a",
"aac",
"-q:a",
"2",
"-movflags",
"+faststart+use_metadata_tags",
]
)
cmd += metadata_options
cmd.append(merged_out_path)
# Insert thread queue size after ffmpeg header
cmd.insert(2, "-thread_queue_size")
cmd.insert(3, "32768")
if self.output_format == "m4b" and cover_path and os.path.exists(cover_path):
# Insert cover image input before the output path
output_path = cmd.pop()
cmd.extend([
"-i", cover_path,
"-map", "0:a",
"-map", "1",
"-c:v", "copy",
"-disposition:v", "attached_pic",
])
cmd.extend(metadata_options)
cmd.append(output_path)
elif self.output_format == "m4b":
output_path = cmd.pop()
cmd.extend(metadata_options)
cmd.append(output_path)
ffmpeg_proc = create_process(cmd, stdin=subprocess.PIPE, text=False)
elif self.output_format == "opus":
static_ffmpeg.add_paths()
cmd = [
"ffmpeg",
"-y",
"-thread_queue_size",
"32768",
"-f",
"f32le",
"-ar",
"24000",
"-ac",
"1",
"-i",
"pipe:0",
]
cmd.extend(["-c:a", "libopus", "-b:a", "24000"])
cmd.append(merged_out_path)
ffmpeg_proc = create_process(cmd, stdin=subprocess.PIPE, text=False)
merged_out_file = None
else:
self.log_updated.emit(
(f"Unsupported output format: {self.output_format}", "red")
@@ -1258,8 +1192,8 @@ class ConversionThread(QThread):
)
if "ass" in subtitle_format:
for start, end, text in new_entries:
start_time = self._ass_time(start)
end_time = self._ass_time(end)
start_time = _format_timestamp(start, ass=True)
end_time = _format_timestamp(end, ass=True)
# Use karaoke effect for highlighting mode
effect = (
"karaoke"
@@ -1274,7 +1208,7 @@ class ConversionThread(QThread):
for entry in new_entries:
start, end, text = entry
merged_subtitle_file.write(
f"{merged_srt_index}\n{self._srt_time(start)} --> {self._srt_time(end)}\n{text}\n\n"
f"{merged_srt_index}\n{_format_timestamp(start)} --> {_format_timestamp(end)}\n{text}\n\n"
)
merged_srt_index += 1
# Per-chapter subtitle processing for both file and ffmpeg_proc
@@ -1292,8 +1226,8 @@ class ConversionThread(QThread):
)
if "ass" in subtitle_format:
for start, end, text in new_chapter_entries:
start_time = self._ass_time(start)
end_time = self._ass_time(end)
start_time = _format_timestamp(start, ass=True)
end_time = _format_timestamp(end, ass=True)
# Use karaoke effect for highlighting mode
effect = (
"karaoke"
@@ -1308,7 +1242,7 @@ class ConversionThread(QThread):
for entry in new_chapter_entries:
start, end, text = entry
chapter_subtitle_file.write(
f"{chapter_srt_index}\n{self._srt_time(start)} --> {self._srt_time(end)}\n{text}\n\n"
f"{chapter_srt_index}\n{_format_timestamp(start)} --> {_format_timestamp(end)}\n{text}\n\n"
)
chapter_srt_index += 1
if merge_chapters_at_end:
@@ -1597,58 +1531,27 @@ class ConversionThread(QThread):
)
else:
static_ffmpeg.add_paths()
cmd = [
"ffmpeg",
"-y",
"-thread_queue_size",
"32768",
"-f",
"f32le",
"-ar",
str(rate),
"-ac",
"1",
"-i",
"pipe:0",
]
cmd = build_ffmpeg_command(
Path(merged_out_path),
self.output_format,
)
cmd.insert(2, "-thread_queue_size")
cmd.insert(3, "32768")
if self.output_format == "m4b":
metadata_options, cover_path = (
self._extract_and_add_metadata_tags_to_ffmpeg_cmd()
)
if cover_path and os.path.exists(cover_path):
cmd.extend(
[
"-i",
cover_path,
"-map",
"0:a",
"-map",
"1",
"-c:v",
"copy",
"-disposition:v",
"attached_pic",
]
)
cmd.extend(
[
"-c:a",
"aac",
"-q:a",
"2",
"-movflags",
"+faststart+use_metadata_tags",
]
)
output_path = cmd.pop()
cmd.extend([
"-i", cover_path,
"-map", "0:a",
"-map", "1",
"-c:v", "copy",
"-disposition:v", "attached_pic",
])
cmd.append(output_path)
cmd.extend(metadata_options)
elif self.output_format == "opus":
cmd.extend(["-c:a", "libopus", "-b:a", "24000"])
else:
self.log_updated.emit(
(f"Unsupported output format: {self.output_format}", "red")
)
return
cmd.append(merged_out_path)
ffmpeg_proc = create_process(cmd, stdin=subprocess.PIPE, text=False)
# Always generate subtitles for subtitle input files
@@ -1938,11 +1841,11 @@ class ConversionThread(QThread):
else processed_text.replace("\n", "\\N")
)
subtitle_file.write(
f"Dialogue: 0,{self._ass_time(start_time)},{self._ass_time(end_time)},Default,,{margin},{margin},0,{effect},{alignment}{ass_text}\n"
f"Dialogue: 0,{_format_timestamp(start_time, ass=True)},{_format_timestamp(end_time, ass=True)},Default,,{margin},{margin},0,{effect},{alignment}{ass_text}\n"
)
else:
subtitle_file.write(
f"{srt_index}\n{self._srt_time(start_time)} --> {self._srt_time(end_time)}\n{processed_text}\n\n"
f"{srt_index}\n{_format_timestamp(start_time)} --> {_format_timestamp(end_time)}\n{processed_text}\n\n"
)
srt_index += 1
@@ -2104,22 +2007,6 @@ class ConversionThread(QThread):
# Add these to ffmpeg command
return metadata_options, cover_path
def _srt_time(self, t):
"""Helper function to format time for SRT files"""
h = int(t // 3600)
m = int((t % 3600) // 60)
s = int(t % 60)
ms = int((t - int(t)) * 1000)
return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"
def _ass_time(self, t):
"""Helper function to format time for ASS files"""
h = int(t // 3600)
m = int((t % 3600) // 60)
s = int(t % 60)
cs = int((t - int(t)) * 100) # Centiseconds for ASS format
return f"{h:01d}:{m:02d}:{s:02d}.{cs:02d}"
def _process_subtitle_tokens(
self,
tokens_with_timestamps,
@@ -2397,6 +2284,10 @@ class ConversionThread(QThread):
self.cancel_requested = True
self.should_cancel = True
self.waiting_for_user_input = False
# Clear voice cache (instance and module-level)
self.voice_cache.clear()
from abogen.voice_cache import clear_voice_cache
clear_voice_cache()
# Terminate subprocess if running
if self.process:
try:
+7 -8
View File
@@ -7,6 +7,7 @@ import base64
import re
from abogen.pyqt.queue_manager_gui import QueueManager
from abogen.pyqt.queued_item import QueuedItem
from abogen.domain.device import select_device as _select_device
import abogen.hf_tracker as hf_tracker
import hashlib # Added for cache path generation
from PyQt6.QtWidgets import (
@@ -2428,10 +2429,7 @@ class abogen(QWidget):
# Determine device based on GPU availability
if gpu_ok:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps"
else:
device = "cuda"
device = _select_device()
else:
device = "cpu"
@@ -2877,10 +2875,7 @@ class abogen(QWidget):
# Determine device based on GPU availability
if self.gpu_ok:
if platform.system() == "Darwin" and platform.processor() == "arm":
device = "mps"
else:
device = "cuda"
device = _select_device()
else:
device = "cpu"
@@ -3236,12 +3231,16 @@ class abogen(QWidget):
)
box.setDefaultButton(QMessageBox.StandardButton.No)
if box.exec() == QMessageBox.StandardButton.Yes:
from abogen import shutdown
shutdown.request_shutdown()
self.cleanup_conversion_thread()
self.cleanup_preview_threads()
event.accept()
else:
event.ignore()
else:
from abogen import shutdown
shutdown.request_shutdown()
self.cleanup_conversion_thread()
self.cleanup_preview_threads()
event.accept()
+4 -22
View File
@@ -1,10 +1,10 @@
import os
import sys
import platform
import atexit
import signal
from abogen.utils import get_resource_path, load_config, prevent_sleep_end
# Initialise global shutdown handling (atexit, signals, Qt) as early as possible.
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
# Fix PyTorch DLL loading issue ([WinError 1114]) on Windows before importing PyQt6
if platform.system() == "Windows":
@@ -94,6 +94,7 @@ os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1" # Disable Hugging Face telemetry
os.environ["HF_HUB_ETAG_TIMEOUT"] = "10" # Metadata request timeout (seconds)
os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = "10" # File download timeout (seconds)
os.environ["HF_HUB_DISABLE_SYMLINKS_WARNING"] = "1" # Disable symlinks warning
from abogen.utils import load_config
if load_config().get("disable_kokoro_internet", False):
print("INFO: Kokoro's internet access is disabled.")
os.environ["HF_HUB_OFFLINE"] = "1" # Disable Hugging Face Hub internet access
@@ -105,25 +106,6 @@ from abogen.constants import PROGRAM_NAME, VERSION
os.environ["MIOPEN_FIND_MODE"] = "FAST"
os.environ["MIOPEN_CONV_PRECISE_ROCM_TUNING"] = "0"
# Reset sleep states
atexit.register(prevent_sleep_end)
# Also handle signals (Ctrl+C, kill, etc.)
def _cleanup_sleep(signum, frame):
prevent_sleep_end()
sys.exit(0)
signal.signal(signal.SIGINT, _cleanup_sleep)
signal.signal(signal.SIGTERM, _cleanup_sleep)
# Ensure sys.stdout and sys.stderr are valid in GUI mode
if sys.stdout is None:
sys.stdout = open(os.devnull, "w")
if sys.stderr is None:
sys.stderr = open(os.devnull, "w")
# Enable MPS GPU acceleration on Mac Apple Silicon
if platform.system() == "Darwin" and platform.processor() == "arm":
os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"
+160
View File
@@ -0,0 +1,160 @@
"""Graceful shutdown - single module, no over-engineering."""
from __future__ import annotations
import atexit
import gc
import signal
import sys
from typing import Callable
_CLEANUP_FUNCS: list[Callable[[], None]] = []
_EXECUTED = False
def register_cleanup(fn: Callable[[], None]) -> None:
"""Register a cleanup function to run on shutdown."""
_CLEANUP_FUNCS.append(fn)
def _run_cleanups() -> None:
global _EXECUTED
if _EXECUTED:
return
_EXECUTED = True
for fn in _CLEANUP_FUNCS:
try:
fn()
except Exception:
pass
# ---- Register built-in cleanup functions ----
# 1. Restore sleep prevention
def _restore_sleep() -> None:
try:
from abogen.utils import prevent_sleep_end
prevent_sleep_end()
except Exception:
pass
register_cleanup(_restore_sleep)
# 2. Shutdown web UI ConversionService
def _shutdown_conversion_service() -> None:
try:
from abogen.webui.service import get_service
svc = get_service()
if svc is not None:
svc.shutdown()
except Exception:
pass
register_cleanup(_shutdown_conversion_service)
# 3. Clear TTS pipelines and GPU memory
def _cleanup_tts_pipelines() -> None:
# Clear web UI pipeline cache
try:
from abogen.webui.conversion_runner import _PIPELINES
_PIPELINES.clear()
except Exception:
pass
# Clear PyQt conversion thread voice cache
try:
from abogen.pyqt.conversion import ConversionThread
if hasattr(ConversionThread, "voice_cache"):
ConversionThread.voice_cache.clear()
except Exception:
pass
gc.collect()
# Release CUDA cache
try:
import torch
if torch.cuda.is_available():
torch.cuda.empty_cache()
torch.cuda.ipc_collect()
except Exception:
pass
register_cleanup(_cleanup_tts_pipelines)
# 4. Clear global voice cache
def _clear_voice_cache() -> None:
try:
from abogen.voice_cache import clear_voice_cache
clear_voice_cache()
except Exception:
pass
register_cleanup(_clear_voice_cache)
# 5. Terminate child processes (ffmpeg, etc.)
def _terminate_subprocesses() -> None:
try:
import psutil
except Exception:
return
try:
current = psutil.Process()
for child in current.children(recursive=True):
try:
child.terminate()
except Exception:
pass
gone, alive = psutil.wait_procs(current.children(recursive=True), timeout=3)
for proc in alive:
try:
proc.kill()
except Exception:
pass
except Exception:
pass
register_cleanup(_terminate_subprocesses)
def register_shutdown() -> None:
"""Install process-wide shutdown hooks (atexit, signals, Qt)."""
if register_shutdown._registered:
return
register_shutdown._registered = True
atexit.register(_run_cleanups)
# POSIX signals
for sig in (signal.SIGINT, signal.SIGTERM):
try:
signal.signal(sig, _on_signal)
except Exception:
pass
# Qt hook
try:
from PyQt6.QtWidgets import QApplication
app = QApplication.instance()
if app is not None:
app.aboutToQuit.connect(_run_cleanups)
except Exception:
pass
register_shutdown._registered = False
def _on_signal(signum: int, _frame) -> None:
_run_cleanups()
sys.exit(0)
def request_shutdown() -> None:
"""Programmatically trigger cleanup (e.g., from GUI closeEvent)."""
_run_cleanups()
__all__ = ["register_shutdown", "request_shutdown", "register_cleanup"]
+8
View File
@@ -144,3 +144,11 @@ def _ensure_single_voice_asset(
hf_hub_download(resume_download=True, **common_kwargs)
return True
def clear_voice_cache() -> None:
"""Clear the inprocess voice cache (used during shutdown)."""
with _CACHE_LOCK:
_CACHED_VOICES.clear()
global _BOOTSTRAPPED
_BOOTSTRAPPED = False
+8 -3
View File
@@ -1,6 +1,5 @@
from __future__ import annotations
import atexit
import logging
import os
from pathlib import Path
@@ -8,6 +7,8 @@ from typing import Any, Optional
from flask import Flask
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
from abogen.utils import get_user_cache_path, get_user_output_path, get_user_settings_dir
from .conversion_runner import run_conversion_job
@@ -83,6 +84,12 @@ def create_app(config: Optional[dict[str, Any]] = None) -> Flask:
"UPLOAD_FOLDER": str(uploads_dir),
"OUTPUT_FOLDER": str(outputs_dir),
"MAX_CONTENT_LENGTH": 1024 * 1024 * 400, # 400 MB uploads
# Large books can submit four form fields per chapter. Werkzeug's
# defaults reject those requests before the wizard route can process
# them, even though the encoded payload is much smaller than the upload
# limit above.
"MAX_FORM_MEMORY_SIZE": 10 * 1024 * 1024,
"MAX_FORM_PARTS": 10_000,
}
if config:
base_config.update(config)
@@ -113,8 +120,6 @@ def create_app(config: Optional[dict[str, Any]] = None) -> Flask:
app.register_blueprint(books_bp, url_prefix="/find-books")
app.register_blueprint(api_bp, url_prefix="/api")
atexit.register(service.shutdown)
global _access_log_filter_attached
if not _access_log_filter_attached:
logging.getLogger("werkzeug").addFilter(_SuppressSuccessfulAccessFilter())
File diff suppressed because it is too large Load Diff
+2 -25
View File
@@ -6,6 +6,8 @@ import soundfile as sf
from flask import current_app, send_file
from flask.typing import ResponseReturnValue
from abogen.domain.device import select_device as _select_device
SPLIT_PATTERN = r"\n+"
SAMPLE_RATE = 24000
@@ -25,31 +27,6 @@ def clear_preview_pipelines() -> None:
_preview_pipelines.clear()
def _select_device() -> str:
import platform
try:
import torch # type: ignore[import-not-found]
except Exception:
return "cpu"
system = platform.system()
if system == "Darwin" and platform.processor() == "arm":
try:
if torch.backends.mps.is_available():
return "mps"
except Exception:
pass
return "cpu"
try:
if torch.cuda.is_available():
return "cuda"
except Exception:
pass
return "cpu"
def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]:
devices: List[str] = ["cpu"]
if use_gpu:
+2
View File
@@ -1609,10 +1609,12 @@ def build_service(
output_root: Optional[Path] = None,
uploads_root: Optional[Path] = None,
) -> ConversionService:
global _service_instance
output_root = output_root or default_storage_root()
service = ConversionService(
output_root=output_root,
uploads_root=uploads_root,
runner=runner,
)
_service_instance = service
return service
+7
View File
@@ -34,6 +34,13 @@ from .engine import KokoroEngine
def _load_kpipeline() -> Any:
"""Lazy-load Kokoro dependencies."""
# Transformers 5.x moved AlbertModel out of top-level imports.
# Monkey-patch before kokoro imports it.
import transformers
if not hasattr(transformers, "AlbertModel"):
from transformers.models.albert import AlbertModel as _AlbertModel
transformers.AlbertModel = _AlbertModel
from kokoro import KPipeline # type: ignore[import-not-found]
return KPipeline
+1 -1
View File
@@ -44,7 +44,7 @@ dependencies = [
"python-dotenv>=1.0.1",
"static_ffmpeg>=2.13",
"Markdown>=3.9",
"Flask>=3.0.3",
"Flask>=3.1.0",
"numpy>=1.24.0",
"gpustat>=1.1.1",
"num2words>=0.5.13",
+128
View File
@@ -0,0 +1,128 @@
"""Tests for audio helper utilities.
Tests import from domain/audio_helpers.py (new module).
"""
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
import numpy as np
import pytest
# ---------------------------------------------------------------------------
# build_ffmpeg_command
# ---------------------------------------------------------------------------
class TestBuildFfmpegCommand:
"""build_ffmpeg_command builds ffmpeg argument list."""
def test_base_structure(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out/audio.wav"), "wav")
assert cmd[0] == "ffmpeg"
assert "-y" in cmd
assert "pipe:0" in cmd
assert str(Path("/out/audio.wav")) in cmd
def test_mp3_codec(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out.mp3"), "mp3")
assert "libmp3lame" in cmd
def test_opus_codec(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out.opus"), "opus")
assert "libopus" in cmd
def test_m4b_codec(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out.m4b"), "m4b")
assert "aac" in cmd
assert "-q:a" in cmd
assert "+faststart+use_metadata_tags" in cmd
def test_wav_copy_codec(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out.wav"), "wav")
assert "copy" in cmd
def test_with_metadata(self):
from abogen.domain.audio_helpers import build_ffmpeg_command
cmd = build_ffmpeg_command(Path("/out.mp3"), "mp3", metadata={"album": "Test"})
assert str(Path("/out.mp3")) in cmd
# ---------------------------------------------------------------------------
# to_float32
# ---------------------------------------------------------------------------
class TestToFloat32:
"""to_float32 converts audio to float32 numpy array."""
def test_none_returns_empty(self):
from abogen.domain.audio_helpers import to_float32
result = to_float32(None)
assert isinstance(result, np.ndarray)
assert result.dtype == np.float32
assert len(result) == 0
def test_numpy_array(self):
from abogen.domain.audio_helpers import to_float32
arr = np.array([1.0, 2.0, 3.0], dtype="float64")
result = to_float32(arr)
assert result.dtype == np.float32
assert len(result) == 3
def test_mock_tensor(self):
from abogen.domain.audio_helpers import to_float32
tensor = MagicMock()
tensor.detach.return_value = tensor
tensor.cpu.return_value = tensor
tensor.numpy.return_value = np.array([1.0, 2.0])
result = to_float32(tensor)
assert result.dtype == np.float32
assert len(result) == 2
def test_list_input(self):
from abogen.domain.audio_helpers import to_float32
result = to_float32([1.0, 2.0])
assert result.dtype == np.float32
assert len(result) == 2
# ---------------------------------------------------------------------------
# apply_m4b_chapters_with_mutagen
# ---------------------------------------------------------------------------
class TestApplyM4bChaptersWithMutagen:
"""apply_m4b_chapters_with_mutagen writes chapter atoms to MP4."""
def test_empty_chapters_returns_false(self):
from abogen.domain.audio_helpers import apply_m4b_chapters_with_mutagen
assert apply_m4b_chapters_with_mutagen(Path("/fake.m4b"), []) is False
def test_missing_mutagen_raises(self):
from abogen.domain.audio_helpers import apply_m4b_chapters_with_mutagen
with patch.dict("sys.modules", {"mutagen": None, "mutagen.mp4": None}):
with pytest.raises((ImportError, KeyError)):
apply_m4b_chapters_with_mutagen(
Path("/fake.m4b"), [{"start": 0, "title": "Ch1"}]
)
+167
View File
@@ -0,0 +1,167 @@
"""Tests for chapter_overrides, merge_metadata, normalize_for_pipeline.
Tests import from domain modules (new location).
"""
from __future__ import annotations
import pytest
from abogen.text_extractor import ExtractedChapter
# ---------------------------------------------------------------------------
# apply_chapter_overrides
# ---------------------------------------------------------------------------
class TestApplyChapterOverrides:
"""apply_chapter_overrides applies chapter overrides to extracted chapters."""
def test_empty_overrides(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
result, updates, diags = apply_chapter_overrides([], [])
assert result == []
assert updates == {}
assert diags == []
def test_basic_override_by_index(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
chapters = [ExtractedChapter(title="Ch1", text="original")]
overrides = [{"index": 0, "title": "New Title", "text": "new text"}]
result, updates, diags = apply_chapter_overrides(chapters, overrides)
assert len(result) == 1
assert result[0].title == "New Title"
assert result[0].text == "new text"
def test_override_by_source_title(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
chapters = [ExtractedChapter(title="Ch1", text="text1")]
overrides = [{"source_title": "Ch1", "title": "Renamed"}]
result, _, _ = apply_chapter_overrides(chapters, overrides)
assert result[0].title == "Renamed"
def test_disabled_override_skipped(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
chapters = [ExtractedChapter(title="Ch1", text="text1")]
overrides = [{"index": 0, "enabled": False}]
result, _, _ = apply_chapter_overrides(chapters, overrides)
assert len(result) == 0
def test_metadata_updates_collected(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
chapters = [ExtractedChapter(title="Ch1", text="text1")]
overrides = [{"index": 0, "metadata": {"album": "New Album"}}]
_, updates, _ = apply_chapter_overrides(chapters, overrides)
assert updates["album"] == "New Album"
def test_no_matching_chapter_diagnostic(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
overrides = [{"index": 99, "title": "X"}]
_, _, diags = apply_chapter_overrides([], overrides)
assert len(diags) == 1
assert "Skipped" in diags[0]
def test_non_dict_override_skipped(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
_, _, diags = apply_chapter_overrides([], ["bad"])
assert len(diags) == 1
def test_text_from_base_when_not_provided(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
chapters = [ExtractedChapter(title="Ch1", text="original text")]
overrides = [{"index": 0, "title": "New Title"}]
result, _, _ = apply_chapter_overrides(chapters, overrides)
assert result[0].text == "original text"
def test_default_title_when_no_base(self):
from abogen.domain.chapter_overrides import apply_chapter_overrides
overrides = [{"text": "some text"}]
result, _, _ = apply_chapter_overrides([], overrides)
assert result[0].title == "Chapter 1"
# ---------------------------------------------------------------------------
# merge_metadata
# ---------------------------------------------------------------------------
class TestMergeMetadata:
"""merge_metadata merges extracted metadata with overrides."""
def test_both_empty(self):
from abogen.domain.metadata_merge import merge_metadata
assert merge_metadata({}, {}) == {}
def test_only_extracted(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata({"album": "Book"}, {})
assert result == {"album": "Book"}
def test_only_overrides(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata(None, {"album": "Override"})
assert result == {"album": "Override"}
def test_override_wins(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata({"album": "Old"}, {"album": "New"})
assert result == {"album": "New"}
def test_none_value_deletes_key(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata({"album": "Book"}, {"album": None})
assert "album" not in result
def test_none_values_in_extracted_skipped(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata({"album": None, "artist": "X"}, {})
assert result == {"artist": "X"}
def test_numeric_values_stringified(self):
from abogen.domain.metadata_merge import merge_metadata
result = merge_metadata({"track": 1}, {})
assert result["track"] == "1"
# ---------------------------------------------------------------------------
# normalize_for_pipeline (thin wrapper)
# ---------------------------------------------------------------------------
class TestNormalizeForPipeline:
"""normalize_for_pipeline normalizes text with runtime settings."""
def test_basic_normalize(self):
from abogen.domain.normalization import normalize_text_for_pipeline
result = normalize_text_for_pipeline("Hello World")
assert isinstance(result, str)
assert len(result) > 0
def test_empty_string(self):
from abogen.domain.normalization import normalize_text_for_pipeline
result = normalize_text_for_pipeline("")
assert result == ""
def test_with_overrides(self):
from abogen.domain.normalization import normalize_text_for_pipeline
result = normalize_text_for_pipeline("test", normalization_overrides={"number_format": "words"})
assert isinstance(result, str)
+144
View File
@@ -0,0 +1,144 @@
"""Tests for domain/chapter_titles.py."""
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from abogen.domain.chapter_titles import (
simplify_heading_text,
headings_equivalent,
strip_duplicate_heading_line,
normalize_caps_word,
normalize_chapter_opening_caps,
format_spoken_chapter_title,
)
class TestSimplifyHeadingText:
def test_empty(self):
assert simplify_heading_text("") == ""
def test_none(self):
assert simplify_heading_text(None) == ""
def test_chapter_prefix_removed(self):
assert simplify_heading_text("Chapter 1") == "1"
def test_lowercase(self):
assert simplify_heading_text("Chapter 1: The Beginning") == "1thebeginning"
def test_strips_special_chars(self):
result = simplify_heading_text("Ch. 1")
assert "1" in result
assert "." not in result
def test_no_chapter_prefix(self):
result = simplify_heading_text("Part 2")
assert "part" in result
class TestHeadingsEquivalent:
def test_exact_match(self):
assert headings_equivalent("Chapter 1", "Chapter 1")
def test_prefix_match(self):
assert headings_equivalent("Chapter 2", "Chapter 2: The Return")
def test_reverse_prefix(self):
assert headings_equivalent("Chapter 2: The Return", "Chapter 2")
def test_different_numbers(self):
assert not headings_equivalent("Chapter 1", "Chapter 2")
def test_empty(self):
assert not headings_equivalent("", "Chapter 1")
def test_long_containment(self):
assert headings_equivalent("Introduction", "Introduction to Everything")
class TestStripDuplicateHeadingLine:
def test_removes_heading(self):
text = "Chapter 1\n\nSome text here"
result, removed = strip_duplicate_heading_line(text, "Chapter 1")
assert removed is True
assert "Chapter 1" not in result
assert "Some text here" in result
def test_no_heading(self):
text = "Just some text"
result, removed = strip_duplicate_heading_line(text, "Chapter 1")
assert removed is False
assert result == text
def test_empty_text(self):
result, removed = strip_duplicate_heading_line("", "Chapter 1")
assert removed is False
def test_strips_leading_empty_lines(self):
text = "Chapter 1\n\n\n\nText"
result, removed = strip_duplicate_heading_line(text, "Chapter 1")
assert removed is True
assert result.startswith("Text")
class TestNormalizeCapsWord:
def test_acronym_kept(self):
assert normalize_caps_word("TTS") == "TTS"
def test_single_letter_kept(self):
assert normalize_caps_word("A") == "A"
def test_roman_numeral_kept(self):
assert normalize_caps_word("IV") == "IV"
def test_all_caps_converted(self):
result = normalize_caps_word("HELLO")
assert result == "Hello"
def test_with_hyphen(self):
result = normalize_caps_word("WELL-KNOWN")
assert result == "Well-Known"
class TestNormalizeChapterOpeningCaps:
def test_all_caps_words(self):
text = "THIS IS A TEST"
result, changed = normalize_chapter_opening_caps(text)
assert changed is True
assert result == "This Is A Test"
def test_already_normal(self):
text = "This is normal"
result, changed = normalize_chapter_opening_caps(text)
assert changed is False
def test_empty(self):
result, changed = normalize_chapter_opening_caps("")
assert changed is False
def test_mixed(self):
text = "HELLO world"
result, changed = normalize_chapter_opening_caps(text)
assert changed is True
class TestFormatSpokenChapterTitle:
def test_empty_no_prefix(self):
assert format_spoken_chapter_title("", 1, False) == ""
def test_empty_with_prefix(self):
assert format_spoken_chapter_title("", 1, True) == "Chapter 1"
def test_no_prefix_returns_base(self):
assert format_spoken_chapter_title("My Chapter", 1, False) == "My Chapter"
def test_already_has_chapter(self):
assert format_spoken_chapter_title("Chapter 5", 1, True) == "Chapter 5"
def test_number_prefix(self):
result = format_spoken_chapter_title("3. The End", 1, True)
assert result == "Chapter 3. The End"
def test_number_only(self):
result = format_spoken_chapter_title("7", 1, True)
assert result == "Chapter 7"
+6 -5
View File
@@ -3,7 +3,8 @@ from __future__ import annotations
from types import SimpleNamespace
from abogen.chunking import chunk_text
from abogen.webui.conversion_runner import _chunk_voice_spec, _group_chunks_by_chapter
from abogen.domain.voice_resolution import chunk_voice_spec
from abogen.domain.chunk_utils import group_chunks_by_chapter
def test_group_chunks_by_chapter_orders_and_groups() -> None:
@@ -13,7 +14,7 @@ def test_group_chunks_by_chapter_orders_and_groups() -> None:
{"chapter_index": 1, "chunk_index": 0, "text": "next"},
]
grouped = _group_chunks_by_chapter(chunks)
grouped = group_chunks_by_chapter(chunks)
assert [entry["text"] for entry in grouped[0]] == ["body", "tail"]
assert grouped[1][0]["text"] == "next"
@@ -23,7 +24,7 @@ def test_chunk_voice_spec_prefers_chunk_overrides() -> None:
job = SimpleNamespace(voice="base_voice", speakers={})
chunk = {"voice": "override_voice", "speaker_id": "narrator"}
assert _chunk_voice_spec(job, chunk, "fallback") == "override_voice"
assert chunk_voice_spec(job, chunk, "fallback") == "override_voice"
def test_chunk_voice_spec_falls_back_to_speaker_voice() -> None:
@@ -32,14 +33,14 @@ def test_chunk_voice_spec_falls_back_to_speaker_voice() -> None:
)
chunk = {"speaker_id": "narrator"}
assert _chunk_voice_spec(job, chunk, "fallback") == "speaker_voice"
assert chunk_voice_spec(job, chunk, "fallback") == "speaker_voice"
def test_chunk_voice_spec_uses_fallback_when_no_overrides() -> None:
job = SimpleNamespace(voice="base_voice", speakers={})
chunk = {"speaker_id": "unknown"}
assert _chunk_voice_spec(job, chunk, "fallback") == "fallback"
assert chunk_voice_spec(job, chunk, "fallback") == "fallback"
def test_chunk_text_merges_title_abbreviations() -> None:
+4 -4
View File
@@ -1,4 +1,4 @@
from abogen.webui.conversion_runner import _chunk_text_for_tts
from abogen.domain.chunk_utils import chunk_text_for_tts
def test_chunk_text_for_tts_prefers_text_over_normalized_text():
@@ -9,7 +9,7 @@ def test_chunk_text_for_tts_prefers_text_over_normalized_text():
"text": "Unfu*k",
}
assert _chunk_text_for_tts(entry) == "Unfu*k"
assert chunk_text_for_tts(entry) == "Unfu*k"
def test_chunk_text_for_tts_falls_back_to_original_text_then_normalized_text():
@@ -17,9 +17,9 @@ def test_chunk_text_for_tts_falls_back_to_original_text_then_normalized_text():
"original_text": "Hello * world",
"normalized_text": "Hello world",
}
assert _chunk_text_for_tts(entry) == "Hello * world"
assert chunk_text_for_tts(entry) == "Hello * world"
entry2 = {
"normalized_text": "Only normalized",
}
assert _chunk_text_for_tts(entry2) == "Only normalized"
assert chunk_text_for_tts(entry2) == "Only normalized"
+123
View File
@@ -0,0 +1,123 @@
"""Tests for chunk processing utilities.
Tests import from domain.chunk_utils (new location).
"""
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import patch
import pytest
# ---------------------------------------------------------------------------
# safe_int
# ---------------------------------------------------------------------------
class TestSafeInt:
"""safe_int safely converts to int with a default."""
def test_int_value(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int(42) == 42
def test_string_number(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int("7") == 7
def test_float_truncated(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int(3.9) == 3
def test_none_returns_default(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int(None) == 0
def test_garbage_returns_default(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int("abc") == 0
def test_custom_default(self):
from abogen.domain.chunk_utils import safe_int
assert safe_int(None, default=-1) == -1
# ---------------------------------------------------------------------------
# chunk_text_for_tts (supplement existing tests)
# ---------------------------------------------------------------------------
class TestChunkTextForTts:
"""chunk_text_for_tts selects the best text source."""
def test_non_mapping_returns_empty(self):
from abogen.domain.chunk_utils import chunk_text_for_tts
assert chunk_text_for_tts("not a dict") == ""
def test_none_returns_empty(self):
from abogen.domain.chunk_utils import chunk_text_for_tts
assert chunk_text_for_tts(None) == ""
def test_empty_dict_returns_empty(self):
from abogen.domain.chunk_utils import chunk_text_for_tts
assert chunk_text_for_tts({}) == ""
def test_whitespace_only_returns_empty(self):
from abogen.domain.chunk_utils import chunk_text_for_tts
assert chunk_text_for_tts({"text": " "}) == ""
# ---------------------------------------------------------------------------
# record_override_usage
# ---------------------------------------------------------------------------
class TestRecordOverrideUsage:
"""record_override_usage records pronunciation override usage."""
def test_noop_when_empty(self):
from abogen.domain.chunk_utils import record_override_usage
job = SimpleNamespace(language="en")
record_override_usage(job, {}, {})
def test_noop_when_all_zero(self):
from abogen.domain.chunk_utils import record_override_usage
job = SimpleNamespace(language="en")
record_override_usage(job, {"hello": 0}, {"hello": "hi"})
def test_records_usage(self):
from abogen.domain.chunk_utils import record_override_usage
job = SimpleNamespace(language="en", add_log=lambda *a, **kw: None)
with patch("abogen.domain.chunk_utils.increment_usage") as mock_inc:
record_override_usage(job, {"hello": 2}, {"hello": "hi"})
mock_inc.assert_called_once_with(language="en", token="hi", amount=2)
def test_fallback_token_from_normalized(self):
from abogen.domain.chunk_utils import record_override_usage
job = SimpleNamespace(language="ja", add_log=lambda *a, **kw: None)
with patch("abogen.domain.chunk_utils.increment_usage") as mock_inc:
record_override_usage(job, {"test": 1}, {})
mock_inc.assert_called_once_with(language="ja", token="test", amount=1)
def test_handles_exception_gracefully(self):
from abogen.domain.chunk_utils import record_override_usage
job = SimpleNamespace(language="en", add_log=lambda *a, **kw: None)
with patch("abogen.domain.chunk_utils.increment_usage", side_effect=RuntimeError("db error")):
record_override_usage(job, {"hello": 1}, {"hello": "hi"})
+114
View File
@@ -124,3 +124,117 @@ def test_normalize_chapter_opening_caps_keeps_mixed_case() -> None:
normalized, changed = _normalize_chapter_opening_caps("Already Mixed Case")
assert normalized == "Already Mixed Case"
assert changed is False
class TestApplyChapterTextTransforms:
"""Tests for the combined heading-strip + opening-caps helper."""
def test_both_enabled_heading_matches(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"Chapter 1: The Beginning\nBody text here",
heading_text="Chapter 1: The Beginning",
raw_title="Chapter 1: The Beginning",
strip_heading=True,
normalize_caps=True,
)
assert heading_removed is True
assert "Body text here" in text
assert "Chapter 1" not in text
def test_heading_fallback_to_number(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"1. The Beginning\nBody text",
heading_text="Chapter 1: The Beginning",
raw_title="1: The Beginning",
strip_heading=True,
normalize_caps=False,
)
assert heading_removed is True
assert "Body text" in text
def test_only_heading_strip(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"Chapter 1: Title\nBody text",
heading_text="Chapter 1: Title",
raw_title="",
strip_heading=True,
normalize_caps=False,
)
assert heading_removed is True
assert caps_changed is False
def test_only_opening_caps(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"ALL CAPS START OF CHAPTER",
heading_text="Chapter 1",
raw_title="",
strip_heading=False,
normalize_caps=True,
)
assert heading_removed is False
assert caps_changed is True
assert text == "All Caps Start Of Chapter"
def test_both_disabled_no_change(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
original = "Some text here"
text, heading_removed, caps_changed = apply_chapter_text_transforms(
original,
heading_text="Chapter 1",
raw_title="",
strip_heading=False,
normalize_caps=False,
)
assert text == original
assert heading_removed is False
assert caps_changed is False
def test_heading_not_matching(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"Completely different text",
heading_text="Chapter 1: Title",
raw_title="",
strip_heading=True,
normalize_caps=False,
)
assert heading_removed is False
assert text == "Completely different text"
def test_empty_text(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"",
heading_text="Chapter 1",
raw_title="",
strip_heading=True,
normalize_caps=True,
)
assert text == ""
assert heading_removed is False
assert caps_changed is False
def test_both_enabled_text_only_has_caps(self) -> None:
from abogen.domain.chapter_titles import apply_chapter_text_transforms
text, heading_removed, caps_changed = apply_chapter_text_transforms(
"NASA MISSION LOG",
heading_text="Chapter 1",
raw_title="",
strip_heading=True,
normalize_caps=True,
)
assert heading_removed is False
assert caps_changed is True
assert text == "NASA Mission Log"
+163
View File
@@ -0,0 +1,163 @@
"""Tests for ExportService FFmpeg metadata methods."""
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from pathlib import Path
from abogen.infrastructure.exporters import ExportService
class TestEscapeFfmetadataValue:
def setup_method(self):
self.svc = ExportService()
def test_simple_string(self):
assert self.svc._escape_ffmetadata_value("hello") == "hello"
def test_escapes_backslash(self):
assert self.svc._escape_ffmetadata_value("a\\b") == "a\\\\b"
def test_escapes_newline(self):
assert self.svc._escape_ffmetadata_value("line1\nline2") == "line1\\nline2"
def test_escapes_equals(self):
assert self.svc._escape_ffmetadata_value("key=value") == "key\\=value"
def test_escapes_semicolon(self):
assert self.svc._escape_ffmetadata_value("a;b") == "a\\;b"
def test_escapes_hash(self):
assert self.svc._escape_ffmetadata_value("#comment") == "\\#comment"
def test_escapes_all_special(self):
result = self.svc._escape_ffmetadata_value("a\\b\nc=d;e#f")
assert "\\\\" in result
assert "\\n" in result
assert "\\=" in result
assert "\\;" in result
assert "\\#" in result
def test_empty_string(self):
assert self.svc._escape_ffmetadata_value("") == ""
class TestRenderFfmetadata:
def setup_method(self):
self.svc = ExportService()
def test_renders_header(self):
result = self.svc.render_ffmetadata({"title": "My Book"}, [])
assert result.startswith(";FFMETADATA1\n")
assert "title=My Book\n" in result
def test_renders_multiple_keys(self):
result = self.svc.render_ffmetadata({"title": "T", "artist": "A"}, [])
assert "title=T\n" in result
assert "artist=A\n" in result
def test_skips_none_values(self):
result = self.svc.render_ffmetadata({"title": None}, [])
assert "title=" not in result
def test_renders_chapters(self):
chapters = [{"start": 0.0, "end": 10.0, "title": "Ch 1"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "[CHAPTER]" in result
assert "TIMEBASE=1/1000" in result
assert "START=0" in result
assert "END=10000" in result
assert "title=Ch 1" in result
def test_renders_voice_in_chapter(self):
chapters = [{"start": 0.0, "end": 5.0, "voice": "af_heart"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "voice=af_heart" in result
def test_skips_chapters_without_times(self):
chapters = [{"title": "No times"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "[CHAPTER]" not in result
def test_end_equals_start_gets_minimum_duration(self):
chapters = [{"start": 5.0, "end": 5.0, "title": "Zero"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "START=5000" in result
assert "END=5001" in result
def test_empty_metadata_and_chapters(self):
result = self.svc.render_ffmetadata({}, [])
assert result.strip() == ";FFMETADATA1"
def test_escapes_special_chars_in_title(self):
chapters = [{"start": 0.0, "end": 1.0, "title": "A=B;C#D"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "\\=" in result
assert "\\;" in result
assert "\\#" in result
def test_negative_start_clamped_to_zero(self):
chapters = [{"start": -1.0, "end": 5.0, "title": "Neg"}]
result = self.svc.render_ffmetadata({}, chapters)
assert "START=0" in result
class TestMetadataToFfmpegArgs:
def setup_method(self):
self.svc = ExportService()
def test_simple_metadata(self):
args = self.svc._metadata_to_ffmpeg_args({"title": "My Book"})
assert args == ["-metadata", "title=My Book"]
def test_year_becomes_date(self):
args = self.svc._metadata_to_ffmpeg_args({"year": "2024"})
assert args == ["-metadata", "date=2024"]
def test_skips_none_and_empty(self):
args = self.svc._metadata_to_ffmpeg_args({"title": None, "artist": ""})
assert args == []
def test_skips_empty_key(self):
args = self.svc._metadata_to_ffmpeg_args({"": "value"})
assert args == []
def test_multiple_keys(self):
args = self.svc._metadata_to_ffmpeg_args({"title": "T", "artist": "A"})
assert "-metadata" in args
assert "title=T" in args
assert "artist=A" in args
def test_empty_metadata(self):
assert self.svc._metadata_to_ffmpeg_args({}) == []
def test_none_metadata(self):
assert self.svc._metadata_to_ffmpeg_args(None) == []
class TestWriteFfmetadataFile:
def setup_method(self):
self.svc = ExportService()
def test_writes_file(self, tmp_path):
audio = tmp_path / "test.mp3"
audio.touch()
meta = {"title": "My Book"}
chapters = [{"start": 0.0, "end": 5.0, "title": "Ch 1"}]
result = self.svc.write_ffmetadata_file(audio, meta, chapters)
assert result is not None
assert result.exists()
content = result.read_text()
assert ";FFMETADATA1" in content
assert "title=My Book" in content
def test_returns_none_for_empty(self, tmp_path):
audio = tmp_path / "test.mp3"
audio.touch()
result = self.svc.write_ffmetadata_file(audio, {}, [])
assert result is None
def test_returns_none_for_only_header(self, tmp_path):
audio = tmp_path / "test.mp3"
audio.touch()
result = self.svc.write_ffmetadata_file(audio, None, None)
assert result is None
+5 -4
View File
@@ -1,8 +1,9 @@
from __future__ import annotations
from pathlib import Path
from abogen.infrastructure.exporters import ExportService
from abogen.webui.conversion_runner import _render_ffmetadata, _write_ffmetadata_file
svc = ExportService()
def test_render_ffmetadata_includes_chapters(tmp_path):
@@ -17,7 +18,7 @@ def test_render_ffmetadata_includes_chapters(tmp_path):
{"start": 5.0, "end": 12.345, "title": "Chapter 2"},
]
rendered = _render_ffmetadata(metadata, chapters)
rendered = svc.render_ffmetadata(metadata, chapters)
assert ";FFMETADATA1" in rendered
assert "title=Sample Book" in rendered
@@ -30,7 +31,7 @@ def test_render_ffmetadata_includes_chapters(tmp_path):
assert "voice=voice_a" in rendered
audio_path = tmp_path / "book.m4b"
metadata_path = _write_ffmetadata_file(audio_path, metadata, chapters)
metadata_path = svc.write_ffmetadata_file(audio_path, metadata, chapters)
assert metadata_path is not None
assert metadata_path.exists()
+133
View File
@@ -0,0 +1,133 @@
"""Tests for domain/metadata_helpers.py."""
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from abogen.domain.metadata_helpers import (
normalize_metadata_map,
format_author_sentence,
ensure_sentence,
normalize_series_number,
extract_series_metadata,
format_series_sentence,
)
class TestNormalizeMetadataMap:
def test_empty(self):
assert normalize_metadata_map({}) == {}
def test_none(self):
assert normalize_metadata_map(None) == {}
def test_normalizes_keys(self):
result = normalize_metadata_map({"Title": "My Book", "artist": "John"})
assert "title" in result
assert "artist" in result
def test_skips_none_values(self):
result = normalize_metadata_map({"title": None, "artist": "John"})
assert "title" not in result
def test_skips_empty_values(self):
result = normalize_metadata_map({"title": "", "artist": "John"})
assert "title" not in result
class TestFormatAuthorSentence:
def test_none(self):
assert format_author_sentence(None) == ""
def test_empty(self):
assert format_author_sentence("") == ""
def test_unknown(self):
assert format_author_sentence("Unknown") == ""
def test_single(self):
assert format_author_sentence("John Doe") == "By John Doe"
def test_two(self):
assert format_author_sentence("John, Jane") == "By John and Jane"
def test_three(self):
assert format_author_sentence("John, Jane, Bob") == "By John, Jane, and Bob"
def test_ampersand(self):
assert format_author_sentence("John & Jane") == "By John and Jane"
class TestEnsureSentence:
def test_empty(self):
assert ensure_sentence("") == ""
def test_already_sentence(self):
assert ensure_sentence("Hello.") == "Hello."
def test_adds_period(self):
assert ensure_sentence("Hello") == "Hello."
def test_exclamation(self):
assert ensure_sentence("Hello!") == "Hello!"
class TestNormalizeSeriesNumber:
def test_empty(self):
assert normalize_series_number("") is None
def test_integer(self):
assert normalize_series_number("3") == "3"
def test_float(self):
assert normalize_series_number("3.5") == "3.5"
def test_float_trailing_zero(self):
assert normalize_series_number("3.10") == "3.1"
def test_comma_as_separator(self):
assert normalize_series_number("3,5") == "3.5"
def test_text_with_number(self):
assert normalize_series_number("Book 3") == "3"
def test_none(self):
assert normalize_series_number(None) is None
class TestExtractSeriesMetadata:
def test_empty(self):
name, number = extract_series_metadata({})
assert name is None
assert number is None
def test_series_name(self):
name, number = extract_series_metadata({"series": "My Series"})
assert name == "My Series"
assert number is None
def test_series_number(self):
name, number = extract_series_metadata({"series_index": "3"})
assert name is None
assert number == "3"
def test_both(self):
name, number = extract_series_metadata({"series": "My Series", "series_index": "3"})
assert name == "My Series"
assert number == "3"
class TestFormatSeriesSentence:
def test_empty(self):
assert format_series_sentence(None, None) == ""
def test_name_only(self):
assert format_series_sentence("My Series", None) == ""
def test_number_only(self):
assert format_series_sentence(None, "3") == ""
def test_both(self):
assert format_series_sentence("My Series", "3") == "Book 3 of the My Series"
def test_with_the(self):
assert format_series_sentence("The Lord of the Rings", "1") == "Book 1 of The Lord of the Rings"
+239 -61
View File
@@ -1,79 +1,257 @@
"""Tests for output path utilities.
Tests import from domain/output_paths.py (new module).
"""
from __future__ import annotations
import time
import re
from datetime import datetime
from pathlib import Path
from types import SimpleNamespace
import pytest
from abogen.webui.conversion_runner import _build_output_path, _prepare_project_layout
from abogen.webui.service import Job
# ---------------------------------------------------------------------------
# slugify
# ---------------------------------------------------------------------------
def _sample_job(tmp_path: Path) -> Job:
source = tmp_path / "sample.txt"
source.write_text("example", encoding="utf-8")
return Job(
id="job-1",
original_filename="Sample Title.txt",
stored_path=source,
language="en",
voice="af_alloy",
speed=1.0,
use_gpu=False,
subtitle_mode="Sentence",
output_format="mp3",
class TestSlugify:
"""slugify converts title to filesystem-safe slug."""
def test_basic_slug(self):
from abogen.domain.output_paths import slugify
assert slugify("Hello World", 0) == "hello_world"
def test_strips_special_chars(self):
from abogen.domain.output_paths import slugify
result = slugify("Chapter 1: The Beginning!", 0)
assert result == "chapter_1_the_beginning"
def test_empty_title_uses_index(self):
from abogen.domain.output_paths import slugify
assert slugify("", 5) == "chapter_05"
def test_truncated_at_80(self):
from abogen.domain.output_paths import slugify
long_title = "a" * 100
assert len(slugify(long_title, 0)) == 80
def test_preserves_hyphens(self):
from abogen.domain.output_paths import slugify
assert slugify("mid-night", 0) == "mid-night"
# ---------------------------------------------------------------------------
# sanitize_output_stem
# ---------------------------------------------------------------------------
class TestSanitizeOutputStem:
"""sanitize_output_stem cleans filename stem."""
def test_basic_sanitize(self):
from abogen.domain.output_paths import sanitize_output_stem
assert sanitize_output_stem("my file.mp3") == "my_file"
def test_empty_returns_default(self):
from abogen.domain.output_paths import sanitize_output_stem
assert sanitize_output_stem("") == "output"
def test_strips_underscores(self):
from abogen.domain.output_paths import sanitize_output_stem
assert sanitize_output_stem("__test__") == "test"
# ---------------------------------------------------------------------------
# output_timestamp_token
# ---------------------------------------------------------------------------
class TestOutputTimestampToken:
"""output_timestamp_token returns timestamp string."""
def test_format(self):
from abogen.domain.output_paths import output_timestamp_token
token = output_timestamp_token()
assert re.match(r"\d{8}-\d{6}", token)
# ---------------------------------------------------------------------------
# build_output_path
# ---------------------------------------------------------------------------
class TestBuildOutputPath:
"""build_output_path builds the output file path."""
def test_basic_path(self, tmp_path):
from abogen.domain.output_paths import build_output_path
result = build_output_path(tmp_path, "test.mp3", "mp3")
assert result.suffix == ".mp3"
assert result.parent == tmp_path
def test_stem_sanitized(self, tmp_path):
from abogen.domain.output_paths import build_output_path
result = build_output_path(tmp_path, "my file.txt", "wav")
assert "my_file" in result.name
# ---------------------------------------------------------------------------
# apply_newline_policy
# ---------------------------------------------------------------------------
class TestApplyNewlinePolicy:
"""apply_newline_policy replaces single newlines in chapter text."""
def test_noop_when_disabled(self):
from abogen.domain.output_paths import apply_newline_policy
from abogen.text_extractor import ExtractedChapter
chapters = [ExtractedChapter(title="t", text="a\nb")]
apply_newline_policy(chapters, False)
assert chapters[0].text == "a\nb"
def test_replaces_single_newlines(self):
from abogen.domain.output_paths import apply_newline_policy
from abogen.text_extractor import ExtractedChapter
chapters = [ExtractedChapter(title="t", text="a\nb\nc")]
apply_newline_policy(chapters, True)
assert chapters[0].text == "a b c"
def test_preserves_double_newlines(self):
from abogen.domain.output_paths import apply_newline_policy
from abogen.text_extractor import ExtractedChapter
chapters = [ExtractedChapter(title="t", text="a\n\nb")]
apply_newline_policy(chapters, True)
assert chapters[0].text == "a\n\nb"
# ---------------------------------------------------------------------------
# resolve_output_directory
# ---------------------------------------------------------------------------
class TestResolveOutputDirectory:
"""resolve_output_directory determines output dir from save_mode."""
def test_save_to_desktop(self, tmp_path):
from abogen.domain.output_paths import resolve_output_directory
result = resolve_output_directory(
save_mode="Save to Desktop",
stored_path=Path("/input/book.epub"),
output_folder=None,
desktop_dir=tmp_path,
user_output_path=None,
user_cache_outputs=None,
)
assert result == tmp_path
def test_save_next_to_input(self, tmp_path):
from abogen.domain.output_paths import resolve_output_directory
stored = tmp_path / "book.epub"
result = resolve_output_directory(
save_mode="Save next to input file",
stored_path=stored,
output_folder=None,
desktop_dir=None,
user_output_path=None,
user_cache_outputs=None,
)
assert result == tmp_path
def test_choose_output_folder(self, tmp_path):
from abogen.domain.output_paths import resolve_output_directory
custom = tmp_path / "custom"
result = resolve_output_directory(
save_mode="Choose output folder",
stored_path=Path("/x/y.epub"),
output_folder=str(custom),
desktop_dir=None,
user_output_path=None,
user_cache_outputs=None,
)
assert result == custom
def test_use_default_save_location(self, tmp_path):
from abogen.domain.output_paths import resolve_output_directory
result = resolve_output_directory(
save_mode="Use default save location",
output_folder=tmp_path,
replace_single_newlines=False,
subtitle_format="srt",
created_at=time.time(),
stored_path=Path("/x/y.epub"),
output_folder=None,
desktop_dir=None,
user_output_path=tmp_path / "default",
user_cache_outputs=None,
)
assert result == tmp_path / "default"
def test_fallback_to_cache(self, tmp_path):
from abogen.domain.output_paths import resolve_output_directory
def test_prepare_project_layout_uses_timestamped_folder(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
job = _sample_job(tmp_path)
monkeypatch.setattr(
"abogen.webui.conversion_runner._output_timestamp_token",
lambda: "20250101-120000",
result = resolve_output_directory(
save_mode="unknown",
stored_path=Path("/x/y.epub"),
output_folder=None,
desktop_dir=None,
user_output_path=None,
user_cache_outputs=tmp_path / "cache",
)
assert result == tmp_path / "cache"
project_root, audio_dir, subtitle_dir, metadata_dir = _prepare_project_layout(
job, tmp_path
# ---------------------------------------------------------------------------
# resolve_project_layout
# ---------------------------------------------------------------------------
class TestResolveProjectLayout:
"""resolve_project_layout computes project folder structure."""
def test_flat_layout(self, tmp_path):
from abogen.domain.output_paths import resolve_project_layout
root, audio, subs, meta = resolve_project_layout(
original_filename="book.epub",
save_as_project=False,
base_dir=tmp_path,
timestamp_fn=lambda: "20260101-000000",
sanitize_fn=lambda n, i: "book",
)
assert audio == root
assert subs == root
assert meta is None
assert project_root.name.startswith(
"20250101-120000_Sample_Title"
), project_root.name
assert audio_dir == project_root
assert subtitle_dir == project_root
assert metadata_dir is None
def test_project_layout(self, tmp_path):
from abogen.domain.output_paths import resolve_project_layout
output_path = _build_output_path(audio_dir, job.original_filename, "mp3")
assert output_path == project_root / "Sample_Title.mp3"
def test_prepare_project_layout_creates_project_subdirs(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
job = _sample_job(tmp_path)
job.save_as_project = True
monkeypatch.setattr(
"abogen.webui.conversion_runner._output_timestamp_token",
lambda: "20250101-120500",
root, audio, subs, meta = resolve_project_layout(
original_filename="book.epub",
save_as_project=True,
base_dir=tmp_path,
timestamp_fn=lambda: "20260101-000000",
sanitize_fn=lambda n, i: "book",
)
project_root, audio_dir, subtitle_dir, metadata_dir = _prepare_project_layout(
job, tmp_path
)
assert audio_dir == project_root / "audio"
assert subtitle_dir == project_root / "subtitles"
assert metadata_dir == project_root / "metadata"
assert audio_dir.is_dir()
assert subtitle_dir.is_dir()
assert metadata_dir is not None and metadata_dir.is_dir()
output_path = _build_output_path(audio_dir, job.original_filename, "wav")
assert output_path == audio_dir / "Sample_Title.wav"
assert root.name == "20260101-000000_book"
assert audio.name == "audio"
assert subs.name == "subtitles"
assert meta.name == "metadata"
+334
View File
@@ -0,0 +1,334 @@
"""Tests for abogen.domain.pronunciation — compile/apply pronunciation rules."""
from __future__ import annotations
import re
import pytest
# ---------------------------------------------------------------------------
# We import the domain functions. The module must be created first.
# For now the tests are written against the expected public API so they can
# serve as the contract during extraction.
# ---------------------------------------------------------------------------
class TestCompilePronunciationRules:
"""compile_pronunciation_rules turns override dicts into regex-based rules."""
def test_empty_input(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
assert compile_pronunciation_rules(None) == []
assert compile_pronunciation_rules([]) == []
def test_single_entry(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [{"token": "albeit", "pronunciation": "all be it"}]
rules = compile_pronunciation_rules(overrides)
assert len(rules) == 1
assert rules[0]["replacement"] == "all be it"
assert rules[0]["pattern"].search("albeit")
def test_skips_entries_without_pronunciation(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [{"token": "hello"}]
assert compile_pronunciation_rules(overrides) == []
def test_skips_entries_without_token(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [{"pronunciation": "foo"}]
assert compile_pronunciation_rules(overrides) == []
def test_deduplication_by_casefold(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [
{"token": "Albeit", "pronunciation": "all be it"},
{"token": "ALBEIT", "pronunciation": "all be it"},
]
rules = compile_pronunciation_rules(overrides)
assert len(rules) == 1
def test_longer_token_sorted_first(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [
{"token": "ice cream", "pronunciation": "ice cream"},
{"token": "ice", "pronunciation": "ais"},
]
rules = compile_pronunciation_rules(overrides)
assert len(rules) == 2
assert len(rules[0]["token"]) >= len(rules[1]["token"])
def test_normalized_fallback_to_entity_token(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [{"normalized": "USA", "pronunciation": "you ess ay"}]
rules = compile_pronunciation_rules(overrides)
assert len(rules) == 1
def test_pattern_is_case_insensitive(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = [{"token": "hello", "pronunciation": "hi"}]
rules = compile_pronunciation_rules(overrides)
assert rules[0]["pattern"].search("Hello")
assert rules[0]["pattern"].search("HELLO")
def test_non_mapping_items_skipped(self):
from abogen.domain.pronunciation import compile_pronunciation_rules
overrides = ["bad", None, 42]
assert compile_pronunciation_rules(overrides) == []
class TestCompileHeteronymSentenceRules:
"""compile_heteronym_sentence_rules builds sentence-level replacements."""
def test_empty_input(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
assert compile_heteronym_sentence_rules(None) == []
assert compile_heteronym_sentence_rules([]) == []
def test_basic_replacement(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [
{
"sentence": "I read the book",
"choice": "past",
"options": [
{"key": "present", "replacement_sentence": "I read the book"},
{"key": "past", "replacement_sentence": "I read the book"},
],
}
]
rules = compile_heteronym_sentence_rules(overrides)
assert len(rules) == 1
assert rules[0]["replacement"] == "I read the book"
def test_skips_without_sentence(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [{"choice": "a", "options": []}]
assert compile_heteronym_sentence_rules(overrides) == []
def test_skips_without_choice(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [{"sentence": "hello", "options": []}]
assert compile_heteronym_sentence_rules(overrides) == []
def test_skips_when_no_matching_option(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [
{
"sentence": "I read the book",
"choice": "past",
"options": [{"key": "present", "replacement_sentence": "I read the book"}],
}
]
assert compile_heteronym_sentence_rules(overrides) == []
def test_deduplication(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
entry = {
"sentence": "I read the book",
"choice": "past",
"options": [{"key": "past", "replacement_sentence": "I red the book"}],
}
rules = compile_heteronym_sentence_rules([entry, entry])
assert len(rules) == 1
def test_longer_sentence_sorted_first(self):
from abogen.domain.pronunciation import compile_heteronym_sentence_rules
overrides = [
{
"sentence": "short",
"choice": "a",
"options": [{"key": "a", "replacement_sentence": "s"}],
},
{
"sentence": "a longer sentence here",
"choice": "b",
"options": [{"key": "b", "replacement_sentence": "l"}],
},
]
rules = compile_heteronym_sentence_rules(overrides)
assert len(rules[0]["pattern"].pattern) >= len(rules[1]["pattern"].pattern)
class TestApplyPronunciationRules:
"""apply_pronunciation_rules applies compiled token-level rules."""
def test_empty_text(self):
from abogen.domain.pronunciation import apply_pronunciation_rules
assert apply_pronunciation_rules("", []) == ""
def test_no_rules(self):
from abogen.domain.pronunciation import apply_pronunciation_rules
assert apply_pronunciation_rules("hello", []) == "hello"
def test_basic_replacement(self):
from abogen.domain.pronunciation import compile_pronunciation_rules, apply_pronunciation_rules
rules = compile_pronunciation_rules([{"token": "albeit", "pronunciation": "all be it"}])
result = apply_pronunciation_rules("albeit it was raining", rules)
assert result == "all be it it was raining"
def test_possessive_preserved(self):
from abogen.domain.pronunciation import compile_pronunciation_rules, apply_pronunciation_rules
rules = compile_pronunciation_rules([{"token": "dog", "pronunciation": "dawg"}])
result = apply_pronunciation_rules("the dog's bone", rules)
assert result == "the dawg's bone"
def test_usage_counter_increments(self):
from abogen.domain.pronunciation import compile_pronunciation_rules, apply_pronunciation_rules
rules = compile_pronunciation_rules([{"token": "hello", "pronunciation": "hi"}])
counter: dict[str, int] = {}
apply_pronunciation_rules("hello hello", rules, usage_counter=counter)
assert counter.get("hello", 0) == 2
def test_case_insensitive_match(self):
from abogen.domain.pronunciation import compile_pronunciation_rules, apply_pronunciation_rules
rules = compile_pronunciation_rules([{"token": "test", "pronunciation": "tst"}])
result = apply_pronunciation_rules("This is a Test", rules)
assert "tst" in result.lower()
class TestApplyHeteronymSentenceRules:
"""apply_heteronym_sentence_rules applies sentence-level replacements."""
def test_empty_text(self):
from abogen.domain.pronunciation import apply_heteronym_sentence_rules
assert apply_heteronym_sentence_rules("", []) == ""
def test_no_rules(self):
from abogen.domain.pronunciation import apply_heteronym_sentence_rules
assert apply_heteronym_sentence_rules("hello", []) == "hello"
def test_basic_replacement(self):
from abogen.domain.pronunciation import (
compile_heteronym_sentence_rules,
apply_heteronym_sentence_rules,
)
rules = compile_heteronym_sentence_rules(
[
{
"sentence": "I read the book",
"choice": "past",
"options": [{"key": "past", "replacement_sentence": "I read the book"}],
}
]
)
result = apply_heteronym_sentence_rules("I read the book.", rules)
assert result == "I read the book."
def test_no_match_left_unchanged(self):
from abogen.domain.pronunciation import (
compile_heteronym_sentence_rules,
apply_heteronym_sentence_rules,
)
rules = compile_heteronym_sentence_rules(
[
{
"sentence": "I read the book",
"choice": "past",
"options": [{"key": "past", "replacement_sentence": "I red the book"}],
}
]
)
result = apply_heteronym_sentence_rules("something else entirely", rules)
assert result == "something else entirely"
class TestMergePronunciationOverrides:
"""merge_pronunciation_overrides consolidates override sources."""
def test_empty_job(self):
from abogen.domain.pronunciation import merge_pronunciation_overrides
class FakeJob:
pronunciation_overrides = None
speakers = None
manual_overrides = None
language = "en"
result = merge_pronunciation_overrides(FakeJob())
assert result == []
def test_pronunciation_overrides_included(self):
from abogen.domain.pronunciation import merge_pronunciation_overrides
class FakeJob:
pronunciation_overrides = [
{"token": "hello", "pronunciation": "hi", "normalized": "hello"}
]
speakers = None
manual_overrides = None
language = "en"
result = merge_pronunciation_overrides(FakeJob())
assert len(result) == 1
assert result[0]["token"] == "hello"
assert result[0]["source"] == "pronunciation"
def test_manual_overrides_win(self):
from abogen.domain.pronunciation import merge_pronunciation_overrides
class FakeJob:
pronunciation_overrides = [
{"token": "hello", "pronunciation": "hi old", "normalized": "hello"}
]
speakers = None
manual_overrides = [
{"token": "hello", "pronunciation": "hi new", "normalized": "hello"}
]
language = "en"
result = merge_pronunciation_overrides(FakeJob())
assert len(result) == 1
assert result[0]["pronunciation"] == "hi new"
assert result[0]["source"] == "manual"
def test_speaker_entries_included(self):
from abogen.domain.pronunciation import merge_pronunciation_overrides
class FakeJob:
pronunciation_overrides = None
speakers = {"narrator": {"token": "war", "pronunciation": "wɔːr"}}
manual_overrides = None
language = "en"
result = merge_pronunciation_overrides(FakeJob())
assert len(result) == 1
assert result[0]["source"] == "speaker"
def test_skips_empty_tokens(self):
from abogen.domain.pronunciation import merge_pronunciation_overrides
class FakeJob:
pronunciation_overrides = [{"token": "", "pronunciation": "foo"}]
speakers = None
manual_overrides = None
language = "en"
result = merge_pronunciation_overrides(FakeJob())
assert result == []
+94
View File
@@ -0,0 +1,94 @@
import sys
from unittest.mock import patch, MagicMock
class TestSelectDevice:
"""Tests for domain.device.select_device."""
def test_returns_mps_on_apple_silicon_when_available(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Darwin"
mock_platform.processor.return_value = "arm"
mock_torch = MagicMock()
mock_torch.backends.mps.is_available.return_value = True
mock_torch.cuda.is_available.return_value = False
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": mock_torch}):
result = select_device()
assert result == "mps"
def test_returns_cpu_on_apple_silicon_when_mps_unavailable(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Darwin"
mock_platform.processor.return_value = "arm"
mock_torch = MagicMock()
mock_torch.backends.mps.is_available.return_value = False
mock_torch.cuda.is_available.return_value = False
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": mock_torch}):
result = select_device()
assert result == "cpu"
def test_returns_cuda_when_available(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Linux"
mock_platform.processor.return_value = "x86_64"
mock_torch = MagicMock()
mock_torch.backends.mps.is_available.return_value = False
mock_torch.cuda.is_available.return_value = True
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": mock_torch}):
result = select_device()
assert result == "cuda"
def test_returns_cpu_when_cuda_unavailable(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Linux"
mock_platform.processor.return_value = "x86_64"
mock_torch = MagicMock()
mock_torch.backends.mps.is_available.return_value = False
mock_torch.cuda.is_available.return_value = False
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": mock_torch}):
result = select_device()
assert result == "cpu"
def test_returns_cpu_when_torch_not_installed(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Linux"
mock_platform.processor.return_value = "x86_64"
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": None}):
result = select_device()
assert result == "cpu"
def test_handles_torch_import_error(self) -> None:
from abogen.domain.device import select_device
mock_platform = MagicMock()
mock_platform.system.return_value = "Windows"
mock_platform.processor.return_value = "AMD64"
with patch("abogen.domain.device._platform", mock_platform), \
patch.dict(sys.modules, {"torch": None}):
result = select_device()
assert result == "cpu"
+101
View File
@@ -0,0 +1,101 @@
"""Tests for split pattern logic (3 identical copies in codebase)."""
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
import pytest
from abogen.domain.split_pattern import get_split_pattern
# --- English always returns \n ---
class TestEnglish:
def test_english_sentence(self):
assert get_split_pattern("a", "Sentence") == "\n"
def test_english_sentence_comma(self):
assert get_split_pattern("a", "Sentence + Comma") == "\n"
def test_english_line(self):
assert get_split_pattern("a", "Line") == "\n"
def test_english_disabled(self):
assert get_split_pattern("a", "Disabled") == "\n"
def test_english_b(self):
assert get_split_pattern("b", "Sentence") == "\n"
# --- CJK languages ---
class TestCJK:
def test_chinese_disabled(self):
pattern = get_split_pattern("z", "Disabled")
assert pattern != "\n"
assert r"\n+" in pattern
def test_chinese_line(self):
pattern = get_split_pattern("z", "Line")
assert pattern != "\n"
assert r"\n+" in pattern
def test_chinese_sentence(self):
pattern = get_split_pattern("z", "Sentence")
assert r"\n+" in pattern
def test_chinese_sentence_comma(self):
pattern = get_split_pattern("z", "Sentence + Comma")
assert r"\n+" in pattern
def test_japanese_disabled(self):
pattern = get_split_pattern("j", "Disabled")
assert pattern != "\n"
assert r"\n+" in pattern
def test_japanese_sentence(self):
pattern = get_split_pattern("j", "Sentence")
assert r"\n+" in pattern
# --- Other languages ---
class TestOtherLanguages:
def test_spanish_sentence(self):
pattern = get_split_pattern("e", "Sentence")
assert r"\n+" in pattern
def test_spanish_line(self):
assert get_split_pattern("e", "Line") == "\n"
def test_spanish_disabled(self):
# canonical: \n+ for non-CJK Disabled
assert get_split_pattern("e", "Disabled") == r"\n+"
def test_french_sentence_comma(self):
pattern = get_split_pattern("f", "Sentence + Comma")
assert r"\n+" in pattern
def test_unknown_lang(self):
pattern = get_split_pattern("x", "Sentence")
assert r"\n+" in pattern
# --- Pattern structure ---
class TestPatternStructure:
def test_sentence_has_lookbehind(self):
pattern = get_split_pattern("e", "Sentence")
assert r"(?<=" in pattern
def test_sentence_comma_has_comma_chars(self):
pattern = get_split_pattern("e", "Sentence + Comma")
assert "," in pattern
def test_cjk_spacing_uses_star(self):
pattern = get_split_pattern("z", "Sentence")
assert r"\s*" in pattern
def test_non_cjk_spacing_uses_plus(self):
pattern = get_split_pattern("e", "Sentence")
assert r"\s+" in pattern
+264
View File
@@ -0,0 +1,264 @@
"""Tests for infrastructure/subtitle_writer.py — SrtWriter, AssWriter, VttWriter."""
from __future__ import annotations
import pytest
from pathlib import Path
from abogen.infrastructure.subtitle_writer import (
AssWriter,
SrtWriter,
SubtitleAlignment,
SubtitleConfig,
SubtitleFormat,
SubtitleMode,
VttWriter,
create_subtitle_writer,
)
# ===================================================================
# SrtWriter._format_time
# ===================================================================
class TestSrtFormatTime:
def test_zero(self):
assert SrtWriter._format_time(0.0) == "00:00:00,000"
def test_simple(self):
assert SrtWriter._format_time(61.5) == "00:01:01,500"
def test_hours(self):
assert SrtWriter._format_time(3661.123) == "01:01:01,123"
def test_large(self):
assert SrtWriter._format_time(7384.0) == "02:03:04,000"
def test_fractional_seconds(self):
assert SrtWriter._format_time(0.999) == "00:00:00,999"
def test_matches_old_format_timestamp(self):
"""Verify matches old _format_timestamp(ass=False) from conversion_runner."""
import math
for t in [0.0, 61.5, 3661.123, 7384.0, 0.999, 125.7]:
h = int(t // 3600)
m = int((t % 3600) // 60)
s = int(t % 60)
ms = int((t - math.floor(t)) * 1000)
expected = f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"
assert SrtWriter._format_time(t) == expected, f"Mismatch at t={t}"
# ===================================================================
# AssWriter._format_time
# ===================================================================
class TestAssFormatTime:
def test_zero(self):
assert AssWriter._format_time(0.0) == "0:00:00.00"
def test_simple(self):
assert AssWriter._format_time(61.5) == "0:01:01.50"
def test_hours(self):
assert AssWriter._format_time(3661.12) == "1:01:01.12"
def test_centiseconds(self):
assert AssWriter._format_time(1.55) == "0:00:01.55"
def test_matches_old_format_timestamp_ass(self):
"""Verify matches old _format_timestamp(ass=True) from conversion_runner.
Note: the old code used int(milliseconds/10) which truncates centiseconds,
while the new code uses float formatting which rounds. For most values they
match; the difference is at most 1 centisecond due to float precision.
"""
import math
for t in [0.0, 61.5, 1.55, 125.7]:
h = int(t // 3600)
m = int((t % 3600) // 60)
s = int(t % 60)
ms = int((t - math.floor(t)) * 1000)
cs = int(ms / 10)
expected = f"{h:01d}:{m:02d}:{s:02d}.{cs:02d}"
assert AssWriter._format_time(t) == expected, f"Mismatch at t={t}"
# ===================================================================
# SrtWriter full write
# ===================================================================
class TestSrtWriter:
def test_single_entry(self, tmp_path):
path = tmp_path / "test.srt"
writer = SrtWriter(path, SubtitleConfig(format=SubtitleFormat.SRT, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=2.5, text="Hello")
writer.close()
content = path.read_text()
assert "1\n" in content
assert "00:00:00,000 --> 00:00:02,500" in content
assert "Hello\n" in content
def test_multiple_entries(self, tmp_path):
path = tmp_path / "test.srt"
writer = SrtWriter(path, SubtitleConfig(format=SubtitleFormat.SRT, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=1.0, text="First")
writer.write_entry(start=1.0, end=2.0, text="Second")
writer.close()
content = path.read_text()
assert "1\n" in content
assert "2\n" in content
assert "First" in content
assert "Second" in content
def test_voice_prefix(self, tmp_path):
path = tmp_path / "test.srt"
writer = SrtWriter(path, SubtitleConfig(format=SubtitleFormat.SRT, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=1.0, text="Hello", voice="af_heart")
writer.close()
content = path.read_text()
assert "[af_heart] Hello" in content
def test_auto_index(self, tmp_path):
path = tmp_path / "test.srt"
writer = SrtWriter(path, SubtitleConfig(format=SubtitleFormat.SRT, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=1.0, text="A")
writer.write_entry(start=1.0, end=2.0, text="B")
writer.write_entry(start=2.0, end=3.0, text="C")
writer.close()
content = path.read_text()
assert "1\n" in content
assert "2\n" in content
assert "3\n" in content
# ===================================================================
# AssWriter full write
# ===================================================================
class TestAssWriter:
def test_header_structure(self, tmp_path):
path = tmp_path / "test.ass"
writer = AssWriter(path, SubtitleConfig(format=SubtitleFormat.ASS, mode=SubtitleMode.LINE))
writer.open()
writer.close()
content = path.read_text()
assert "[Script Info]" in content
assert "[V4+ Styles]" in content
assert "[Events]" in content
assert "Format: Layer, Start, End" in content
def test_single_entry(self, tmp_path):
path = tmp_path / "test.ass"
writer = AssWriter(path, SubtitleConfig(format=SubtitleFormat.ASS, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=2.5, text="Hello")
writer.close()
content = path.read_text()
assert "Dialogue:" in content
assert "Hello" in content
def test_voice_prefix(self, tmp_path):
path = tmp_path / "test.ass"
writer = AssWriter(path, SubtitleConfig(format=SubtitleFormat.ASS, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=1.0, text="Hello", voice="af_heart")
writer.close()
content = path.read_text()
assert "[af_heart] Hello" in content
def test_highlight_mode(self, tmp_path):
path = tmp_path / "test.ass"
config = SubtitleConfig(
format=SubtitleFormat.ASS,
mode=SubtitleMode.SENTENCE_HIGHLIGHT,
)
writer = AssWriter(path, config)
writer.write_entry(start=0.0, end=1.0, text="Hello world")
writer.close()
content = path.read_text()
assert "Highlight" in content
assert r"{\k100}" in content
def test_centered_alignment(self, tmp_path):
path = tmp_path / "test.ass"
config = SubtitleConfig(
format=SubtitleFormat.ASS,
mode=SubtitleMode.LINE,
alignment=SubtitleAlignment.CENTER,
)
writer = AssWriter(path, config)
writer.open()
writer.close()
content = path.read_text()
# Centered uses alignment=5
assert ",5," in content or ",5\n" in content
# ===================================================================
# VttWriter
# ===================================================================
class TestVttWriter:
def test_header(self, tmp_path):
path = tmp_path / "test.vtt"
writer = VttWriter(path, SubtitleConfig(format=SubtitleFormat.VTT, mode=SubtitleMode.LINE))
writer.open()
writer.close()
content = path.read_text()
assert content.startswith("WEBVTT")
def test_single_entry(self, tmp_path):
path = tmp_path / "test.vtt"
writer = VttWriter(path, SubtitleConfig(format=SubtitleFormat.VTT, mode=SubtitleMode.LINE))
writer.write_entry(start=0.0, end=2.5, text="Hello")
writer.close()
content = path.read_text()
assert "1\n" in content
assert "Hello" in content
# ===================================================================
# create_subtitle_writer factory
# ===================================================================
class TestCreateSubtitleWriter:
def test_srt(self, tmp_path):
path = tmp_path / "test.srt"
writer = create_subtitle_writer(path, "srt", "Line")
assert isinstance(writer, SrtWriter)
writer.close()
def test_ass(self, tmp_path):
path = tmp_path / "test.ass"
writer = create_subtitle_writer(path, "ass", "Line")
assert isinstance(writer, AssWriter)
writer.close()
def test_vtt(self, tmp_path):
path = tmp_path / "test.vtt"
writer = create_subtitle_writer(path, "vtt", "Line")
assert isinstance(writer, VttWriter)
writer.close()
def test_unsupported_raises(self, tmp_path):
path = tmp_path / "test.xyz"
with pytest.raises(ValueError):
create_subtitle_writer(path, "xyz", "Line")
# ===================================================================
# Context manager
# ===================================================================
class TestContextManager:
def test_srt_context_manager(self, tmp_path):
path = tmp_path / "test.srt"
with SrtWriter(path, SubtitleConfig(format=SubtitleFormat.SRT, mode=SubtitleMode.LINE)) as writer:
writer.write_entry(start=0.0, end=1.0, text="Hello")
content = path.read_text()
assert "Hello" in content
def test_ass_context_manager(self, tmp_path):
path = tmp_path / "test.ass"
with AssWriter(path, SubtitleConfig(format=SubtitleFormat.ASS, mode=SubtitleMode.LINE)) as writer:
writer.write_entry(start=0.0, end=1.0, text="Hello")
content = path.read_text()
assert "Dialogue:" in content
+59
View File
@@ -0,0 +1,59 @@
"""Tests for domain/title_builder.py."""
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from abogen.domain.title_builder import build_title_intro_text, build_outro_text
class TestBuildTitleIntroText:
def test_empty_metadata(self):
result = build_title_intro_text({}, "book.epub")
assert result == "book."
def test_title_from_metadata(self):
result = build_title_intro_text({"title": "My Book"}, "book.epub")
assert result == "My Book."
def test_title_fallback_basename(self):
result = build_title_intro_text({}, "my_book.epub")
assert result == "my_book."
def test_with_author(self):
result = build_title_intro_text({"title": "My Book", "author": "John Doe"}, "book.epub")
assert result == "My Book. By John Doe."
def test_with_subtitle(self):
result = build_title_intro_text({"title": "My Book", "subtitle": "A Tale"}, "book.epub")
assert result == "My Book. A Tale."
def test_duplicate_title_subtitle(self):
result = build_title_intro_text({"title": "My Book", "subtitle": "My Book"}, "book.epub")
assert result == "My Book."
def test_with_series(self):
result = build_title_intro_text({"title": "My Book", "series": "Series", "series_index": "3"}, "book.epub")
assert result == "Book 3 of the Series. My Book."
class TestBuildOutroText:
def test_empty(self):
result = build_outro_text({}, "book.epub")
assert result == "The end of book."
def test_title_only(self):
result = build_outro_text({"title": "My Book"}, "book.epub")
assert result == "The end of My Book."
def test_author_only(self):
result = build_outro_text({"author": "John Doe"}, "book.epub")
assert result == "The end of book from John Doe."
def test_title_and_author(self):
result = build_outro_text({"title": "My Book", "author": "John Doe"}, "book.epub")
assert result == "The end of My Book from John Doe."
def test_with_series(self):
result = build_outro_text({"title": "My Book", "series": "Series", "series_index": "3"}, "book.epub")
assert "The end of My Book." in result
assert "Book 3 of the Series." in result
+139
View File
@@ -0,0 +1,139 @@
import sys
import types
if "soundfile" not in sys.modules:
soundfile_stub = types.ModuleType("soundfile")
class _SoundFileStub: # pragma: no cover - placeholder to satisfy imports
def __init__(self, *args: object, **kwargs: object) -> None:
raise RuntimeError("soundfile is not installed in the test environment")
soundfile_stub.SoundFile = _SoundFileStub # type: ignore[attr-defined]
sys.modules["soundfile"] = soundfile_stub
if "static_ffmpeg" not in sys.modules:
sys.modules["static_ffmpeg"] = types.ModuleType("static_ffmpeg")
if "ebooklib" not in sys.modules:
ebooklib_stub = types.ModuleType("ebooklib")
ebooklib_epub_stub = types.ModuleType("ebooklib.epub")
ebooklib_stub.epub = ebooklib_epub_stub # type: ignore[attr-defined]
sys.modules["ebooklib"] = ebooklib_stub
sys.modules["ebooklib.epub"] = ebooklib_epub_stub
if "fitz" not in sys.modules:
sys.modules["fitz"] = types.ModuleType("fitz")
if "markdown" not in sys.modules:
markdown_stub = types.ModuleType("markdown")
class _MarkdownStub:
def __init__(self, *args: object, **kwargs: object) -> None:
self.toc_tokens = []
def convert(self, text: str) -> str:
return text
markdown_stub.Markdown = _MarkdownStub # type: ignore[attr-defined]
sys.modules["markdown"] = markdown_stub
if "bs4" not in sys.modules:
bs4_stub = types.ModuleType("bs4")
class _BeautifulSoupStub:
def __init__(self, *args: object, **kwargs: object) -> None:
self._text = ""
def find(self, *args: object, **kwargs: object) -> None:
return None
def get_text(self) -> str:
return self._text
def decompose(self) -> None: # pragma: no cover - compatibility shim
return None
class _NavigableStringStub(str):
pass
bs4_stub.BeautifulSoup = _BeautifulSoupStub # type: ignore[attr-defined]
bs4_stub.NavigableString = _NavigableStringStub # type: ignore[attr-defined]
sys.modules["bs4"] = bs4_stub
from unittest.mock import patch, MagicMock
class TestResolveFallbackVoiceSpec:
"""Tests for the voice fallback resolution helper."""
def test_uses_base_voice_spec(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="af_heart",
job_voice="af_bella",
voice_cache_keys=[],
)
assert result == "af_heart"
def test_falls_back_to_job_voice(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="",
job_voice="af_bella",
voice_cache_keys=[],
)
assert result == "af_bella"
def test_skips_custom_mix_uses_job_voice(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="__custom_mix",
job_voice="af_bella",
voice_cache_keys=[],
)
assert result == "af_bella"
def test_falls_back_to_voice_cache(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="",
job_voice="",
voice_cache_keys=["kokoro:af_heart"],
)
assert result == "af_heart"
def test_skips_custom_mix_in_cache(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="",
job_voice="",
voice_cache_keys=["__custom_mix", "kokoro:af_heart"],
)
assert result == "af_heart"
def test_falls_back_to_default_voice(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
with patch("abogen.domain.voice_resolution.get_default_voice", return_value="af_heart"):
result = resolve_fallback_voice_spec(
base_spec="",
job_voice="",
voice_cache_keys=[],
)
assert result == "af_heart"
def test_empty_base_and_job_with_cache(self) -> None:
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
result = resolve_fallback_voice_spec(
base_spec="",
job_voice="",
voice_cache_keys=["kokoro:af_bella", "kokoro:af_heart"],
)
assert result == "af_bella"
+274
View File
@@ -0,0 +1,274 @@
"""Tests for voice resolution helpers.
Tests import from domain.voice_resolution (new location).
"""
from __future__ import annotations
from types import SimpleNamespace
from typing import Any, Dict
from unittest.mock import patch
import pytest
# ---------------------------------------------------------------------------
# spec_to_voice_ids
# ---------------------------------------------------------------------------
class TestSpecToVoiceIds:
"""spec_to_voice_ids extracts voice identifiers from a spec string."""
def test_empty_string(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
assert spec_to_voice_ids("") == set()
def test_none(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
assert spec_to_voice_ids(None) == set()
def test_custom_mix_returns_empty(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
assert spec_to_voice_ids("__custom_mix") == set()
def test_single_known_voice(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
with patch("abogen.domain.voice_resolution.get_voices", return_value={"af_heart"}):
assert spec_to_voice_ids("af_heart") == {"af_heart"}
def test_unknown_single_voice_returns_empty(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
with patch("abogen.domain.voice_resolution.get_voices", return_value=set()):
assert spec_to_voice_ids("nonexistent") == set()
def test_formula_with_star(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
with patch("abogen.domain.voice_resolution.extract_voice_ids", return_value=["v1", "v2"]):
result = spec_to_voice_ids("v1*v2")
assert result == {"v1", "v2"}
def test_formula_value_error_returns_empty(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
with patch("abogen.domain.voice_resolution.extract_voice_ids", side_effect=ValueError("bad")):
assert spec_to_voice_ids("bad*spec") == set()
def test_whitespace_stripped(self):
from abogen.domain.voice_resolution import spec_to_voice_ids
assert spec_to_voice_ids(" ") == set()
# ---------------------------------------------------------------------------
# job_voice_fallback
# ---------------------------------------------------------------------------
class TestJobVoiceFallback:
"""job_voice_fallback resolves a fallback voice from job attributes."""
def test_direct_voice(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(voice="af_heart", speakers=None, chapters=[])
assert job_voice_fallback(job) == "af_heart"
def test_custom_mix_ignored(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(voice="__custom_mix", speakers=None, chapters=[])
assert job_voice_fallback(job) == ""
def test_narrator_speaker(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(
voice="__custom_mix",
speakers={"narrator": {"resolved_voice": "af_heart"}},
chapters=[],
)
assert job_voice_fallback(job) == "af_heart"
def test_speaker_voice_formula(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(
voice="",
speakers={"speaker1": {"voice_formula": "v1*v2"}},
chapters=[],
)
assert job_voice_fallback(job) == "v1*v2"
def test_chapter_voice(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(
voice="",
speakers=None,
chapters=[{"resolved_voice": "af_bella"}],
)
assert job_voice_fallback(job) == "af_bella"
def test_empty_job(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(voice="", speakers=None, chapters=[])
assert job_voice_fallback(job) == ""
def test_narrator_custom_mix_falls_through(self):
from abogen.domain.voice_resolution import job_voice_fallback
job = SimpleNamespace(
voice="",
speakers={"narrator": {"voice": "__custom_mix"}},
chapters=[{"voice": "af_heart"}],
)
assert job_voice_fallback(job) == "af_heart"
# ---------------------------------------------------------------------------
# chapter_voice_spec
# ---------------------------------------------------------------------------
class TestChapterVoiceSpec:
"""chapter_voice_spec resolves voice for a chapter override."""
def test_no_override_uses_fallback(self):
from abogen.domain.voice_resolution import chapter_voice_spec
job = SimpleNamespace(voice="af_heart", speakers=None, chapters=[])
assert chapter_voice_spec(job, None) == "af_heart"
def test_resolved_voice_wins(self):
from abogen.domain.voice_resolution import chapter_voice_spec
job = SimpleNamespace(voice="af_heart", speakers=None, chapters=[])
override = {"resolved_voice": "af_bella", "voice_formula": "x", "voice": "y"}
assert chapter_voice_spec(job, override) == "af_bella"
def test_formula_second(self):
from abogen.domain.voice_resolution import chapter_voice_spec
job = SimpleNamespace(voice="", speakers=None, chapters=[])
override = {"voice_formula": "v1*v2", "voice": "y"}
assert chapter_voice_spec(job, override) == "v1*v2"
def test_voice_third(self):
from abogen.domain.voice_resolution import chapter_voice_spec
job = SimpleNamespace(voice="", speakers=None, chapters=[])
override = {"voice": "af_nicole"}
assert chapter_voice_spec(job, override) == "af_nicole"
def test_empty_override_falls_to_fallback(self):
from abogen.domain.voice_resolution import chapter_voice_spec
job = SimpleNamespace(voice="af_heart", speakers=None, chapters=[])
assert chapter_voice_spec(job, {}) == "af_heart"
# ---------------------------------------------------------------------------
# chunk_voice_spec
# ---------------------------------------------------------------------------
class TestChunkVoiceSpec:
"""chunk_voice_spec resolves voice for a TTS chunk."""
def test_chunk_direct_voice(self):
from abogen.domain.voice_resolution import chunk_voice_spec
job = SimpleNamespace(speakers=None)
chunk = {"resolved_voice": "af_heart"}
assert chunk_voice_spec(job, chunk, "fallback") == "af_heart"
def test_chunk_speaker_lookup(self):
from abogen.domain.voice_resolution import chunk_voice_spec
job = SimpleNamespace(speakers={"narrator": {"resolved_voice": "af_bella"}})
chunk = {"speaker_id": "narrator"}
assert chunk_voice_spec(job, chunk, "") == "af_bella"
def test_chunk_voice_profile_lookup(self):
from abogen.domain.voice_resolution import chunk_voice_spec
job = SimpleNamespace(speakers={"角色A": {"voice": "af_nicole"}})
chunk = {"voice_profile": "角色A"}
assert chunk_voice_spec(job, chunk, "") == "af_nicole"
def test_uses_fallback_string(self):
from abogen.domain.voice_resolution import chunk_voice_spec
job = SimpleNamespace(speakers=None)
chunk = {}
assert chunk_voice_spec(job, chunk, "my_fallback") == "my_fallback"
def test_fallback_to_job(self):
from abogen.domain.voice_resolution import chunk_voice_spec
job = SimpleNamespace(voice="af_heart", speakers=None, chapters=[])
chunk = {}
assert chunk_voice_spec(job, chunk, "") == "af_heart"
# ---------------------------------------------------------------------------
# collect_required_voice_ids
# ---------------------------------------------------------------------------
class TestCollectRequiredVoiceIds:
"""collect_required_voice_ids gathers all voice IDs from a job."""
def test_includes_job_voice(self):
from abogen.domain.voice_resolution import collect_required_voice_ids
job = SimpleNamespace(voice="af_heart", chapters=[], chunks=[], speakers={})
with patch("abogen.domain.voice_resolution.get_voices", return_value={"af_heart"}), \
patch("abogen.domain.voice_resolution.job_voice_fallback", return_value=""):
result = collect_required_voice_ids(job)
assert "af_heart" in result
def test_includes_chapter_voices(self):
from abogen.domain.voice_resolution import collect_required_voice_ids
job = SimpleNamespace(
voice="",
chapters=[{"resolved_voice": "af_bella"}],
chunks=[],
speakers={},
)
with patch("abogen.domain.voice_resolution.get_voices", return_value={"af_bella"}), \
patch("abogen.domain.voice_resolution.job_voice_fallback", return_value=""):
result = collect_required_voice_ids(job)
assert "af_bella" in result
def test_includes_chunk_voices(self):
from abogen.domain.voice_resolution import collect_required_voice_ids
job = SimpleNamespace(
voice="",
chapters=[],
chunks=[{"voice": "af_nicole"}],
speakers={},
)
with patch("abogen.domain.voice_resolution.get_voices", return_value={"af_nicole"}), \
patch("abogen.domain.voice_resolution.job_voice_fallback", return_value=""):
result = collect_required_voice_ids(job)
assert "af_nicole" in result
def test_always_includes_kokoro_voices(self):
from abogen.domain.voice_resolution import collect_required_voice_ids
job = SimpleNamespace(voice="", chapters=[], chunks=[], speakers={})
with patch("abogen.domain.voice_resolution.get_voices", return_value={"af_heart", "af_bella"}), \
patch("abogen.domain.voice_resolution.job_voice_fallback", return_value=""):
result = collect_required_voice_ids(job)
assert {"af_heart", "af_bella"}.issubset(result)
+118
View File
@@ -0,0 +1,118 @@
"""Tests for domain/voice_utils.py."""
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
from abogen.domain.voice_utils import (
infer_provider_from_spec,
supertonic_voice_from_spec,
split_speaker_reference,
formula_from_kokoro_entry,
coerce_truthy,
)
class TestInferProviderFromSpec:
def test_empty_returns_fallback(self):
assert infer_provider_from_spec("", "kokoro") == "kokoro"
def test_supertonic_uppercase(self):
assert infer_provider_from_spec("M1", "kokoro") == "supertonic"
def test_kokoro_voice(self):
assert infer_provider_from_spec("af_bella", "kokoro") == "kokoro"
def test_custom_mix(self):
assert infer_provider_from_spec("__custom_mix", "kokoro") == "kokoro"
def test_formula(self):
assert infer_provider_from_spec("af_bella*0.5+am_adam*0.5", "kokoro") == "kokoro"
class TestSupertonicVoiceFromSpec:
def test_normal(self):
assert supertonic_voice_from_spec("m1", "m2") == "M1"
def test_empty_uses_fallback(self):
assert supertonic_voice_from_spec("", "m2") == "M2"
def test_formula_uses_fallback(self):
assert supertonic_voice_from_spec("m1*0.5", "m2") == "M2"
def test_both_empty_uses_default(self):
assert supertonic_voice_from_spec("", "") == "M1"
class TestSplitSpeakerReference:
def test_speaker(self):
name, original = split_speaker_reference("speaker:John")
assert name == "John"
assert original == "speaker:John"
def test_profile(self):
name, original = split_speaker_reference("profile:Main")
assert name == "Main"
assert original == "profile:Main"
def test_invalid_prefix(self):
name, original = split_speaker_reference("voice:John")
assert name is None
assert original == "voice:John"
def test_no_colon(self):
name, original = split_speaker_reference("John")
assert name is None
assert original == "John"
def test_empty(self):
name, original = split_speaker_reference("")
assert name is None
assert original == ""
class TestFormulaFromKokoroEntry:
def test_normal(self):
entry = {"voices": [["af_bella", 0.5], ["am_adam", 0.5]]}
result = formula_from_kokoro_entry(entry)
assert "af_bella" in result
assert "am_adam" in result
def test_empty(self):
assert formula_from_kokoro_entry({}) == ""
def test_invalid_items(self):
entry = {"voices": [["af_bella", "invalid"], ["am_adam", 0.5]]}
result = formula_from_kokoro_entry(entry)
assert "am_adam" in result
assert "af_bella" not in result
class TestCoerceTruthy:
def test_bool_true(self):
assert coerce_truthy(True) is True
def test_bool_false(self):
assert coerce_truthy(False) is False
def test_string_true(self):
assert coerce_truthy("true") is True
assert coerce_truthy("yes") is True
assert coerce_truthy("1") is True
assert coerce_truthy("on") is True
def test_string_false(self):
assert coerce_truthy("false") is False
assert coerce_truthy("no") is False
assert coerce_truthy("0") is False
assert coerce_truthy("off") is False
assert coerce_truthy("") is False
def test_none_default_true(self):
assert coerce_truthy(None, True) is True
def test_none_default_false(self):
assert coerce_truthy(None, False) is False
def test_int(self):
assert coerce_truthy(1) is True
assert coerce_truthy(0) is False
+53
View File
@@ -0,0 +1,53 @@
from abogen.webui.app import create_app
def _large_chapter_form() -> dict[str, str]:
data = {"step": "chapters"}
for index in range(370):
prefix = f"chapter-{index}"
data[f"{prefix}-enabled"] = "on"
data[f"{prefix}-title"] = f"Chapter {index} " + ("x" * 1400)
data[f"{prefix}-voice"] = "af_heart"
data[f"{prefix}-formula"] = "default"
return data
def test_large_chapter_form_reaches_wizard_route(tmp_path):
app = create_app(
{
"TESTING": True,
"SECRET_KEY": "test",
"OUTPUT_FOLDER": str(tmp_path / "output"),
"UPLOAD_FOLDER": str(tmp_path / "uploads"),
}
)
with app.test_client() as client:
response = client.post(
"/wizard/update?format=json",
data=_large_chapter_form(),
)
assert response.status_code == 400
assert response.get_json()["error"] == "Missing job ID"
def test_large_multipart_chapter_form_reaches_wizard_route(tmp_path):
app = create_app(
{
"TESTING": True,
"SECRET_KEY": "test",
"OUTPUT_FOLDER": str(tmp_path / "output"),
"UPLOAD_FOLDER": str(tmp_path / "uploads"),
}
)
with app.test_client() as client:
response = client.post(
"/wizard/update?format=json",
data=_large_chapter_form(),
content_type="multipart/form-data",
)
assert response.status_code == 400
assert response.get_json()["error"] == "Missing job ID"