mirror of
https://github.com/denizsafak/abogen.git
synced 2026-09-20 19:50:59 +02:00
Compare commits
50
Commits
9201f58770
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
08e2ee8b85 | ||
|
|
be74c69507 | ||
|
|
ffac4a4da9 | ||
|
|
5432de7ac5 | ||
|
|
823f5be029 | ||
|
|
aaa6ac112b | ||
|
|
11274ad6bf | ||
|
|
f340b976db | ||
|
|
9da15aefa4 | ||
|
|
919aea9295 | ||
|
|
ce1fc0c880 | ||
|
|
94e6b3f62e | ||
|
|
d334266238 | ||
|
|
2d18501839 | ||
|
|
464bf8e17d | ||
|
|
9bf4f8e809 | ||
|
|
f802fb2af6 | ||
|
|
c293cc90f6 | ||
|
|
696ce1ebd0 | ||
|
|
d3ded8af0e | ||
|
|
c706f7714a | ||
|
|
2b70b9ca45 | ||
|
|
953bef1e71 | ||
|
|
146cc81271 | ||
|
|
61204cc389 | ||
|
|
f7a224cc46 | ||
|
|
98ab2d925e | ||
|
|
625b6610e2 | ||
|
|
0dc491e420 | ||
|
|
713abdfd73 | ||
|
|
73f42e9563 | ||
|
|
2c61f55f81 | ||
|
|
6497e8c47a | ||
|
|
0b953d48e8 | ||
|
|
f516cf1985 | ||
|
|
3857c27aae | ||
|
|
7ed2addb11 | ||
|
|
cfc7de7abf | ||
|
|
654c395943 | ||
|
|
d1a84cfb8b | ||
|
|
332934c0cf | ||
|
|
c79838a5a1 | ||
|
|
d51a9118e4 | ||
|
|
3311bef2f7 | ||
|
|
4123cadd87 | ||
|
|
2f83d10a1e | ||
|
|
a1241ee9ca | ||
|
|
0ee5bb0496 | ||
|
|
7d28b7eb52 | ||
|
|
7340d52ebb |
@@ -40,3 +40,6 @@ test_assets/
|
|||||||
dev_notes/
|
dev_notes/
|
||||||
.claude/
|
.claude/
|
||||||
.coverage
|
.coverage
|
||||||
|
|
||||||
|
# CodeGraph index (local, machine-specific)
|
||||||
|
.codegraph/
|
||||||
|
|||||||
@@ -0,0 +1,82 @@
|
|||||||
|
# AGENTS.md — Segmentation & Subtitle System Contract
|
||||||
|
|
||||||
|
This document is the source of truth for how text is split for **voice
|
||||||
|
processing** (TTS engine segmentation) and **subtitle processing**, across
|
||||||
|
languages, TTS engines, and subtitle modes. It was written after a bug where
|
||||||
|
sentence modes "processed all text as a whole" (one merged engine segment →
|
||||||
|
one giant subtitle). **Do not change this behavior without updating this
|
||||||
|
table.**
|
||||||
|
|
||||||
|
## Voice processing — split pattern passed to the TTS engine
|
||||||
|
|
||||||
|
`get_split_pattern(language, mode)` in `abogen/domain/split_pattern.py` is the
|
||||||
|
default; the spaCy pre-TTS path overrides it. Both UIs must stay in sync:
|
||||||
|
`spacy_pre_tts_segmentation` (`abogen/domain/conversion_pipeline.py`, WebUI)
|
||||||
|
and the inline branch in `abogen/pyqt/conversion.py` (~line 860, PyQt).
|
||||||
|
|
||||||
|
| Subtitle mode | English (en-US/en-GB) | Non-English, spaCy ON | Non-English, spaCy OFF | CJK (ja/zh) |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| Disabled | `\n` | spaCy pre-split, engine `\n` | `\n+` | `(?<=[.!?؟。!?।])\s*\|\n+` |
|
||||||
|
| Line | `\n` | spaCy pre-split, engine `\n` | `\n` | `(?<=[.!?؟。!?।])\s*\|\n+` |
|
||||||
|
| Sentence | `\n` | spaCy pre-split, engine `\n` | `(?<=[.!?؟。!?।])\s+\|\n+` | `(?<=[.!?؟。!?।])\s*\|\n+` |
|
||||||
|
| Sentence + Comma | `\n` | spaCy pre-split, engine `\n` | `(?<=[.!?,؟。!?،،、।])\s+\|\n+` (commas kept) | `(?<=[.!?,؟。!?،،、।])\s*\|\n+` |
|
||||||
|
| Sentence + Highlighting | `\n+` | `\n+` | `\n+` | `\n+` |
|
||||||
|
| N words ("5 words") | `\n` (→ Disabled) | `\n+` | `\n+` | Disabled CJK pattern |
|
||||||
|
|
||||||
|
Rules baked into this table:
|
||||||
|
|
||||||
|
- **English voice splitting is ALWAYS newline-only** for Disabled, Line,
|
||||||
|
Sentence, and Sentence + Comma. English sentence/comma boundaries are
|
||||||
|
produced ONLY at subtitle time (spaCy post-TTS / regex fallback). Never add
|
||||||
|
punctuation to the English engine pattern.
|
||||||
|
- **Non-English + spaCy ON**: spaCy pre-segments the text (pre-TTS); the
|
||||||
|
engine pattern is `\n` for Sentence AND Sentence + Comma — **never commas**.
|
||||||
|
spaCy is skipped when the toggle is off, mode is Disabled/Line, or input is
|
||||||
|
a subtitle file.
|
||||||
|
- **Non-English + spaCy OFF** (toggle off, spaCy failure, subtitle input): the
|
||||||
|
default pattern is used — Sentence + Comma KEEPS its commas here. This is
|
||||||
|
the intentional fallback, not a bug.
|
||||||
|
- CJK: punctuation-based patterns for Disabled/Line (historical); spacing is
|
||||||
|
`\s*` (no spaces needed between CJK chars).
|
||||||
|
- Engine-level extra chunking (applies after the pattern): kokoro English
|
||||||
|
re-chunks at ~510 phonemes; kokoro non-English at ~400 chars; supertonic
|
||||||
|
caps each part at 300 chars.
|
||||||
|
|
||||||
|
## Subtitle processing — post-TTS, from tokens
|
||||||
|
|
||||||
|
| Mode | Behavior |
|
||||||
|
|---|---|
|
||||||
|
| Disabled | no subtitles |
|
||||||
|
| Line | one entry per TTS segment (line) |
|
||||||
|
| Sentence | sentence boundaries: English → spaCy; others → regex on `[.!?…]` |
|
||||||
|
| Sentence + Comma | sentence + comma boundaries at subtitle time (both languages) — commas never affect voice |
|
||||||
|
| Sentence + Highlighting | karaoke `{\kf…}` per word, grouped by sentence |
|
||||||
|
| N words | groups of N words by whitespace counting |
|
||||||
|
|
||||||
|
Token granularity (timing quality): kokoro English emits **per-word tokens**
|
||||||
|
with timestamps; kokoro non-English and supertonic emit **no tokens** → each
|
||||||
|
engine segment becomes one FakeToken, split by regex with proportional timing
|
||||||
|
when it contains multiple sentences.
|
||||||
|
|
||||||
|
## Hard invariants (breaking these reintroduces the original bug)
|
||||||
|
|
||||||
|
1. `Pipeline.__call__` (`abogen/tts_plugin/utils.py`) must yield ONE `Segment`
|
||||||
|
per engine segment (with tokens) — never merge segments back into the
|
||||||
|
whole text. `SynthesizedAudio.segments` carries the per-segment data;
|
||||||
|
engines expose it in `plugins/kokoro/engine.py` and
|
||||||
|
`plugins/supertonic/engine.py`.
|
||||||
|
2. `tts_segments` (`abogen/domain/conversion_pipeline.py`) restores trailing
|
||||||
|
whitespace on segment-boundary tokens ONLY for real per-word tokens, never
|
||||||
|
for FakeToken fallbacks.
|
||||||
|
3. `_to_language_enum` must return `lang_code` as-is when it is already a
|
||||||
|
`Language` enum (`str(Language.ES)` is `"Language.ES"`, which silently
|
||||||
|
resolved to EN_US and disabled spaCy pre-TTS for every language in WebUI).
|
||||||
|
4. English must never use spaCy for PRE-TTS segmentation — only for subtitles.
|
||||||
|
|
||||||
|
## Guarded by tests
|
||||||
|
|
||||||
|
- `tests/test_split_pattern.py` — English newline-only; non-English sentence
|
||||||
|
patterns; CJK behavior.
|
||||||
|
- `tests/test_domain_conversion_pipeline.py` — `tts_segments` / spaCy
|
||||||
|
segmentation helpers.
|
||||||
|
- Full suite: `python -m pytest tests/ -q` (expect 1566+ passing).
|
||||||
@@ -721,7 +721,7 @@ This project is available under the MIT License - see the [LICENSE](https://gith
|
|||||||
[Kokoro](https://github.com/hexgrad/kokoro) is licensed under [Apache-2.0](https://github.com/hexgrad/kokoro/blob/main/LICENSE) which allows commercial use, modification, distribution, and private use.
|
[Kokoro](https://github.com/hexgrad/kokoro) is licensed under [Apache-2.0](https://github.com/hexgrad/kokoro/blob/main/LICENSE) which allows commercial use, modification, distribution, and private use.
|
||||||
|
|
||||||
## `Star History`
|
## `Star History`
|
||||||
[](https://www.star-history.com/#denizsafak/abogen&Date)
|
[](https://star-history.dera.page/#denizsafak/abogen&Date)
|
||||||
|
|
||||||
> [!NOTE]
|
> [!NOTE]
|
||||||
> Abogen supports subtitle generation for all languages. However, word-level subtitle modes (e.g., "1 word", "2 words", "3 words", etc.) are only available for English because [Kokoro provides timestamp tokens only for English text](https://github.com/hexgrad/kokoro/blob/6d87f4ae7abc2d14dbc4b3ef2e5f19852e861ac2/kokoro/pipeline.py#L383). For non-English languages, Abogen uses a duration-based fallback that supports sentence-level and comma-based subtitle modes ("Line", "Sentence", "Sentence + Comma"). If you need word-level subtitles for other languages, please request that feature in the [Kokoro project](https://github.com/hexgrad/kokoro).
|
> Abogen supports subtitle generation for all languages. However, word-level subtitle modes (e.g., "1 word", "2 words", "3 words", etc.) are only available for English because [Kokoro provides timestamp tokens only for English text](https://github.com/hexgrad/kokoro/blob/6d87f4ae7abc2d14dbc4b3ef2e5f19852e861ac2/kokoro/pipeline.py#L383). For non-English languages, Abogen uses a duration-based fallback that supports sentence-level and comma-based subtitle modes ("Line", "Sentence", "Sentence + Comma"). If you need word-level subtitles for other languages, please request that feature in the [Kokoro project](https://github.com/hexgrad/kokoro).
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
"""Chapter selection helpers for the application layer.
|
||||||
|
|
||||||
|
Builds chapter payloads with smart defaults (preselection based on
|
||||||
|
supplement score) and character counts. Used by both WebUI and PyQt.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Dict, List
|
||||||
|
|
||||||
|
from abogen.domain.chapter_classification import (
|
||||||
|
ensure_at_least_one_chapter_enabled,
|
||||||
|
should_preselect_chapter,
|
||||||
|
)
|
||||||
|
from abogen.domain.text_utils import calculate_text_length
|
||||||
|
|
||||||
|
|
||||||
|
def build_chapter_payload(
|
||||||
|
chapters: List[Any],
|
||||||
|
source_name: str = "",
|
||||||
|
) -> List[Dict[str, Any]]:
|
||||||
|
"""Build a chapter payload with preselection and character counts.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
chapters: List of chapter-like objects with ``title`` and ``text`` attributes.
|
||||||
|
source_name: Fallback title for the placeholder chapter when *chapters* is empty.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of chapter dicts ready for ``PendingJob.chapters`` or ``ChapterChunkConfig``.
|
||||||
|
"""
|
||||||
|
total = len(chapters)
|
||||||
|
payload: List[Dict[str, Any]] = []
|
||||||
|
|
||||||
|
for index, chapter in enumerate(chapters):
|
||||||
|
title = getattr(chapter, "title", "") or ""
|
||||||
|
text = getattr(chapter, "text", "") or ""
|
||||||
|
enabled = should_preselect_chapter(title, text, index, total)
|
||||||
|
payload.append(
|
||||||
|
{
|
||||||
|
"id": f"{index:04d}",
|
||||||
|
"index": index,
|
||||||
|
"title": title,
|
||||||
|
"text": text,
|
||||||
|
"characters": calculate_text_length(text),
|
||||||
|
"enabled": enabled,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
if not payload:
|
||||||
|
payload.append(
|
||||||
|
{
|
||||||
|
"id": "0000",
|
||||||
|
"index": 0,
|
||||||
|
"title": source_name,
|
||||||
|
"text": "",
|
||||||
|
"characters": 0,
|
||||||
|
"enabled": True,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
ensure_at_least_one_chapter_enabled(payload)
|
||||||
|
return payload
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
"""Application-layer cleanup — global resource disposal.
|
||||||
|
|
||||||
|
Handles:
|
||||||
|
- GPU/CUDA memory flush
|
||||||
|
- TTS engine disposal (PluginManager)
|
||||||
|
- UI-specific cleanup callbacks (registered by entry points)
|
||||||
|
|
||||||
|
Called by shutdown.py at process exit and by run_conversion() per-conversion.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import gc
|
||||||
|
import sys
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
_UI_CLEANUPS: list[Callable[[], None]] = []
|
||||||
|
|
||||||
|
|
||||||
|
def flush_cuda() -> None:
|
||||||
|
"""Run GC and release CUDA cache. Safe to call multiple times."""
|
||||||
|
gc.collect()
|
||||||
|
# Skip entirely if torch was never imported — importing it here just to
|
||||||
|
# check would add several seconds to shutdown with nothing to flush.
|
||||||
|
if "torch" not in sys.modules:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
torch = sys.modules["torch"]
|
||||||
|
if torch.cuda.is_available():
|
||||||
|
torch.cuda.empty_cache()
|
||||||
|
torch.cuda.ipc_collect()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def dispose_engines() -> None:
|
||||||
|
"""Dispose all cached TTS engines via PluginManager."""
|
||||||
|
try:
|
||||||
|
from abogen.tts_plugin.plugin_manager import get_plugin_manager
|
||||||
|
get_plugin_manager().dispose_all()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _clear_global_voice_cache() -> None:
|
||||||
|
"""Reset the global voice download cache state."""
|
||||||
|
try:
|
||||||
|
from abogen.voice_cache import clear_voice_cache
|
||||||
|
clear_voice_cache()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def register_ui_cleanup(fn: Callable[[], None]) -> None:
|
||||||
|
"""Register a UI-specific cleanup callback (e.g. preview threads, temp files)."""
|
||||||
|
_UI_CLEANUPS.append(fn)
|
||||||
|
|
||||||
|
|
||||||
|
def cleanup() -> None:
|
||||||
|
"""Run all application-level cleanups. Idempotent."""
|
||||||
|
dispose_engines()
|
||||||
|
flush_cuda()
|
||||||
|
_clear_global_voice_cache()
|
||||||
|
|
||||||
|
for fn in _UI_CLEANUPS:
|
||||||
|
try:
|
||||||
|
fn()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
_UI_CLEANUPS.clear()
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"flush_cuda",
|
||||||
|
"dispose_engines",
|
||||||
|
"register_ui_cleanup",
|
||||||
|
"cleanup",
|
||||||
|
]
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
"""Feature config objects for ConversionRequest.
|
||||||
|
|
||||||
|
Each config object groups parameters for a specific feature.
|
||||||
|
If the object is None, the feature is disabled.
|
||||||
|
|
||||||
|
This keeps ConversionRequest clean: no boolean flags for feature toggles,
|
||||||
|
no scattered parameters across unrelated fields.
|
||||||
|
|
||||||
|
Domain config types (PronunciationConfig, SubtitleConfig) live in
|
||||||
|
domain/config_types.py — domain defines the contract, app fills them.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
from abogen.domain.config_types import CoverConfig, PronunciationConfig, SubtitleConfig
|
||||||
|
from abogen.domain.enums import OutputFormat, SaveMode
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class WordSubstitutionConfig:
|
||||||
|
"""Word substitution settings.
|
||||||
|
|
||||||
|
When present on ConversionRequest, word substitution is applied
|
||||||
|
to the source text before chapter parsing.
|
||||||
|
"""
|
||||||
|
substitutions_list: str = ""
|
||||||
|
case_sensitive: bool = False
|
||||||
|
replace_caps: bool = False
|
||||||
|
replace_numerals: bool = False
|
||||||
|
fix_punctuation: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class SubtitleInputConfig:
|
||||||
|
"""Subtitle file input settings.
|
||||||
|
|
||||||
|
When present on ConversionRequest, the source is treated as a
|
||||||
|
subtitle file (.srt/.ass/.vtt) or timestamp text, and the
|
||||||
|
subtitle-to-audio pipeline is used instead of normal text conversion.
|
||||||
|
"""
|
||||||
|
is_timestamp_text: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Epub3ExportConfig:
|
||||||
|
"""EPUB3 export settings.
|
||||||
|
|
||||||
|
When present on ConversionRequest, an EPUB3 package with
|
||||||
|
synchronized audio narration is generated after conversion.
|
||||||
|
"""
|
||||||
|
book_id: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ChapterChunkConfig:
|
||||||
|
"""Chapter and chunk configuration.
|
||||||
|
|
||||||
|
Groups chapter overrides, chunk data, and speaker settings
|
||||||
|
used by the planner to build segments.
|
||||||
|
"""
|
||||||
|
chapter_overrides: List[Dict[str, Any]] = field(default_factory=list)
|
||||||
|
chunks: List[Dict[str, Any]] = field(default_factory=list)
|
||||||
|
chunk_level: str = "paragraph"
|
||||||
|
speaker_mode: str = "single"
|
||||||
|
speakers: Dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
_VALID_CHUNK_LEVELS = ("paragraph", "sentence")
|
||||||
|
_VALID_SPEAKER_MODES = ("single", "multi")
|
||||||
|
if self.chunk_level not in _VALID_CHUNK_LEVELS:
|
||||||
|
raise ValueError(
|
||||||
|
f"chunk_level must be one of {_VALID_CHUNK_LEVELS}, got {self.chunk_level!r}"
|
||||||
|
)
|
||||||
|
if self.speaker_mode not in _VALID_SPEAKER_MODES:
|
||||||
|
raise ValueError(
|
||||||
|
f"speaker_mode must be one of {_VALID_SPEAKER_MODES}, got {self.speaker_mode!r}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class SaveConfig:
|
||||||
|
"""Save/output settings.
|
||||||
|
|
||||||
|
Groups save mode, output folder, chapter splitting, and merge options.
|
||||||
|
"""
|
||||||
|
mode: SaveMode = SaveMode.SAVE_NEXT_TO_INPUT
|
||||||
|
output_folder: Optional[Path] = None
|
||||||
|
save_chapters_separately: bool = False
|
||||||
|
merge_chapters_at_end: bool = True
|
||||||
|
separate_chapters_format: OutputFormat = OutputFormat.WAV
|
||||||
|
save_as_project: bool = False
|
||||||
@@ -8,15 +8,13 @@ This is Stage 6 of the conversion flow unification plan.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
import time
|
import time
|
||||||
from contextlib import ExitStack
|
from contextlib import ExitStack
|
||||||
from typing import Any, Callable, Dict, List, Optional, Tuple
|
from typing import Any, Callable, Dict, List, Optional, Set, Tuple
|
||||||
|
|
||||||
from abogen.application.conversion_models import (
|
from abogen.application.conversion_models import (
|
||||||
ChapterPlan,
|
|
||||||
ConversionPlan,
|
ConversionPlan,
|
||||||
IntroOutroSpec,
|
|
||||||
SegmentPlan,
|
|
||||||
)
|
)
|
||||||
from abogen.application.conversion_ports import (
|
from abogen.application.conversion_ports import (
|
||||||
AudioSink,
|
AudioSink,
|
||||||
@@ -35,10 +33,117 @@ from abogen.domain.conversion_engine import (
|
|||||||
)
|
)
|
||||||
from abogen.domain.enums import OutputFormat, SubtitleMode
|
from abogen.domain.enums import OutputFormat, SubtitleMode
|
||||||
from abogen.domain.normalization import TTSContext
|
from abogen.domain.normalization import TTSContext
|
||||||
|
from abogen.domain.chapter_titles import (
|
||||||
|
apply_chapter_text_transforms,
|
||||||
|
headings_equivalent as _headings_equivalent,
|
||||||
|
)
|
||||||
from abogen.domain.output_paths import sanitize_filename_for_chapter
|
from abogen.domain.output_paths import sanitize_filename_for_chapter
|
||||||
from abogen.infrastructure.subtitle_writer import make_subtitle_writer
|
from abogen.infrastructure.subtitle_writer import make_subtitle_writer
|
||||||
|
|
||||||
|
|
||||||
|
# ─── MarkerCollector ───
|
||||||
|
|
||||||
|
|
||||||
|
class MarkerCollector:
|
||||||
|
"""Observes execution events and accumulates chapter/chunk markers.
|
||||||
|
|
||||||
|
Separates marker collection from synthesis logic.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._chapter_markers: List[Dict[str, Any]] = []
|
||||||
|
self._chunk_markers: List[Dict[str, Any]] = []
|
||||||
|
self._current_chapter_voices: Set[Tuple[str, str]] = set()
|
||||||
|
self._current_chapter_index: int = 0
|
||||||
|
self._current_chapter_title: str = ""
|
||||||
|
self._current_chapter_start: float = 0.0
|
||||||
|
|
||||||
|
def on_chapter_start(
|
||||||
|
self, index: int, title: str, start_time: float
|
||||||
|
) -> None:
|
||||||
|
"""Record chapter start."""
|
||||||
|
self._current_chapter_index = index
|
||||||
|
self._current_chapter_title = title
|
||||||
|
self._current_chapter_start = start_time
|
||||||
|
self._current_chapter_voices.clear()
|
||||||
|
|
||||||
|
def on_segment(
|
||||||
|
self,
|
||||||
|
provider: str,
|
||||||
|
voice: Any,
|
||||||
|
voice_spec: str,
|
||||||
|
speaker_id: str = "narrator",
|
||||||
|
) -> None:
|
||||||
|
"""Record a voice used in this chapter (for multi-speaker tracking)."""
|
||||||
|
self._current_chapter_voices.add((provider, voice_spec))
|
||||||
|
|
||||||
|
def on_chunk(
|
||||||
|
self,
|
||||||
|
chunk_id: str,
|
||||||
|
chapter_index: int,
|
||||||
|
chunk_index: int,
|
||||||
|
start: float,
|
||||||
|
end: float,
|
||||||
|
speaker_id: str,
|
||||||
|
provider: str,
|
||||||
|
voice_spec: str,
|
||||||
|
level: str,
|
||||||
|
characters: int,
|
||||||
|
) -> None:
|
||||||
|
"""Record a chunk marker."""
|
||||||
|
self._chunk_markers.append({
|
||||||
|
"id": chunk_id,
|
||||||
|
"chapter_index": chapter_index,
|
||||||
|
"chunk_index": chunk_index,
|
||||||
|
"start": start,
|
||||||
|
"end": end,
|
||||||
|
"speaker_id": speaker_id,
|
||||||
|
"voice": {"provider": provider, "voice": voice_spec},
|
||||||
|
"level": level,
|
||||||
|
"characters": characters,
|
||||||
|
})
|
||||||
|
|
||||||
|
def on_chapter_end(self, end_time: float) -> None:
|
||||||
|
"""Record chapter end and build chapter marker."""
|
||||||
|
voices = [
|
||||||
|
{"provider": p, "voice": v}
|
||||||
|
for p, v in sorted(self._current_chapter_voices)
|
||||||
|
]
|
||||||
|
self._chapter_markers.append({
|
||||||
|
"chapter_index": self._current_chapter_index,
|
||||||
|
"index": self._current_chapter_index + 1,
|
||||||
|
"title": self._current_chapter_title,
|
||||||
|
"start": self._current_chapter_start,
|
||||||
|
"end": end_time,
|
||||||
|
"voices": voices,
|
||||||
|
})
|
||||||
|
|
||||||
|
def on_outro(
|
||||||
|
self,
|
||||||
|
start_time: float,
|
||||||
|
end_time: float,
|
||||||
|
provider: str,
|
||||||
|
voice_spec: str,
|
||||||
|
) -> None:
|
||||||
|
"""Record outro chapter marker."""
|
||||||
|
self._chapter_markers.append({
|
||||||
|
"chapter_index": len(self._chapter_markers),
|
||||||
|
"index": len(self._chapter_markers) + 1,
|
||||||
|
"title": "Outro",
|
||||||
|
"start": start_time,
|
||||||
|
"end": end_time,
|
||||||
|
"voices": [{"provider": provider, "voice": voice_spec}],
|
||||||
|
})
|
||||||
|
|
||||||
|
@property
|
||||||
|
def chapter_markers(self) -> List[Dict[str, Any]]:
|
||||||
|
return self._chapter_markers
|
||||||
|
|
||||||
|
@property
|
||||||
|
def chunk_markers(self) -> List[Dict[str, Any]]:
|
||||||
|
return self._chunk_markers
|
||||||
|
|
||||||
|
|
||||||
def execute_conversion(
|
def execute_conversion(
|
||||||
plan: ConversionPlan,
|
plan: ConversionPlan,
|
||||||
events: ConversionEvents,
|
events: ConversionEvents,
|
||||||
@@ -66,6 +171,15 @@ def execute_conversion(
|
|||||||
"""
|
"""
|
||||||
request = plan.request
|
request = plan.request
|
||||||
result = ConversionResult(metadata=plan.metadata)
|
result = ConversionResult(metadata=plan.metadata)
|
||||||
|
collector = MarkerCollector()
|
||||||
|
|
||||||
|
logging.info(
|
||||||
|
"[executor] Starting: chapters=%d intro=%s outro=%s merge=%s",
|
||||||
|
len(plan.chapters),
|
||||||
|
bool(plan.intro and plan.intro.enabled),
|
||||||
|
bool(plan.outro and plan.outro.enabled),
|
||||||
|
request.save.merge_chapters_at_end,
|
||||||
|
)
|
||||||
|
|
||||||
# Determine cancellation checker
|
# Determine cancellation checker
|
||||||
if check_cancelled is None:
|
if check_cancelled is None:
|
||||||
@@ -88,7 +202,7 @@ def execute_conversion(
|
|||||||
)
|
)
|
||||||
|
|
||||||
# Compute subtitle flag once (used in every synthesize_text call)
|
# Compute subtitle flag once (used in every synthesize_text call)
|
||||||
use_spacy = request.subtitle_mode not in (SubtitleMode.DISABLED, SubtitleMode.LINE)
|
use_spacy = request.subtitle.mode not in (SubtitleMode.DISABLED, SubtitleMode.LINE)
|
||||||
|
|
||||||
# Output paths
|
# Output paths
|
||||||
output_layout = plan.output_layout
|
output_layout = plan.output_layout
|
||||||
@@ -96,15 +210,18 @@ def execute_conversion(
|
|||||||
raise ValueError("ConversionPlan must have an output_layout")
|
raise ValueError("ConversionPlan must have an output_layout")
|
||||||
|
|
||||||
# Determine if merged output is needed
|
# Determine if merged output is needed
|
||||||
merge_chapters = request.merge_chapters_at_end or not request.save_chapters_separately
|
merge_chapters = request.save.merge_chapters_at_end or not request.save.save_chapters_separately
|
||||||
if request.output_format == OutputFormat.M4B:
|
if request.output_format == OutputFormat.M4B:
|
||||||
merge_chapters = True
|
merge_chapters = True
|
||||||
|
|
||||||
# Resolve voices
|
# Resolve voices
|
||||||
base_voice_spec = request.voice or "M1"
|
base_voice_spec = request.voice or "M1"
|
||||||
|
logging.info("[executor] Resolving base voice: spec=%s", base_voice_spec)
|
||||||
base_provider, base_voice_choice, base_speed, base_steps = _resolve_voice(
|
base_provider, base_voice_choice, base_speed, base_steps = _resolve_voice(
|
||||||
voice_resolver, base_voice_spec, request
|
voice_resolver, base_voice_spec, request,
|
||||||
|
log_callback=lambda msg: events.log(msg, level="warning"),
|
||||||
)
|
)
|
||||||
|
logging.info("[executor] Base voice resolved: provider=%s voice=%s speed=%.2f", base_provider, base_voice_choice, base_speed)
|
||||||
|
|
||||||
# Use ExitStack for resource management
|
# Use ExitStack for resource management
|
||||||
with ExitStack() as stack:
|
with ExitStack() as stack:
|
||||||
@@ -112,7 +229,7 @@ def execute_conversion(
|
|||||||
audio_sink: Optional[AudioSink] = None
|
audio_sink: Optional[AudioSink] = None
|
||||||
audio_path = None
|
audio_path = None
|
||||||
if merge_chapters:
|
if merge_chapters:
|
||||||
audio_path = output_layout.audio_dir / f"{_base_name(request)}.{request.output_format}"
|
audio_path = output_layout.audio_dir / f"{_base_name(request)}{request.output_format.dot_ext}"
|
||||||
meta = plan.metadata if plan.metadata else None
|
meta = plan.metadata if plan.metadata else None
|
||||||
audio_sink = stack.enter_context(
|
audio_sink = stack.enter_context(
|
||||||
open_audio_sink(
|
open_audio_sink(
|
||||||
@@ -126,19 +243,17 @@ def execute_conversion(
|
|||||||
|
|
||||||
# Open subtitle writer if needed
|
# Open subtitle writer if needed
|
||||||
subtitle_writer: Optional[SubtitleWriter] = None
|
subtitle_writer: Optional[SubtitleWriter] = None
|
||||||
if request.subtitle_mode != SubtitleMode.DISABLED and audio_sink:
|
if request.subtitle.mode != SubtitleMode.DISABLED and audio_sink:
|
||||||
subtitle_writer = make_subtitle_writer(
|
subtitle_writer = make_subtitle_writer(
|
||||||
audio_path,
|
audio_path,
|
||||||
request.subtitle_format,
|
request.subtitle,
|
||||||
request.subtitle_mode,
|
|
||||||
max_words=request.max_subtitle_words,
|
|
||||||
)
|
)
|
||||||
if subtitle_writer:
|
if subtitle_writer:
|
||||||
subtitle_writer.open()
|
subtitle_writer.open()
|
||||||
stack.callback(subtitle_writer.close)
|
stack.callback(subtitle_writer.close)
|
||||||
result.subtitle_paths.append(subtitle_writer.path)
|
result.subtitle_paths.append(subtitle_writer.path)
|
||||||
|
|
||||||
effective_subtitle_mode = request.subtitle_mode if subtitle_writer else SubtitleMode.DISABLED
|
effective_subtitle_mode = request.subtitle.mode if subtitle_writer else SubtitleMode.DISABLED
|
||||||
|
|
||||||
synth = SynthParams(
|
synth = SynthParams(
|
||||||
tts_context=tts_context,
|
tts_context=tts_context,
|
||||||
@@ -147,14 +262,14 @@ def execute_conversion(
|
|||||||
on_progress=lambda pct, etr: events.progress(pct, etr),
|
on_progress=lambda pct, etr: events.progress(pct, etr),
|
||||||
audio_sink=audio_sink,
|
audio_sink=audio_sink,
|
||||||
subtitle_mode=effective_subtitle_mode,
|
subtitle_mode=effective_subtitle_mode,
|
||||||
max_subtitle_words=request.max_subtitle_words,
|
max_subtitle_words=request.subtitle.max_words,
|
||||||
lang_code=request.language,
|
language=request.language,
|
||||||
use_spacy_segmentation=use_spacy,
|
use_spacy_segmentation=use_spacy,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Chapter directory
|
# Chapter directory
|
||||||
chapter_dir = None
|
chapter_dir = None
|
||||||
if request.save_chapters_separately and len(plan.chapters) > 1:
|
if request.save.save_chapters_separately and len(plan.chapters) > 1:
|
||||||
chapter_dir = output_layout.audio_dir / "chapters"
|
chapter_dir = output_layout.audio_dir / "chapters"
|
||||||
chapter_dir.mkdir(parents=True, exist_ok=True)
|
chapter_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
@@ -163,7 +278,8 @@ def execute_conversion(
|
|||||||
if plan.intro and plan.intro.enabled and merge_chapters:
|
if plan.intro and plan.intro.enabled and merge_chapters:
|
||||||
events.log(f"Title intro: {plan.intro.text[:80]}")
|
events.log(f"Title intro: {plan.intro.text[:80]}")
|
||||||
intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice(
|
intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice(
|
||||||
voice_resolver, plan.intro.voice_spec, request
|
voice_resolver, plan.intro.voice_spec, request,
|
||||||
|
log_callback=lambda msg: events.log(msg, level="warning"),
|
||||||
)
|
)
|
||||||
intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu)
|
intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu)
|
||||||
synthesize_text(
|
synthesize_text(
|
||||||
@@ -172,6 +288,7 @@ def execute_conversion(
|
|||||||
backend=intro_backend,
|
backend=intro_backend,
|
||||||
voice=intro_voice,
|
voice=intro_voice,
|
||||||
speed=intro_speed or request.speed,
|
speed=intro_speed or request.speed,
|
||||||
|
total_steps=intro_steps,
|
||||||
chapter_sink=None,
|
chapter_sink=None,
|
||||||
preview_callback=lambda text: events.log(f" {text[:80]}"),
|
preview_callback=lambda text: events.log(f" {text[:80]}"),
|
||||||
)
|
)
|
||||||
@@ -184,33 +301,58 @@ def execute_conversion(
|
|||||||
|
|
||||||
chapter_display = f"Chapter {chapter_idx}/{len(plan.chapters)}: {chapter.title}"
|
chapter_display = f"Chapter {chapter_idx}/{len(plan.chapters)}: {chapter.title}"
|
||||||
events.log(f"Processing {chapter_display}")
|
events.log(f"Processing {chapter_display}")
|
||||||
|
logging.info("[executor] Chapter %d/%d: %s", chapter_idx, len(plan.chapters), chapter.title)
|
||||||
|
|
||||||
# Resolve chapter voice
|
# Resolve chapter voice
|
||||||
chapter_provider, chapter_voice, chapter_speed, chapter_steps = _resolve_voice(
|
chapter_provider, chapter_voice, chapter_speed, chapter_steps = _resolve_voice(
|
||||||
voice_resolver, chapter.voice_spec, request
|
voice_resolver, chapter.voice_spec, request,
|
||||||
|
log_callback=lambda msg: events.log(msg, level="warning"),
|
||||||
)
|
)
|
||||||
|
logging.info("[executor] Chapter %d voice: provider=%s voice=%s speed=%.2f", chapter_idx, chapter_provider, chapter_voice, chapter_speed)
|
||||||
chapter_backend = pipeline_provider.get(chapter_provider, request.language, request.use_gpu)
|
chapter_backend = pipeline_provider.get(chapter_provider, request.language, request.use_gpu)
|
||||||
|
|
||||||
|
# Record chapter start for markers
|
||||||
|
collector.on_chapter_start(chapter_idx - 1, chapter.title, stats.current_time)
|
||||||
|
|
||||||
# Per-chapter sink
|
# Per-chapter sink
|
||||||
chapter_sink: Optional[AudioSink] = None
|
chapter_sink: Optional[AudioSink] = None
|
||||||
chapter_path = None
|
chapter_path = None
|
||||||
if chapter_dir:
|
if chapter_dir:
|
||||||
chapter_filename = sanitize_filename_for_chapter(chapter.title, chapter_idx)
|
chapter_filename = sanitize_filename_for_chapter(chapter.title, chapter_idx)
|
||||||
chapter_path = chapter_dir / f"{chapter_filename}.{request.separate_chapters_format}"
|
chapter_path = chapter_dir / f"{chapter_filename}.{request.save.separate_chapters_format}"
|
||||||
chapter_sink = stack.enter_context(
|
chapter_sink = stack.enter_context(
|
||||||
open_audio_sink(
|
open_audio_sink(
|
||||||
chapter_path,
|
chapter_path,
|
||||||
request.separate_chapters_format,
|
request.save.separate_chapters_format,
|
||||||
cancel_check=check_cancelled,
|
cancel_check=check_cancelled,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
result.chapter_paths.append(chapter_path)
|
result.chapter_paths.append(chapter_path)
|
||||||
|
|
||||||
|
# Per-chapter subtitle writer
|
||||||
|
chapter_subtitle_writer: Optional[SubtitleWriter] = None
|
||||||
|
if chapter_dir and request.subtitle.mode != SubtitleMode.DISABLED and chapter_sink:
|
||||||
|
from abogen.infrastructure.subtitle_writer import resolve_subtitle_format
|
||||||
|
|
||||||
|
chapter_filename = sanitize_filename_for_chapter(chapter.title, chapter_idx)
|
||||||
|
subtitle_ext, _ = resolve_subtitle_format(
|
||||||
|
request.subtitle
|
||||||
|
)
|
||||||
|
chapter_subtitle_path = chapter_dir / f"{chapter_filename}.{subtitle_ext}"
|
||||||
|
chapter_subtitle_writer = make_subtitle_writer(
|
||||||
|
chapter_subtitle_path,
|
||||||
|
request.subtitle,
|
||||||
|
)
|
||||||
|
if chapter_subtitle_writer:
|
||||||
|
chapter_subtitle_writer.open()
|
||||||
|
result.subtitle_paths.append(chapter_subtitle_writer.path)
|
||||||
|
|
||||||
# Intro delay before first chapter
|
# Intro delay before first chapter
|
||||||
if not intro_emitted and plan.intro and plan.intro.enabled:
|
if not intro_emitted and plan.intro and plan.intro.enabled:
|
||||||
# Intro will be emitted with first chapter
|
# Intro will be emitted with first chapter
|
||||||
intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice(
|
intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice(
|
||||||
voice_resolver, plan.intro.voice_spec, request
|
voice_resolver, plan.intro.voice_spec, request,
|
||||||
|
log_callback=lambda msg: events.log(msg, level="warning"),
|
||||||
)
|
)
|
||||||
intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu)
|
intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu)
|
||||||
synthesize_text(
|
synthesize_text(
|
||||||
@@ -219,6 +361,7 @@ def execute_conversion(
|
|||||||
backend=intro_backend,
|
backend=intro_backend,
|
||||||
voice=intro_voice,
|
voice=intro_voice,
|
||||||
speed=intro_speed or request.speed,
|
speed=intro_speed or request.speed,
|
||||||
|
total_steps=intro_steps,
|
||||||
chapter_sink=chapter_sink,
|
chapter_sink=chapter_sink,
|
||||||
preview_callback=lambda text: events.log(f" Intro: {text[:80]}"),
|
preview_callback=lambda text: events.log(f" Intro: {text[:80]}"),
|
||||||
)
|
)
|
||||||
@@ -232,6 +375,7 @@ def execute_conversion(
|
|||||||
)
|
)
|
||||||
|
|
||||||
# Process heading
|
# Process heading
|
||||||
|
heading_text = ""
|
||||||
if chapter.title:
|
if chapter.title:
|
||||||
heading_text = _format_heading(chapter.title, chapter_idx, request)
|
heading_text = _format_heading(chapter.title, chapter_idx, request)
|
||||||
if heading_text:
|
if heading_text:
|
||||||
@@ -252,59 +396,120 @@ def execute_conversion(
|
|||||||
stats=stats,
|
stats=stats,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Heading dedup: check if first line of body matches heading
|
||||||
|
pending_heading_strip = False
|
||||||
|
if heading_text and chapter.body_text:
|
||||||
|
first_line = next(
|
||||||
|
(line.strip() for line in chapter.body_text.splitlines() if line.strip()),
|
||||||
|
"",
|
||||||
|
)
|
||||||
|
if first_line and _headings_equivalent(first_line, heading_text):
|
||||||
|
pending_heading_strip = True
|
||||||
|
|
||||||
# Process body segments
|
# Process body segments
|
||||||
chapter_chunk_markers: List[Dict[str, Any]] = []
|
|
||||||
for seg_idx, segment in enumerate(chapter.segments):
|
for seg_idx, segment in enumerate(chapter.segments):
|
||||||
check_cancelled()
|
check_cancelled()
|
||||||
|
|
||||||
|
# Apply heading dedup to first segment (consume-once)
|
||||||
|
seg_text = segment.text
|
||||||
|
if pending_heading_strip and seg_text.strip():
|
||||||
|
seg_text, heading_removed, _ = apply_chapter_text_transforms(
|
||||||
|
seg_text,
|
||||||
|
heading_text=heading_text,
|
||||||
|
raw_title=chapter.title,
|
||||||
|
strip_heading=True,
|
||||||
|
normalize_caps=False,
|
||||||
|
)
|
||||||
|
if heading_removed:
|
||||||
|
pending_heading_strip = False
|
||||||
|
if not seg_text.strip():
|
||||||
|
continue
|
||||||
|
|
||||||
# Resolve segment voice (may differ from chapter voice)
|
# Resolve segment voice (may differ from chapter voice)
|
||||||
if segment.voice_spec != chapter.voice_spec:
|
if segment.voice_spec != chapter.voice_spec:
|
||||||
seg_provider, seg_voice, seg_speed, seg_steps = _resolve_voice(
|
seg_provider, seg_voice, seg_speed, seg_steps = _resolve_voice(
|
||||||
voice_resolver, segment.voice_spec, request
|
voice_resolver, segment.voice_spec, request,
|
||||||
|
log_callback=lambda msg: events.log(msg, level="warning"),
|
||||||
)
|
)
|
||||||
seg_backend = pipeline_provider.get(seg_provider, request.language, request.use_gpu)
|
seg_backend = pipeline_provider.get(seg_provider, request.language, request.use_gpu)
|
||||||
else:
|
else:
|
||||||
seg_provider = chapter_provider
|
seg_provider = chapter_provider
|
||||||
seg_voice = chapter_voice
|
seg_voice = chapter_voice
|
||||||
seg_speed = chapter_speed
|
seg_speed = chapter_speed
|
||||||
|
seg_steps = chapter_steps
|
||||||
seg_backend = chapter_backend
|
seg_backend = chapter_backend
|
||||||
|
|
||||||
seg_start_time = stats.current_time
|
# Track voice for chapter marker
|
||||||
local_segments, accumulated_tokens = synthesize_text(
|
collector.on_segment(seg_provider, seg_voice, segment.voice_spec)
|
||||||
text=segment.text,
|
|
||||||
params=synth,
|
# spaCy pre-TTS segmentation
|
||||||
backend=seg_backend,
|
from abogen.domain.conversion_pipeline import spacy_pre_tts_segmentation
|
||||||
voice=seg_voice,
|
|
||||||
speed=seg_speed or request.speed,
|
is_subtitle_input = bool(
|
||||||
chapter_sink=chapter_sink,
|
request.subtitle_input
|
||||||
preview_callback=lambda text: events.log(f" {text[:80]}"),
|
)
|
||||||
|
spacy_segments, active_split = spacy_pre_tts_segmentation(
|
||||||
|
seg_text,
|
||||||
|
request.language,
|
||||||
|
request.subtitle.mode,
|
||||||
|
is_subtitle_input=is_subtitle_input,
|
||||||
|
use_spacy_segmentation=use_spacy,
|
||||||
|
log_callback=lambda msg: events.log(msg),
|
||||||
)
|
)
|
||||||
|
|
||||||
# Process subtitles
|
seg_start_time = stats.current_time
|
||||||
if subtitle_writer and audio_sink and accumulated_tokens:
|
accumulated_tokens: List[Dict[str, Any]] = []
|
||||||
process_and_write_subtitles(
|
for spacy_seg in spacy_segments:
|
||||||
accumulated_tokens,
|
if not spacy_seg.strip():
|
||||||
subtitle_writer,
|
continue
|
||||||
subtitle_mode=request.subtitle_mode,
|
_, seg_tokens = synthesize_text(
|
||||||
max_subtitle_words=request.max_subtitle_words,
|
text=spacy_seg,
|
||||||
lang_code=request.language,
|
params=synth,
|
||||||
use_spacy_segmentation=use_spacy,
|
backend=seg_backend,
|
||||||
fallback_end_time=stats.current_time,
|
voice=seg_voice,
|
||||||
|
speed=seg_speed or request.speed,
|
||||||
|
total_steps=seg_steps,
|
||||||
|
chapter_sink=chapter_sink,
|
||||||
|
preview_callback=lambda text: events.log(f" {text[:80]}"),
|
||||||
|
split_pattern_override=active_split,
|
||||||
)
|
)
|
||||||
|
accumulated_tokens.extend(seg_tokens)
|
||||||
|
|
||||||
|
# Process subtitles
|
||||||
|
if audio_sink and accumulated_tokens:
|
||||||
|
if subtitle_writer:
|
||||||
|
process_and_write_subtitles(
|
||||||
|
accumulated_tokens,
|
||||||
|
subtitle_writer,
|
||||||
|
subtitle=request.subtitle,
|
||||||
|
language=request.language,
|
||||||
|
use_spacy_segmentation=use_spacy,
|
||||||
|
fallback_end_time=stats.current_time,
|
||||||
|
)
|
||||||
|
if chapter_subtitle_writer:
|
||||||
|
process_and_write_subtitles(
|
||||||
|
accumulated_tokens,
|
||||||
|
chapter_subtitle_writer,
|
||||||
|
subtitle=request.subtitle,
|
||||||
|
language=request.language,
|
||||||
|
use_spacy_segmentation=use_spacy,
|
||||||
|
fallback_end_time=stats.current_time,
|
||||||
|
)
|
||||||
|
|
||||||
# Record chunk marker
|
# Record chunk marker
|
||||||
if segment.source in ("chunk", "voice_marker"):
|
if segment.source in ("chunk", "voice_marker"):
|
||||||
chapter_chunk_markers.append({
|
collector.on_chunk(
|
||||||
"id": segment.chunk_id,
|
chunk_id=segment.chunk_id or "",
|
||||||
"chapter_index": chapter_idx - 1,
|
chapter_index=chapter_idx - 1,
|
||||||
"chunk_index": segment.chunk_index or seg_idx,
|
chunk_index=segment.chunk_index or seg_idx,
|
||||||
"start": seg_start_time,
|
start=seg_start_time,
|
||||||
"end": stats.current_time,
|
end=stats.current_time,
|
||||||
"speaker_id": segment.speaker_id,
|
speaker_id=segment.speaker_id or "narrator",
|
||||||
"voice": segment.voice_spec,
|
provider=seg_provider,
|
||||||
"level": segment.level or request.chunk_level,
|
voice_spec=segment.voice_spec,
|
||||||
"characters": len(segment.text),
|
level=segment.level or (request.chapter_chunk.chunk_level if request.chapter_chunk else "paragraph"),
|
||||||
})
|
characters=len(segment.text),
|
||||||
|
)
|
||||||
|
|
||||||
# Silence between chapters
|
# Silence between chapters
|
||||||
if chapter_idx < len(plan.chapters) and request.silence_between_chapters > 0:
|
if chapter_idx < len(plan.chapters) and request.silence_between_chapters > 0:
|
||||||
@@ -319,21 +524,22 @@ def execute_conversion(
|
|||||||
if chapter_sink:
|
if chapter_sink:
|
||||||
chapter_sink.close()
|
chapter_sink.close()
|
||||||
|
|
||||||
# Add chapter marker
|
# Close chapter subtitle writer
|
||||||
result.chapter_markers.append({
|
if chapter_subtitle_writer:
|
||||||
"chapter_index": chapter_idx - 1,
|
chapter_subtitle_writer.close()
|
||||||
"title": chapter.title,
|
|
||||||
"start": stats.current_time - (stats.current_time - seg_start_time) if chapter.segments else stats.current_time,
|
|
||||||
"end": stats.current_time,
|
|
||||||
})
|
|
||||||
|
|
||||||
result.chunk_markers.extend(chapter_chunk_markers)
|
# Record chapter end for markers
|
||||||
|
collector.on_chapter_end(stats.current_time)
|
||||||
|
logging.info("[executor] Chapter %d/%d done: time=%.1fs", chapter_idx, len(plan.chapters), stats.current_time)
|
||||||
|
|
||||||
|
logging.info("[executor] All chapters done: total=%.1fs", stats.current_time)
|
||||||
|
|
||||||
# Process outro
|
# Process outro
|
||||||
if plan.outro and plan.outro.enabled and merge_chapters:
|
if plan.outro and plan.outro.enabled and merge_chapters:
|
||||||
events.log(f"Closing outro: {plan.outro.text[:80]}")
|
events.log(f"Closing outro: {plan.outro.text[:80]}")
|
||||||
outro_provider, outro_voice, outro_speed, outro_steps = _resolve_voice(
|
outro_provider, outro_voice, outro_speed, outro_steps = _resolve_voice(
|
||||||
voice_resolver, plan.outro.voice_spec, request
|
voice_resolver, plan.outro.voice_spec, request,
|
||||||
|
log_callback=lambda msg: events.log(msg, level="warning"),
|
||||||
)
|
)
|
||||||
outro_backend = pipeline_provider.get(outro_provider, request.language, request.use_gpu)
|
outro_backend = pipeline_provider.get(outro_provider, request.language, request.use_gpu)
|
||||||
|
|
||||||
@@ -346,18 +552,24 @@ def execute_conversion(
|
|||||||
stats=stats,
|
stats=stats,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
outro_start = stats.current_time
|
||||||
synthesize_text(
|
synthesize_text(
|
||||||
text=plan.outro.text,
|
text=plan.outro.text,
|
||||||
params=synth,
|
params=synth,
|
||||||
backend=outro_backend,
|
backend=outro_backend,
|
||||||
voice=outro_voice,
|
voice=outro_voice,
|
||||||
speed=outro_speed or request.speed,
|
speed=outro_speed or request.speed,
|
||||||
|
total_steps=outro_steps,
|
||||||
chapter_sink=None,
|
chapter_sink=None,
|
||||||
preview_callback=lambda text: events.log(f" {text[:80]}"),
|
preview_callback=lambda text: events.log(f" {text[:80]}"),
|
||||||
)
|
)
|
||||||
|
# Record outro marker
|
||||||
|
collector.on_outro(outro_start, stats.current_time, outro_provider, plan.outro.voice_spec)
|
||||||
events.log("Outro synthesized.")
|
events.log("Outro synthesized.")
|
||||||
|
|
||||||
# Set result metadata
|
# Set result metadata
|
||||||
|
result.chapter_markers = collector.chapter_markers
|
||||||
|
result.chunk_markers = collector.chunk_markers
|
||||||
result.total_chapters = len(plan.chapters)
|
result.total_chapters = len(plan.chapters)
|
||||||
result.total_segments = sum(len(ch.segments) for ch in plan.chapters)
|
result.total_segments = sum(len(ch.segments) for ch in plan.chapters)
|
||||||
result.total_characters = total_characters
|
result.total_characters = total_characters
|
||||||
@@ -375,6 +587,8 @@ def _resolve_voice(
|
|||||||
resolver: VoiceResolver,
|
resolver: VoiceResolver,
|
||||||
voice_spec: str,
|
voice_spec: str,
|
||||||
request: Any,
|
request: Any,
|
||||||
|
*,
|
||||||
|
log_callback: Optional[Callable[[str], None]] = None,
|
||||||
) -> Tuple[str, Any, Optional[float], Optional[int]]:
|
) -> Tuple[str, Any, Optional[float], Optional[int]]:
|
||||||
"""Resolve a voice spec and return (provider, voice, speed, steps)."""
|
"""Resolve a voice spec and return (provider, voice, speed, steps)."""
|
||||||
try:
|
try:
|
||||||
@@ -385,9 +599,21 @@ def _resolve_voice(
|
|||||||
resolved.speed,
|
resolved.speed,
|
||||||
resolved.supertonic_steps,
|
resolved.supertonic_steps,
|
||||||
)
|
)
|
||||||
except Exception:
|
except Exception as exc:
|
||||||
# Fallback to base voice
|
# Fallback to base voice
|
||||||
resolved = resolver.resolve(request.voice or "M1")
|
base_spec = request.voice or "M1"
|
||||||
|
if log_callback:
|
||||||
|
log_callback(
|
||||||
|
f"Voice '{voice_spec}' failed to resolve: {exc}. "
|
||||||
|
f"Falling back to '{base_spec}'."
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
resolved = resolver.resolve(base_spec)
|
||||||
|
except Exception as fallback_exc:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"Both voice '{voice_spec}' and fallback '{base_spec}' failed to resolve. "
|
||||||
|
f"Primary error: {exc}; Fallback error: {fallback_exc}"
|
||||||
|
) from fallback_exc
|
||||||
return (
|
return (
|
||||||
resolved.provider,
|
resolved.provider,
|
||||||
resolved.voice,
|
resolved.voice,
|
||||||
|
|||||||
@@ -15,12 +15,13 @@ The planning flow:
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional
|
from typing import TYPE_CHECKING, Any, Dict, List, Optional
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from abogen.application.conversion_request import ConversionRequest
|
from abogen.application.conversion_request import ConversionRequest
|
||||||
|
from abogen.text_extractor import ExtractionResult
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -93,3 +94,4 @@ class ConversionPlan:
|
|||||||
intro: Optional[IntroOutroSpec] = None
|
intro: Optional[IntroOutroSpec] = None
|
||||||
outro: Optional[IntroOutroSpec] = None
|
outro: Optional[IntroOutroSpec] = None
|
||||||
output_layout: Optional[OutputLayout] = None
|
output_layout: Optional[OutputLayout] = None
|
||||||
|
extraction: Optional[ExtractionResult] = None
|
||||||
|
|||||||
@@ -8,14 +8,13 @@ This is Stage 2 of the conversion flow unification plan.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from pathlib import Path
|
import logging
|
||||||
from typing import Any, Dict, List, Optional, Tuple
|
from typing import Any, Dict, List, Optional, Tuple
|
||||||
|
|
||||||
from abogen.application.conversion_models import (
|
from abogen.application.conversion_models import (
|
||||||
ChapterPlan,
|
ChapterPlan,
|
||||||
ConversionPlan,
|
ConversionPlan,
|
||||||
IntroOutroSpec,
|
IntroOutroSpec,
|
||||||
OutputLayout,
|
|
||||||
SegmentPlan,
|
SegmentPlan,
|
||||||
)
|
)
|
||||||
from abogen.application.conversion_request import ConversionRequest
|
from abogen.application.conversion_request import ConversionRequest
|
||||||
@@ -25,7 +24,7 @@ from abogen.domain.file_type import auto_select_relevant_chapters
|
|||||||
from abogen.domain.intro_outro import resolve_intro, resolve_outro
|
from abogen.domain.intro_outro import resolve_intro, resolve_outro
|
||||||
from abogen.domain.metadata_extraction import extract_metadata_for_file
|
from abogen.domain.metadata_extraction import extract_metadata_for_file
|
||||||
from abogen.domain.metadata_merge import merge_metadata
|
from abogen.domain.metadata_merge import merge_metadata
|
||||||
from abogen.subtitle_utils import split_text_by_voice_markers
|
from abogen.domain.voice_markers import split_text_by_voice_markers
|
||||||
|
|
||||||
|
|
||||||
def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
|
def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
|
||||||
@@ -50,7 +49,7 @@ def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
|
|||||||
raise ValueError("No text content to convert")
|
raise ValueError("No text content to convert")
|
||||||
|
|
||||||
# 2. Extract metadata
|
# 2. Extract metadata
|
||||||
metadata = _extract_metadata(request)
|
metadata, extraction = _extract_metadata(request)
|
||||||
|
|
||||||
# 3. Parse chapters
|
# 3. Parse chapters
|
||||||
raw_chapters = _parse_chapters(source_text, request)
|
raw_chapters = _parse_chapters(source_text, request)
|
||||||
@@ -67,6 +66,13 @@ def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
|
|||||||
# 7. Resolve output layout
|
# 7. Resolve output layout
|
||||||
output_layout = resolve_output_layout(request)
|
output_layout = resolve_output_layout(request)
|
||||||
|
|
||||||
|
logging.info(
|
||||||
|
"[planner] Plan built: chapters=%d intro=%s outro=%s",
|
||||||
|
len(chapters),
|
||||||
|
bool(intro and intro.enabled),
|
||||||
|
bool(outro and outro.enabled),
|
||||||
|
)
|
||||||
|
|
||||||
return ConversionPlan(
|
return ConversionPlan(
|
||||||
request=request,
|
request=request,
|
||||||
metadata=metadata,
|
metadata=metadata,
|
||||||
@@ -74,6 +80,7 @@ def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
|
|||||||
intro=intro,
|
intro=intro,
|
||||||
outro=outro,
|
outro=outro,
|
||||||
output_layout=output_layout,
|
output_layout=output_layout,
|
||||||
|
extraction=extraction,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -82,22 +89,44 @@ def _extract_source_text(request: ConversionRequest) -> Optional[str]:
|
|||||||
from abogen.subtitle_utils import clean_text
|
from abogen.subtitle_utils import clean_text
|
||||||
|
|
||||||
if request.direct_text:
|
if request.direct_text:
|
||||||
return clean_text(request.direct_text)
|
text = clean_text(request.direct_text)
|
||||||
if request.source_path and request.source_path.exists():
|
elif request.source_path and request.source_path.exists():
|
||||||
encoding = "utf-8"
|
encoding = "utf-8"
|
||||||
try:
|
try:
|
||||||
with open(request.source_path, "r", encoding=encoding, errors="replace") as f:
|
with open(request.source_path, "r", encoding=encoding, errors="replace") as f:
|
||||||
text = f.read()
|
text = f.read()
|
||||||
except Exception:
|
except Exception:
|
||||||
return None
|
return None
|
||||||
return clean_text(text)
|
text = clean_text(text)
|
||||||
return None
|
else:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Apply word substitutions if configured
|
||||||
|
if request.word_substitution:
|
||||||
|
from abogen.word_substitution import apply_word_substitutions
|
||||||
|
|
||||||
|
ws = request.word_substitution
|
||||||
|
text = apply_word_substitutions(
|
||||||
|
text,
|
||||||
|
ws.substitutions_list,
|
||||||
|
ws.case_sensitive,
|
||||||
|
ws.replace_caps,
|
||||||
|
ws.replace_numerals,
|
||||||
|
ws.fix_punctuation,
|
||||||
|
)
|
||||||
|
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
def _extract_metadata(request: ConversionRequest) -> Dict[str, Any]:
|
def _extract_metadata(
|
||||||
"""Extract metadata from source file."""
|
request: ConversionRequest,
|
||||||
|
) -> Tuple[Dict[str, Any], Optional[Any]]:
|
||||||
|
"""Extract metadata from source file.
|
||||||
|
|
||||||
|
Returns (metadata, extraction) tuple.
|
||||||
|
"""
|
||||||
if request.direct_text:
|
if request.direct_text:
|
||||||
return dict(request.metadata_tags)
|
return dict(request.metadata_tags), None
|
||||||
|
|
||||||
if request.source_path and request.source_path.exists():
|
if request.source_path and request.source_path.exists():
|
||||||
try:
|
try:
|
||||||
@@ -106,11 +135,12 @@ def _extract_metadata(request: ConversionRequest) -> Dict[str, Any]:
|
|||||||
)
|
)
|
||||||
metadata = dict(extraction.metadata) if extraction.metadata else {}
|
metadata = dict(extraction.metadata) if extraction.metadata else {}
|
||||||
except Exception:
|
except Exception:
|
||||||
|
extraction = None
|
||||||
metadata = {}
|
metadata = {}
|
||||||
metadata = merge_metadata(metadata, request.metadata_tags)
|
metadata = merge_metadata(metadata, request.metadata_tags)
|
||||||
return metadata
|
return metadata, extraction
|
||||||
|
|
||||||
return dict(request.metadata_tags)
|
return dict(request.metadata_tags), None
|
||||||
|
|
||||||
|
|
||||||
def _parse_chapters(
|
def _parse_chapters(
|
||||||
@@ -144,8 +174,9 @@ def _apply_selection(
|
|||||||
]
|
]
|
||||||
|
|
||||||
# If user specified chapters, apply overrides
|
# If user specified chapters, apply overrides
|
||||||
if request.chapter_overrides:
|
chapter_chunk = request.chapter_chunk
|
||||||
selected, _, diagnostics = apply_chapter_overrides(extracted, request.chapter_overrides)
|
if chapter_chunk and chapter_chunk.chapter_overrides:
|
||||||
|
selected, _, diagnostics = apply_chapter_overrides(extracted, chapter_chunk.chapter_overrides)
|
||||||
if selected:
|
if selected:
|
||||||
# Map back to (title, text, voice) tuples
|
# Map back to (title, text, voice) tuples
|
||||||
result = []
|
result = []
|
||||||
@@ -187,11 +218,17 @@ def _build_chapters(
|
|||||||
selected_chapters: List[Tuple[str, str, str]], request: ConversionRequest
|
selected_chapters: List[Tuple[str, str, str]], request: ConversionRequest
|
||||||
) -> List[ChapterPlan]:
|
) -> List[ChapterPlan]:
|
||||||
"""Build ChapterPlan with SegmentPlan for each chapter."""
|
"""Build ChapterPlan with SegmentPlan for each chapter."""
|
||||||
|
from abogen.domain.chapter_titles import normalize_chapter_opening_caps
|
||||||
|
|
||||||
chapters = []
|
chapters = []
|
||||||
|
|
||||||
for idx, (title, body_text, default_voice) in enumerate(selected_chapters, 1):
|
for idx, (title, body_text, default_voice) in enumerate(selected_chapters, 1):
|
||||||
# Build segments for this chapter
|
# Apply caps normalization to body text if enabled
|
||||||
segments = _build_segments(body_text, default_voice, request)
|
if request.normalize_chapter_opening_caps and body_text:
|
||||||
|
body_text, _ = normalize_chapter_opening_caps(body_text)
|
||||||
|
|
||||||
|
# Build segments for this chapter (idx is 1-based, chunks use 0-based)
|
||||||
|
segments = _build_segments(body_text, default_voice, request, chapter_index=idx - 1)
|
||||||
|
|
||||||
chapter = ChapterPlan(
|
chapter = ChapterPlan(
|
||||||
index=idx,
|
index=idx,
|
||||||
@@ -207,7 +244,8 @@ def _build_chapters(
|
|||||||
|
|
||||||
|
|
||||||
def _build_segments(
|
def _build_segments(
|
||||||
body_text: str, default_voice: str, request: ConversionRequest
|
body_text: str, default_voice: str, request: ConversionRequest,
|
||||||
|
chapter_index: int = 0,
|
||||||
) -> List[SegmentPlan]:
|
) -> List[SegmentPlan]:
|
||||||
"""Build SegmentPlan list for a chapter's body text.
|
"""Build SegmentPlan list for a chapter's body text.
|
||||||
|
|
||||||
@@ -216,9 +254,15 @@ def _build_segments(
|
|||||||
segments = []
|
segments = []
|
||||||
|
|
||||||
# Check for chunks (WebUI style)
|
# Check for chunks (WebUI style)
|
||||||
if request.chunks:
|
chapter_chunk = request.chapter_chunk
|
||||||
# Group chunks by chapter (simplified — assume chunks are for current chapter)
|
if chapter_chunk and chapter_chunk.chunks:
|
||||||
for chunk_idx, chunk in enumerate(request.chunks):
|
# Group chunks by chapter index
|
||||||
|
from abogen.domain.chunk_utils import group_chunks_by_chapter
|
||||||
|
|
||||||
|
chunk_groups = group_chunks_by_chapter(chapter_chunk.chunks)
|
||||||
|
chunks_for_chapter = chunk_groups.get(chapter_index, [])
|
||||||
|
|
||||||
|
for chunk_idx, chunk in enumerate(chunks_for_chapter):
|
||||||
chunk_text = chunk.get("normalized_text") or chunk.get("text", "")
|
chunk_text = chunk.get("normalized_text") or chunk.get("text", "")
|
||||||
if not chunk_text or not chunk_text.strip():
|
if not chunk_text or not chunk_text.strip():
|
||||||
continue
|
continue
|
||||||
@@ -234,7 +278,7 @@ def _build_segments(
|
|||||||
speaker_id=speaker_id,
|
speaker_id=speaker_id,
|
||||||
chunk_id=chunk.get("id"),
|
chunk_id=chunk.get("id"),
|
||||||
chunk_index=chunk.get("chunk_index", chunk_idx),
|
chunk_index=chunk.get("chunk_index", chunk_idx),
|
||||||
level=chunk.get("level", request.chunk_level),
|
level=chunk.get("level", chapter_chunk.chunk_level),
|
||||||
source="chunk",
|
source="chunk",
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
@@ -242,7 +286,7 @@ def _build_segments(
|
|||||||
|
|
||||||
# Check for voice markers (PyQt style)
|
# Check for voice markers (PyQt style)
|
||||||
# Detect markers even if validation fails (voice names may not be loaded yet)
|
# Detect markers even if validation fails (voice names may not be loaded yet)
|
||||||
from abogen.subtitle_utils import _VOICE_MARKER_SEARCH_PATTERN
|
from abogen.domain.voice_markers import _VOICE_MARKER_SEARCH_PATTERN
|
||||||
|
|
||||||
has_voice_markers = bool(_VOICE_MARKER_SEARCH_PATTERN.search(body_text))
|
has_voice_markers = bool(_VOICE_MARKER_SEARCH_PATTERN.search(body_text))
|
||||||
voice_segments, last_voice, valid_count, invalid_count = split_text_by_voice_markers(
|
voice_segments, last_voice, valid_count, invalid_count = split_text_by_voice_markers(
|
||||||
@@ -284,8 +328,9 @@ def _resolve_chunk_voice(
|
|||||||
"""Resolve voice for a chunk."""
|
"""Resolve voice for a chunk."""
|
||||||
# Check for speaker-based voice
|
# Check for speaker-based voice
|
||||||
speaker_id = chunk.get("speaker_id", "narrator")
|
speaker_id = chunk.get("speaker_id", "narrator")
|
||||||
if speaker_id and speaker_id != "narrator" and request.speakers:
|
speakers = request.chapter_chunk.speakers if request.chapter_chunk else {}
|
||||||
speaker_config = request.speakers.get(speaker_id, {})
|
if speaker_id and speaker_id != "narrator" and speakers:
|
||||||
|
speaker_config = speakers.get(speaker_id, {})
|
||||||
if isinstance(speaker_config, dict):
|
if isinstance(speaker_config, dict):
|
||||||
voice = speaker_config.get("voice")
|
voice = speaker_config.get("voice")
|
||||||
if voice:
|
if voice:
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ implementations (PyQt signals, Flask Job, etc.).
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from typing import Any, List, Optional, Protocol, runtime_checkable
|
from typing import Any, Protocol
|
||||||
|
|
||||||
|
|
||||||
class ConversionCancelled(Exception):
|
class ConversionCancelled(Exception):
|
||||||
|
|||||||
@@ -14,7 +14,17 @@ from dataclasses import dataclass, field
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Dict, List, Optional
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
from abogen.domain.enums import Language, OutputFormat, SaveMode, SubtitleFormat, SubtitleMode
|
from abogen.application.conversion_config import (
|
||||||
|
ChapterChunkConfig,
|
||||||
|
CoverConfig,
|
||||||
|
Epub3ExportConfig,
|
||||||
|
PronunciationConfig,
|
||||||
|
SaveConfig,
|
||||||
|
SubtitleConfig,
|
||||||
|
SubtitleInputConfig,
|
||||||
|
WordSubstitutionConfig,
|
||||||
|
)
|
||||||
|
from abogen.domain.enums import Language, OutputFormat
|
||||||
|
|
||||||
|
|
||||||
class ConversionRequestError(ValueError):
|
class ConversionRequestError(ValueError):
|
||||||
@@ -23,19 +33,12 @@ class ConversionRequestError(ValueError):
|
|||||||
|
|
||||||
# Numeric field constraints: attr -> (min, max)
|
# Numeric field constraints: attr -> (min, max)
|
||||||
_NUMERIC_CONSTRAINTS: dict[str, tuple[float, float | None]] = {
|
_NUMERIC_CONSTRAINTS: dict[str, tuple[float, float | None]] = {
|
||||||
"max_subtitle_words": (1, 500),
|
|
||||||
"speed": (0.5, 3.0),
|
"speed": (0.5, 3.0),
|
||||||
"supertonic_total_steps": (2, 15),
|
"supertonic_total_steps": (2, 15),
|
||||||
"silence_between_chapters": (0.0, None),
|
"silence_between_chapters": (0.0, None),
|
||||||
"chapter_intro_delay": (0.0, None),
|
"chapter_intro_delay": (0.0, None),
|
||||||
}
|
}
|
||||||
|
|
||||||
# Enum-like fields that must be in allowed set
|
|
||||||
_ENUM_CONSTRAINTS: dict[str, tuple[str, ...]] = {
|
|
||||||
"chunk_level": ("paragraph", "sentence"),
|
|
||||||
"speaker_mode": ("single", "multi"),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class ConversionRequest:
|
class ConversionRequest:
|
||||||
@@ -44,10 +47,14 @@ class ConversionRequest:
|
|||||||
Only contains fields that describe the conversion task itself.
|
Only contains fields that describe the conversion task itself.
|
||||||
UI-only fields (display, logging, user prompts) stay in adapters.
|
UI-only fields (display, logging, user prompts) stay in adapters.
|
||||||
|
|
||||||
|
Feature toggles use config objects (None = disabled):
|
||||||
|
- word_substitution, subtitle_input, chapter_chunk, epub3_export
|
||||||
|
- pronunciation (raw data, compiled by app layer)
|
||||||
|
- subtitle, save, cover (grouped parameters)
|
||||||
|
|
||||||
Validation runs on creation via __post_init__:
|
Validation runs on creation via __post_init__:
|
||||||
- None values → replaced with field default (from declaration)
|
- None values → replaced with field default (from declaration)
|
||||||
- Numeric fields → clamped to valid range
|
- Numeric fields → clamped to valid range
|
||||||
- String enums → validated against allowed set
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# --- Source ---
|
# --- Source ---
|
||||||
@@ -66,17 +73,6 @@ class ConversionRequest:
|
|||||||
|
|
||||||
# --- Output Format ---
|
# --- Output Format ---
|
||||||
output_format: OutputFormat = OutputFormat.WAV
|
output_format: OutputFormat = OutputFormat.WAV
|
||||||
subtitle_mode: SubtitleMode = SubtitleMode.DISABLED
|
|
||||||
subtitle_format: SubtitleFormat = SubtitleFormat.SRT
|
|
||||||
max_subtitle_words: int = 50
|
|
||||||
|
|
||||||
# --- Save Options ---
|
|
||||||
save_mode: SaveMode = SaveMode.SAVE_NEXT_TO_INPUT
|
|
||||||
output_folder: Optional[Path] = None
|
|
||||||
save_chapters_separately: bool = False
|
|
||||||
merge_chapters_at_end: bool = True
|
|
||||||
separate_chapters_format: OutputFormat = OutputFormat.WAV
|
|
||||||
save_as_project: bool = False
|
|
||||||
|
|
||||||
# --- Timing ---
|
# --- Timing ---
|
||||||
silence_between_chapters: float = 2.0
|
silence_between_chapters: float = 2.0
|
||||||
@@ -89,34 +85,28 @@ class ConversionRequest:
|
|||||||
auto_prefix_chapter_titles: bool = True
|
auto_prefix_chapter_titles: bool = True
|
||||||
normalize_chapter_opening_caps: bool = False
|
normalize_chapter_opening_caps: bool = False
|
||||||
|
|
||||||
# --- Pronunciation / Normalization ---
|
|
||||||
pronunciation_overrides: List[Dict[str, Any]] = field(default_factory=list)
|
|
||||||
manual_overrides: List[Dict[str, Any]] = field(default_factory=list)
|
|
||||||
heteronym_overrides: List[Dict[str, Any]] = field(default_factory=list)
|
|
||||||
normalization_overrides: Optional[Dict[str, Any]] = None
|
|
||||||
|
|
||||||
# --- Chapter/Chunk Configuration ---
|
|
||||||
chapter_overrides: List[Dict[str, Any]] = field(default_factory=list)
|
|
||||||
chunks: List[Dict[str, Any]] = field(default_factory=list)
|
|
||||||
chunk_level: str = "paragraph"
|
|
||||||
speaker_mode: str = "single"
|
|
||||||
speakers: Dict[str, Any] = field(default_factory=dict)
|
|
||||||
|
|
||||||
# --- Metadata ---
|
# --- Metadata ---
|
||||||
metadata_tags: Dict[str, Any] = field(default_factory=dict)
|
metadata_tags: Dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
# --- Artifacts ---
|
# --- Grouped configs ---
|
||||||
cover_image_path: Optional[Path] = None
|
subtitle: SubtitleConfig = field(default_factory=SubtitleConfig)
|
||||||
cover_image_mime: Optional[str] = None
|
save: SaveConfig = field(default_factory=SaveConfig)
|
||||||
generate_epub3: bool = False
|
cover: CoverConfig = field(default_factory=CoverConfig)
|
||||||
|
pronunciation: PronunciationConfig = field(default_factory=PronunciationConfig)
|
||||||
|
|
||||||
|
# --- Feature configs (None = disabled) ---
|
||||||
|
epub3_export: Optional[Epub3ExportConfig] = None
|
||||||
|
word_substitution: Optional[WordSubstitutionConfig] = None
|
||||||
|
subtitle_input: Optional[SubtitleInputConfig] = None
|
||||||
|
chapter_chunk: Optional[ChapterChunkConfig] = None
|
||||||
|
|
||||||
def __post_init__(self) -> None:
|
def __post_init__(self) -> None:
|
||||||
"""Resolve None → default, then validate and clamp."""
|
"""Resolve None → default, then validate and clamp."""
|
||||||
_apply_none_defaults(self)
|
_apply_none_defaults(self)
|
||||||
if not self.tts_provider:
|
if not self.tts_provider:
|
||||||
self.tts_provider = "kokoro"
|
self.tts_provider = "kokoro"
|
||||||
|
_coerce_enums(self)
|
||||||
_clamp_numerics(self)
|
_clamp_numerics(self)
|
||||||
_validate_enums(self)
|
|
||||||
|
|
||||||
|
|
||||||
def _apply_none_defaults(obj: ConversionRequest) -> None:
|
def _apply_none_defaults(obj: ConversionRequest) -> None:
|
||||||
@@ -130,6 +120,25 @@ def _apply_none_defaults(obj: ConversionRequest) -> None:
|
|||||||
setattr(obj, f.name, f.default_factory())
|
setattr(obj, f.name, f.default_factory())
|
||||||
|
|
||||||
|
|
||||||
|
# Enum fields that accept string coercion: attr -> (enum_class, fallback)
|
||||||
|
_ENUM_COERCIONS: dict[str, tuple[type, Any]] = {
|
||||||
|
"language": (Language, Language.EN_US),
|
||||||
|
"output_format": (OutputFormat, OutputFormat.WAV),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _coerce_enums(obj: ConversionRequest) -> None:
|
||||||
|
"""Coerce string values to their expected enum types."""
|
||||||
|
for attr, (enum_cls, fallback) in _ENUM_COERCIONS.items():
|
||||||
|
val = getattr(obj, attr)
|
||||||
|
if isinstance(val, enum_cls):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
setattr(obj, attr, enum_cls.from_str(str(val)))
|
||||||
|
except (ValueError, AttributeError):
|
||||||
|
setattr(obj, attr, fallback)
|
||||||
|
|
||||||
|
|
||||||
def _clamp_numerics(obj: ConversionRequest) -> None:
|
def _clamp_numerics(obj: ConversionRequest) -> None:
|
||||||
"""Clamp numeric fields to valid ranges."""
|
"""Clamp numeric fields to valid ranges."""
|
||||||
for attr, (min_v, max_v) in _NUMERIC_CONSTRAINTS.items():
|
for attr, (min_v, max_v) in _NUMERIC_CONSTRAINTS.items():
|
||||||
@@ -144,13 +153,3 @@ def _clamp_numerics(obj: ConversionRequest) -> None:
|
|||||||
if max_v is not None:
|
if max_v is not None:
|
||||||
clamped = min(max_v, clamped)
|
clamped = min(max_v, clamped)
|
||||||
setattr(obj, attr, clamped)
|
setattr(obj, attr, clamped)
|
||||||
|
|
||||||
|
|
||||||
def _validate_enums(obj: ConversionRequest) -> None:
|
|
||||||
"""Validate string enum fields against allowed values."""
|
|
||||||
for attr, allowed in _ENUM_CONSTRAINTS.items():
|
|
||||||
val = getattr(obj, attr)
|
|
||||||
if val not in allowed:
|
|
||||||
raise ConversionRequestError(
|
|
||||||
f"{attr} must be one of {allowed}, got {val!r}"
|
|
||||||
)
|
|
||||||
|
|||||||
@@ -37,6 +37,9 @@ class ConversionResult:
|
|||||||
total_segments: int = 0
|
total_segments: int = 0
|
||||||
total_characters: int = 0
|
total_characters: int = 0
|
||||||
|
|
||||||
|
# --- Override usage tracking ---
|
||||||
|
usage_counter: Dict[str, int] = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class ConversionError:
|
class ConversionError:
|
||||||
|
|||||||
@@ -15,42 +15,35 @@ The service NEVER imports from PyQt or WebUI.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from typing import Any, Callable, Dict, Optional
|
import logging
|
||||||
|
from collections import defaultdict
|
||||||
|
from typing import Any, Dict
|
||||||
|
|
||||||
from abogen.application.conversion_executor import execute_conversion
|
from abogen.application.conversion_executor import execute_conversion
|
||||||
from abogen.application.conversion_models import ConversionPlan
|
from abogen.application.conversion_models import ConversionPlan
|
||||||
from abogen.application.conversion_planner import build_conversion_plan
|
from abogen.application.conversion_planner import build_conversion_plan
|
||||||
from abogen.application.conversion_ports import (
|
from abogen.application.conversion_ports import ConversionEvents
|
||||||
ConversionEvents,
|
|
||||||
PipelineProvider,
|
|
||||||
VoiceResolver,
|
|
||||||
)
|
|
||||||
from abogen.application.conversion_request import ConversionRequest
|
from abogen.application.conversion_request import ConversionRequest
|
||||||
from abogen.application.conversion_result import ConversionResult
|
from abogen.application.conversion_result import ConversionResult
|
||||||
from abogen.domain.enums import SubtitleMode
|
from abogen.domain.normalization import build_tts_context
|
||||||
from abogen.domain.normalization import TTSContext
|
|
||||||
from abogen.domain.split_pattern import get_split_pattern
|
|
||||||
|
|
||||||
|
|
||||||
def run_conversion(
|
def run_conversion(
|
||||||
request: ConversionRequest,
|
request: ConversionRequest,
|
||||||
events: ConversionEvents,
|
events: ConversionEvents,
|
||||||
pipeline_provider: PipelineProvider,
|
|
||||||
voice_resolver: VoiceResolver,
|
|
||||||
) -> ConversionResult:
|
) -> ConversionResult:
|
||||||
"""Execute a conversion request and return the result.
|
"""Execute a conversion request and return the result.
|
||||||
|
|
||||||
This is the single entry point for both UIs. It orchestrates:
|
This is the single entry point for both UIs. It orchestrates:
|
||||||
1. TTS context preparation
|
1. Voice infrastructure setup (pool, cache, resolver)
|
||||||
2. Conversion planning
|
2. TTS context preparation
|
||||||
3. Conversion execution
|
3. Conversion planning
|
||||||
4. Resource cleanup
|
4. Conversion execution
|
||||||
|
5. Resource cleanup
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
request: Normalized conversion request
|
request: Normalized conversion request
|
||||||
events: UI-specific callbacks (log, progress, check_cancelled)
|
events: UI-specific callbacks (log, progress, check_cancelled)
|
||||||
pipeline_provider: Provides TTS backends
|
|
||||||
voice_resolver: Resolves voice specs into loaded voices
|
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
ConversionResult with paths and markers
|
ConversionResult with paths and markers
|
||||||
@@ -60,10 +53,30 @@ def run_conversion(
|
|||||||
ValueError: If request is invalid
|
ValueError: If request is invalid
|
||||||
Exception: On TTS or I/O errors
|
Exception: On TTS or I/O errors
|
||||||
"""
|
"""
|
||||||
|
from abogen.domain.pipeline_factory import PipelinePool
|
||||||
|
from abogen.domain.voice_loader import VoiceCache
|
||||||
|
|
||||||
|
pool = PipelinePool()
|
||||||
|
voice_cache = VoiceCache()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Stage 1: Prepare TTS context
|
# Stage 0: Create voice resolver
|
||||||
events.log("Preparing conversion pipeline")
|
events.log("Preparing conversion pipeline")
|
||||||
tts_context = _prepare_tts_context(request, events)
|
logging.info(
|
||||||
|
"[app] run_conversion: provider=%s language=%s voice=%s speed=%.2f",
|
||||||
|
request.tts_provider, request.language, request.voice, request.speed,
|
||||||
|
)
|
||||||
|
resolver = _create_voice_resolver(request, pool, voice_cache)
|
||||||
|
|
||||||
|
# Stage 1: Prepare TTS context
|
||||||
|
usage_counter: Dict[str, int] = defaultdict(int)
|
||||||
|
tts_context = build_tts_context(
|
||||||
|
language=request.language,
|
||||||
|
subtitle=request.subtitle,
|
||||||
|
pronunciation=request.pronunciation,
|
||||||
|
usage_counter=usage_counter,
|
||||||
|
log_callback=lambda level, msg: events.log(msg, level=level),
|
||||||
|
)
|
||||||
|
|
||||||
# Stage 2: Build conversion plan
|
# Stage 2: Build conversion plan
|
||||||
events.log("Building conversion plan")
|
events.log("Building conversion plan")
|
||||||
@@ -74,99 +87,164 @@ def run_conversion(
|
|||||||
result = execute_conversion(
|
result = execute_conversion(
|
||||||
plan=plan,
|
plan=plan,
|
||||||
events=events,
|
events=events,
|
||||||
pipeline_provider=pipeline_provider,
|
pipeline_provider=pool,
|
||||||
voice_resolver=voice_resolver,
|
voice_resolver=resolver,
|
||||||
tts_context=tts_context,
|
tts_context=tts_context,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Stage 4: Finalize
|
# Propagate usage counter to result
|
||||||
|
result.usage_counter = dict(usage_counter)
|
||||||
|
|
||||||
|
# Stage 4: Finalize (m4b metadata embedding, EPUB3 generation)
|
||||||
|
_finalize(request, result, plan, events)
|
||||||
|
|
||||||
events.log("Conversion complete")
|
events.log("Conversion complete")
|
||||||
|
logging.info("[app] run_conversion completed successfully")
|
||||||
return result
|
return result
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
events.log(f"Conversion failed: {e}", level="error")
|
events.log(f"Conversion failed: {e}", level="error")
|
||||||
|
logging.exception("[app] run_conversion failed: %s", e)
|
||||||
raise
|
raise
|
||||||
|
finally:
|
||||||
|
pool.dispose_all()
|
||||||
|
voice_cache.clear()
|
||||||
|
from abogen.application.cleanup import flush_cuda
|
||||||
|
flush_cuda()
|
||||||
|
|
||||||
|
|
||||||
def _prepare_tts_context(
|
def _create_voice_resolver(
|
||||||
request: ConversionRequest,
|
request: ConversionRequest,
|
||||||
events: ConversionEvents,
|
pool: Any,
|
||||||
) -> TTSContext:
|
cache: Any,
|
||||||
"""Prepare TTSContext with normalization settings.
|
) -> Any:
|
||||||
|
"""Create AppVoiceResolver with loaded profiles.
|
||||||
|
|
||||||
This compiles pronunciation/heteronym rules and creates the
|
Loads voice profiles from disk, normalizes them, and creates
|
||||||
normalization context used during conversion.
|
an AppVoiceResolver that can resolve voice specs into loaded voices.
|
||||||
|
|
||||||
Args:
|
|
||||||
request: Conversion request with override settings
|
|
||||||
events: For logging warnings about missing features
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
TTSContext ready for text normalization
|
|
||||||
"""
|
"""
|
||||||
from abogen.domain.normalization import (
|
from abogen.application.voice_resolver import AppVoiceResolver
|
||||||
build_apostrophe_config,
|
from abogen.voice_profiles import load_profiles, normalize_profile_entry
|
||||||
get_runtime_settings,
|
|
||||||
)
|
|
||||||
from abogen.domain.pronunciation import (
|
|
||||||
compile_heteronym_sentence_rules,
|
|
||||||
compile_pronunciation_rules,
|
|
||||||
merge_pronunciation_overrides,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Get runtime normalization settings
|
try:
|
||||||
normalization_settings = get_runtime_settings()
|
profiles = load_profiles()
|
||||||
|
except Exception:
|
||||||
|
profiles = {}
|
||||||
|
|
||||||
# Build apostrophe config
|
normalized_profiles: Dict[str, Dict[str, Any]] = {}
|
||||||
apostrophe_config = build_apostrophe_config(
|
for name, entry in (profiles or {}).items():
|
||||||
settings=normalization_settings,
|
normalized = normalize_profile_entry(entry)
|
||||||
)
|
if normalized:
|
||||||
|
normalized_profiles[str(name)] = normalized
|
||||||
|
|
||||||
|
return AppVoiceResolver(request, normalized_profiles, pool, cache)
|
||||||
|
|
||||||
|
|
||||||
|
def _finalize(
|
||||||
|
request: ConversionRequest,
|
||||||
|
result: ConversionResult,
|
||||||
|
plan: ConversionPlan,
|
||||||
|
events: ConversionEvents,
|
||||||
|
) -> None:
|
||||||
|
"""Post-conversion finalization (m4b metadata embedding, EPUB3 generation, etc.)."""
|
||||||
|
from abogen.domain.enums import OutputFormat
|
||||||
|
|
||||||
|
# m4b metadata embedding
|
||||||
|
if (
|
||||||
|
result.audio_path
|
||||||
|
and request.output_format == OutputFormat.M4B
|
||||||
|
):
|
||||||
|
|
||||||
|
from abogen.infrastructure.exporters import ExportService
|
||||||
|
|
||||||
|
export_svc = ExportService()
|
||||||
|
|
||||||
# Check for num2words availability
|
|
||||||
if apostrophe_config.convert_numbers:
|
|
||||||
try:
|
try:
|
||||||
import num2words # noqa: F401
|
export_svc.embed_m4b_metadata(
|
||||||
except ImportError:
|
audio_path=result.audio_path,
|
||||||
events.log(
|
metadata=result.metadata or {},
|
||||||
"Number normalization is enabled but 'num2words' library is not available. "
|
chapters=result.chapter_markers or [],
|
||||||
"Numbers will NOT be converted to words.",
|
cover=request.cover,
|
||||||
level="warning",
|
log_callback=lambda msg, level="info": events.log(msg, level=level),
|
||||||
)
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
events.log(f"Failed to embed m4b metadata: {exc}", level="error")
|
||||||
|
raise RuntimeError(f"Failed to embed m4b metadata: {exc}") from exc
|
||||||
|
|
||||||
# Compute split pattern
|
# EPUB3 generation
|
||||||
split_pattern = get_split_pattern(
|
if request.epub3_export and plan.extraction:
|
||||||
request.language or Language.EN_US,
|
audio_asset = result.audio_path
|
||||||
request.subtitle_mode or SubtitleMode.DISABLED,
|
if not audio_asset and result.chapter_paths:
|
||||||
)
|
audio_asset = result.chapter_paths[0]
|
||||||
|
|
||||||
# Merge pronunciation overrides (manual + pronunciation)
|
if audio_asset:
|
||||||
# Create a mock job-like object for merge_pronunciation_overrides
|
try:
|
||||||
class _MockJob:
|
|
||||||
def __init__(self, req):
|
|
||||||
self.pronunciation_overrides = req.pronunciation_overrides
|
|
||||||
self.manual_overrides = req.manual_overrides
|
|
||||||
self.heteronym_overrides = req.heteronym_overrides
|
|
||||||
|
|
||||||
merged_overrides = merge_pronunciation_overrides(_MockJob(request))
|
from abogen.epub3.exporter import build_epub3_package
|
||||||
|
|
||||||
# Compile rules
|
epub_root = result.project_root or plan.output_layout.parent_dir
|
||||||
pronunciation_rules = compile_pronunciation_rules(merged_overrides)
|
from abogen.domain.output_paths import build_output_path
|
||||||
heteronym_rules = compile_heteronym_sentence_rules(request.heteronym_overrides)
|
|
||||||
|
|
||||||
if heteronym_rules:
|
epub_output_path = build_output_path(epub_root, request.original_filename, "epub")
|
||||||
events.log(
|
events.log("Generating EPUB 3 package...")
|
||||||
f"Applying {len(heteronym_rules)} heteronym override(s) during conversion.",
|
epub_path = build_epub3_package(
|
||||||
level="debug",
|
output_path=epub_output_path,
|
||||||
)
|
book_id=request.epub3_export.book_id,
|
||||||
if pronunciation_rules:
|
extraction=plan.extraction,
|
||||||
events.log(
|
metadata_tags=result.metadata or {},
|
||||||
f"Applying {len(pronunciation_rules)} pronunciation override(s) during conversion.",
|
chapter_markers=result.chapter_markers or [],
|
||||||
level="debug",
|
chunk_markers=result.chunk_markers or [],
|
||||||
|
chunks=request.chapter_chunk.chunks if request.chapter_chunk else [],
|
||||||
|
audio_path=audio_asset,
|
||||||
|
speaker_mode=request.chapter_chunk.speaker_mode if request.chapter_chunk else "single",
|
||||||
|
cover=request.cover,
|
||||||
|
)
|
||||||
|
result.epub_path = epub_path
|
||||||
|
result.artifacts["epub3"] = epub_path
|
||||||
|
events.log(f"EPUB 3 package created at {epub_path}")
|
||||||
|
except Exception as exc:
|
||||||
|
events.log(f"Failed to generate EPUB 3: {exc}", level="error")
|
||||||
|
else:
|
||||||
|
events.log("Skipped EPUB 3 generation: audio output unavailable.", level="warning")
|
||||||
|
|
||||||
|
# Build metadata payload and write metadata.json
|
||||||
|
if plan.output_layout and plan.output_layout.metadata_dir:
|
||||||
|
from abogen.domain.metadata_helpers import build_metadata_payload
|
||||||
|
|
||||||
|
metadata_payload = build_metadata_payload(
|
||||||
|
metadata=result.metadata,
|
||||||
|
chapter_markers=result.chapter_markers,
|
||||||
|
chunk_markers=result.chunk_markers,
|
||||||
|
chunk_level=request.chapter_chunk.chunk_level if request.chapter_chunk else None,
|
||||||
|
speaker_mode=request.chapter_chunk.speaker_mode if request.chapter_chunk else None,
|
||||||
|
speakers=request.chapter_chunk.speakers if request.chapter_chunk else None,
|
||||||
|
generate_epub3=bool(request.epub3_export),
|
||||||
)
|
)
|
||||||
|
|
||||||
return TTSContext(
|
metadata_dir = plan.output_layout.metadata_dir
|
||||||
split_pattern=split_pattern,
|
metadata_dir.mkdir(parents=True, exist_ok=True)
|
||||||
pronunciation_rules=pronunciation_rules,
|
metadata_file = metadata_dir / "metadata.json"
|
||||||
heteronym_rules=heteronym_rules,
|
|
||||||
normalization_overrides=request.normalization_overrides,
|
import json
|
||||||
)
|
|
||||||
|
metadata_file.write_text(json.dumps(metadata_payload, indent=2), encoding="utf-8")
|
||||||
|
result.artifacts["metadata"] = metadata_file
|
||||||
|
events.log(f"Metadata written to {metadata_file}")
|
||||||
|
|
||||||
|
# Record override usage
|
||||||
|
if result.usage_counter:
|
||||||
|
try:
|
||||||
|
from abogen.normalization_settings import record_override_usage
|
||||||
|
|
||||||
|
record_override_usage(result.usage_counter)
|
||||||
|
except Exception as exc:
|
||||||
|
events.log(f"Failed to record override usage: {exc}", level="debug")
|
||||||
|
|
||||||
|
# Post-conversion hooks (Audiobookshelf, etc.)
|
||||||
|
from abogen.application.integration_hooks import PostConversionHooks
|
||||||
|
|
||||||
|
hooks = PostConversionHooks()
|
||||||
|
hooks.run(request, result, events)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,164 @@
|
|||||||
|
"""Post-conversion integration hooks.
|
||||||
|
|
||||||
|
Called by ConversionService after finalization.
|
||||||
|
Each integration is a method on PostConversionHooks — isolated, testable,
|
||||||
|
and easy to extend with new hooks (Plex, Navidrome, etc.).
|
||||||
|
|
||||||
|
The service NEVER imports from PyQt or WebUI.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Mapping, Optional
|
||||||
|
|
||||||
|
from abogen.application.conversion_ports import ConversionEvents
|
||||||
|
from abogen.application.conversion_request import ConversionRequest
|
||||||
|
from abogen.application.conversion_result import ConversionResult
|
||||||
|
from abogen.domain.metadata_helpers import (
|
||||||
|
build_audiobookshelf_metadata as _build_abs_metadata,
|
||||||
|
load_audiobookshelf_chapters as _load_abs_chapters,
|
||||||
|
)
|
||||||
|
from abogen.domain.settings_core import (
|
||||||
|
build_audiobookshelf_config,
|
||||||
|
coerce_bool,
|
||||||
|
load_audiobookshelf_config,
|
||||||
|
stored_integration_config,
|
||||||
|
)
|
||||||
|
from abogen.integrations.audiobookshelf import (
|
||||||
|
AudiobookshelfClient,
|
||||||
|
AudiobookshelfUploadError,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class PostConversionHooks:
|
||||||
|
"""Runs post-conversion integrations (Audiobookshelf, etc.).
|
||||||
|
|
||||||
|
Usage::
|
||||||
|
|
||||||
|
hooks = PostConversionHooks()
|
||||||
|
hooks.run(request, result, events)
|
||||||
|
"""
|
||||||
|
|
||||||
|
def run(
|
||||||
|
self,
|
||||||
|
request: ConversionRequest,
|
||||||
|
result: ConversionResult,
|
||||||
|
events: ConversionEvents,
|
||||||
|
) -> None:
|
||||||
|
"""Run all registered post-conversion hooks."""
|
||||||
|
self._maybe_send_to_audiobookshelf(request, result, events)
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Audiobookshelf
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _maybe_send_to_audiobookshelf(
|
||||||
|
self,
|
||||||
|
request: ConversionRequest,
|
||||||
|
result: ConversionResult,
|
||||||
|
events: ConversionEvents,
|
||||||
|
) -> None:
|
||||||
|
"""Upload finished audiobook to Audiobookshelf if enabled."""
|
||||||
|
abs_settings = stored_integration_config("audiobookshelf")
|
||||||
|
if not abs_settings:
|
||||||
|
return
|
||||||
|
|
||||||
|
enabled = coerce_bool(abs_settings.get("enabled"), False)
|
||||||
|
auto_send = coerce_bool(abs_settings.get("auto_send"), False)
|
||||||
|
if not (enabled and auto_send):
|
||||||
|
return
|
||||||
|
|
||||||
|
config = build_audiobookshelf_config(abs_settings)
|
||||||
|
if config is None:
|
||||||
|
events.log(
|
||||||
|
"Audiobookshelf upload skipped: configure base URL, API token, "
|
||||||
|
"library ID, and folder ID first.",
|
||||||
|
level="warning",
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
audio_path = result.audio_path
|
||||||
|
if not audio_path or not audio_path.exists():
|
||||||
|
events.log(
|
||||||
|
"Audiobookshelf upload skipped: audio output not found.",
|
||||||
|
level="warning",
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Build metadata
|
||||||
|
filename = request.original_filename or "Audiobook"
|
||||||
|
lang = request.language.value if hasattr(request.language, "value") else str(request.language)
|
||||||
|
metadata = _build_abs_metadata(
|
||||||
|
result.metadata or {},
|
||||||
|
language=lang,
|
||||||
|
filename=Path(filename).stem,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Load chapters from metadata artifact
|
||||||
|
chapters = None
|
||||||
|
if config.send_chapters:
|
||||||
|
metadata_artifact = result.artifacts.get("metadata")
|
||||||
|
if metadata_artifact:
|
||||||
|
metadata_path = (
|
||||||
|
metadata_artifact
|
||||||
|
if isinstance(metadata_artifact, Path)
|
||||||
|
else Path(str(metadata_artifact))
|
||||||
|
)
|
||||||
|
chapters = _load_abs_chapters(metadata_path)
|
||||||
|
|
||||||
|
# Resolve cover
|
||||||
|
cover_path = None
|
||||||
|
if config.send_cover and request.cover and request.cover.path:
|
||||||
|
candidate = request.cover.path
|
||||||
|
if isinstance(candidate, Path) and candidate.exists():
|
||||||
|
cover_path = candidate
|
||||||
|
|
||||||
|
# Resolve subtitles
|
||||||
|
subtitles = None
|
||||||
|
if config.send_subtitles and result.subtitle_paths:
|
||||||
|
subtitles = [
|
||||||
|
p for p in result.subtitle_paths
|
||||||
|
if isinstance(p, Path) and p.exists()
|
||||||
|
]
|
||||||
|
|
||||||
|
# Upload
|
||||||
|
client = AudiobookshelfClient(config)
|
||||||
|
display_title = metadata.get("title") or audio_path.stem
|
||||||
|
|
||||||
|
try:
|
||||||
|
existing_items = client.find_existing_items(
|
||||||
|
display_title, folder_id=config.folder_id,
|
||||||
|
)
|
||||||
|
except AudiobookshelfUploadError as exc:
|
||||||
|
events.log(f"Audiobookshelf lookup failed: {exc}", level="error")
|
||||||
|
return
|
||||||
|
|
||||||
|
if existing_items:
|
||||||
|
events.log(
|
||||||
|
f"Removing existing Audiobookshelf item(s) for '{display_title}'.",
|
||||||
|
level="info",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
client.delete_items(existing_items)
|
||||||
|
except Exception as exc:
|
||||||
|
events.log(
|
||||||
|
f"Failed to remove existing item(s): {exc}", level="warning",
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
client.upload_audiobook(
|
||||||
|
audio_path,
|
||||||
|
metadata=metadata,
|
||||||
|
cover_path=cover_path,
|
||||||
|
chapters=chapters,
|
||||||
|
subtitles=subtitles,
|
||||||
|
)
|
||||||
|
events.log("Audiobookshelf upload queued.", level="info")
|
||||||
|
except AudiobookshelfUploadError as exc:
|
||||||
|
events.log(f"Audiobookshelf upload failed: {exc}", level="error")
|
||||||
|
except Exception as exc:
|
||||||
|
events.log(f"Audiobookshelf integration error: {exc}", level="error")
|
||||||
@@ -15,7 +15,6 @@ Responsibilities:
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Optional
|
|
||||||
|
|
||||||
from abogen.application.conversion_models import OutputLayout
|
from abogen.application.conversion_models import OutputLayout
|
||||||
from abogen.application.conversion_request import ConversionRequest
|
from abogen.application.conversion_request import ConversionRequest
|
||||||
@@ -40,8 +39,8 @@ def resolve_output_layout(request: ConversionRequest) -> OutputLayout:
|
|||||||
OutputLayout with resolved paths
|
OutputLayout with resolved paths
|
||||||
"""
|
"""
|
||||||
# Determine base output directory
|
# Determine base output directory
|
||||||
if request.save_mode == SaveMode.CUSTOM_FOLDER and request.output_folder:
|
if request.save.mode == SaveMode.CUSTOM_FOLDER and request.save.output_folder:
|
||||||
parent_dir = Path(request.output_folder)
|
parent_dir = Path(request.save.output_folder)
|
||||||
elif request.source_path:
|
elif request.source_path:
|
||||||
parent_dir = request.source_path.parent
|
parent_dir = request.source_path.parent
|
||||||
else:
|
else:
|
||||||
@@ -67,7 +66,7 @@ def resolve_output_layout(request: ConversionRequest) -> OutputLayout:
|
|||||||
subtitle_dir = None
|
subtitle_dir = None
|
||||||
metadata_dir = None
|
metadata_dir = None
|
||||||
|
|
||||||
if request.save_as_project:
|
if request.save.save_as_project:
|
||||||
project_root, audio_dir, subtitle_dir, metadata_dir = resolve_project_layout(
|
project_root, audio_dir, subtitle_dir, metadata_dir = resolve_project_layout(
|
||||||
original_filename=request.original_filename,
|
original_filename=request.original_filename,
|
||||||
save_as_project=True,
|
save_as_project=True,
|
||||||
@@ -99,7 +98,7 @@ def resolve_merged_path(
|
|||||||
base_name = sanitize_output_stem(
|
base_name = sanitize_output_stem(
|
||||||
request.original_filename or "output"
|
request.original_filename or "output"
|
||||||
)
|
)
|
||||||
return layout.audio_dir / f"{base_name}.{request.output_format}"
|
return layout.audio_dir / f"{base_name}{request.output_format.dot_ext}"
|
||||||
|
|
||||||
|
|
||||||
def resolve_chapter_path(
|
def resolve_chapter_path(
|
||||||
@@ -125,7 +124,7 @@ def resolve_chapter_path(
|
|||||||
slug = re.sub(r'[\s_]+', '_', slug).strip('_')
|
slug = re.sub(r'[\s_]+', '_', slug).strip('_')
|
||||||
if not slug:
|
if not slug:
|
||||||
slug = f"chapter_{chapter_index}"
|
slug = f"chapter_{chapter_index}"
|
||||||
filename = f"{chapter_index:02d}_{slug}.{request.separate_chapters_format}"
|
filename = f"{chapter_index:02d}_{slug}.{request.save.separate_chapters_format}"
|
||||||
return layout.audio_dir / "chapters" / filename
|
return layout.audio_dir / "chapters" / filename
|
||||||
|
|
||||||
|
|
||||||
@@ -145,6 +144,6 @@ def should_merge_output(request: ConversionRequest) -> bool:
|
|||||||
"""
|
"""
|
||||||
if request.output_format == OutputFormat.M4B:
|
if request.output_format == OutputFormat.M4B:
|
||||||
return True
|
return True
|
||||||
if not request.save_chapters_separately:
|
if not request.save.save_chapters_separately:
|
||||||
return True
|
return True
|
||||||
return request.merge_chapters_at_end
|
return request.save.merge_chapters_at_end
|
||||||
|
|||||||
@@ -0,0 +1,81 @@
|
|||||||
|
"""AppVoiceResolver — voice resolution inside the application layer.
|
||||||
|
|
||||||
|
Resolves voice specs into loaded voices using profiles, pipeline pool,
|
||||||
|
and voice cache. Replaces UI-specific resolvers (WebUIVoiceResolver,
|
||||||
|
PyQtVoiceResolver) with a single app-layer implementation.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from typing import Any, Dict, Optional
|
||||||
|
|
||||||
|
from abogen.application.conversion_ports import ResolvedVoice, VoiceResolver
|
||||||
|
from abogen.application.conversion_request import ConversionRequest
|
||||||
|
from abogen.domain.pipeline_factory import PipelinePool
|
||||||
|
from abogen.domain.voice_loader import VoiceCache, resolve_voice
|
||||||
|
from abogen.domain.voice_utils import resolve_voice_target
|
||||||
|
|
||||||
|
|
||||||
|
class AppVoiceResolver:
|
||||||
|
"""App-layer implementation of VoiceResolver protocol.
|
||||||
|
|
||||||
|
Uses ConversionRequest instead of Job. Loads profiles, creates
|
||||||
|
resolver internally — UIs don't need to manage this.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
request: ConversionRequest,
|
||||||
|
normalized_profiles: Dict[str, Dict[str, Any]],
|
||||||
|
pool: PipelinePool,
|
||||||
|
cache: VoiceCache,
|
||||||
|
):
|
||||||
|
self._request = request
|
||||||
|
self._profiles = normalized_profiles
|
||||||
|
self._cache = cache
|
||||||
|
self._pool = pool
|
||||||
|
|
||||||
|
def resolve(self, voice_spec: str) -> ResolvedVoice:
|
||||||
|
"""Resolve a voice spec into a loaded voice."""
|
||||||
|
provider, resolved, speed, steps = resolve_voice_target(
|
||||||
|
voice_spec,
|
||||||
|
self._profiles,
|
||||||
|
job_voice=self._request.voice,
|
||||||
|
job_tts_provider=self._request.tts_provider,
|
||||||
|
job_supertonic_total_steps=self._request.supertonic_total_steps,
|
||||||
|
job_speed=self._request.speed,
|
||||||
|
)
|
||||||
|
|
||||||
|
cache_key = f"{provider}:{resolved}" if resolved else provider
|
||||||
|
cached = self._cache.get(cache_key)
|
||||||
|
if cached is not None:
|
||||||
|
logging.info("[resolver] Cache hit: spec=%s -> provider=%s resolved=%s", voice_spec, provider, resolved)
|
||||||
|
return ResolvedVoice(
|
||||||
|
provider=provider,
|
||||||
|
resolved_spec=resolved,
|
||||||
|
voice=cached,
|
||||||
|
speed=speed,
|
||||||
|
supertonic_steps=steps or 0,
|
||||||
|
)
|
||||||
|
|
||||||
|
if provider == "kokoro":
|
||||||
|
kokoro_backend = self._pool.get(
|
||||||
|
"kokoro", self._request.language, self._request.use_gpu,
|
||||||
|
)
|
||||||
|
loaded = resolve_voice(
|
||||||
|
resolved, kokoro_backend, self._request.use_gpu, cache=self._cache,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
loaded = resolved
|
||||||
|
|
||||||
|
self._cache.set(cache_key, loaded)
|
||||||
|
logging.info("[resolver] Resolved: spec=%s -> provider=%s resolved=%s speed=%.2f steps=%s",
|
||||||
|
voice_spec, provider, resolved, speed, steps)
|
||||||
|
return ResolvedVoice(
|
||||||
|
provider=provider,
|
||||||
|
resolved_spec=resolved,
|
||||||
|
voice=loaded,
|
||||||
|
speed=speed,
|
||||||
|
supertonic_steps=steps or 0,
|
||||||
|
)
|
||||||
@@ -12,7 +12,8 @@ import fitz # PyMuPDF
|
|||||||
import markdown
|
import markdown
|
||||||
|
|
||||||
from abogen.utils import detect_encoding
|
from abogen.utils import detect_encoding
|
||||||
from abogen.subtitle_utils import clean_text, calculate_text_length
|
from abogen.subtitle_utils import clean_text
|
||||||
|
from abogen.domain.text_utils import calculate_text_length
|
||||||
|
|
||||||
# Pre-compile frequently used regex patterns
|
# Pre-compile frequently used regex patterns
|
||||||
_BRACKETED_NUMBERS_PATTERN = re.compile(r"\[\s*\d+\s*\]")
|
_BRACKETED_NUMBERS_PATTERN = re.compile(r"\[\s*\d+\s*\]")
|
||||||
|
|||||||
+31
-17
@@ -1,4 +1,5 @@
|
|||||||
from abogen.utils import get_version
|
from abogen.utils import get_version
|
||||||
|
from abogen.domain.enums import Language
|
||||||
|
|
||||||
# Program Information
|
# Program Information
|
||||||
PROGRAM_NAME = "abogen"
|
PROGRAM_NAME = "abogen"
|
||||||
@@ -16,8 +17,22 @@ SUBTITLE_FORMATS = [
|
|||||||
("ass_centered_narrow", "ASS (centered narrow)"),
|
("ass_centered_narrow", "ASS (centered narrow)"),
|
||||||
]
|
]
|
||||||
|
|
||||||
# Language description mapping
|
# Language description mapping (Language enum → human-readable label).
|
||||||
LANGUAGE_DESCRIPTIONS = {
|
LANGUAGE_DESCRIPTIONS = {
|
||||||
|
Language.EN_US: "American English",
|
||||||
|
Language.EN_GB: "British English",
|
||||||
|
Language.ES: "Spanish",
|
||||||
|
Language.FR: "French",
|
||||||
|
Language.HI: "Hindi",
|
||||||
|
Language.IT: "Italian",
|
||||||
|
Language.JA: "Japanese",
|
||||||
|
Language.PT_BR: "Brazilian Portuguese",
|
||||||
|
Language.ZH: "Mandarin Chinese",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Display-only mapping for kokoro codes → labels.
|
||||||
|
# Used by voice catalog and PyQt (legacy) where kokoro codes are still present.
|
||||||
|
KOKORO_CODE_LABELS = {
|
||||||
"a": "American English",
|
"a": "American English",
|
||||||
"b": "British English",
|
"b": "British English",
|
||||||
"e": "Spanish",
|
"e": "Spanish",
|
||||||
@@ -55,25 +70,24 @@ SUPPORTED_INPUT_FORMATS = [
|
|||||||
"vtt",
|
"vtt",
|
||||||
]
|
]
|
||||||
|
|
||||||
# Supported languages for subtitle generation
|
# Supported languages for subtitle generation.
|
||||||
# Currently, only 'a (American English)' and 'b (British English)' are supported for subtitle generation.
|
# All languages are supported: only English emits per-word timestamped tokens
|
||||||
# This is because tokens that contain timestamps are not generated for other languages in the Kokoro pipeline.
|
# in the Kokoro pipeline, but other languages fall back to segment-level fake
|
||||||
# Please refer to: https://github.com/hexgrad/kokoro/blob/6d87f4ae7abc2d14dbc4b3ef2e5f19852e861ac2/kokoro/pipeline.py
|
# tokens (see abogen.domain.tokens.FakeToken), so subtitles are still
|
||||||
# 383 English processing (unchanged)
|
# generated at segment granularity.
|
||||||
# 384 if self.lang_code in 'ab':
|
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION = list(Language)
|
||||||
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION = list(LANGUAGE_DESCRIPTIONS.keys())
|
|
||||||
|
|
||||||
# Voice and sample text mapping
|
# Voice and sample text mapping
|
||||||
SAMPLE_VOICE_TEXTS = {
|
SAMPLE_VOICE_TEXTS = {
|
||||||
"a": "This is a sample of the selected voice.",
|
Language.EN_US: "This is a sample of the selected voice.",
|
||||||
"b": "This is a sample of the selected voice.",
|
Language.EN_GB: "This is a sample of the selected voice.",
|
||||||
"e": "Este es una muestra de la voz seleccionada.",
|
Language.ES: "Este es una muestra de la voz seleccionada.",
|
||||||
"f": "Ceci est un exemple de la voix sélectionnée.",
|
Language.FR: "Ceci est un exemple de la voix sélectionnée.",
|
||||||
"h": "यह चयनित आवाज़ का एक नमूना है।",
|
Language.HI: "यह चयनित आवाज़ का एक नमूना है।",
|
||||||
"i": "Questo è un esempio della voce selezionata.",
|
Language.IT: "Questo è un esempio della voce selezionata.",
|
||||||
"j": "これは選択した声のサンプルです。",
|
Language.JA: "これは選択した声のサンプルです。",
|
||||||
"p": "Este é um exemplo da voz selecionada.",
|
Language.PT_BR: "Este é um exemplo da voz selecionada.",
|
||||||
"z": "这是所选语音的示例。",
|
Language.ZH: "这是所选语音的示例。",
|
||||||
}
|
}
|
||||||
|
|
||||||
COLORS = {
|
COLORS = {
|
||||||
|
|||||||
@@ -33,7 +33,7 @@ def build_ffmpeg_command(path: Path, fmt: str, metadata: Optional[Dict[str, str]
|
|||||||
if fmt == "mp3":
|
if fmt == "mp3":
|
||||||
base += ["-c:a", "libmp3lame", "-qscale:a", "2"]
|
base += ["-c:a", "libmp3lame", "-qscale:a", "2"]
|
||||||
elif fmt == "opus":
|
elif fmt == "opus":
|
||||||
base += ["-c:a", "libopus", "-b:a", "24000"]
|
base += ["-c:a", "libopus", "-b:a", "128000"]
|
||||||
elif fmt == "m4b":
|
elif fmt == "m4b":
|
||||||
base += ["-c:a", "aac", "-q:a", "2", "-movflags", "+faststart+use_metadata_tags"]
|
base += ["-c:a", "aac", "-q:a", "2", "-movflags", "+faststart+use_metadata_tags"]
|
||||||
else:
|
else:
|
||||||
|
|||||||
@@ -7,8 +7,9 @@ text for TTS synthesis.
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from collections import defaultdict
|
from collections import defaultdict
|
||||||
from typing import Any, Dict, Iterable, Mapping, Optional
|
from typing import Any, Dict, Iterable, Mapping
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.pronunciation_store import increment_usage
|
from abogen.pronunciation_store import increment_usage
|
||||||
|
|
||||||
|
|
||||||
@@ -44,7 +45,7 @@ def record_override_usage(
|
|||||||
if not usage_counter:
|
if not usage_counter:
|
||||||
return
|
return
|
||||||
|
|
||||||
language = getattr(job, "language", "") or "a"
|
language = getattr(job, "language", Language.EN_US) or Language.EN_US
|
||||||
for normalized, amount in usage_counter.items():
|
for normalized, amount in usage_counter.items():
|
||||||
if amount <= 0:
|
if amount <= 0:
|
||||||
continue
|
continue
|
||||||
|
|||||||
@@ -0,0 +1,52 @@
|
|||||||
|
"""Domain config types — shared contracts for domain functions.
|
||||||
|
|
||||||
|
These dataclasses group parameters that domain functions receive.
|
||||||
|
Domain defines them, app layer fills them.
|
||||||
|
|
||||||
|
Why here (domain) and not application:
|
||||||
|
- build_tts_context() is in domain → needs PronunciationConfig
|
||||||
|
- make_subtitle_writer() is in infrastructure → needs SubtitleConfig
|
||||||
|
- embed_m4b_metadata() is in infrastructure → needs CoverConfig
|
||||||
|
- Domain should not depend on application layer (DIP)
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
|
from abogen.domain.enums import SubtitleFormat, SubtitleMode
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PronunciationConfig:
|
||||||
|
"""Pronunciation and normalization override settings.
|
||||||
|
|
||||||
|
Used by build_tts_context() to compile override rules.
|
||||||
|
"""
|
||||||
|
pronunciation_overrides: List[Dict[str, Any]] = field(default_factory=list)
|
||||||
|
manual_overrides: List[Dict[str, Any]] = field(default_factory=list)
|
||||||
|
heteronym_overrides: List[Dict[str, Any]] = field(default_factory=list)
|
||||||
|
normalization_overrides: Optional[Dict[str, Any]] = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class SubtitleConfig:
|
||||||
|
"""Subtitle output settings.
|
||||||
|
|
||||||
|
Used by make_subtitle_writer() and process_and_write_subtitles().
|
||||||
|
"""
|
||||||
|
mode: SubtitleMode = SubtitleMode.DISABLED
|
||||||
|
format: SubtitleFormat = SubtitleFormat.SRT
|
||||||
|
max_words: int = 50
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class CoverConfig:
|
||||||
|
"""Cover image settings.
|
||||||
|
|
||||||
|
Used by embed_m4b_metadata() and build_epub3_package().
|
||||||
|
"""
|
||||||
|
path: Optional[Path] = None
|
||||||
|
mime: Optional[str] = None
|
||||||
@@ -19,11 +19,11 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import time
|
import time
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any, Callable, List, Optional, Protocol
|
from typing import Any, Callable, Optional, Protocol
|
||||||
|
|
||||||
from abogen.domain.audio_sink import AudioSink
|
from abogen.domain.audio_sink import AudioSink
|
||||||
from abogen.domain.conversion_pipeline import tts_segments
|
from abogen.domain.conversion_pipeline import tts_segments
|
||||||
from abogen.domain.enums import SubtitleMode
|
from abogen.domain.enums import Language, SubtitleMode
|
||||||
from abogen.domain.normalization import TTSContext
|
from abogen.domain.normalization import TTSContext
|
||||||
from abogen.domain.progress import calc_etr_str
|
from abogen.domain.progress import calc_etr_str
|
||||||
from abogen.domain.subtitle_generation import process_subtitle_tokens
|
from abogen.domain.subtitle_generation import process_subtitle_tokens
|
||||||
@@ -61,6 +61,7 @@ def run_tts_segment_loop(
|
|||||||
voice: Any,
|
voice: Any,
|
||||||
speed: float,
|
speed: float,
|
||||||
split_pattern: str,
|
split_pattern: str,
|
||||||
|
total_steps: Optional[int] = None,
|
||||||
chapter_sink: Optional[AudioSink] = None,
|
chapter_sink: Optional[AudioSink] = None,
|
||||||
preview_callback: Optional[Callable[[str], None]] = None,
|
preview_callback: Optional[Callable[[str], None]] = None,
|
||||||
on_segment: Optional[Callable[[SegmentInfo], None]] = None,
|
on_segment: Optional[Callable[[SegmentInfo], None]] = None,
|
||||||
@@ -74,6 +75,7 @@ def run_tts_segment_loop(
|
|||||||
voice: Voice name/id for the backend.
|
voice: Voice name/id for the backend.
|
||||||
speed: Speech speed multiplier.
|
speed: Speech speed multiplier.
|
||||||
split_pattern: Regex pattern used by the TTS engine for sentence splitting.
|
split_pattern: Regex pattern used by the TTS engine for sentence splitting.
|
||||||
|
total_steps: Inference quality steps (Supertonic only, ignored by Kokoro).
|
||||||
preview_callback: Called with a short preview string per segment.
|
preview_callback: Called with a short preview string per segment.
|
||||||
on_segment: Called with a SegmentInfo for each segment *before*
|
on_segment: Called with a SegmentInfo for each segment *before*
|
||||||
audio is written. Useful for callers that need per-segment
|
audio is written. Useful for callers that need per-segment
|
||||||
@@ -95,6 +97,7 @@ def run_tts_segment_loop(
|
|||||||
speed=speed,
|
speed=speed,
|
||||||
split_pattern=split_pattern,
|
split_pattern=split_pattern,
|
||||||
current_time=params.stats.current_time,
|
current_time=params.stats.current_time,
|
||||||
|
total_steps=total_steps,
|
||||||
):
|
):
|
||||||
if params.check_cancel():
|
if params.check_cancel():
|
||||||
break
|
break
|
||||||
@@ -151,25 +154,35 @@ def process_and_write_subtitles(
|
|||||||
accumulated_tokens: list[dict],
|
accumulated_tokens: list[dict],
|
||||||
subtitle_writer: Any,
|
subtitle_writer: Any,
|
||||||
*,
|
*,
|
||||||
subtitle_mode: str,
|
subtitle: "SubtitleConfig | str",
|
||||||
max_subtitle_words: int,
|
max_subtitle_words: int | None = None,
|
||||||
lang_code: str,
|
language: Language,
|
||||||
use_spacy_segmentation: bool,
|
use_spacy_segmentation: bool,
|
||||||
fallback_end_time: float,
|
fallback_end_time: float,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Process accumulated subtitle tokens and write entries to a subtitle writer.
|
"""Process accumulated subtitle tokens and write entries to a subtitle writer.
|
||||||
|
|
||||||
This is the standard subtitle post-processing step shared by both UIs.
|
Accepts a SubtitleConfig object or a subtitle mode string
|
||||||
|
for backward compatibility.
|
||||||
"""
|
"""
|
||||||
|
from abogen.domain.config_types import SubtitleConfig
|
||||||
|
|
||||||
|
if isinstance(subtitle, SubtitleConfig):
|
||||||
|
mode_str = subtitle.mode.value
|
||||||
|
words = subtitle.max_words
|
||||||
|
else:
|
||||||
|
mode_str = subtitle
|
||||||
|
words = max_subtitle_words or 50
|
||||||
|
|
||||||
if not accumulated_tokens or not subtitle_writer:
|
if not accumulated_tokens or not subtitle_writer:
|
||||||
return
|
return
|
||||||
new_entries: list[tuple] = []
|
new_entries: list[tuple] = []
|
||||||
process_subtitle_tokens(
|
process_subtitle_tokens(
|
||||||
accumulated_tokens,
|
accumulated_tokens,
|
||||||
new_entries,
|
new_entries,
|
||||||
max_subtitle_words,
|
words,
|
||||||
subtitle_mode,
|
mode_str,
|
||||||
lang_code,
|
language,
|
||||||
use_spacy_segmentation=use_spacy_segmentation,
|
use_spacy_segmentation=use_spacy_segmentation,
|
||||||
fallback_end_time=fallback_end_time,
|
fallback_end_time=fallback_end_time,
|
||||||
)
|
)
|
||||||
@@ -191,7 +204,7 @@ class SynthParams:
|
|||||||
audio_sink: Optional[AudioSink] = None
|
audio_sink: Optional[AudioSink] = None
|
||||||
subtitle_mode: str = "Disabled"
|
subtitle_mode: str = "Disabled"
|
||||||
max_subtitle_words: int = 50
|
max_subtitle_words: int = 50
|
||||||
lang_code: str = "a"
|
language: Language = Language.EN_US
|
||||||
use_spacy_segmentation: bool = False
|
use_spacy_segmentation: bool = False
|
||||||
|
|
||||||
|
|
||||||
@@ -202,6 +215,7 @@ def synthesize_text(
|
|||||||
backend: Any,
|
backend: Any,
|
||||||
voice: Any,
|
voice: Any,
|
||||||
speed: float,
|
speed: float,
|
||||||
|
total_steps: Optional[int] = None,
|
||||||
chapter_sink: Optional[AudioSink] = None,
|
chapter_sink: Optional[AudioSink] = None,
|
||||||
preview_callback: Optional[Callable[[str], None]] = None,
|
preview_callback: Optional[Callable[[str], None]] = None,
|
||||||
on_segment: Optional[Callable[[SegmentInfo], None]] = None,
|
on_segment: Optional[Callable[[SegmentInfo], None]] = None,
|
||||||
@@ -219,6 +233,7 @@ def synthesize_text(
|
|||||||
backend=backend,
|
backend=backend,
|
||||||
voice=voice,
|
voice=voice,
|
||||||
speed=speed,
|
speed=speed,
|
||||||
|
total_steps=total_steps,
|
||||||
split_pattern=split_pattern_override or params.tts_context.split_pattern,
|
split_pattern=split_pattern_override or params.tts_context.split_pattern,
|
||||||
chapter_sink=chapter_sink,
|
chapter_sink=chapter_sink,
|
||||||
preview_callback=preview_callback,
|
preview_callback=preview_callback,
|
||||||
|
|||||||
@@ -9,8 +9,8 @@ from __future__ import annotations
|
|||||||
import logging
|
import logging
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
from abogen.domain.enums import SubtitleMode
|
from abogen.domain.enums import Language, SubtitleMode
|
||||||
from typing import Any, Callable, Dict, Iterator, List, Optional
|
from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple
|
||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
@@ -21,6 +21,109 @@ from abogen.domain.audio_buffer import SAMPLE_RATE
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# Languages where spaCy is used for pre-TTS segmentation
|
||||||
|
# English ("a", "b") is excluded — spaCy only used for post-TTS subtitles
|
||||||
|
_SPACY_EXCLUDED_LANGS = {Language.EN_US, Language.EN_GB}
|
||||||
|
|
||||||
|
# CJK languages — different spacing pattern
|
||||||
|
_CJK_LANGS = {Language.ZH, Language.JA}
|
||||||
|
|
||||||
|
|
||||||
|
def spacy_pre_tts_segmentation(
|
||||||
|
text: str,
|
||||||
|
lang_code: Any,
|
||||||
|
subtitle_mode: Any,
|
||||||
|
*,
|
||||||
|
is_subtitle_input: bool = False,
|
||||||
|
use_spacy_segmentation: bool = True,
|
||||||
|
log_callback: Optional[Callable[[str], None]] = None,
|
||||||
|
) -> Tuple[List[str], str]:
|
||||||
|
"""Segment text using spaCy before TTS, with split_pattern override.
|
||||||
|
|
||||||
|
For non-English languages, spaCy sentence segmentation produces better
|
||||||
|
sentence boundaries than regex. This function:
|
||||||
|
1. Checks if spaCy should be used (toggle on, not disabled mode, not subtitle input)
|
||||||
|
2. For non-English: runs spaCy segmentation, computes split_pattern override
|
||||||
|
3. For English: returns single segment with default pattern (spaCy only for subtitles)
|
||||||
|
4. If spaCy fails: falls back to default pattern
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Text to segment.
|
||||||
|
lang_code: Language code (Language enum or string like "a", "de", "fr").
|
||||||
|
subtitle_mode: SubtitleMode enum or string.
|
||||||
|
is_subtitle_input: True if source is .srt/.ass/.vtt file.
|
||||||
|
use_spacy_segmentation: User toggle for spaCy segmentation.
|
||||||
|
log_callback: Optional logging function.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tuple of (text_segments, active_split_pattern).
|
||||||
|
text_segments is a list of sentences (always at least one element).
|
||||||
|
active_split_pattern is the regex to use for TTS backend splitting.
|
||||||
|
"""
|
||||||
|
from abogen.domain.split_pattern import get_split_pattern
|
||||||
|
|
||||||
|
def _log(msg: str) -> None:
|
||||||
|
if log_callback:
|
||||||
|
log_callback(msg)
|
||||||
|
|
||||||
|
# Normalize language
|
||||||
|
lang_enum = _to_language_enum(lang_code)
|
||||||
|
|
||||||
|
# Default split pattern
|
||||||
|
default_split = get_split_pattern(lang_code, subtitle_mode)
|
||||||
|
|
||||||
|
# Check conditions
|
||||||
|
if not use_spacy_segmentation:
|
||||||
|
return [text], default_split
|
||||||
|
|
||||||
|
subtitle_mode_str = _to_subtitle_mode_str(subtitle_mode)
|
||||||
|
if subtitle_mode_str in ("Disabled", "Line"):
|
||||||
|
return [text], default_split
|
||||||
|
|
||||||
|
if is_subtitle_input:
|
||||||
|
return [text], default_split
|
||||||
|
|
||||||
|
# English: spaCy only for post-TTS subtitles, not pre-TTS
|
||||||
|
if lang_enum in _SPACY_EXCLUDED_LANGS:
|
||||||
|
return [text], default_split
|
||||||
|
|
||||||
|
# Non-English: run spaCy pre-TTS segmentation
|
||||||
|
from abogen.spacy_utils import segment_sentences
|
||||||
|
|
||||||
|
_log("Using spaCy for sentence segmentation (pre-TTS)...")
|
||||||
|
spacy_sentences = segment_sentences(text, lang_code, log_callback=log_callback)
|
||||||
|
|
||||||
|
if not spacy_sentences:
|
||||||
|
_log("spaCy: Fallback to default segmentation...")
|
||||||
|
return [text], default_split
|
||||||
|
|
||||||
|
_log(f"spaCy: Text segmented into {len(spacy_sentences)} sentences...")
|
||||||
|
|
||||||
|
# spaCy already split at sentence boundaries; the engine only needs to
|
||||||
|
# split on newlines. Commas are never used in the engine split pattern
|
||||||
|
# for non-English (Sentence + Comma splits at commas only at subtitle
|
||||||
|
# time, like English).
|
||||||
|
active_split = "\n"
|
||||||
|
|
||||||
|
return spacy_sentences, active_split
|
||||||
|
|
||||||
|
|
||||||
|
def _to_language_enum(lang_code: Any) -> Language:
|
||||||
|
"""Convert lang_code to Language enum (ISO code or Language enum)."""
|
||||||
|
if isinstance(lang_code, Language):
|
||||||
|
return lang_code
|
||||||
|
try:
|
||||||
|
return Language.from_str(str(lang_code))
|
||||||
|
except ValueError:
|
||||||
|
return Language.EN_US
|
||||||
|
|
||||||
|
|
||||||
|
def _to_subtitle_mode_str(subtitle_mode: Any) -> str:
|
||||||
|
"""Convert subtitle_mode to string."""
|
||||||
|
if isinstance(subtitle_mode, SubtitleMode):
|
||||||
|
return subtitle_mode.value
|
||||||
|
return str(subtitle_mode)
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class SegmentResult:
|
class SegmentResult:
|
||||||
@@ -40,6 +143,7 @@ def tts_segments(
|
|||||||
speed: float,
|
speed: float,
|
||||||
split_pattern: str,
|
split_pattern: str,
|
||||||
current_time: float = 0.0,
|
current_time: float = 0.0,
|
||||||
|
total_steps: Optional[int] = None,
|
||||||
) -> Iterator[SegmentResult]:
|
) -> Iterator[SegmentResult]:
|
||||||
"""Invoke TTS backend on (already normalized) text and yield SegmentResults.
|
"""Invoke TTS backend on (already normalized) text and yield SegmentResults.
|
||||||
|
|
||||||
@@ -53,18 +157,24 @@ def tts_segments(
|
|||||||
speed: TTS speed multiplier.
|
speed: TTS speed multiplier.
|
||||||
split_pattern: Regex pattern for sentence splitting.
|
split_pattern: Regex pattern for sentence splitting.
|
||||||
current_time: Current position in the audio timeline (seconds).
|
current_time: Current position in the audio timeline (seconds).
|
||||||
|
total_steps: Inference quality steps (Supertonic only, ignored by Kokoro).
|
||||||
|
|
||||||
Yields:
|
Yields:
|
||||||
SegmentResult for each non-empty TTS segment.
|
SegmentResult for each non-empty TTS segment.
|
||||||
"""
|
"""
|
||||||
segment_iter = backend(
|
kwargs: dict[str, Any] = dict(
|
||||||
text,
|
|
||||||
voice=voice,
|
voice=voice,
|
||||||
speed=speed,
|
speed=speed,
|
||||||
split_pattern=split_pattern,
|
split_pattern=split_pattern,
|
||||||
)
|
)
|
||||||
|
if total_steps is not None:
|
||||||
|
kwargs["total_steps"] = total_steps
|
||||||
|
|
||||||
|
segment_iter = backend(text, **kwargs)
|
||||||
|
|
||||||
chunk_start = current_time
|
chunk_start = current_time
|
||||||
|
prev_tokens: Optional[List[Dict[str, Any]]] = None
|
||||||
|
prev_was_fallback = True
|
||||||
|
|
||||||
for segment in segment_iter:
|
for segment in segment_iter:
|
||||||
graphemes_raw = getattr(segment, "graphemes", "") or ""
|
graphemes_raw = getattr(segment, "graphemes", "") or ""
|
||||||
@@ -77,8 +187,10 @@ def tts_segments(
|
|||||||
duration = len(audio) / SAMPLE_RATE
|
duration = len(audio) / SAMPLE_RATE
|
||||||
|
|
||||||
tokens_list = getattr(segment, "tokens", [])
|
tokens_list = getattr(segment, "tokens", [])
|
||||||
|
was_fallback = False
|
||||||
if not tokens_list and graphemes:
|
if not tokens_list and graphemes:
|
||||||
tokens_list = [FakeToken(graphemes, 0, duration)]
|
tokens_list = [FakeToken(graphemes, 0, duration)]
|
||||||
|
was_fallback = True
|
||||||
|
|
||||||
tokens = [
|
tokens = [
|
||||||
{
|
{
|
||||||
@@ -90,6 +202,18 @@ def tts_segments(
|
|||||||
for tok in tokens_list
|
for tok in tokens_list
|
||||||
]
|
]
|
||||||
|
|
||||||
|
# When the engine splits text on a punctuation pattern, the
|
||||||
|
# whitespace between segments is consumed by the split. Restore a
|
||||||
|
# trailing space on the boundary token of the previous segment so
|
||||||
|
# subtitle processing sees the original spacing (only for real
|
||||||
|
# per-word tokens; FakeToken fallbacks split via their own logic).
|
||||||
|
if (
|
||||||
|
not prev_was_fallback
|
||||||
|
and prev_tokens
|
||||||
|
and not prev_tokens[-1].get("whitespace")
|
||||||
|
):
|
||||||
|
prev_tokens[-1]["whitespace"] = " "
|
||||||
|
|
||||||
yield SegmentResult(
|
yield SegmentResult(
|
||||||
graphemes=graphemes,
|
graphemes=graphemes,
|
||||||
audio=audio,
|
audio=audio,
|
||||||
@@ -98,6 +222,8 @@ def tts_segments(
|
|||||||
tokens=tokens,
|
tokens=tokens,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
prev_tokens = tokens
|
||||||
|
prev_was_fallback = was_fallback
|
||||||
chunk_start += duration
|
chunk_start += duration
|
||||||
|
|
||||||
|
|
||||||
@@ -109,6 +235,7 @@ def emit_text_segments(
|
|||||||
speed: float,
|
speed: float,
|
||||||
split_pattern: str,
|
split_pattern: str,
|
||||||
current_time: float = 0.0,
|
current_time: float = 0.0,
|
||||||
|
total_steps: Optional[int] = None,
|
||||||
# normalization
|
# normalization
|
||||||
heteronym_rules: Any = None,
|
heteronym_rules: Any = None,
|
||||||
pronunciation_rules: Any = None,
|
pronunciation_rules: Any = None,
|
||||||
@@ -159,6 +286,7 @@ def emit_text_segments(
|
|||||||
speed=speed,
|
speed=speed,
|
||||||
split_pattern=split_pattern,
|
split_pattern=split_pattern,
|
||||||
current_time=current_time,
|
current_time=current_time,
|
||||||
|
total_steps=total_steps,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -176,7 +304,7 @@ def emit_text_to_sinks(
|
|||||||
# subtitle
|
# subtitle
|
||||||
subtitle_writer: Any = None,
|
subtitle_writer: Any = None,
|
||||||
subtitle_mode: str = "Disabled",
|
subtitle_mode: str = "Disabled",
|
||||||
subtitle_lang: str = "a",
|
subtitle_lang: Language = Language.EN_US,
|
||||||
max_subtitle_words: int = 50,
|
max_subtitle_words: int = 50,
|
||||||
use_spacy_segmentation: bool = True,
|
use_spacy_segmentation: bool = True,
|
||||||
# normalization
|
# normalization
|
||||||
|
|||||||
+58
-6
@@ -128,8 +128,8 @@ class InputFormat(str, Enum):
|
|||||||
class Language(str, Enum):
|
class Language(str, Enum):
|
||||||
"""TTS language code (ISO 639-1 with region where needed).
|
"""TTS language code (ISO 639-1 with region where needed).
|
||||||
|
|
||||||
Each engine (Kokoro, Supertonic) maps these to its own
|
Each engine maps these to its own internal language identifiers.
|
||||||
internal language identifiers.
|
Engines report which languages they support via ``supported_languages()``.
|
||||||
"""
|
"""
|
||||||
EN_US = "en-US"
|
EN_US = "en-US"
|
||||||
EN_GB = "en-GB"
|
EN_GB = "en-GB"
|
||||||
@@ -140,6 +140,30 @@ class Language(str, Enum):
|
|||||||
JA = "ja"
|
JA = "ja"
|
||||||
PT_BR = "pt-BR"
|
PT_BR = "pt-BR"
|
||||||
ZH = "zh"
|
ZH = "zh"
|
||||||
|
AR = "ar"
|
||||||
|
BG = "bg"
|
||||||
|
CS = "cs"
|
||||||
|
DA = "da"
|
||||||
|
DE = "de"
|
||||||
|
EL = "el"
|
||||||
|
ET = "et"
|
||||||
|
FI = "fi"
|
||||||
|
HR = "hr"
|
||||||
|
HU = "hu"
|
||||||
|
ID = "id"
|
||||||
|
KO = "ko"
|
||||||
|
LT = "lt"
|
||||||
|
LV = "lv"
|
||||||
|
NL = "nl"
|
||||||
|
PL = "pl"
|
||||||
|
RO = "ro"
|
||||||
|
RU = "ru"
|
||||||
|
SK = "sk"
|
||||||
|
SL = "sl"
|
||||||
|
SV = "sv"
|
||||||
|
TR = "tr"
|
||||||
|
UK = "uk"
|
||||||
|
VI = "vi"
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def display_name(self) -> str:
|
def display_name(self) -> str:
|
||||||
@@ -154,18 +178,46 @@ class Language(str, Enum):
|
|||||||
"ja": "Japanese",
|
"ja": "Japanese",
|
||||||
"pt-BR": "Brazilian Portuguese",
|
"pt-BR": "Brazilian Portuguese",
|
||||||
"zh": "Mandarin Chinese",
|
"zh": "Mandarin Chinese",
|
||||||
|
"ar": "Arabic",
|
||||||
|
"bg": "Bulgarian",
|
||||||
|
"cs": "Czech",
|
||||||
|
"da": "Danish",
|
||||||
|
"de": "German",
|
||||||
|
"el": "Greek",
|
||||||
|
"et": "Estonian",
|
||||||
|
"fi": "Finnish",
|
||||||
|
"hr": "Croatian",
|
||||||
|
"hu": "Hungarian",
|
||||||
|
"id": "Indonesian",
|
||||||
|
"ko": "Korean",
|
||||||
|
"lt": "Lithuanian",
|
||||||
|
"lv": "Latvian",
|
||||||
|
"nl": "Dutch",
|
||||||
|
"pl": "Polish",
|
||||||
|
"ro": "Romanian",
|
||||||
|
"ru": "Russian",
|
||||||
|
"sk": "Slovak",
|
||||||
|
"sl": "Slovenian",
|
||||||
|
"sv": "Swedish",
|
||||||
|
"tr": "Turkish",
|
||||||
|
"uk": "Ukrainian",
|
||||||
|
"vi": "Vietnamese",
|
||||||
}
|
}
|
||||||
return _names[self.value]
|
return _names[self.value]
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def is_cjk(self) -> bool:
|
def is_cjk(self) -> bool:
|
||||||
"""True for CJK languages (Chinese, Japanese)."""
|
"""True for CJK languages (Chinese, Japanese, Korean)."""
|
||||||
return self in (self.ZH, self.JA)
|
return self in (self.ZH, self.JA, self.KO)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def supports_subtitle_tokens(self) -> bool:
|
def supports_subtitle_tokens(self) -> bool:
|
||||||
"""True if this language generates timestamped tokens for subtitles."""
|
"""True if this language supports subtitle generation.
|
||||||
return self in (self.EN_US, self.EN_GB)
|
|
||||||
|
All languages are supported: languages without per-word timestamped
|
||||||
|
tokens fall back to segment-level fake tokens in the pipeline.
|
||||||
|
"""
|
||||||
|
return True
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def from_str(cls, value: str) -> Language:
|
def from_str(cls, value: str) -> Language:
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ import logging
|
|||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
import uuid
|
import uuid
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Dict, List, Optional, Tuple
|
from typing import Any, Dict, List, Optional, Tuple
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|||||||
@@ -21,6 +21,60 @@ _SERIES_NUMBER_KEYS = (
|
|||||||
)
|
)
|
||||||
_SERIES_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?")
|
_SERIES_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?")
|
||||||
|
|
||||||
|
_SERIES_NAME_ALIASES = ("series", "series_name", "seriesname", "series_title", "seriestitle")
|
||||||
|
_SERIES_INDEX_ALIASES = ("series_index", "series_sequence", "series_position", "book_number")
|
||||||
|
_AUTHOR_ALIASES = ("author", "authors")
|
||||||
|
_DESCRIPTION_ALIASES = ("description", "summary")
|
||||||
|
_TAGS_ALIASES = ("tags", "keywords", "genre")
|
||||||
|
|
||||||
|
|
||||||
|
def expand_metadata_aliases(tags: Mapping[str, Any]) -> Dict[str, Any]:
|
||||||
|
"""Expand concept aliases so each concept has all canonical keys set.
|
||||||
|
|
||||||
|
One input concept fans out to multiple keys so that downstream consumers
|
||||||
|
can look up any variant and find the value.
|
||||||
|
|
||||||
|
Expanded concepts:
|
||||||
|
series -> series, series_name, seriesname, series_title, seriestitle
|
||||||
|
series_index -> series_index, series_sequence, series_position, book_number
|
||||||
|
author -> author, authors
|
||||||
|
description -> description, summary
|
||||||
|
tags -> tags, keywords, genre
|
||||||
|
"""
|
||||||
|
if not tags:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
result: Dict[str, Any] = {}
|
||||||
|
for key, value in tags.items():
|
||||||
|
if value is None:
|
||||||
|
continue
|
||||||
|
text = str(value).strip() if not isinstance(value, (list, tuple, set)) else value
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
key_lower = str(key).strip().lower()
|
||||||
|
if not key_lower:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if key_lower in _SERIES_NAME_ALIASES:
|
||||||
|
for alias in _SERIES_NAME_ALIASES:
|
||||||
|
result[alias] = text
|
||||||
|
elif key_lower in _SERIES_INDEX_ALIASES:
|
||||||
|
for alias in _SERIES_INDEX_ALIASES:
|
||||||
|
result[alias] = text
|
||||||
|
elif key_lower in _AUTHOR_ALIASES:
|
||||||
|
for alias in _AUTHOR_ALIASES:
|
||||||
|
result[alias] = text
|
||||||
|
elif key_lower in _DESCRIPTION_ALIASES:
|
||||||
|
for alias in _DESCRIPTION_ALIASES:
|
||||||
|
result[alias] = text
|
||||||
|
elif key_lower in _TAGS_ALIASES:
|
||||||
|
for alias in _TAGS_ALIASES:
|
||||||
|
result[alias] = text
|
||||||
|
else:
|
||||||
|
result[key_lower] = text
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
def normalize_metadata_map(values: Optional[Mapping[str, Any]]) -> Dict[str, str]:
|
def normalize_metadata_map(values: Optional[Mapping[str, Any]]) -> Dict[str, str]:
|
||||||
normalized: Dict[str, str] = {}
|
normalized: Dict[str, str] = {}
|
||||||
@@ -403,3 +457,40 @@ def load_audiobookshelf_chapters(
|
|||||||
if title and start is not None and end is not None:
|
if title and start is not None and end is not None:
|
||||||
cleaned.append({"title": str(title), "start": start, "end": end})
|
cleaned.append({"title": str(title), "start": start, "end": end})
|
||||||
return cleaned or None
|
return cleaned or None
|
||||||
|
|
||||||
|
|
||||||
|
def build_metadata_payload(
|
||||||
|
metadata: Optional[Dict[str, Any]] = None,
|
||||||
|
chapter_markers: Optional[List[Dict[str, Any]]] = None,
|
||||||
|
chunk_markers: Optional[List[Dict[str, Any]]] = None,
|
||||||
|
chunk_level: Optional[str] = None,
|
||||||
|
speaker_mode: Optional[str] = None,
|
||||||
|
speakers: Optional[Dict[str, Any]] = None,
|
||||||
|
generate_epub3: bool = False,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
"""Build the canonical metadata payload dict for persistence and downstream use.
|
||||||
|
|
||||||
|
This is the single source of truth for metadata assembly. Both PyQt and WebUI
|
||||||
|
runners should call this instead of building the dict manually.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
metadata: Normalized metadata tags dict.
|
||||||
|
chapter_markers: List of chapter marker dicts with title/start/end.
|
||||||
|
chunk_markers: List of chunk marker dicts.
|
||||||
|
chunk_level: Chunk granularity level (e.g. 'chapter', 'chunk').
|
||||||
|
speaker_mode: Speaker mode ('single', 'multi', etc.).
|
||||||
|
speakers: Speaker profile mapping.
|
||||||
|
generate_epub3: Whether EPUB3 generation is enabled.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Complete metadata payload dict.
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
"metadata": dict(metadata or {}),
|
||||||
|
"chapters": chapter_markers or [],
|
||||||
|
"chunks": chunk_markers or [],
|
||||||
|
"chunk_level": chunk_level,
|
||||||
|
"speaker_mode": speaker_mode,
|
||||||
|
"speakers": dict(speakers or {}),
|
||||||
|
"generate_epub3": generate_epub3,
|
||||||
|
}
|
||||||
|
|||||||
@@ -8,23 +8,22 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from typing import Any, Dict, Mapping
|
from typing import Any, Dict, Mapping
|
||||||
|
|
||||||
|
from abogen.domain.metadata_helpers import expand_metadata_aliases
|
||||||
|
|
||||||
|
|
||||||
def normalize_opds_metadata(metadata_payload: Mapping[str, Any]) -> Dict[str, Any]:
|
def normalize_opds_metadata(metadata_payload: Mapping[str, Any]) -> Dict[str, Any]:
|
||||||
"""Normalize OPDS/Calibre metadata into canonical override keys.
|
"""Normalize OPDS/Calibre metadata into canonical override keys.
|
||||||
|
|
||||||
Takes a metadata payload with various key aliases (e.g. 'series'/'series_name',
|
Takes a metadata payload with various key aliases (e.g. 'series'/'series_name',
|
||||||
'tags'/'keywords', 'authors'/'creator') and returns a dict with canonical
|
'tags'/'keywords', 'authors'/'creator') and returns a dict with all
|
||||||
keys set.
|
concept aliases expanded.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
metadata_payload: Raw metadata dict from OPDS/Calibre import.
|
metadata_payload: Raw metadata dict from OPDS/Calibre import.
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Dict with canonical metadata keys (series, series_index, tags,
|
Dict with all canonical metadata key aliases expanded.
|
||||||
description, subtitle, publisher, authors).
|
|
||||||
"""
|
"""
|
||||||
metadata_overrides: Dict[str, Any] = {}
|
|
||||||
|
|
||||||
def _stringify(value: Any) -> str:
|
def _stringify(value: Any) -> str:
|
||||||
if value is None:
|
if value is None:
|
||||||
return ""
|
return ""
|
||||||
@@ -33,67 +32,25 @@ def normalize_opds_metadata(metadata_payload: Mapping[str, Any]) -> Dict[str, An
|
|||||||
return ", ".join(part for part in parts if part)
|
return ", ".join(part for part in parts if part)
|
||||||
return str(value).strip()
|
return str(value).strip()
|
||||||
|
|
||||||
raw_series = metadata_payload.get("series") or metadata_payload.get("series_name")
|
# Map OPDS-specific keys to common concept keys before expansion
|
||||||
series_name = str(raw_series or "").strip()
|
normalized_input: Dict[str, Any] = {}
|
||||||
if series_name:
|
for key, value in metadata_payload.items():
|
||||||
metadata_overrides["series"] = series_name
|
if value is None:
|
||||||
metadata_overrides.setdefault("series_name", series_name)
|
continue
|
||||||
|
key_lower = str(key).strip().lower()
|
||||||
|
if not key_lower:
|
||||||
|
continue
|
||||||
|
text = _stringify(value)
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
|
||||||
series_index_value = (
|
# Map OPDS-specific author aliases
|
||||||
metadata_payload.get("series_index")
|
if key_lower in ("creator", "dc_creator"):
|
||||||
or metadata_payload.get("series_position")
|
normalized_input["author"] = text
|
||||||
or metadata_payload.get("series_sequence")
|
# Map OPDS-specific subtitle aliases
|
||||||
or metadata_payload.get("book_number")
|
elif key_lower in ("sub_title", "calibre_subtitle"):
|
||||||
)
|
normalized_input["subtitle"] = text
|
||||||
if series_index_value is not None:
|
else:
|
||||||
series_index_text = str(series_index_value).strip()
|
normalized_input[key_lower] = text
|
||||||
if series_index_text:
|
|
||||||
metadata_overrides.setdefault("series_index", series_index_text)
|
|
||||||
metadata_overrides.setdefault("series_position", series_index_text)
|
|
||||||
metadata_overrides.setdefault("series_sequence", series_index_text)
|
|
||||||
metadata_overrides.setdefault("book_number", series_index_text)
|
|
||||||
|
|
||||||
tags_value = metadata_payload.get("tags") or metadata_payload.get("keywords")
|
return expand_metadata_aliases(normalized_input)
|
||||||
if tags_value:
|
|
||||||
tags_text = _stringify(tags_value)
|
|
||||||
if tags_text:
|
|
||||||
metadata_overrides.setdefault("tags", tags_text)
|
|
||||||
metadata_overrides.setdefault("keywords", tags_text)
|
|
||||||
metadata_overrides.setdefault("genre", tags_text)
|
|
||||||
|
|
||||||
description_value = metadata_payload.get("description") or metadata_payload.get("summary")
|
|
||||||
if description_value:
|
|
||||||
description_text = _stringify(description_value)
|
|
||||||
if description_text:
|
|
||||||
metadata_overrides.setdefault("description", description_text)
|
|
||||||
metadata_overrides.setdefault("summary", description_text)
|
|
||||||
|
|
||||||
subtitle_value = (
|
|
||||||
metadata_payload.get("subtitle")
|
|
||||||
or metadata_payload.get("sub_title")
|
|
||||||
or metadata_payload.get("calibre_subtitle")
|
|
||||||
)
|
|
||||||
if subtitle_value:
|
|
||||||
subtitle_text = _stringify(subtitle_value)
|
|
||||||
if subtitle_text:
|
|
||||||
metadata_overrides.setdefault("subtitle", subtitle_text)
|
|
||||||
|
|
||||||
publisher_value = metadata_payload.get("publisher")
|
|
||||||
if publisher_value:
|
|
||||||
publisher_text = _stringify(publisher_value)
|
|
||||||
if publisher_text:
|
|
||||||
metadata_overrides.setdefault("publisher", publisher_text)
|
|
||||||
|
|
||||||
authors_value = (
|
|
||||||
metadata_payload.get("authors")
|
|
||||||
or metadata_payload.get("author")
|
|
||||||
or metadata_payload.get("creator")
|
|
||||||
or metadata_payload.get("dc_creator")
|
|
||||||
)
|
|
||||||
if authors_value:
|
|
||||||
authors_text = _stringify(authors_value)
|
|
||||||
if authors_text:
|
|
||||||
metadata_overrides.setdefault("authors", authors_text)
|
|
||||||
metadata_overrides.setdefault("author", authors_text)
|
|
||||||
|
|
||||||
return metadata_overrides
|
|
||||||
|
|||||||
@@ -13,8 +13,9 @@ resources so they can be created once and passed as a single object.
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any, Dict, List, Mapping, Optional
|
from typing import Any, Callable, Dict, List, Mapping, Optional
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.kokoro_text_normalization import (
|
from abogen.kokoro_text_normalization import (
|
||||||
ApostropheConfig,
|
ApostropheConfig,
|
||||||
normalize_for_pipeline as _normalize_for_pipeline,
|
normalize_for_pipeline as _normalize_for_pipeline,
|
||||||
@@ -123,3 +124,122 @@ def prepare_text_for_tts(
|
|||||||
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
|
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
|
||||||
|
|
||||||
return _normalize_for_pipeline(result, config=apostrophe_config, settings=runtime_settings)
|
return _normalize_for_pipeline(result, config=apostrophe_config, settings=runtime_settings)
|
||||||
|
|
||||||
|
|
||||||
|
def build_tts_context(
|
||||||
|
*,
|
||||||
|
language: Language,
|
||||||
|
subtitle: "SubtitleConfig | str" = "Disabled",
|
||||||
|
pronunciation: Optional["PronunciationConfig"] = None,
|
||||||
|
speakers: Optional[Dict[str, Any]] = None,
|
||||||
|
usage_counter: Optional[Dict[str, int]] = None,
|
||||||
|
log_callback: Optional[Callable[[str, str], None]] = None,
|
||||||
|
) -> TTSContext:
|
||||||
|
"""Build a TTSContext from raw data. Single entry point for both UIs.
|
||||||
|
|
||||||
|
Loads normalization settings, applies overrides, validates configuration,
|
||||||
|
merges pronunciation overrides, and compiles all rules.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
language: Language enum value.
|
||||||
|
subtitle: SubtitleConfig object or subtitle mode string.
|
||||||
|
pronunciation: PronunciationConfig with override rules.
|
||||||
|
speakers: Speaker profile mapping.
|
||||||
|
usage_counter: Mutable dict for tracking override usage.
|
||||||
|
log_callback: Callable(level, message) for warnings.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
TTSContext ready for text normalization.
|
||||||
|
"""
|
||||||
|
from abogen.domain.config_types import PronunciationConfig, SubtitleConfig
|
||||||
|
from abogen.domain.enums import SubtitleMode
|
||||||
|
from abogen.domain.pronunciation import (
|
||||||
|
compile_heteronym_sentence_rules,
|
||||||
|
compile_pronunciation_rules,
|
||||||
|
merge_pronunciation_overrides,
|
||||||
|
)
|
||||||
|
from abogen.domain.split_pattern import get_split_pattern
|
||||||
|
|
||||||
|
def _log(msg: str, level: str = "warning") -> None:
|
||||||
|
if log_callback:
|
||||||
|
log_callback(level, msg)
|
||||||
|
|
||||||
|
# Resolve subtitle mode
|
||||||
|
if isinstance(subtitle, SubtitleConfig):
|
||||||
|
resolved_subtitle = subtitle.mode
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
resolved_subtitle = SubtitleMode.from_str(subtitle) if not isinstance(subtitle, SubtitleMode) else subtitle
|
||||||
|
except ValueError:
|
||||||
|
resolved_subtitle = SubtitleMode.DISABLED
|
||||||
|
|
||||||
|
# Resolve pronunciation config
|
||||||
|
if pronunciation is None:
|
||||||
|
pronunciation = PronunciationConfig()
|
||||||
|
|
||||||
|
# Get runtime normalization settings
|
||||||
|
runtime_settings = get_runtime_settings()
|
||||||
|
|
||||||
|
# Apply per-job normalization overrides
|
||||||
|
if pronunciation.normalization_overrides:
|
||||||
|
runtime_settings = _apply_overrides(runtime_settings, pronunciation.normalization_overrides)
|
||||||
|
|
||||||
|
# Build apostrophe config
|
||||||
|
apostrophe_config = build_apostrophe_config(settings=runtime_settings)
|
||||||
|
|
||||||
|
# Validate LLM apostrophe mode
|
||||||
|
apostrophe_mode = str(runtime_settings.get("normalization_apostrophe_mode", "spacy")).lower()
|
||||||
|
if apostrophe_mode == "llm":
|
||||||
|
from abogen.normalization_settings import build_llm_configuration
|
||||||
|
llm_config = build_llm_configuration(runtime_settings)
|
||||||
|
if not llm_config.is_configured():
|
||||||
|
raise RuntimeError(
|
||||||
|
"LLM-based apostrophe normalization is selected, but the LLM configuration is incomplete."
|
||||||
|
)
|
||||||
|
|
||||||
|
# Check for num2words availability
|
||||||
|
if apostrophe_config.convert_numbers:
|
||||||
|
try:
|
||||||
|
import num2words # noqa: F401
|
||||||
|
except ImportError:
|
||||||
|
_log(
|
||||||
|
"Number normalization is enabled but 'num2words' library is not available. "
|
||||||
|
"Numbers will NOT be converted to words."
|
||||||
|
)
|
||||||
|
|
||||||
|
# Compute split pattern
|
||||||
|
if not isinstance(language, Language):
|
||||||
|
raise TypeError(f"language must be Language enum, got {type(language).__name__}: {language!r}")
|
||||||
|
split_pattern = get_split_pattern(language, resolved_subtitle)
|
||||||
|
|
||||||
|
# Merge pronunciation overrides
|
||||||
|
source = {
|
||||||
|
"pronunciation_overrides": pronunciation.pronunciation_overrides,
|
||||||
|
"manual_overrides": pronunciation.manual_overrides,
|
||||||
|
"speakers": speakers or {},
|
||||||
|
"language": language,
|
||||||
|
}
|
||||||
|
merged_overrides = merge_pronunciation_overrides(source)
|
||||||
|
|
||||||
|
# Compile rules
|
||||||
|
pronunciation_rules = compile_pronunciation_rules(merged_overrides)
|
||||||
|
heteronym_rules = compile_heteronym_sentence_rules(pronunciation.heteronym_overrides)
|
||||||
|
|
||||||
|
if heteronym_rules:
|
||||||
|
_log(
|
||||||
|
f"Applying {len(heteronym_rules)} heteronym override(s) during conversion.",
|
||||||
|
level="debug",
|
||||||
|
)
|
||||||
|
if pronunciation_rules:
|
||||||
|
_log(
|
||||||
|
f"Applying {len(pronunciation_rules)} pronunciation override(s) during conversion.",
|
||||||
|
level="debug",
|
||||||
|
)
|
||||||
|
|
||||||
|
return TTSContext(
|
||||||
|
split_pattern=split_pattern,
|
||||||
|
pronunciation_rules=pronunciation_rules,
|
||||||
|
heteronym_rules=heteronym_rules,
|
||||||
|
normalization_overrides=pronunciation.normalization_overrides,
|
||||||
|
usage_counter=usage_counter if usage_counter is not None else {},
|
||||||
|
)
|
||||||
|
|||||||
@@ -11,9 +11,8 @@ import platform
|
|||||||
import re
|
import re
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Callable, List, Optional, Tuple
|
from typing import Callable, List, Optional, Tuple
|
||||||
|
|
||||||
from abogen.subtitle_utils import sanitize_name_for_os
|
|
||||||
from abogen.text_extractor import ExtractedChapter
|
from abogen.text_extractor import ExtractedChapter
|
||||||
|
|
||||||
|
|
||||||
@@ -21,6 +20,9 @@ _OUTPUT_SANITIZE_RE = re.compile(r"[^\w\-_.]+")
|
|||||||
|
|
||||||
# OS-specific illegal characters for filenames
|
# OS-specific illegal characters for filenames
|
||||||
_WINDOWS_ILLEGAL_CHARS_RE = re.compile(r'[<>:"/\\|?*\x00-\x1f]')
|
_WINDOWS_ILLEGAL_CHARS_RE = re.compile(r'[<>:"/\\|?*\x00-\x1f]')
|
||||||
|
_MACOS_ILLEGAL_CHARS_RE = re.compile(r"[:]")
|
||||||
|
_LINUX_ILLEGAL_CHARS_RE = re.compile(r"[/\x00]")
|
||||||
|
_CONTROL_CHARS_RE = re.compile(r"[\x00-\x1f]")
|
||||||
_UNIX_CONTROL_CHARS_RE = re.compile(r'[\x00-\x1f]')
|
_UNIX_CONTROL_CHARS_RE = re.compile(r'[\x00-\x1f]')
|
||||||
_RESERVED_NAMES = frozenset(
|
_RESERVED_NAMES = frozenset(
|
||||||
{"CON", "PRN", "AUX", "NUL"}
|
{"CON", "PRN", "AUX", "NUL"}
|
||||||
@@ -29,6 +31,47 @@ _RESERVED_NAMES = frozenset(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def sanitize_name_for_os(name: str, is_folder: bool = True) -> str:
|
||||||
|
"""Sanitize a filename or folder name based on the operating system.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
name: The name to sanitize
|
||||||
|
is_folder: Whether this is a folder name (default: True)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Sanitized name safe for the current OS
|
||||||
|
"""
|
||||||
|
if not name:
|
||||||
|
return "audiobook"
|
||||||
|
|
||||||
|
system = platform.system()
|
||||||
|
|
||||||
|
if system == "Windows":
|
||||||
|
sanitized = _WINDOWS_ILLEGAL_CHARS_RE.sub("_", name)
|
||||||
|
sanitized = _CONTROL_CHARS_RE.sub("_", sanitized)
|
||||||
|
sanitized = sanitized.rstrip(". ")
|
||||||
|
if sanitized.upper() in _RESERVED_NAMES or sanitized.upper().split(".")[0] in _RESERVED_NAMES:
|
||||||
|
sanitized = f"_{sanitized}"
|
||||||
|
elif system == "Darwin":
|
||||||
|
sanitized = _MACOS_ILLEGAL_CHARS_RE.sub("_", name)
|
||||||
|
sanitized = _CONTROL_CHARS_RE.sub("_", sanitized)
|
||||||
|
if is_folder and sanitized.startswith("."):
|
||||||
|
sanitized = "_" + sanitized[1:]
|
||||||
|
else:
|
||||||
|
sanitized = _LINUX_ILLEGAL_CHARS_RE.sub("_", name)
|
||||||
|
sanitized = _UNIX_CONTROL_CHARS_RE.sub("_", sanitized)
|
||||||
|
if is_folder and sanitized.startswith("."):
|
||||||
|
sanitized = "_" + sanitized[1:]
|
||||||
|
|
||||||
|
if not sanitized or sanitized.strip() == "":
|
||||||
|
sanitized = "audiobook"
|
||||||
|
|
||||||
|
if len(sanitized) > 255:
|
||||||
|
sanitized = sanitized[:255].rstrip(". ")
|
||||||
|
|
||||||
|
return sanitized
|
||||||
|
|
||||||
|
|
||||||
def slugify(title: str, index: int) -> str:
|
def slugify(title: str, index: int) -> str:
|
||||||
sanitized = re.sub(r"[^\w\-]+", "_", title.lower()).strip("_")
|
sanitized = re.sub(r"[^\w\-]+", "_", title.lower()).strip("_")
|
||||||
if not sanitized:
|
if not sanitized:
|
||||||
|
|||||||
@@ -2,30 +2,21 @@
|
|||||||
|
|
||||||
Provides a unified interface for creating and managing TTS pipelines
|
Provides a unified interface for creating and managing TTS pipelines
|
||||||
across all UI layers (WebUI, PyQt, CLI).
|
across all UI layers (WebUI, PyQt, CLI).
|
||||||
|
|
||||||
|
Language handling: the engine owns the mapping between Language enum
|
||||||
|
and its internal format. Callers pass Language enum; the engine
|
||||||
|
converts internally. No engine-specific codes leak outside the engine.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from typing import Any, Dict, Optional
|
from typing import Any, Dict
|
||||||
|
|
||||||
from abogen.domain.device import select_device
|
from abogen.domain.device import select_device
|
||||||
from abogen.domain.enums import Language
|
from abogen.domain.enums import Language
|
||||||
from abogen.domain.voice_resolution import initialize_voice_cache
|
from abogen.domain.voice_resolution import initialize_voice_cache
|
||||||
from abogen.tts_plugin.utils import create_pipeline, is_plugin_registered
|
from abogen.tts_plugin.utils import create_pipeline, is_plugin_registered
|
||||||
|
|
||||||
# Kokoro-specific language mapping (engine's responsibility)
|
|
||||||
_KOKORO_LANG_MAP = {
|
|
||||||
Language.EN_US: "a",
|
|
||||||
Language.EN_GB: "b",
|
|
||||||
Language.ES: "e",
|
|
||||||
Language.FR: "f",
|
|
||||||
Language.HI: "h",
|
|
||||||
Language.IT: "i",
|
|
||||||
Language.JA: "j",
|
|
||||||
Language.PT_BR: "p",
|
|
||||||
Language.ZH: "z",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_device(use_gpu: bool) -> str:
|
def resolve_device(use_gpu: bool) -> str:
|
||||||
"""Determine compute device from job and global config flags."""
|
"""Determine compute device from job and global config flags."""
|
||||||
@@ -39,29 +30,25 @@ def resolve_device(use_gpu: bool) -> str:
|
|||||||
|
|
||||||
def create_pipeline_for_job(
|
def create_pipeline_for_job(
|
||||||
provider: str,
|
provider: str,
|
||||||
language: str,
|
language: Language,
|
||||||
use_gpu: bool,
|
use_gpu: bool,
|
||||||
) -> Any:
|
) -> Any:
|
||||||
"""Create a TTS pipeline with proper device selection.
|
"""Create a TTS pipeline with proper device selection.
|
||||||
|
|
||||||
Handles provider validation, GPU decision, and plugin checks.
|
Args:
|
||||||
|
provider: TTS provider name ("kokoro" or "supertonic").
|
||||||
|
language: Language enum (app-layer type, not engine-specific).
|
||||||
|
use_gpu: Whether GPU acceleration is requested.
|
||||||
"""
|
"""
|
||||||
provider = str(provider or "kokoro").strip().lower() or "kokoro"
|
provider = str(provider or "kokoro").strip().lower() or "kokoro"
|
||||||
if not is_plugin_registered(provider):
|
if not is_plugin_registered(provider):
|
||||||
provider = "kokoro"
|
provider = "kokoro"
|
||||||
|
|
||||||
# Convert Language enum to Kokoro single-letter code
|
|
||||||
try:
|
|
||||||
lang = Language.from_str(language) if not isinstance(language, Language) else language
|
|
||||||
except ValueError:
|
|
||||||
lang = Language.EN_US # fallback for unknown languages
|
|
||||||
kokoro_code = _KOKORO_LANG_MAP.get(lang, "a")
|
|
||||||
|
|
||||||
if provider == "supertonic":
|
if provider == "supertonic":
|
||||||
return create_pipeline("supertonic")
|
return create_pipeline("supertonic", language=language)
|
||||||
|
|
||||||
device = resolve_device(use_gpu)
|
device = resolve_device(use_gpu)
|
||||||
return create_pipeline("kokoro", lang_code=kokoro_code, device=device)
|
return create_pipeline("kokoro", language=language, device=device)
|
||||||
|
|
||||||
|
|
||||||
def dispose_pipelines(pipelines: Dict[str, Any]) -> None:
|
def dispose_pipelines(pipelines: Dict[str, Any]) -> None:
|
||||||
@@ -80,7 +67,7 @@ class PipelinePool:
|
|||||||
Usage::
|
Usage::
|
||||||
|
|
||||||
pool = PipelinePool()
|
pool = PipelinePool()
|
||||||
backend = pool.get("kokoro", "en", use_gpu=True)
|
backend = pool.get("kokoro", Language.EN_US, use_gpu=True)
|
||||||
# ... use backend ...
|
# ... use backend ...
|
||||||
pool.dispose_all()
|
pool.dispose_all()
|
||||||
"""
|
"""
|
||||||
@@ -92,18 +79,20 @@ class PipelinePool:
|
|||||||
def get(
|
def get(
|
||||||
self,
|
self,
|
||||||
provider: str,
|
provider: str,
|
||||||
language: str,
|
language: Language,
|
||||||
use_gpu: bool,
|
use_gpu: bool,
|
||||||
*,
|
*,
|
||||||
job: Any = None,
|
request: Any = None,
|
||||||
|
events: Any = None,
|
||||||
) -> Any:
|
) -> Any:
|
||||||
"""Get or create a cached pipeline for the given provider.
|
"""Get or create a cached pipeline for the given provider.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
provider: TTS provider name ("kokoro" or "supertonic").
|
provider: TTS provider name ("kokoro" or "supertonic").
|
||||||
language: Language code (for kokoro).
|
language: Language enum (app-layer type).
|
||||||
use_gpu: Whether GPU acceleration is requested.
|
use_gpu: Whether GPU acceleration is requested.
|
||||||
job: Optional job object for voice cache initialization.
|
request: ConversionRequest for voice cache initialization.
|
||||||
|
events: ConversionEvents for logging during cache init.
|
||||||
"""
|
"""
|
||||||
provider = str(provider or "kokoro").strip().lower() or "kokoro"
|
provider = str(provider or "kokoro").strip().lower() or "kokoro"
|
||||||
if not is_plugin_registered(provider):
|
if not is_plugin_registered(provider):
|
||||||
@@ -116,8 +105,8 @@ class PipelinePool:
|
|||||||
pipeline = create_pipeline_for_job(provider, language, use_gpu)
|
pipeline = create_pipeline_for_job(provider, language, use_gpu)
|
||||||
self._pipelines[provider] = pipeline
|
self._pipelines[provider] = pipeline
|
||||||
|
|
||||||
if provider == "kokoro" and not self._voice_cache_initialized and job is not None:
|
if provider == "kokoro" and not self._voice_cache_initialized and request is not None:
|
||||||
initialize_voice_cache(job)
|
initialize_voice_cache(request, events=events)
|
||||||
self._voice_cache_initialized = True
|
self._voice_cache_initialized = True
|
||||||
|
|
||||||
return pipeline
|
return pipeline
|
||||||
|
|||||||
@@ -180,11 +180,20 @@ def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
|
|||||||
we must merge manual overrides so they always apply (before TTS).
|
we must merge manual overrides so they always apply (before TTS).
|
||||||
|
|
||||||
Precedence: manual overrides win over existing entries for the same normalized key.
|
Precedence: manual overrides win over existing entries for the same normalized key.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
job: Either a job-like object with attributes, or a dict with keys:
|
||||||
|
``pronunciation_overrides``, ``manual_overrides``, ``speakers``, ``language``.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
collected: Dict[str, Dict[str, Any]] = {}
|
collected: Dict[str, Dict[str, Any]] = {}
|
||||||
|
|
||||||
existing = getattr(job, "pronunciation_overrides", None)
|
def _get(key: str, default: Any = None) -> Any:
|
||||||
|
if isinstance(job, Mapping):
|
||||||
|
return job.get(key, default)
|
||||||
|
return getattr(job, key, default)
|
||||||
|
|
||||||
|
existing = _get("pronunciation_overrides")
|
||||||
if isinstance(existing, list):
|
if isinstance(existing, list):
|
||||||
for entry in existing:
|
for entry in existing:
|
||||||
if not isinstance(entry, Mapping):
|
if not isinstance(entry, Mapping):
|
||||||
@@ -204,10 +213,10 @@ def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
|
|||||||
"notes": str(entry.get("notes") or "").strip() or None,
|
"notes": str(entry.get("notes") or "").strip() or None,
|
||||||
"context": str(entry.get("context") or "").strip() or None,
|
"context": str(entry.get("context") or "").strip() or None,
|
||||||
"source": str(entry.get("source") or "pronunciation"),
|
"source": str(entry.get("source") or "pronunciation"),
|
||||||
"language": getattr(job, "language", None),
|
"language": _get("language"),
|
||||||
}
|
}
|
||||||
|
|
||||||
speakers = getattr(job, "speakers", None)
|
speakers = _get("speakers")
|
||||||
if isinstance(speakers, dict):
|
if isinstance(speakers, dict):
|
||||||
for payload in speakers.values():
|
for payload in speakers.values():
|
||||||
if not isinstance(payload, Mapping):
|
if not isinstance(payload, Mapping):
|
||||||
@@ -226,16 +235,16 @@ def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
|
|||||||
"voice": str(
|
"voice": str(
|
||||||
payload.get("resolved_voice")
|
payload.get("resolved_voice")
|
||||||
or payload.get("voice")
|
or payload.get("voice")
|
||||||
or getattr(job, "voice", "")
|
or _get("voice", "")
|
||||||
).strip()
|
).strip()
|
||||||
or None,
|
or None,
|
||||||
"notes": None,
|
"notes": None,
|
||||||
"context": None,
|
"context": None,
|
||||||
"source": "speaker",
|
"source": "speaker",
|
||||||
"language": getattr(job, "language", None),
|
"language": _get("language"),
|
||||||
}
|
}
|
||||||
|
|
||||||
manual = getattr(job, "manual_overrides", None)
|
manual = _get("manual_overrides")
|
||||||
if isinstance(manual, list):
|
if isinstance(manual, list):
|
||||||
for entry in manual:
|
for entry in manual:
|
||||||
if not isinstance(entry, Mapping):
|
if not isinstance(entry, Mapping):
|
||||||
@@ -255,7 +264,7 @@ def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
|
|||||||
"notes": str(entry.get("notes") or "").strip() or None,
|
"notes": str(entry.get("notes") or "").strip() or None,
|
||||||
"context": str(entry.get("context") or "").strip() or None,
|
"context": str(entry.get("context") or "").strip() or None,
|
||||||
"source": str(entry.get("source") or "manual"),
|
"source": str(entry.get("source") or "manual"),
|
||||||
"language": getattr(job, "language", None),
|
"language": _get("language"),
|
||||||
}
|
}
|
||||||
|
|
||||||
return list(collected.values())
|
return list(collected.values())
|
||||||
|
|||||||
@@ -9,11 +9,11 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass
|
||||||
from typing import Any, Callable, Dict, Mapping, Optional, Sequence
|
from typing import Any, Callable, Dict, Mapping, Optional
|
||||||
|
|
||||||
from abogen.constants import (
|
from abogen.constants import (
|
||||||
LANGUAGE_DESCRIPTIONS,
|
KOKORO_CODE_LABELS,
|
||||||
SUBTITLE_FORMATS,
|
SUBTITLE_FORMATS,
|
||||||
SUPPORTED_SOUND_FORMATS,
|
SUPPORTED_SOUND_FORMATS,
|
||||||
)
|
)
|
||||||
@@ -135,10 +135,10 @@ def _norm_speaker_spec(value: Any, default: str) -> str:
|
|||||||
|
|
||||||
def _norm_language_list(value: Any, default: list) -> list:
|
def _norm_language_list(value: Any, default: list) -> list:
|
||||||
if isinstance(value, (list, tuple, set)):
|
if isinstance(value, (list, tuple, set)):
|
||||||
return [code for code in value if isinstance(code, str) and code in LANGUAGE_DESCRIPTIONS]
|
return [code for code in value if isinstance(code, str) and code in KOKORO_CODE_LABELS]
|
||||||
if isinstance(value, str):
|
if isinstance(value, str):
|
||||||
parts = [item.strip().lower() for item in value.split(",") if item.strip()]
|
parts = [item.strip().lower() for item in value.split(",") if item.strip()]
|
||||||
return [code for code in parts if code in LANGUAGE_DESCRIPTIONS]
|
return [code for code in parts if code in KOKORO_CODE_LABELS]
|
||||||
return default
|
return default
|
||||||
|
|
||||||
|
|
||||||
@@ -578,3 +578,64 @@ def integration_defaults() -> Dict[str, Dict[str, Any]]:
|
|||||||
"timeout": 30.0,
|
"timeout": 30.0,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def stored_integration_config(name: str) -> Dict[str, Any]:
|
||||||
|
"""Read raw integration config from config.json.
|
||||||
|
|
||||||
|
Reads ``config["integrations"][name]``.
|
||||||
|
"""
|
||||||
|
from abogen.utils import load_config
|
||||||
|
|
||||||
|
cfg = load_config() or {}
|
||||||
|
integrations = cfg.get("integrations")
|
||||||
|
if isinstance(integrations, Mapping):
|
||||||
|
entry = integrations.get(name)
|
||||||
|
if isinstance(entry, Mapping):
|
||||||
|
return dict(entry)
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def load_audiobookshelf_config() -> Optional["AudiobookshelfConfig"]:
|
||||||
|
"""Read Audiobookshelf settings from config.json and build typed config.
|
||||||
|
|
||||||
|
Returns ``None`` when the integration is not configured or required
|
||||||
|
fields are missing.
|
||||||
|
"""
|
||||||
|
raw = stored_integration_config("audiobookshelf")
|
||||||
|
if not raw:
|
||||||
|
return None
|
||||||
|
return build_audiobookshelf_config(raw)
|
||||||
|
|
||||||
|
|
||||||
|
def build_audiobookshelf_config(
|
||||||
|
settings: Mapping[str, Any],
|
||||||
|
) -> Optional["AudiobookshelfConfig"]:
|
||||||
|
"""Build :class:`AudiobookshelfConfig` from a settings dict.
|
||||||
|
|
||||||
|
Returns ``None`` when required fields (base_url, api_token, library_id)
|
||||||
|
are missing.
|
||||||
|
"""
|
||||||
|
from abogen.integrations.audiobookshelf import AudiobookshelfConfig
|
||||||
|
|
||||||
|
base_url = str(settings.get("base_url") or "").strip()
|
||||||
|
api_token = str(settings.get("api_token") or "").strip()
|
||||||
|
library_id = str(settings.get("library_id") or "").strip()
|
||||||
|
if not (base_url and api_token and library_id):
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
timeout = float(settings.get("timeout", 3600.0))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
timeout = 3600.0
|
||||||
|
return AudiobookshelfConfig(
|
||||||
|
base_url=base_url,
|
||||||
|
api_token=api_token,
|
||||||
|
library_id=library_id,
|
||||||
|
collection_id=(str(settings.get("collection_id") or "").strip() or None),
|
||||||
|
folder_id=(str(settings.get("folder_id") or "").strip() or None),
|
||||||
|
verify_ssl=coerce_bool(settings.get("verify_ssl"), True),
|
||||||
|
send_cover=coerce_bool(settings.get("send_cover"), True),
|
||||||
|
send_chapters=coerce_bool(settings.get("send_chapters"), True),
|
||||||
|
send_subtitles=coerce_bool(settings.get("send_subtitles"), False),
|
||||||
|
timeout=timeout,
|
||||||
|
)
|
||||||
|
|||||||
@@ -0,0 +1,381 @@
|
|||||||
|
"""Speaker metadata functions for building and applying speaker rosters.
|
||||||
|
|
||||||
|
This module contains the core logic for:
|
||||||
|
- Building narrator and speaker rosters from analysis results
|
||||||
|
- Matching speakers to configured presets
|
||||||
|
- Applying speaker config presets to rosters
|
||||||
|
- Preparing full speaker metadata for conversion
|
||||||
|
|
||||||
|
Moved from webui/routes/utils/voice.py to be available across all UIs.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
|
||||||
|
|
||||||
|
from abogen.speaker_analysis import analyze_speakers
|
||||||
|
from abogen.speaker_configs import slugify_label
|
||||||
|
from abogen.domain.settings_core import load_settings
|
||||||
|
|
||||||
|
|
||||||
|
def build_narrator_roster(
|
||||||
|
voice: str,
|
||||||
|
voice_profile: Optional[str],
|
||||||
|
existing: Optional[Mapping[str, Any]] = None,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
roster: Dict[str, Any] = {
|
||||||
|
"narrator": {
|
||||||
|
"id": "narrator",
|
||||||
|
"label": "Narrator",
|
||||||
|
"voice": voice,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if voice_profile:
|
||||||
|
roster["narrator"]["voice_profile"] = voice_profile
|
||||||
|
existing_entry: Optional[Mapping[str, Any]] = None
|
||||||
|
if existing is not None:
|
||||||
|
existing_entry = existing.get("narrator") if isinstance(existing, Mapping) else None
|
||||||
|
if isinstance(existing_entry, Mapping):
|
||||||
|
roster_entry = roster["narrator"]
|
||||||
|
for key in ("label", "voice", "voice_profile", "voice_formula", "pronunciation"):
|
||||||
|
value = existing_entry.get(key)
|
||||||
|
if value is not None and value != "":
|
||||||
|
roster_entry[key] = value
|
||||||
|
return roster
|
||||||
|
|
||||||
|
|
||||||
|
def build_speaker_roster(
|
||||||
|
analysis: Dict[str, Any],
|
||||||
|
base_voice: str,
|
||||||
|
voice_profile: Optional[str],
|
||||||
|
existing: Optional[Mapping[str, Any]] = None,
|
||||||
|
order: Optional[Iterable[str]] = None,
|
||||||
|
) -> Dict[str, Any]:
|
||||||
|
roster = build_narrator_roster(base_voice, voice_profile, existing)
|
||||||
|
existing_map: Dict[str, Any] = dict(existing) if isinstance(existing, Mapping) else {}
|
||||||
|
speakers = analysis.get("speakers", {}) if isinstance(analysis, dict) else {}
|
||||||
|
ordered_ids: Iterable[str]
|
||||||
|
if order is not None:
|
||||||
|
ordered_ids = [sid for sid in order if sid in speakers]
|
||||||
|
else:
|
||||||
|
ordered_ids = speakers.keys()
|
||||||
|
|
||||||
|
for speaker_id in ordered_ids:
|
||||||
|
payload = speakers.get(speaker_id, {})
|
||||||
|
if speaker_id == "narrator":
|
||||||
|
continue
|
||||||
|
if isinstance(payload, Mapping) and payload.get("suppressed"):
|
||||||
|
continue
|
||||||
|
previous = existing_map.get(speaker_id)
|
||||||
|
roster[speaker_id] = {
|
||||||
|
"id": speaker_id,
|
||||||
|
"label": payload.get("label") or speaker_id.replace("_", " ").title(),
|
||||||
|
"analysis_confidence": payload.get("confidence"),
|
||||||
|
"analysis_count": payload.get("count"),
|
||||||
|
"gender": payload.get("gender", "unknown"),
|
||||||
|
}
|
||||||
|
detected_gender = payload.get("detected_gender")
|
||||||
|
if detected_gender:
|
||||||
|
roster[speaker_id]["detected_gender"] = detected_gender
|
||||||
|
samples = payload.get("sample_quotes")
|
||||||
|
if isinstance(samples, list):
|
||||||
|
roster[speaker_id]["sample_quotes"] = samples
|
||||||
|
if isinstance(previous, Mapping):
|
||||||
|
for key in ("voice", "voice_profile", "voice_formula", "resolved_voice", "pronunciation"):
|
||||||
|
value = previous.get(key)
|
||||||
|
if value is not None and value != "":
|
||||||
|
roster[speaker_id][key] = value
|
||||||
|
if "sample_quotes" not in roster[speaker_id]:
|
||||||
|
prev_samples = previous.get("sample_quotes")
|
||||||
|
if isinstance(prev_samples, list):
|
||||||
|
roster[speaker_id]["sample_quotes"] = prev_samples
|
||||||
|
if "detected_gender" not in roster[speaker_id]:
|
||||||
|
prev_detected = previous.get("detected_gender")
|
||||||
|
if isinstance(prev_detected, str) and prev_detected:
|
||||||
|
roster[speaker_id]["detected_gender"] = prev_detected
|
||||||
|
return roster
|
||||||
|
|
||||||
|
|
||||||
|
def match_configured_speaker(
|
||||||
|
config_speakers: Mapping[str, Any],
|
||||||
|
roster_id: str,
|
||||||
|
roster_label: str,
|
||||||
|
) -> Optional[Mapping[str, Any]]:
|
||||||
|
if not config_speakers:
|
||||||
|
return None
|
||||||
|
entry = config_speakers.get(roster_id)
|
||||||
|
if entry:
|
||||||
|
return cast(Mapping[str, Any], entry)
|
||||||
|
slug = slugify_label(roster_label)
|
||||||
|
if slug != roster_id and slug in config_speakers:
|
||||||
|
return cast(Mapping[str, Any], config_speakers[slug])
|
||||||
|
lower_label = roster_label.strip().lower()
|
||||||
|
for record in config_speakers.values():
|
||||||
|
if not isinstance(record, Mapping):
|
||||||
|
continue
|
||||||
|
if str(record.get("label", "")).strip().lower() == lower_label:
|
||||||
|
return record
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def apply_speaker_config_to_roster(
|
||||||
|
roster: Mapping[str, Any],
|
||||||
|
config: Optional[Mapping[str, Any]],
|
||||||
|
*,
|
||||||
|
persist_changes: bool = False,
|
||||||
|
fallback_languages: Optional[Iterable[str]] = None,
|
||||||
|
) -> Tuple[Dict[str, Any], List[str], Optional[Dict[str, Any]]]:
|
||||||
|
if not isinstance(roster, Mapping):
|
||||||
|
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
|
||||||
|
return {}, effective_languages, None
|
||||||
|
updated_roster: Dict[str, Any] = {key: dict(value) for key, value in roster.items() if isinstance(value, Mapping)}
|
||||||
|
if not config:
|
||||||
|
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
|
||||||
|
return updated_roster, effective_languages, None
|
||||||
|
|
||||||
|
speakers_map = config.get("speakers")
|
||||||
|
if not isinstance(speakers_map, Mapping):
|
||||||
|
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
|
||||||
|
return updated_roster, effective_languages, None
|
||||||
|
|
||||||
|
config_languages = config.get("languages")
|
||||||
|
if isinstance(config_languages, list):
|
||||||
|
allowed_languages = [code for code in config_languages if isinstance(code, str) and code]
|
||||||
|
else:
|
||||||
|
allowed_languages = []
|
||||||
|
if not allowed_languages and fallback_languages:
|
||||||
|
allowed_languages = [code for code in fallback_languages if isinstance(code, str) and code]
|
||||||
|
|
||||||
|
default_voice = config.get("default_voice") if isinstance(config.get("default_voice"), str) else ""
|
||||||
|
used_voices = {entry.get("resolved_voice") or entry.get("voice") for entry in updated_roster.values()} - {None}
|
||||||
|
narrator_voice = ""
|
||||||
|
narrator_entry = updated_roster.get("narrator") if isinstance(updated_roster, Mapping) else None
|
||||||
|
if isinstance(narrator_entry, Mapping):
|
||||||
|
narrator_voice = str(
|
||||||
|
narrator_entry.get("resolved_voice")
|
||||||
|
or narrator_entry.get("default_voice")
|
||||||
|
or ""
|
||||||
|
).strip()
|
||||||
|
if narrator_voice:
|
||||||
|
used_voices.add(narrator_voice)
|
||||||
|
|
||||||
|
config_changed = False
|
||||||
|
new_config_payload: Dict[str, Any] = {
|
||||||
|
"language": config.get("language", "a"),
|
||||||
|
"languages": allowed_languages,
|
||||||
|
"default_voice": default_voice,
|
||||||
|
"speakers": dict(speakers_map),
|
||||||
|
"version": config.get("version", 1),
|
||||||
|
"notes": config.get("notes", ""),
|
||||||
|
}
|
||||||
|
|
||||||
|
speakers_payload = new_config_payload["speakers"]
|
||||||
|
|
||||||
|
for speaker_id, roster_entry in updated_roster.items():
|
||||||
|
if speaker_id == "narrator":
|
||||||
|
continue
|
||||||
|
label = str(roster_entry.get("label") or speaker_id)
|
||||||
|
config_entry = match_configured_speaker(speakers_map, speaker_id, label)
|
||||||
|
if config_entry is None:
|
||||||
|
continue
|
||||||
|
voice_id = str(config_entry.get("voice") or "").strip()
|
||||||
|
voice_profile = str(config_entry.get("voice_profile") or "").strip()
|
||||||
|
voice_formula = str(config_entry.get("voice_formula") or "").strip()
|
||||||
|
resolved_voice = str(config_entry.get("resolved_voice") or "").strip()
|
||||||
|
languages = config_entry.get("languages") if isinstance(config_entry.get("languages"), list) else []
|
||||||
|
chosen_voice = resolved_voice or voice_formula or voice_id or roster_entry.get("voice")
|
||||||
|
usable_languages = languages or allowed_languages
|
||||||
|
|
||||||
|
if chosen_voice:
|
||||||
|
roster_entry["resolved_voice"] = chosen_voice
|
||||||
|
roster_entry["voice"] = chosen_voice if not voice_profile and not voice_formula else roster_entry.get("voice", chosen_voice)
|
||||||
|
if voice_profile:
|
||||||
|
roster_entry["voice_profile"] = voice_profile
|
||||||
|
if voice_formula:
|
||||||
|
roster_entry["voice_formula"] = voice_formula
|
||||||
|
roster_entry["resolved_voice"] = voice_formula
|
||||||
|
if not voice_formula and not voice_profile and resolved_voice:
|
||||||
|
roster_entry["resolved_voice"] = resolved_voice
|
||||||
|
roster_entry["config_languages"] = usable_languages or []
|
||||||
|
|
||||||
|
if chosen_voice:
|
||||||
|
used_voices.add(chosen_voice)
|
||||||
|
|
||||||
|
# persist updates back to config payload if required
|
||||||
|
if persist_changes:
|
||||||
|
slug = config_entry.get("id") or slugify_label(label)
|
||||||
|
speakers_payload[slug] = {
|
||||||
|
"id": slug,
|
||||||
|
"label": label,
|
||||||
|
"gender": config_entry.get("gender", "unknown"),
|
||||||
|
"voice": voice_id,
|
||||||
|
"voice_profile": voice_profile,
|
||||||
|
"voice_formula": voice_formula,
|
||||||
|
"resolved_voice": roster_entry.get("resolved_voice", resolved_voice or voice_id),
|
||||||
|
"languages": usable_languages,
|
||||||
|
}
|
||||||
|
|
||||||
|
new_config = new_config_payload if (persist_changes and config_changed) else None
|
||||||
|
return updated_roster, allowed_languages, new_config
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_speaker_metadata(
|
||||||
|
*,
|
||||||
|
chapters: List[Dict[str, Any]],
|
||||||
|
chunks: List[Dict[str, Any]],
|
||||||
|
analysis_chunks: Optional[List[Dict[str, Any]]] = None,
|
||||||
|
voice: str,
|
||||||
|
voice_profile: Optional[str],
|
||||||
|
threshold: int,
|
||||||
|
existing_roster: Optional[Mapping[str, Any]] = None,
|
||||||
|
run_analysis: bool = True,
|
||||||
|
speaker_config: Optional[Mapping[str, Any]] = None,
|
||||||
|
apply_config: bool = False,
|
||||||
|
persist_config: bool = False,
|
||||||
|
inject_recommended: Optional[Any] = None,
|
||||||
|
) -> tuple[List[Dict[str, Any]], Dict[str, Any], Dict[str, Any], List[str], Optional[Dict[str, Any]]]:
|
||||||
|
chunk_list = [dict(chunk) for chunk in chunks]
|
||||||
|
analysis_source = [dict(chunk) for chunk in (analysis_chunks or chunks)]
|
||||||
|
threshold_value = max(1, int(threshold))
|
||||||
|
analysis_enabled = run_analysis
|
||||||
|
settings_state = load_settings()
|
||||||
|
global_random_languages = [
|
||||||
|
code
|
||||||
|
for code in settings_state.get("speaker_random_languages", [])
|
||||||
|
if isinstance(code, str) and code
|
||||||
|
]
|
||||||
|
|
||||||
|
if not analysis_enabled:
|
||||||
|
for chunk in chunk_list:
|
||||||
|
chunk["speaker_id"] = "narrator"
|
||||||
|
chunk["speaker_label"] = "Narrator"
|
||||||
|
analysis_payload = {
|
||||||
|
"version": "1.0",
|
||||||
|
"narrator": "narrator",
|
||||||
|
"assignments": {str(chunk.get("id")): "narrator" for chunk in chunk_list},
|
||||||
|
"speakers": {
|
||||||
|
"narrator": {
|
||||||
|
"id": "narrator",
|
||||||
|
"label": "Narrator",
|
||||||
|
"count": len(chunk_list),
|
||||||
|
"confidence": "low",
|
||||||
|
"sample_quotes": [],
|
||||||
|
"suppressed": False,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"suppressed": [],
|
||||||
|
"stats": {
|
||||||
|
"total_chunks": len(chunk_list),
|
||||||
|
"explicit_chunks": 0,
|
||||||
|
"active_speakers": 0,
|
||||||
|
"unique_speakers": 1,
|
||||||
|
"suppressed": 0,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
roster = build_narrator_roster(voice, voice_profile, existing_roster)
|
||||||
|
narrator_pron = roster["narrator"].get("pronunciation")
|
||||||
|
if narrator_pron:
|
||||||
|
analysis_payload["speakers"]["narrator"]["pronunciation"] = narrator_pron
|
||||||
|
return chunk_list, roster, analysis_payload, [], None
|
||||||
|
|
||||||
|
analysis_result = analyze_speakers(
|
||||||
|
chapters,
|
||||||
|
analysis_source,
|
||||||
|
threshold=threshold_value,
|
||||||
|
max_speakers=0,
|
||||||
|
)
|
||||||
|
analysis_payload = analysis_result.to_dict()
|
||||||
|
speakers_payload = analysis_payload.get("speakers", {})
|
||||||
|
ordered_ids = [
|
||||||
|
sid
|
||||||
|
for sid, meta in sorted(
|
||||||
|
(
|
||||||
|
(sid, meta)
|
||||||
|
for sid, meta in speakers_payload.items()
|
||||||
|
if sid != "narrator" and isinstance(meta, Mapping) and not meta.get("suppressed")
|
||||||
|
),
|
||||||
|
key=lambda item: item[1].get("count", 0),
|
||||||
|
reverse=True,
|
||||||
|
)
|
||||||
|
]
|
||||||
|
analysis_payload["ordered_speakers"] = ordered_ids
|
||||||
|
assignments = analysis_payload.get("assignments", {})
|
||||||
|
suppressed_ids = analysis_payload.get("suppressed", [])
|
||||||
|
suppressed_details: List[Dict[str, Any]] = []
|
||||||
|
speakers_payload = analysis_payload.get("speakers", {})
|
||||||
|
if isinstance(suppressed_ids, Iterable):
|
||||||
|
for suppressed_id in suppressed_ids:
|
||||||
|
speaker_meta = speakers_payload.get(suppressed_id) if isinstance(speakers_payload, dict) else None
|
||||||
|
if isinstance(speaker_meta, dict):
|
||||||
|
suppressed_details.append(
|
||||||
|
{
|
||||||
|
"id": suppressed_id,
|
||||||
|
"label": speaker_meta.get("label")
|
||||||
|
or str(suppressed_id).replace("_", " ").title(),
|
||||||
|
"pronunciation": speaker_meta.get("pronunciation"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
suppressed_details.append(
|
||||||
|
{
|
||||||
|
"id": suppressed_id,
|
||||||
|
"label": str(suppressed_id).replace("_", " ").title(),
|
||||||
|
"pronunciation": None,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
analysis_payload["suppressed_details"] = suppressed_details
|
||||||
|
roster = build_speaker_roster(
|
||||||
|
analysis_payload,
|
||||||
|
voice,
|
||||||
|
voice_profile,
|
||||||
|
existing=existing_roster,
|
||||||
|
order=analysis_payload.get("ordered_speakers"),
|
||||||
|
)
|
||||||
|
applied_languages: List[str] = []
|
||||||
|
updated_config: Optional[Dict[str, Any]] = None
|
||||||
|
if apply_config and speaker_config:
|
||||||
|
roster, applied_languages, updated_config = apply_speaker_config_to_roster(
|
||||||
|
roster,
|
||||||
|
speaker_config,
|
||||||
|
persist_changes=persist_config,
|
||||||
|
fallback_languages=global_random_languages,
|
||||||
|
)
|
||||||
|
speakers_payload = analysis_payload.get("speakers")
|
||||||
|
if isinstance(speakers_payload, dict):
|
||||||
|
for roster_id, roster_payload in roster.items():
|
||||||
|
speaker_meta = speakers_payload.get(roster_id)
|
||||||
|
if isinstance(speaker_meta, dict):
|
||||||
|
for key in ("voice", "voice_profile", "voice_formula", "resolved_voice"):
|
||||||
|
value = roster_payload.get(key)
|
||||||
|
if value:
|
||||||
|
speaker_meta[key] = value
|
||||||
|
effective_languages: List[str] = []
|
||||||
|
if applied_languages:
|
||||||
|
effective_languages = applied_languages
|
||||||
|
elif isinstance(analysis_payload.get("config_languages"), list):
|
||||||
|
effective_languages = [
|
||||||
|
code for code in analysis_payload.get("config_languages", []) if isinstance(code, str) and code
|
||||||
|
]
|
||||||
|
elif global_random_languages:
|
||||||
|
effective_languages = list(global_random_languages)
|
||||||
|
|
||||||
|
if effective_languages:
|
||||||
|
analysis_payload["config_languages"] = effective_languages
|
||||||
|
speakers_payload = analysis_payload.get("speakers")
|
||||||
|
if isinstance(speakers_payload, dict):
|
||||||
|
for roster_id, roster_payload in roster.items():
|
||||||
|
if roster_id in speakers_payload and isinstance(roster_payload, dict):
|
||||||
|
pronunciation_value = roster_payload.get("pronunciation")
|
||||||
|
if pronunciation_value:
|
||||||
|
speakers_payload[roster_id]["pronunciation"] = pronunciation_value
|
||||||
|
|
||||||
|
fallback_languages = effective_languages or []
|
||||||
|
if callable(inject_recommended):
|
||||||
|
inject_recommended(roster, fallback_languages=fallback_languages)
|
||||||
|
|
||||||
|
for chunk in chunk_list:
|
||||||
|
chunk_id = str(chunk.get("id"))
|
||||||
|
speaker_id = assignments.get(chunk_id, "narrator")
|
||||||
|
chunk["speaker_id"] = speaker_id
|
||||||
|
speaker_meta = roster.get(speaker_id)
|
||||||
|
chunk["speaker_label"] = speaker_meta.get("label") if isinstance(speaker_meta, dict) else speaker_id
|
||||||
|
|
||||||
|
return chunk_list, roster, analysis_payload, applied_languages, updated_config
|
||||||
@@ -1,43 +1,51 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
"""Unified split pattern logic extracted from 3 copies."""
|
"""Unified split pattern logic extracted from 3 copies."""
|
||||||
import re
|
|
||||||
|
|
||||||
from abogen.domain.enums import Language, SubtitleMode
|
from abogen.domain.enums import Language, SubtitleMode
|
||||||
|
|
||||||
PUNCTUATION_SENTENCE = r".!?。!?"
|
# Canonical punctuation sets covering all supported scripts:
|
||||||
PUNCTUATION_SENTENCE_COMMA = r".!?,。!?、,"
|
# ASCII (. ! ?), ellipsis (…), Arabic ؟, CJK (。!?), Devanagari ।
|
||||||
|
PUNCTUATION_SENTENCE = r".!?…؟。!?।"
|
||||||
|
# Commas: ASCII , CJK fullwidth ,CJK ideographic 、
|
||||||
|
PUNCTUATION_SENTENCE_COMMA = r".!?…,?。!?،,、।"
|
||||||
|
PUNCTUATION_COMMAS = ",,、"
|
||||||
|
|
||||||
|
|
||||||
def get_split_pattern(language: str, subtitle_mode: str) -> str:
|
def get_split_pattern(language: Language, subtitle_mode: str) -> str:
|
||||||
"""Get the appropriate split pattern based on language and subtitle mode.
|
"""Get the appropriate split pattern based on language and subtitle mode.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
language: Language code (a, b, e, f, etc.)
|
language: Language enum value, ISO code, or kokoro letter code.
|
||||||
subtitle_mode: Subtitle mode ("Sentence", "Sentence + Comma", "Line", etc.)
|
subtitle_mode: Subtitle mode ("Sentence", "Sentence + Comma", "Line", etc.)
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Split pattern string
|
Split pattern string
|
||||||
"""
|
"""
|
||||||
try:
|
|
||||||
lang = Language.from_str(language) if not isinstance(language, Language) else language
|
|
||||||
except ValueError:
|
|
||||||
lang = None # unknown language — treat as non-English, non-CJK
|
|
||||||
try:
|
try:
|
||||||
mode = SubtitleMode.from_str(subtitle_mode) if not isinstance(subtitle_mode, SubtitleMode) else subtitle_mode
|
mode = SubtitleMode.from_str(subtitle_mode) if not isinstance(subtitle_mode, SubtitleMode) else subtitle_mode
|
||||||
except ValueError:
|
except ValueError:
|
||||||
mode = SubtitleMode.DISABLED
|
mode = SubtitleMode.DISABLED
|
||||||
|
|
||||||
# For English, always use newline splitting only
|
# English: spaCy is NOT used for pre-TTS segmentation (it is only used
|
||||||
if lang in (Language.EN_US, Language.EN_GB):
|
# for post-TTS subtitle boundaries), so sentence boundaries for English
|
||||||
return "\n"
|
# are applied at subtitle time, not in the TTS engine. Disabled, Line,
|
||||||
|
# Sentence, and Sentence + Comma all keep newline-only engine splitting.
|
||||||
|
if language in (Language.EN_US, Language.EN_GB):
|
||||||
|
if mode in (
|
||||||
|
SubtitleMode.DISABLED,
|
||||||
|
SubtitleMode.LINE,
|
||||||
|
SubtitleMode.SENTENCE,
|
||||||
|
SubtitleMode.SENTENCE_COMMA,
|
||||||
|
):
|
||||||
|
return "\n"
|
||||||
|
|
||||||
# Determine spacing pattern based on language
|
# Determine spacing pattern based on language
|
||||||
spacing = r"\s*" if lang and lang.is_cjk else r"\s+"
|
spacing = r"\s*" if language.is_cjk else r"\s+"
|
||||||
|
|
||||||
# For CJK languages, when subtitle mode is Disabled or Line, prefer
|
# For CJK languages, when subtitle mode is Disabled or Line, prefer
|
||||||
# punctuation-based splitting instead of plain newline splitting.
|
# punctuation-based splitting instead of plain newline splitting.
|
||||||
if mode in (SubtitleMode.DISABLED, SubtitleMode.LINE) and lang and lang.is_cjk:
|
if mode in (SubtitleMode.DISABLED, SubtitleMode.LINE) and language.is_cjk:
|
||||||
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
|
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
|
||||||
|
|
||||||
if mode == SubtitleMode.LINE:
|
if mode == SubtitleMode.LINE:
|
||||||
|
|||||||
@@ -11,11 +11,35 @@ import re
|
|||||||
from typing import List, Optional, Tuple
|
from typing import List, Optional, Tuple
|
||||||
|
|
||||||
from abogen.domain.enums import Language, SubtitleMode
|
from abogen.domain.enums import Language, SubtitleMode
|
||||||
|
from abogen.domain.split_pattern import PUNCTUATION_SENTENCE, PUNCTUATION_SENTENCE_COMMA
|
||||||
|
|
||||||
|
_CLOSING_DELIMS = "\"\"\"\"'\"”’»›)]}」』"
|
||||||
|
|
||||||
|
|
||||||
# Punctuation constants for sentence splitting
|
def _is_sentence_boundary(
|
||||||
PUNCTUATION_SENTENCE = ".!?\u061f\u3002\uff01\uff1f" # .!? .?. ??
|
token: dict,
|
||||||
PUNCTUATION_SENTENCE_COMMA = ".!?,\u3001\u061f\u3002\uff01\uff0c\uff1f" # .!?, ,. ??
|
current_sentence: List[dict],
|
||||||
|
separator: str,
|
||||||
|
) -> bool:
|
||||||
|
"""Check whether token ends a sentence, considering closing quotes and brackets."""
|
||||||
|
ws = token.get("whitespace", "") or ""
|
||||||
|
if not ws:
|
||||||
|
return False
|
||||||
|
|
||||||
|
# For Line mode, a newline in whitespace or text marks line boundary
|
||||||
|
if separator == r"\n":
|
||||||
|
return "\n" in ws or "\n" in str(token.get("text", ""))
|
||||||
|
|
||||||
|
text = str(token.get("text", ""))
|
||||||
|
if re.search(rf"{separator}[{re.escape(_CLOSING_DELIMS)}]*$", text):
|
||||||
|
return True
|
||||||
|
|
||||||
|
if len(current_sentence) >= 2 and text and all(c in _CLOSING_DELIMS for c in text):
|
||||||
|
prev_text = str(current_sentence[-2].get("text", ""))
|
||||||
|
if re.search(rf"{separator}$", prev_text):
|
||||||
|
return True
|
||||||
|
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
def process_subtitle_tokens(
|
def process_subtitle_tokens(
|
||||||
@@ -23,7 +47,7 @@ def process_subtitle_tokens(
|
|||||||
subtitle_entries: List[Tuple[float, float, str]],
|
subtitle_entries: List[Tuple[float, float, str]],
|
||||||
max_subtitle_words: int,
|
max_subtitle_words: int,
|
||||||
subtitle_mode: str,
|
subtitle_mode: str,
|
||||||
lang_code: str,
|
language: Language,
|
||||||
use_spacy_segmentation: bool = False,
|
use_spacy_segmentation: bool = False,
|
||||||
fallback_end_time: Optional[float] = None,
|
fallback_end_time: Optional[float] = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
@@ -39,44 +63,62 @@ def process_subtitle_tokens(
|
|||||||
max_subtitle_words: Maximum number of words per subtitle entry.
|
max_subtitle_words: Maximum number of words per subtitle entry.
|
||||||
subtitle_mode: One of "Disabled", "Line", "Sentence", "Sentence + Comma",
|
subtitle_mode: One of "Disabled", "Line", "Sentence", "Sentence + Comma",
|
||||||
"Sentence + Highlighting", or a string like "5" for word-count mode.
|
"Sentence + Highlighting", or a string like "5" for word-count mode.
|
||||||
lang_code: Language code for spaCy processing (e.g., "a" for English).
|
language: Language enum value for spaCy processing.
|
||||||
use_spacy_segmentation: Whether to use spaCy for sentence boundary detection.
|
use_spacy_segmentation: Whether to use spaCy for sentence boundary detection.
|
||||||
fallback_end_time: Fallback end time for the last entry if none is available.
|
fallback_end_time: Fallback end time for the last entry if none is available.
|
||||||
"""
|
"""
|
||||||
if not tokens_with_timestamps:
|
if not tokens_with_timestamps:
|
||||||
return
|
return
|
||||||
|
|
||||||
|
if not isinstance(language, Language):
|
||||||
|
try:
|
||||||
|
language = Language.from_str(str(language))
|
||||||
|
except ValueError:
|
||||||
|
language = Language.EN_US
|
||||||
|
|
||||||
|
if isinstance(subtitle_mode, SubtitleMode):
|
||||||
|
subtitle_mode_str = subtitle_mode.value
|
||||||
|
else:
|
||||||
|
subtitle_mode_str = str(subtitle_mode)
|
||||||
|
|
||||||
processed_tokens = tokens_with_timestamps
|
processed_tokens = tokens_with_timestamps
|
||||||
|
|
||||||
# For English with spaCy enabled and sentence-based modes, use spaCy for sentence boundaries
|
# For English with spaCy enabled and sentence-based modes, use spaCy for sentence boundaries
|
||||||
# spaCy is disabled when subtitle mode is "Disabled" or "Line"
|
# spaCy is disabled when subtitle mode is "Disabled" or "Line"
|
||||||
use_spacy_for_english = (
|
use_spacy_for_english = (
|
||||||
use_spacy_segmentation
|
use_spacy_segmentation
|
||||||
and subtitle_mode not in [SubtitleMode.DISABLED, SubtitleMode.LINE]
|
and subtitle_mode_str not in [SubtitleMode.DISABLED.value, SubtitleMode.LINE.value, "Disabled", "Line"]
|
||||||
and lang_code in [Language.EN_US, Language.EN_GB]
|
and language in [Language.EN_US, Language.EN_GB]
|
||||||
and subtitle_mode in [SubtitleMode.SENTENCE, SubtitleMode.SENTENCE_COMMA]
|
and subtitle_mode_str in [SubtitleMode.SENTENCE.value, SubtitleMode.SENTENCE_COMMA.value, "Sentence", "Sentence + Comma"]
|
||||||
)
|
)
|
||||||
|
|
||||||
if subtitle_mode == SubtitleMode.SENTENCE_HIGHLIGHT:
|
if subtitle_mode_str in (SubtitleMode.SENTENCE_HIGHLIGHT.value, "Sentence + Highlighting"):
|
||||||
_process_karaoke_highlighting(
|
_process_karaoke_highlighting(
|
||||||
processed_tokens, subtitle_entries, max_subtitle_words, fallback_end_time
|
processed_tokens, subtitle_entries, max_subtitle_words, fallback_end_time
|
||||||
)
|
)
|
||||||
elif subtitle_mode in [SubtitleMode.SENTENCE, SubtitleMode.SENTENCE_COMMA, SubtitleMode.LINE]:
|
elif subtitle_mode_str in [
|
||||||
if use_spacy_for_english and subtitle_mode != SubtitleMode.LINE:
|
SubtitleMode.SENTENCE.value,
|
||||||
|
SubtitleMode.SENTENCE_COMMA.value,
|
||||||
|
SubtitleMode.LINE.value,
|
||||||
|
"Sentence",
|
||||||
|
"Sentence + Comma",
|
||||||
|
"Line",
|
||||||
|
]:
|
||||||
|
if use_spacy_for_english and subtitle_mode_str not in (SubtitleMode.LINE.value, "Line"):
|
||||||
_process_spacy_sentences(
|
_process_spacy_sentences(
|
||||||
processed_tokens, subtitle_entries, max_subtitle_words,
|
processed_tokens, subtitle_entries, max_subtitle_words,
|
||||||
subtitle_mode, lang_code, fallback_end_time
|
subtitle_mode_str, language, fallback_end_time
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
_process_regex_sentences(
|
_process_regex_sentences(
|
||||||
processed_tokens, subtitle_entries, max_subtitle_words,
|
processed_tokens, subtitle_entries, max_subtitle_words,
|
||||||
subtitle_mode, fallback_end_time
|
subtitle_mode_str, fallback_end_time
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
# Word count-based grouping (e.g., "5" for 5-word groups)
|
# Word count-based grouping (e.g., "5" for 5-word groups)
|
||||||
_process_word_count(
|
_process_word_count(
|
||||||
processed_tokens, subtitle_entries, max_subtitle_words,
|
processed_tokens, subtitle_entries, max_subtitle_words,
|
||||||
subtitle_mode, fallback_end_time
|
subtitle_mode_str, fallback_end_time
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -87,7 +129,7 @@ def _process_karaoke_highlighting(
|
|||||||
fallback_end_time: Optional[float],
|
fallback_end_time: Optional[float],
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Process tokens for Sentence + Highlighting mode (karaoke effect)."""
|
"""Process tokens for Sentence + Highlighting mode (karaoke effect)."""
|
||||||
separator = rf"[{re.escape(PUNCTUATION_SENTENCE)}]"
|
separator = rf"[{PUNCTUATION_SENTENCE}]"
|
||||||
current_sentence = []
|
current_sentence = []
|
||||||
word_count = 0
|
word_count = 0
|
||||||
|
|
||||||
@@ -95,10 +137,8 @@ def _process_karaoke_highlighting(
|
|||||||
current_sentence.append(token)
|
current_sentence.append(token)
|
||||||
word_count += 1
|
word_count += 1
|
||||||
|
|
||||||
# Split sentences based on separator or word count
|
is_boundary = _is_sentence_boundary(token, current_sentence, separator)
|
||||||
if (
|
if is_boundary or word_count >= max_subtitle_words:
|
||||||
re.search(separator, token["text"]) and token.get("whitespace") == " "
|
|
||||||
) or word_count >= max_subtitle_words:
|
|
||||||
if current_sentence:
|
if current_sentence:
|
||||||
# Create karaoke subtitle entry for this sentence
|
# Create karaoke subtitle entry for this sentence
|
||||||
start_time = current_sentence[0]["start"]
|
start_time = current_sentence[0]["start"]
|
||||||
@@ -113,13 +153,18 @@ def _process_karaoke_highlighting(
|
|||||||
if t.get("end") is not None and t.get("start") is not None
|
if t.get("end") is not None and t.get("start") is not None
|
||||||
else 0.5
|
else 0.5
|
||||||
)
|
)
|
||||||
duration_cs = int(duration * 100)
|
try:
|
||||||
|
duration_cs = int(duration * 100)
|
||||||
|
except (ValueError, OverflowError, TypeError):
|
||||||
|
duration_cs = 50
|
||||||
# Add karaoke effect
|
# Add karaoke effect
|
||||||
karaoke_text += f"{{\\kf{duration_cs}}}{t['text']}{t.get('whitespace', '') or ''}"
|
karaoke_text += f"{{\\kf{duration_cs}}}{t.get('text', '')}{t.get('whitespace', '') or ''}"
|
||||||
|
|
||||||
subtitle_entries.append(
|
text_stripped = karaoke_text.strip()
|
||||||
(start_time, end_time, karaoke_text.strip())
|
if text_stripped:
|
||||||
)
|
subtitle_entries.append(
|
||||||
|
(start_time, end_time, text_stripped)
|
||||||
|
)
|
||||||
current_sentence = []
|
current_sentence = []
|
||||||
word_count = 0
|
word_count = 0
|
||||||
|
|
||||||
@@ -132,9 +177,14 @@ def _process_karaoke_highlighting(
|
|||||||
karaoke_text = ""
|
karaoke_text = ""
|
||||||
for t in current_sentence:
|
for t in current_sentence:
|
||||||
duration = t["end"] - t["start"] if t.get("end") and t.get("start") else 0.5
|
duration = t["end"] - t["start"] if t.get("end") and t.get("start") else 0.5
|
||||||
duration_cs = int(duration * 100)
|
try:
|
||||||
karaoke_text += f"{{\\kf{duration_cs}}}{t['text']}{t.get('whitespace', '') or ''}"
|
duration_cs = int(duration * 100)
|
||||||
subtitle_entries.append((start_time, end_time, karaoke_text.strip()))
|
except (ValueError, OverflowError, TypeError):
|
||||||
|
duration_cs = 50
|
||||||
|
karaoke_text += f"{{\\kf{duration_cs}}}{t.get('text', '')}{t.get('whitespace', '') or ''}"
|
||||||
|
text_stripped = karaoke_text.strip()
|
||||||
|
if text_stripped:
|
||||||
|
subtitle_entries.append((start_time, end_time, text_stripped))
|
||||||
|
|
||||||
# Fallback for last entry
|
# Fallback for last entry
|
||||||
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
|
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
|
||||||
@@ -145,7 +195,7 @@ def _process_spacy_sentences(
|
|||||||
subtitle_entries: List[Tuple[float, float, str]],
|
subtitle_entries: List[Tuple[float, float, str]],
|
||||||
max_subtitle_words: int,
|
max_subtitle_words: int,
|
||||||
subtitle_mode: str,
|
subtitle_mode: str,
|
||||||
lang_code: str,
|
language: Language,
|
||||||
fallback_end_time: Optional[float],
|
fallback_end_time: Optional[float],
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Process tokens using spaCy for sentence boundary detection."""
|
"""Process tokens using spaCy for sentence boundary detection."""
|
||||||
@@ -159,7 +209,7 @@ def _process_spacy_sentences(
|
|||||||
)
|
)
|
||||||
return
|
return
|
||||||
|
|
||||||
nlp = get_spacy_model(lang_code)
|
nlp = get_spacy_model(language)
|
||||||
if not nlp:
|
if not nlp:
|
||||||
_process_regex_sentences(
|
_process_regex_sentences(
|
||||||
tokens, subtitle_entries, max_subtitle_words,
|
tokens, subtitle_entries, max_subtitle_words,
|
||||||
@@ -170,7 +220,7 @@ def _process_spacy_sentences(
|
|||||||
# Build full text and track character positions to token indices
|
# Build full text and track character positions to token indices
|
||||||
full_text = ""
|
full_text = ""
|
||||||
for token in tokens:
|
for token in tokens:
|
||||||
text_part = token["text"] + (token.get("whitespace") or "")
|
text_part = str(token.get("text", "")) + (token.get("whitespace") or "")
|
||||||
full_text += text_part
|
full_text += text_part
|
||||||
|
|
||||||
# Get sentence boundaries from spaCy
|
# Get sentence boundaries from spaCy
|
||||||
@@ -178,7 +228,7 @@ def _process_spacy_sentences(
|
|||||||
sentence_boundaries = [sent.end_char for sent in doc.sents]
|
sentence_boundaries = [sent.end_char for sent in doc.sents]
|
||||||
|
|
||||||
# For "Sentence + Comma" mode, also split on commas
|
# For "Sentence + Comma" mode, also split on commas
|
||||||
if subtitle_mode == SubtitleMode.SENTENCE_COMMA:
|
if subtitle_mode in (SubtitleMode.SENTENCE_COMMA.value, "Sentence + Comma"):
|
||||||
comma_positions = [
|
comma_positions = [
|
||||||
i + 1 for i, c in enumerate(full_text) if c == ","
|
i + 1 for i, c in enumerate(full_text) if c == ","
|
||||||
]
|
]
|
||||||
@@ -186,6 +236,56 @@ def _process_spacy_sentences(
|
|||||||
set(sentence_boundaries + comma_positions)
|
set(sentence_boundaries + comma_positions)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# spaCy does not treat ellipsis ("...", "..", "…") as a sentence
|
||||||
|
# boundary ("Lorem ipsum... Lorem..." stays one sentence), so ellipsis
|
||||||
|
# runs followed by whitespace/end would merge into a single subtitle
|
||||||
|
# entry. Add explicit boundaries after them. Single dots ("Mr.") stay
|
||||||
|
# spaCy's responsibility so abbreviations don't regress.
|
||||||
|
for m in re.finditer(r"\.{2,}(?=[\s\"'”’»›)\]}]|$)|…(?=[\s\"'”’»›)\]}]|$)", full_text):
|
||||||
|
sentence_boundaries.append(m.end())
|
||||||
|
# Double newlines are paragraph breaks: always split, even when spaCy
|
||||||
|
# sees no sentence boundary.
|
||||||
|
for m in re.finditer(r"\n{2,}", full_text):
|
||||||
|
sentence_boundaries.append(m.end())
|
||||||
|
sentence_boundaries = sorted(set(sentence_boundaries))
|
||||||
|
|
||||||
|
# Multi-sentence single FakeToken handling
|
||||||
|
if len(tokens) == 1 and len(sentence_boundaries) > 1:
|
||||||
|
single = tokens[0]
|
||||||
|
start_time = single.get("start", 0.0) or 0.0
|
||||||
|
end_time = single.get("end")
|
||||||
|
duration = (end_time - start_time) if (end_time is not None and end_time > start_time) else 0.0
|
||||||
|
|
||||||
|
prev_pos = 0
|
||||||
|
cur_start = start_time
|
||||||
|
total_chars = max(len(full_text), 1)
|
||||||
|
|
||||||
|
for i, b_pos in enumerate(sentence_boundaries):
|
||||||
|
piece = full_text[prev_pos:b_pos].strip()
|
||||||
|
if not piece:
|
||||||
|
prev_pos = b_pos
|
||||||
|
continue
|
||||||
|
if i == len(sentence_boundaries) - 1:
|
||||||
|
cur_end = end_time if end_time is not None else (cur_start + 1.0)
|
||||||
|
else:
|
||||||
|
cur_end = cur_start + duration * len(piece) / total_chars
|
||||||
|
subtitle_entries.append((cur_start, cur_end, piece))
|
||||||
|
cur_start = cur_end
|
||||||
|
prev_pos = b_pos
|
||||||
|
|
||||||
|
if prev_pos < len(full_text):
|
||||||
|
remainder = full_text[prev_pos:].strip()
|
||||||
|
if remainder:
|
||||||
|
remainder_end = end_time
|
||||||
|
if remainder_end is None:
|
||||||
|
remainder_end = fallback_end_time
|
||||||
|
if remainder_end is None:
|
||||||
|
remainder_end = cur_start
|
||||||
|
subtitle_entries.append((cur_start, remainder_end, remainder))
|
||||||
|
|
||||||
|
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
|
||||||
|
return
|
||||||
|
|
||||||
# Group tokens by sentence boundaries
|
# Group tokens by sentence boundaries
|
||||||
current_sentence = []
|
current_sentence = []
|
||||||
word_count = 0
|
word_count = 0
|
||||||
@@ -195,7 +295,7 @@ def _process_spacy_sentences(
|
|||||||
for token in tokens:
|
for token in tokens:
|
||||||
current_sentence.append(token)
|
current_sentence.append(token)
|
||||||
word_count += 1
|
word_count += 1
|
||||||
text_len = len(token["text"]) + len(token.get("whitespace") or "")
|
text_len = len(str(token.get("text", ""))) + len(token.get("whitespace") or "")
|
||||||
current_char_pos += text_len
|
current_char_pos += text_len
|
||||||
|
|
||||||
# Check if we've hit a sentence boundary or max words
|
# Check if we've hit a sentence boundary or max words
|
||||||
@@ -208,15 +308,19 @@ def _process_spacy_sentences(
|
|||||||
start_time = current_sentence[0]["start"]
|
start_time = current_sentence[0]["start"]
|
||||||
end_time = current_sentence[-1]["end"]
|
end_time = current_sentence[-1]["end"]
|
||||||
sentence_text = "".join(
|
sentence_text = "".join(
|
||||||
t["text"] + (t.get("whitespace") or "")
|
str(t.get("text", "")) + (t.get("whitespace") or "")
|
||||||
for t in current_sentence
|
for t in current_sentence
|
||||||
)
|
).strip()
|
||||||
subtitle_entries.append(
|
if sentence_text:
|
||||||
(start_time, end_time, sentence_text.strip())
|
subtitle_entries.append(
|
||||||
)
|
(start_time, end_time, sentence_text)
|
||||||
|
)
|
||||||
current_sentence = []
|
current_sentence = []
|
||||||
word_count = 0
|
word_count = 0
|
||||||
if at_boundary:
|
while (
|
||||||
|
boundary_idx < len(sentence_boundaries)
|
||||||
|
and current_char_pos >= sentence_boundaries[boundary_idx]
|
||||||
|
):
|
||||||
boundary_idx += 1
|
boundary_idx += 1
|
||||||
|
|
||||||
# Add remaining tokens
|
# Add remaining tokens
|
||||||
@@ -224,12 +328,13 @@ def _process_spacy_sentences(
|
|||||||
start_time = current_sentence[0]["start"]
|
start_time = current_sentence[0]["start"]
|
||||||
end_time = current_sentence[-1]["end"]
|
end_time = current_sentence[-1]["end"]
|
||||||
sentence_text = "".join(
|
sentence_text = "".join(
|
||||||
t["text"] + (t.get("whitespace") or "")
|
str(t.get("text", "")) + (t.get("whitespace") or "")
|
||||||
for t in current_sentence
|
for t in current_sentence
|
||||||
)
|
).strip()
|
||||||
subtitle_entries.append(
|
if sentence_text:
|
||||||
(start_time, end_time, sentence_text.strip())
|
subtitle_entries.append(
|
||||||
)
|
(start_time, end_time, sentence_text)
|
||||||
|
)
|
||||||
|
|
||||||
# Fallback for last entry
|
# Fallback for last entry
|
||||||
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
|
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
|
||||||
@@ -244,14 +349,12 @@ def _process_regex_sentences(
|
|||||||
) -> None:
|
) -> None:
|
||||||
"""Process tokens using regex for sentence boundary detection."""
|
"""Process tokens using regex for sentence boundary detection."""
|
||||||
# Define separator pattern based on mode
|
# Define separator pattern based on mode
|
||||||
if subtitle_mode == SubtitleMode.LINE:
|
if subtitle_mode in (SubtitleMode.LINE.value, "Line"):
|
||||||
separator = r"\n"
|
separator = r"\n"
|
||||||
elif subtitle_mode == SubtitleMode.SENTENCE:
|
elif subtitle_mode in (SubtitleMode.SENTENCE.value, "Sentence"):
|
||||||
# Use punctuation without comma
|
separator = rf"[{PUNCTUATION_SENTENCE}]"
|
||||||
separator = rf"[{re.escape(PUNCTUATION_SENTENCE)}]"
|
|
||||||
else: # Sentence + Comma
|
else: # Sentence + Comma
|
||||||
# Use punctuation with comma
|
separator = rf"[{PUNCTUATION_SENTENCE_COMMA}]"
|
||||||
separator = rf"[{re.escape(PUNCTUATION_SENTENCE_COMMA)}]"
|
|
||||||
|
|
||||||
current_sentence = []
|
current_sentence = []
|
||||||
word_count = 0
|
word_count = 0
|
||||||
@@ -261,22 +364,22 @@ def _process_regex_sentences(
|
|||||||
word_count += 1
|
word_count += 1
|
||||||
|
|
||||||
# Split sentences based on separator or word count
|
# Split sentences based on separator or word count
|
||||||
if (
|
is_boundary = _is_sentence_boundary(token, current_sentence, separator)
|
||||||
re.search(separator, token["text"]) and token.get("whitespace") == " "
|
if is_boundary or word_count >= max_subtitle_words:
|
||||||
) or word_count >= max_subtitle_words:
|
|
||||||
if current_sentence:
|
if current_sentence:
|
||||||
# Create subtitle entry for this sentence
|
# Create subtitle entry for this sentence
|
||||||
start_time = current_sentence[0]["start"]
|
start_time = current_sentence[0]["start"]
|
||||||
end_time = current_sentence[-1]["end"]
|
end_time = current_sentence[-1]["end"]
|
||||||
|
|
||||||
# Simplified text joining logic
|
sentence_text = "".join(
|
||||||
sentence_text = ""
|
str(t.get("text", "")) + (t.get("whitespace") or "")
|
||||||
for t in current_sentence:
|
for t in current_sentence
|
||||||
sentence_text += t["text"] + (t.get("whitespace") or "")
|
).strip()
|
||||||
|
|
||||||
subtitle_entries.append(
|
if sentence_text:
|
||||||
(start_time, end_time, sentence_text.strip())
|
subtitle_entries.append(
|
||||||
)
|
(start_time, end_time, sentence_text)
|
||||||
|
)
|
||||||
current_sentence = []
|
current_sentence = []
|
||||||
word_count = 0
|
word_count = 0
|
||||||
|
|
||||||
@@ -285,23 +388,39 @@ def _process_regex_sentences(
|
|||||||
start_time = current_sentence[0]["start"]
|
start_time = current_sentence[0]["start"]
|
||||||
end_time = current_sentence[-1]["end"]
|
end_time = current_sentence[-1]["end"]
|
||||||
|
|
||||||
sentence_text = ""
|
sentence_text = "".join(
|
||||||
for t in current_sentence:
|
str(t.get("text", "")) + (t.get("whitespace") or "")
|
||||||
sentence_text += t["text"] + (t.get("whitespace") or "")
|
for t in current_sentence
|
||||||
sentence_text = sentence_text.strip()
|
).strip()
|
||||||
|
|
||||||
if len(current_sentence) == 1:
|
if len(current_sentence) == 1:
|
||||||
parts = re.split(rf"(?<={separator})\s+", sentence_text)
|
split_pat = (
|
||||||
|
r"\n+"
|
||||||
|
if separator == r"\n"
|
||||||
|
else rf"(?<={separator})\s+|(?<={separator}[{re.escape(_CLOSING_DELIMS)}])\s+"
|
||||||
|
)
|
||||||
|
parts = [p.strip() for p in re.split(split_pat, sentence_text) if p.strip()]
|
||||||
if len(parts) > 1:
|
if len(parts) > 1:
|
||||||
d = end_time - start_time
|
d = (end_time - start_time) if (end_time is not None and start_time is not None and end_time > start_time) else 0.0
|
||||||
|
total_len = max(len(sentence_text), 1)
|
||||||
|
cur_s = start_time if start_time is not None else 0.0
|
||||||
for i, p in enumerate(parts):
|
for i, p in enumerate(parts):
|
||||||
e = end_time if i == len(parts) - 1 else start_time + d * len(p) / len(sentence_text)
|
if i == len(parts) - 1 and end_time is not None:
|
||||||
subtitle_entries.append((start_time, e, p.strip()))
|
e = end_time
|
||||||
start_time = e
|
else:
|
||||||
|
e = cur_s + d * len(p) / total_len
|
||||||
|
subtitle_entries.append((cur_s, e, p))
|
||||||
|
cur_s = e
|
||||||
current_sentence = []
|
current_sentence = []
|
||||||
|
|
||||||
if current_sentence:
|
if current_sentence and sentence_text:
|
||||||
subtitle_entries.append((start_time, end_time, sentence_text))
|
safe_start = start_time if start_time is not None else 0.0
|
||||||
|
safe_end = end_time
|
||||||
|
if safe_end is None:
|
||||||
|
safe_end = fallback_end_time
|
||||||
|
if safe_end is None:
|
||||||
|
safe_end = safe_start
|
||||||
|
subtitle_entries.append((safe_start, safe_end, sentence_text))
|
||||||
|
|
||||||
# Fallback for last entry
|
# Fallback for last entry
|
||||||
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
|
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
|
||||||
@@ -334,27 +453,29 @@ def _process_word_count(
|
|||||||
# Split after counting N spaces
|
# Split after counting N spaces
|
||||||
if space_count >= word_count:
|
if space_count >= word_count:
|
||||||
text = "".join(
|
text = "".join(
|
||||||
t["text"] + (t.get("whitespace") or "")
|
str(t.get("text", "")) + (t.get("whitespace") or "")
|
||||||
for t in current_group
|
for t in current_group
|
||||||
)
|
).strip()
|
||||||
subtitle_entries.append(
|
if text:
|
||||||
(
|
subtitle_entries.append(
|
||||||
current_group[0]["start"],
|
(
|
||||||
current_group[-1]["end"],
|
current_group[0]["start"],
|
||||||
text.strip(),
|
current_group[-1]["end"],
|
||||||
|
text,
|
||||||
|
)
|
||||||
)
|
)
|
||||||
)
|
|
||||||
current_group = []
|
current_group = []
|
||||||
space_count = 0
|
space_count = 0
|
||||||
|
|
||||||
# Add any remaining tokens
|
# Add any remaining tokens
|
||||||
if current_group:
|
if current_group:
|
||||||
text = "".join(
|
text = "".join(
|
||||||
t["text"] + (t.get("whitespace") or "") for t in current_group
|
str(t.get("text", "")) + (t.get("whitespace") or "") for t in current_group
|
||||||
)
|
).strip()
|
||||||
subtitle_entries.append(
|
if text:
|
||||||
(current_group[0]["start"], current_group[-1]["end"], text.strip())
|
subtitle_entries.append(
|
||||||
)
|
(current_group[0]["start"], current_group[-1]["end"], text)
|
||||||
|
)
|
||||||
|
|
||||||
# Fallback for last entry
|
# Fallback for last entry
|
||||||
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
|
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
|
||||||
|
|||||||
@@ -14,7 +14,6 @@ from typing import Any, Callable, List, Optional, Tuple
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
from abogen.domain.audio_buffer import (
|
from abogen.domain.audio_buffer import (
|
||||||
create_silence,
|
|
||||||
fit_audio_to_duration,
|
fit_audio_to_duration,
|
||||||
ffmpeg_time_stretch,
|
ffmpeg_time_stretch,
|
||||||
mix_audio,
|
mix_audio,
|
||||||
|
|||||||
@@ -0,0 +1,22 @@
|
|||||||
|
"""Text utility functions for the domain layer."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
# Pre-compiled patterns for calculate_text_length
|
||||||
|
_METADATA_TAG_PATTERN = re.compile(r"<<METADATA_[^:]+:[^>]*>>")
|
||||||
|
_CHAPTER_MARKER_PATTERN = re.compile(r"<<CHAPTER_MARKER:[^>]*>>")
|
||||||
|
_VOICE_MARKER_PATTERN = re.compile(r"<<VOICE:[^>]*>>")
|
||||||
|
|
||||||
|
|
||||||
|
def calculate_text_length(text: str) -> int:
|
||||||
|
"""Calculate character count, ignoring internal markers and newlines.
|
||||||
|
|
||||||
|
Strips chapter markers, voice markers, and metadata tags before counting.
|
||||||
|
"""
|
||||||
|
text = _CHAPTER_MARKER_PATTERN.sub("", text)
|
||||||
|
text = _VOICE_MARKER_PATTERN.sub("", text)
|
||||||
|
text = _METADATA_TAG_PATTERN.sub("", text)
|
||||||
|
text = text.replace("\n", "").strip()
|
||||||
|
return len(text)
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Dict, List, Mapping, Optional
|
from typing import Any, List, Mapping, Optional
|
||||||
|
|
||||||
from .metadata_helpers import (
|
from .metadata_helpers import (
|
||||||
ensure_sentence,
|
ensure_sentence,
|
||||||
|
|||||||
@@ -0,0 +1,112 @@
|
|||||||
|
"""Voice catalog — shared voice metadata for all UIs.
|
||||||
|
|
||||||
|
Builds a unified catalog of available voices with metadata (language,
|
||||||
|
gender, display name). Used by both WebUI and PyQt for voice selection UIs.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Dict, Iterable, List, Mapping, Optional
|
||||||
|
|
||||||
|
from abogen.constants import LANGUAGE_DESCRIPTIONS
|
||||||
|
from abogen.tts_plugin.utils import get_voices
|
||||||
|
|
||||||
|
|
||||||
|
def build_voice_catalog() -> List[Dict[str, str]]:
|
||||||
|
"""Build voice catalog with metadata for all available voices.
|
||||||
|
|
||||||
|
Returns a list of dicts, each containing:
|
||||||
|
- id: voice ID (e.g. "af_heart")
|
||||||
|
- language: language code (e.g. "a", "e")
|
||||||
|
- language_label: human-readable language name
|
||||||
|
- gender: "Female", "Male", or "Unknown"
|
||||||
|
- gender_code: "f", "m", or ""
|
||||||
|
- display_name: human-readable voice name
|
||||||
|
"""
|
||||||
|
from plugins.kokoro.engine import language_for_voice_id
|
||||||
|
|
||||||
|
catalog: List[Dict[str, str]] = []
|
||||||
|
gender_map = {"f": "Female", "m": "Male"}
|
||||||
|
for voice_id in get_voices("kokoro"):
|
||||||
|
prefix, _, rest = voice_id.partition("_")
|
||||||
|
gender_code = prefix[1] if len(prefix) > 1 else ""
|
||||||
|
lang = language_for_voice_id(voice_id)
|
||||||
|
catalog.append(
|
||||||
|
{
|
||||||
|
"id": voice_id,
|
||||||
|
"language": lang.value,
|
||||||
|
"language_label": LANGUAGE_DESCRIPTIONS.get(lang, lang.value.upper()),
|
||||||
|
"gender": gender_map.get(gender_code, "Unknown"),
|
||||||
|
"gender_code": gender_code,
|
||||||
|
"display_name": rest.replace("_", " ").title() if rest else voice_id,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return catalog
|
||||||
|
|
||||||
|
|
||||||
|
def filter_voice_catalog(
|
||||||
|
catalog: Iterable[Mapping[str, Any]],
|
||||||
|
*,
|
||||||
|
gender: str,
|
||||||
|
allowed_languages: Optional[Iterable[str]] = None,
|
||||||
|
) -> List[str]:
|
||||||
|
"""Filter voice catalog by gender and language.
|
||||||
|
|
||||||
|
Returns voice IDs that match the criteria. Falls back to broader
|
||||||
|
matches if no exact matches are found.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
catalog: Voice catalog entries (from build_voice_catalog).
|
||||||
|
gender: Gender filter ("male", "female", or "unknown").
|
||||||
|
allowed_languages: Optional list of allowed language codes.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of matching voice IDs.
|
||||||
|
"""
|
||||||
|
allowed_set = {code.lower() for code in (allowed_languages or []) if isinstance(code, str) and code}
|
||||||
|
gender_normalized = (gender or "unknown").lower()
|
||||||
|
gender_code = ""
|
||||||
|
if gender_normalized == "male":
|
||||||
|
gender_code = "m"
|
||||||
|
elif gender_normalized == "female":
|
||||||
|
gender_code = "f"
|
||||||
|
|
||||||
|
matches: List[str] = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
|
||||||
|
def _consider(entry: Mapping[str, Any]) -> None:
|
||||||
|
voice_id = entry.get("id")
|
||||||
|
if not isinstance(voice_id, str) or not voice_id:
|
||||||
|
return
|
||||||
|
if voice_id in seen:
|
||||||
|
return
|
||||||
|
seen.add(voice_id)
|
||||||
|
matches.append(voice_id)
|
||||||
|
|
||||||
|
primary: List[Mapping[str, Any]] = []
|
||||||
|
fallback: List[Mapping[str, Any]] = []
|
||||||
|
for entry in catalog:
|
||||||
|
if not isinstance(entry, Mapping):
|
||||||
|
continue
|
||||||
|
voice_lang = str(entry.get("language", "")).lower()
|
||||||
|
voice_gender_code = str(entry.get("gender_code", "")).lower()
|
||||||
|
if allowed_set and voice_lang not in allowed_set:
|
||||||
|
continue
|
||||||
|
if gender_code and voice_gender_code != gender_code:
|
||||||
|
fallback.append(entry)
|
||||||
|
continue
|
||||||
|
primary.append(entry)
|
||||||
|
|
||||||
|
for entry in primary:
|
||||||
|
_consider(entry)
|
||||||
|
|
||||||
|
if not matches:
|
||||||
|
for entry in fallback:
|
||||||
|
_consider(entry)
|
||||||
|
|
||||||
|
if not matches:
|
||||||
|
for entry in catalog:
|
||||||
|
if isinstance(entry, Mapping):
|
||||||
|
_consider(entry)
|
||||||
|
|
||||||
|
return matches
|
||||||
@@ -6,7 +6,7 @@ PyQt and WebUI interfaces.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from typing import Any, Dict, Optional, Tuple
|
from typing import Any, Dict, Optional
|
||||||
|
|
||||||
from abogen.voice_formulas import get_new_voice
|
from abogen.voice_formulas import get_new_voice
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,117 @@
|
|||||||
|
"""Voice marker parsing and text splitting.
|
||||||
|
|
||||||
|
Handles <<VOICE:name>> markers in text, splitting text into voice-specific
|
||||||
|
segments. This is domain logic about text segmentation by voice, not subtitle
|
||||||
|
processing.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from typing import List, Tuple
|
||||||
|
|
||||||
|
_VOICE_MARKER_PATTERN = re.compile(r"<<VOICE:[^>]*>>")
|
||||||
|
_VOICE_MARKER_SEARCH_PATTERN = re.compile(r"<<VOICE:(.*?)>>")
|
||||||
|
|
||||||
|
|
||||||
|
def validate_voice_name(voice_name: str) -> Tuple[bool, str | None]:
|
||||||
|
"""Validate voice name against available voices (case-insensitive).
|
||||||
|
|
||||||
|
Handles both single voices and formulas like 'af_heart*0.5 + am_echo*0.5'.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tuple of (is_valid, invalid_voice_name):
|
||||||
|
- is_valid: True if all voices in the name/formula are valid
|
||||||
|
- invalid_voice_name: The first invalid voice found, or None if all valid
|
||||||
|
"""
|
||||||
|
from abogen.tts_plugin.utils import get_voices
|
||||||
|
|
||||||
|
voice_lookup_lower = {v.lower() for v in get_voices("kokoro")}
|
||||||
|
voice_name = voice_name.strip()
|
||||||
|
|
||||||
|
if "*" in voice_name:
|
||||||
|
voices = voice_name.split("+")
|
||||||
|
for term in voices:
|
||||||
|
if "*" in term:
|
||||||
|
base_voice = term.split("*")[0].strip()
|
||||||
|
if base_voice.lower() not in voice_lookup_lower:
|
||||||
|
return False, base_voice
|
||||||
|
return True, None
|
||||||
|
else:
|
||||||
|
if voice_name.lower() not in voice_lookup_lower:
|
||||||
|
return False, voice_name
|
||||||
|
return True, None
|
||||||
|
|
||||||
|
|
||||||
|
def split_text_by_voice_markers(
|
||||||
|
text: str, default_voice: str
|
||||||
|
) -> Tuple[List[Tuple[str, str]], str, int, int]:
|
||||||
|
"""Split text by voice markers, returning list of (voice, text) tuples.
|
||||||
|
|
||||||
|
Returns the last voice used so it can persist across chapters.
|
||||||
|
Voice names are normalized to lowercase to match canonical voice names.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Text potentially containing <<VOICE:name>> markers
|
||||||
|
default_voice: Voice to use if no markers found or before first marker
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Tuple of (segments_list, last_voice_used, valid_count, invalid_count):
|
||||||
|
- segments_list: List of (voice_name, segment_text) tuples
|
||||||
|
- last_voice_used: The voice that should continue into next chapter
|
||||||
|
- valid_count: Number of valid voice markers processed
|
||||||
|
- invalid_count: Number of invalid voice markers skipped
|
||||||
|
"""
|
||||||
|
from abogen.tts_plugin.utils import get_voices
|
||||||
|
|
||||||
|
voice_splits = list(_VOICE_MARKER_SEARCH_PATTERN.finditer(text))
|
||||||
|
|
||||||
|
if not voice_splits:
|
||||||
|
return [(default_voice, text)], default_voice, 0, 0
|
||||||
|
|
||||||
|
segments: List[Tuple[str, str]] = []
|
||||||
|
current_voice = default_voice
|
||||||
|
valid_markers = 0
|
||||||
|
invalid_markers = 0
|
||||||
|
|
||||||
|
first_start = voice_splits[0].start()
|
||||||
|
if first_start > 0:
|
||||||
|
intro_text = text[:first_start].strip()
|
||||||
|
if intro_text:
|
||||||
|
segments.append((current_voice, intro_text))
|
||||||
|
|
||||||
|
for idx, match in enumerate(voice_splits):
|
||||||
|
voice_name = match.group(1).strip()
|
||||||
|
start = match.end()
|
||||||
|
end = voice_splits[idx + 1].start() if idx + 1 < len(voice_splits) else len(text)
|
||||||
|
segment_text = text[start:end].strip()
|
||||||
|
|
||||||
|
is_valid, invalid_voice = validate_voice_name(voice_name)
|
||||||
|
if is_valid:
|
||||||
|
if "*" in voice_name:
|
||||||
|
normalized_parts = []
|
||||||
|
for part in voice_name.split("+"):
|
||||||
|
part = part.strip()
|
||||||
|
if "*" in part:
|
||||||
|
voice_part, weight = part.split("*", 1)
|
||||||
|
voice_part_lower = voice_part.strip().lower()
|
||||||
|
canonical_voice = next(
|
||||||
|
(v for v in get_voices("kokoro") if v.lower() == voice_part_lower),
|
||||||
|
voice_part.strip()
|
||||||
|
)
|
||||||
|
normalized_parts.append(f"{canonical_voice}*{weight.strip()}")
|
||||||
|
current_voice = " + ".join(normalized_parts)
|
||||||
|
else:
|
||||||
|
voice_name_lower = voice_name.lower()
|
||||||
|
current_voice = next(
|
||||||
|
(v for v in get_voices("kokoro") if v.lower() == voice_name_lower),
|
||||||
|
voice_name
|
||||||
|
)
|
||||||
|
valid_markers += 1
|
||||||
|
else:
|
||||||
|
invalid_markers += 1
|
||||||
|
|
||||||
|
if segment_text:
|
||||||
|
segments.append((current_voice, segment_text))
|
||||||
|
|
||||||
|
return segments, current_voice, valid_markers, invalid_markers
|
||||||
@@ -2,14 +2,17 @@
|
|||||||
|
|
||||||
Functions for resolving voice specifications, collecting required voice IDs,
|
Functions for resolving voice specifications, collecting required voice IDs,
|
||||||
and determining the voice to use for chapters and chunks.
|
and determining the voice to use for chapters and chunks.
|
||||||
|
|
||||||
|
All functions accept ConversionRequest (the app-layer contract) instead of
|
||||||
|
UI-specific objects. This keeps the domain layer UI-agnostic.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from typing import Any, Dict, Optional, Set
|
from typing import Any, Dict, Mapping, Optional, Set, Tuple
|
||||||
|
|
||||||
from abogen.tts_plugin.utils import get_voices, get_default_voice
|
from abogen.tts_plugin.utils import get_voices, get_default_voice
|
||||||
from abogen.voice_formulas import extract_voice_ids
|
from abogen.voice_formulas import extract_voice_ids, pairs_to_formula
|
||||||
from abogen.voice_cache import ensure_voice_assets
|
from abogen.voice_cache import ensure_voice_assets
|
||||||
|
|
||||||
|
|
||||||
@@ -29,12 +32,28 @@ def spec_to_voice_ids(spec: Any) -> Set[str]:
|
|||||||
return set()
|
return set()
|
||||||
|
|
||||||
|
|
||||||
def job_voice_fallback(job: Any) -> str:
|
def _get_chapter_overrides(request: Any) -> list:
|
||||||
base = str(getattr(job, "voice", "") or "").strip()
|
"""Extract chapter overrides from ConversionRequest."""
|
||||||
|
cc = getattr(request, "chapter_chunk", None)
|
||||||
|
if cc is not None:
|
||||||
|
return getattr(cc, "chapter_overrides", []) or []
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def _get_chunks(request: Any) -> list:
|
||||||
|
"""Extract chunks from ConversionRequest."""
|
||||||
|
cc = getattr(request, "chapter_chunk", None)
|
||||||
|
if cc is not None:
|
||||||
|
return getattr(cc, "chunks", []) or []
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def job_voice_fallback(request: Any) -> str:
|
||||||
|
base = str(getattr(request, "voice", "") or "").strip()
|
||||||
if base and base != "__custom_mix":
|
if base and base != "__custom_mix":
|
||||||
return base
|
return base
|
||||||
|
|
||||||
speakers = getattr(job, "speakers", None)
|
speakers = getattr(request, "speakers", None)
|
||||||
if isinstance(speakers, dict):
|
if isinstance(speakers, dict):
|
||||||
narrator = speakers.get("narrator")
|
narrator = speakers.get("narrator")
|
||||||
if isinstance(narrator, dict):
|
if isinstance(narrator, dict):
|
||||||
@@ -52,7 +71,7 @@ def job_voice_fallback(job: Any) -> str:
|
|||||||
if candidate and candidate != "__custom_mix":
|
if candidate and candidate != "__custom_mix":
|
||||||
return candidate
|
return candidate
|
||||||
|
|
||||||
for chapter in getattr(job, "chapters", []) or []:
|
for chapter in _get_chapter_overrides(request):
|
||||||
if not isinstance(chapter, dict):
|
if not isinstance(chapter, dict):
|
||||||
continue
|
continue
|
||||||
for key in ("resolved_voice", "voice_formula", "voice"):
|
for key in ("resolved_voice", "voice_formula", "voice"):
|
||||||
@@ -63,24 +82,24 @@ def job_voice_fallback(job: Any) -> str:
|
|||||||
return ""
|
return ""
|
||||||
|
|
||||||
|
|
||||||
def collect_required_voice_ids(job: Any) -> Set[str]:
|
def collect_required_voice_ids(request: Any) -> Set[str]:
|
||||||
voices: Set[str] = set()
|
voices: Set[str] = set()
|
||||||
voices.update(spec_to_voice_ids(job.voice))
|
voices.update(spec_to_voice_ids(request.voice))
|
||||||
voices.update(spec_to_voice_ids(job_voice_fallback(job)))
|
voices.update(spec_to_voice_ids(job_voice_fallback(request)))
|
||||||
|
|
||||||
for chapter in getattr(job, "chapters", []) or []:
|
for chapter in _get_chapter_overrides(request):
|
||||||
if not isinstance(chapter, dict):
|
if not isinstance(chapter, dict):
|
||||||
continue
|
continue
|
||||||
for key in ("resolved_voice", "voice_formula", "voice"):
|
for key in ("resolved_voice", "voice_formula", "voice"):
|
||||||
voices.update(spec_to_voice_ids(chapter.get(key)))
|
voices.update(spec_to_voice_ids(chapter.get(key)))
|
||||||
|
|
||||||
for chunk in getattr(job, "chunks", []) or []:
|
for chunk in _get_chunks(request):
|
||||||
if not isinstance(chunk, dict):
|
if not isinstance(chunk, dict):
|
||||||
continue
|
continue
|
||||||
for key in ("resolved_voice", "voice_formula", "voice"):
|
for key in ("resolved_voice", "voice_formula", "voice"):
|
||||||
voices.update(spec_to_voice_ids(chunk.get(key)))
|
voices.update(spec_to_voice_ids(chunk.get(key)))
|
||||||
|
|
||||||
speakers = getattr(job, "speakers", {})
|
speakers = getattr(request, "speakers", {})
|
||||||
if isinstance(speakers, dict):
|
if isinstance(speakers, dict):
|
||||||
for payload in speakers.values() or []:
|
for payload in speakers.values() or []:
|
||||||
if not isinstance(payload, dict):
|
if not isinstance(payload, dict):
|
||||||
@@ -92,30 +111,38 @@ def collect_required_voice_ids(job: Any) -> Set[str]:
|
|||||||
return voices
|
return voices
|
||||||
|
|
||||||
|
|
||||||
def initialize_voice_cache(job: Any) -> None:
|
def initialize_voice_cache(request: Any, events: Any = None) -> None:
|
||||||
|
"""Initialize voice cache by downloading required voice assets.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
request: ConversionRequest with voice/chapter/chunk/speaker info.
|
||||||
|
events: ConversionEvents for logging (optional, for backward compat).
|
||||||
|
"""
|
||||||
|
log = (lambda msg, level="info": events.log(msg, level=level)) if events else (lambda msg, level="info": None)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
targets = collect_required_voice_ids(job)
|
targets = collect_required_voice_ids(request)
|
||||||
downloaded, errors = ensure_voice_assets(
|
downloaded, errors = ensure_voice_assets(
|
||||||
targets,
|
targets,
|
||||||
on_progress=lambda message: job.add_log(message, level="debug"),
|
on_progress=lambda message: log(message, level="debug"),
|
||||||
)
|
)
|
||||||
except RuntimeError as exc:
|
except RuntimeError as exc:
|
||||||
job.add_log(f"Voice cache unavailable: {exc}", level="warning")
|
log(f"Voice cache unavailable: {exc}", level="warning")
|
||||||
return
|
return
|
||||||
|
|
||||||
if downloaded:
|
if downloaded:
|
||||||
job.add_log(
|
log(
|
||||||
f"Cached {len(downloaded)} voice asset{'s' if len(downloaded) != 1 else ''} locally.",
|
f"Cached {len(downloaded)} voice asset{'s' if len(downloaded) != 1 else ''} locally.",
|
||||||
level="info",
|
level="info",
|
||||||
)
|
)
|
||||||
|
|
||||||
for voice_id, error in errors.items():
|
for voice_id, error in errors.items():
|
||||||
job.add_log(f"Failed to cache voice '{voice_id}': {error}", level="warning")
|
log(f"Failed to cache voice '{voice_id}': {error}", level="warning")
|
||||||
|
|
||||||
|
|
||||||
def chapter_voice_spec(job: Any, override: Optional[Dict[str, Any]]) -> str:
|
def chapter_voice_spec(request: Any, override: Optional[Dict[str, Any]]) -> str:
|
||||||
if not override:
|
if not override:
|
||||||
return job_voice_fallback(job)
|
return job_voice_fallback(request)
|
||||||
|
|
||||||
resolved = str(override.get("resolved_voice", "")).strip()
|
resolved = str(override.get("resolved_voice", "")).strip()
|
||||||
if resolved:
|
if resolved:
|
||||||
@@ -129,17 +156,17 @@ def chapter_voice_spec(job: Any, override: Optional[Dict[str, Any]]) -> str:
|
|||||||
if voice:
|
if voice:
|
||||||
return voice
|
return voice
|
||||||
|
|
||||||
return job_voice_fallback(job)
|
return job_voice_fallback(request)
|
||||||
|
|
||||||
|
|
||||||
def chunk_voice_spec(job: Any, chunk: Dict[str, Any], fallback: str) -> str:
|
def chunk_voice_spec(request: Any, chunk: Dict[str, Any], fallback: str) -> str:
|
||||||
for key in ("resolved_voice", "voice_formula", "voice"):
|
for key in ("resolved_voice", "voice_formula", "voice"):
|
||||||
value = chunk.get(key)
|
value = chunk.get(key)
|
||||||
if value:
|
if value:
|
||||||
return str(value)
|
return str(value)
|
||||||
|
|
||||||
speaker_id = chunk.get("speaker_id")
|
speaker_id = chunk.get("speaker_id")
|
||||||
speakers = getattr(job, "speakers", None)
|
speakers = getattr(request, "speakers", None)
|
||||||
if isinstance(speakers, dict) and speaker_id in speakers:
|
if isinstance(speakers, dict) and speaker_id in speakers:
|
||||||
speaker_entry = speakers.get(speaker_id) or {}
|
speaker_entry = speakers.get(speaker_id) or {}
|
||||||
if isinstance(speaker_entry, dict):
|
if isinstance(speaker_entry, dict):
|
||||||
@@ -163,7 +190,7 @@ def chunk_voice_spec(job: Any, chunk: Dict[str, Any], fallback: str) -> str:
|
|||||||
|
|
||||||
if fallback:
|
if fallback:
|
||||||
return fallback
|
return fallback
|
||||||
return job_voice_fallback(job)
|
return job_voice_fallback(request)
|
||||||
|
|
||||||
|
|
||||||
def resolve_fallback_voice_spec(
|
def resolve_fallback_voice_spec(
|
||||||
@@ -188,3 +215,141 @@ def resolve_fallback_voice_spec(
|
|||||||
if not spec:
|
if not spec:
|
||||||
spec = get_default_voice(provider)
|
spec = get_default_voice(provider)
|
||||||
return spec
|
return spec
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Voice choice resolution (shared by all UIs)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def formula_from_profile(entry: Dict[str, Any]) -> Optional[str]:
|
||||||
|
"""Convert a voice profile entry to a voice formula string.
|
||||||
|
|
||||||
|
Handles both Kokoro (voices list) and SuperTonic (single voice) profiles.
|
||||||
|
Returns None if the entry has no usable voice data.
|
||||||
|
"""
|
||||||
|
if not isinstance(entry, dict):
|
||||||
|
return None
|
||||||
|
voices = entry.get("voices") or []
|
||||||
|
if not voices:
|
||||||
|
return None
|
||||||
|
return pairs_to_formula(voices)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_profile_voice(
|
||||||
|
profile_name: Optional[str],
|
||||||
|
*,
|
||||||
|
profiles: Optional[Mapping[str, Any]] = None,
|
||||||
|
) -> Tuple[str, Optional[str]]:
|
||||||
|
"""Resolve a profile name to (formula, language).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
profile_name: Name of the profile to resolve.
|
||||||
|
profiles: Pre-loaded profiles dict. If None, loads from disk.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
(formula_string, language_code) or ("", None) if not found.
|
||||||
|
"""
|
||||||
|
if not profile_name:
|
||||||
|
return "", None
|
||||||
|
source = profiles if isinstance(profiles, Mapping) else None
|
||||||
|
if source is None:
|
||||||
|
from abogen.voice_profiles import load_profiles
|
||||||
|
source = load_profiles()
|
||||||
|
entry = source.get(profile_name) if isinstance(source, Mapping) else None
|
||||||
|
if not isinstance(entry, Mapping):
|
||||||
|
return "", None
|
||||||
|
formula = formula_from_profile(dict(entry)) or ""
|
||||||
|
language = entry.get("language") if isinstance(entry.get("language"), str) else None
|
||||||
|
if isinstance(language, str):
|
||||||
|
language = language.strip().lower() or None
|
||||||
|
return formula, language
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_voice_setting(
|
||||||
|
value: Any,
|
||||||
|
*,
|
||||||
|
profiles: Optional[Mapping[str, Any]] = None,
|
||||||
|
) -> Tuple[str, Optional[str], Optional[str]]:
|
||||||
|
"""Resolve a raw voice setting value into (spec, profile_name, language).
|
||||||
|
|
||||||
|
Parses 'profile:name' or 'speaker:name' prefixes and resolves
|
||||||
|
the profile to a formula string.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
value: Raw voice value from user input (e.g. "af_heart", "profile:MyMix").
|
||||||
|
profiles: Pre-loaded profiles dict. If None, loads from disk.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
(resolved_spec, profile_name, language) — profile_name and language
|
||||||
|
are None when the input is a plain voice spec.
|
||||||
|
"""
|
||||||
|
from abogen.domain.settings_core import split_profile_spec
|
||||||
|
|
||||||
|
base_spec, profile_name = split_profile_spec(value)
|
||||||
|
if profile_name:
|
||||||
|
formula, language = resolve_profile_voice(profile_name, profiles=profiles)
|
||||||
|
return formula or "", profile_name, language
|
||||||
|
return base_spec, None, None
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_voice_choice(
|
||||||
|
language: str,
|
||||||
|
base_voice: str,
|
||||||
|
profile_name: str,
|
||||||
|
custom_formula: str,
|
||||||
|
profiles: Dict[str, Any],
|
||||||
|
) -> Tuple[str, str, Optional[str]]:
|
||||||
|
"""Resolve a user's voice selection into (resolved_voice, resolved_language, selected_profile).
|
||||||
|
|
||||||
|
Handles three input modes:
|
||||||
|
1. Profile selection → resolves to formula (Kokoro) or speaker reference (SuperTonic)
|
||||||
|
2. Custom formula → used directly
|
||||||
|
3. Plain voice spec → passed through
|
||||||
|
|
||||||
|
Args:
|
||||||
|
language: Current language code (e.g. "a", "e").
|
||||||
|
base_voice: Base voice spec (voice ID or formula).
|
||||||
|
profile_name: Selected profile name (empty string if none).
|
||||||
|
custom_formula: Custom formula string (empty string if none).
|
||||||
|
profiles: Dict of all available profiles.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
(resolved_voice, resolved_language, selected_profile)
|
||||||
|
"""
|
||||||
|
from abogen.voice_profiles import normalize_profile_entry
|
||||||
|
|
||||||
|
resolved_voice = base_voice
|
||||||
|
resolved_language = language
|
||||||
|
selected_profile = None
|
||||||
|
|
||||||
|
if profile_name:
|
||||||
|
entry_raw = profiles.get(profile_name)
|
||||||
|
entry = normalize_profile_entry(entry_raw)
|
||||||
|
provider = str((entry or {}).get("provider") or "").strip().lower()
|
||||||
|
|
||||||
|
# Provider-aware behavior:
|
||||||
|
# - Kokoro profiles typically represent mixes (formula strings).
|
||||||
|
# - SuperTonic profiles represent a discrete voice id + settings.
|
||||||
|
# In that case, we return a speaker reference so downstream can
|
||||||
|
# resolve provider per-speaker and allow mixed-provider casting.
|
||||||
|
if provider == "supertonic":
|
||||||
|
resolved_voice = f"speaker:{profile_name}"
|
||||||
|
selected_profile = profile_name
|
||||||
|
profile_language = (entry or {}).get("language")
|
||||||
|
if profile_language:
|
||||||
|
resolved_language = str(profile_language)
|
||||||
|
else:
|
||||||
|
formula = formula_from_profile(entry or {}) if entry else None
|
||||||
|
if formula:
|
||||||
|
resolved_voice = formula
|
||||||
|
selected_profile = profile_name
|
||||||
|
profile_language = (entry or {}).get("language")
|
||||||
|
if profile_language:
|
||||||
|
resolved_language = profile_language
|
||||||
|
|
||||||
|
if custom_formula:
|
||||||
|
resolved_voice = custom_formula
|
||||||
|
selected_profile = None
|
||||||
|
|
||||||
|
return resolved_voice, resolved_language, selected_profile
|
||||||
|
|||||||
@@ -1,8 +1,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from typing import Any, Dict, Mapping, Optional, Tuple, Set
|
from typing import Any, Dict, Mapping, Optional, Tuple
|
||||||
|
|
||||||
from abogen.voice_formulas import extract_voice_ids, get_new_voice
|
|
||||||
from abogen.tts_plugin.utils import get_voices
|
from abogen.tts_plugin.utils import get_voices
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -9,15 +9,26 @@ from collections import Counter
|
|||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple
|
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple
|
||||||
|
|
||||||
try: # pragma: no cover - fallback when spaCy not available during tests
|
|
||||||
import spacy # type: ignore[import-not-found]
|
|
||||||
except Exception: # pragma: no cover - spaCy optional during runtime bootstrap
|
|
||||||
spacy = None
|
|
||||||
|
|
||||||
_Language = Any # type: ignore[misc,assignment]
|
_Language = Any # type: ignore[misc,assignment]
|
||||||
Doc = Any # type: ignore[misc,assignment]
|
Doc = Any # type: ignore[misc,assignment]
|
||||||
Span = Any # type: ignore[misc,assignment]
|
Span = Any # type: ignore[misc,assignment]
|
||||||
|
|
||||||
|
_SPACY: Any = None
|
||||||
|
_SPACY_LOADED = False
|
||||||
|
|
||||||
|
|
||||||
|
def _get_spacy() -> Any:
|
||||||
|
"""Import spaCy lazily (it pulls in torch/thinc, ~2s at startup)."""
|
||||||
|
global _SPACY, _SPACY_LOADED
|
||||||
|
if not _SPACY_LOADED:
|
||||||
|
_SPACY_LOADED = True
|
||||||
|
try: # pragma: no cover - fallback when spaCy not available during tests
|
||||||
|
import spacy # type: ignore[import-not-found]
|
||||||
|
except Exception: # pragma: no cover - spaCy optional during runtime bootstrap
|
||||||
|
spacy = None
|
||||||
|
_SPACY = spacy
|
||||||
|
return _SPACY
|
||||||
|
|
||||||
|
|
||||||
_TITLE_PREFIXES = (
|
_TITLE_PREFIXES = (
|
||||||
"mr",
|
"mr",
|
||||||
@@ -167,6 +178,7 @@ def _resolve_model_name(language: str) -> str:
|
|||||||
|
|
||||||
|
|
||||||
def _load_model(language: str) -> Any:
|
def _load_model(language: str) -> Any:
|
||||||
|
spacy = _get_spacy()
|
||||||
if spacy is None:
|
if spacy is None:
|
||||||
raise EntityModelError(
|
raise EntityModelError(
|
||||||
"spaCy is not available. Install spaCy to enable entity extraction."
|
"spaCy is not available. Install spaCy to enable entity extraction."
|
||||||
|
|||||||
+15
-13
@@ -12,6 +12,7 @@ from typing import Any, Dict, Iterable, List, Optional, Pattern, Sequence, Tuple
|
|||||||
import zipfile
|
import zipfile
|
||||||
|
|
||||||
from abogen.text_extractor import ExtractedChapter, ExtractionResult
|
from abogen.text_extractor import ExtractedChapter, ExtractionResult
|
||||||
|
from abogen.domain.metadata_helpers import normalize_metadata_map
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
@@ -22,7 +23,7 @@ class ChunkOverlay:
|
|||||||
start: Optional[float]
|
start: Optional[float]
|
||||||
end: Optional[float]
|
end: Optional[float]
|
||||||
speaker_id: str
|
speaker_id: str
|
||||||
voice: Optional[str]
|
voice: Optional[Dict[str, str]]
|
||||||
level: Optional[str] = None
|
level: Optional[str] = None
|
||||||
group_id: Optional[str] = None
|
group_id: Optional[str] = None
|
||||||
|
|
||||||
@@ -59,7 +60,7 @@ class EPUB3PackageBuilder:
|
|||||||
self.output_path = output_path
|
self.output_path = output_path
|
||||||
self.book_id = book_id or str(uuid.uuid4())
|
self.book_id = book_id or str(uuid.uuid4())
|
||||||
self.extraction = extraction
|
self.extraction = extraction
|
||||||
self.metadata_tags = _normalize_metadata(metadata_tags)
|
self.metadata_tags = normalize_metadata_map(metadata_tags)
|
||||||
self.chapter_markers = list(chapter_markers or [])
|
self.chapter_markers = list(chapter_markers or [])
|
||||||
self.chunk_markers = list(chunk_markers or [])
|
self.chunk_markers = list(chunk_markers or [])
|
||||||
self.chunks = list(chunks or [])
|
self.chunks = list(chunks or [])
|
||||||
@@ -273,7 +274,7 @@ class EPUB3PackageBuilder:
|
|||||||
start=_safe_float(marker.get("start")),
|
start=_safe_float(marker.get("start")),
|
||||||
end=_safe_float(marker.get("end")),
|
end=_safe_float(marker.get("end")),
|
||||||
speaker_id=speaker_id,
|
speaker_id=speaker_id,
|
||||||
voice=str(voice) if voice else None,
|
voice=voice if isinstance(voice, dict) else None,
|
||||||
level=str(level) if level else None,
|
level=str(level) if level else None,
|
||||||
group_id=normalized_group_id,
|
group_id=normalized_group_id,
|
||||||
)
|
)
|
||||||
@@ -516,9 +517,14 @@ def build_epub3_package(
|
|||||||
chunks: Iterable[Dict[str, Any]],
|
chunks: Iterable[Dict[str, Any]],
|
||||||
audio_path: Path,
|
audio_path: Path,
|
||||||
speaker_mode: str = "single",
|
speaker_mode: str = "single",
|
||||||
|
cover: "CoverConfig | None" = None,
|
||||||
cover_image_path: Optional[Path] = None,
|
cover_image_path: Optional[Path] = None,
|
||||||
cover_image_mime: Optional[str] = None,
|
cover_image_mime: Optional[str] = None,
|
||||||
) -> Path:
|
) -> Path:
|
||||||
|
from abogen.domain.config_types import CoverConfig
|
||||||
|
if isinstance(cover, CoverConfig):
|
||||||
|
cover_image_path = cover.path
|
||||||
|
cover_image_mime = cover.mime
|
||||||
builder = EPUB3PackageBuilder(
|
builder = EPUB3PackageBuilder(
|
||||||
output_path=output_path,
|
output_path=output_path,
|
||||||
book_id=book_id,
|
book_id=book_id,
|
||||||
@@ -545,15 +551,6 @@ class ChunkLookup:
|
|||||||
by_chapter: Dict[int, List[Dict[str, Any]]]
|
by_chapter: Dict[int, List[Dict[str, Any]]]
|
||||||
|
|
||||||
|
|
||||||
def _normalize_metadata(metadata: Optional[Dict[str, Any]]) -> Dict[str, str]:
|
|
||||||
normalized: Dict[str, str] = {}
|
|
||||||
for key, value in (metadata or {}).items():
|
|
||||||
if value is None:
|
|
||||||
continue
|
|
||||||
normalized[str(key).lower()] = str(value)
|
|
||||||
return normalized
|
|
||||||
|
|
||||||
|
|
||||||
def _combine_metadata(*sources: Dict[str, Any]) -> Dict[str, str]:
|
def _combine_metadata(*sources: Dict[str, Any]) -> Dict[str, str]:
|
||||||
combined: Dict[str, str] = {}
|
combined: Dict[str, str] = {}
|
||||||
for source in sources:
|
for source in sources:
|
||||||
@@ -696,7 +693,12 @@ def _group_chunks_for_render(chunks: Sequence[ChunkOverlay]) -> List[Tuple[Optio
|
|||||||
def _render_chunk_inline(chunk: ChunkOverlay) -> str:
|
def _render_chunk_inline(chunk: ChunkOverlay) -> str:
|
||||||
escaped_id = html.escape(chunk.id)
|
escaped_id = html.escape(chunk.id)
|
||||||
speaker_attr = f" data-speaker=\"{html.escape(chunk.speaker_id)}\"" if chunk.speaker_id else ""
|
speaker_attr = f" data-speaker=\"{html.escape(chunk.speaker_id)}\"" if chunk.speaker_id else ""
|
||||||
voice_attr = f" data-voice=\"{html.escape(chunk.voice)}\"" if chunk.voice else ""
|
voice_str = None
|
||||||
|
if chunk.voice and isinstance(chunk.voice, dict):
|
||||||
|
name = chunk.voice.get("voice", "")
|
||||||
|
provider = chunk.voice.get("provider", "")
|
||||||
|
voice_str = f"{name}@{provider}" if name and provider else name or None
|
||||||
|
voice_attr = f" data-voice=\"{html.escape(voice_str)}\"" if voice_str else ""
|
||||||
level_attr = f" data-level=\"{html.escape(chunk.level)}\"" if chunk.level else ""
|
level_attr = f" data-level=\"{html.escape(chunk.level)}\"" if chunk.level else ""
|
||||||
raw_text = chunk.text or ""
|
raw_text = chunk.text or ""
|
||||||
escaped_text = html.escape(raw_text)
|
escaped_text = html.escape(raw_text)
|
||||||
|
|||||||
@@ -5,10 +5,21 @@ import re
|
|||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from typing import Any, Dict, Iterable, List, Mapping, Optional, Sequence, Tuple
|
from typing import Any, Dict, Iterable, List, Mapping, Optional, Sequence, Tuple
|
||||||
|
|
||||||
try: # pragma: no cover - optional dependency
|
_SPACY: Any = None
|
||||||
import spacy # type: ignore
|
_SPACY_LOADED = False
|
||||||
except Exception: # pragma: no cover - spaCy may be unavailable in minimal environments
|
|
||||||
spacy = None
|
|
||||||
|
def _get_spacy() -> Any:
|
||||||
|
"""Import spaCy lazily (it pulls in torch/thinc, ~2s at startup)."""
|
||||||
|
global _SPACY, _SPACY_LOADED
|
||||||
|
if not _SPACY_LOADED:
|
||||||
|
_SPACY_LOADED = True
|
||||||
|
try: # pragma: no cover - optional dependency
|
||||||
|
import spacy # type: ignore
|
||||||
|
except Exception: # pragma: no cover - spaCy may be unavailable in minimal environments
|
||||||
|
spacy = None
|
||||||
|
_SPACY = spacy
|
||||||
|
return _SPACY
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -184,6 +195,7 @@ def _build_replacement_sentence(
|
|||||||
|
|
||||||
|
|
||||||
def _load_spacy(language: str) -> Any:
|
def _load_spacy(language: str) -> Any:
|
||||||
|
spacy = _get_spacy()
|
||||||
if spacy is None:
|
if spacy is None:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@@ -221,7 +233,7 @@ def extract_heteronym_overrides(
|
|||||||
if not lang.startswith("en"):
|
if not lang.startswith("en"):
|
||||||
return []
|
return []
|
||||||
|
|
||||||
if spacy is None:
|
if _get_spacy() is None:
|
||||||
return []
|
return []
|
||||||
|
|
||||||
nlp = _load_spacy(lang)
|
nlp = _load_spacy(lang)
|
||||||
|
|||||||
@@ -10,22 +10,14 @@ from typing import Any, Dict, List, Optional, Mapping, Sequence
|
|||||||
import static_ffmpeg
|
import static_ffmpeg
|
||||||
|
|
||||||
from abogen.domain.metadata_helpers import (
|
from abogen.domain.metadata_helpers import (
|
||||||
normalize_metadata_casefold,
|
|
||||||
split_people_field,
|
split_people_field,
|
||||||
split_simple_list,
|
split_simple_list,
|
||||||
first_nonempty,
|
first_nonempty,
|
||||||
extract_year,
|
extract_year,
|
||||||
normalize_series_sequence,
|
normalize_series_sequence,
|
||||||
build_audiobookshelf_metadata as _build_abs_metadata,
|
|
||||||
load_audiobookshelf_chapters as _load_abs_chapters,
|
|
||||||
_SERIES_SEQUENCE_TAG_KEYS,
|
_SERIES_SEQUENCE_TAG_KEYS,
|
||||||
)
|
)
|
||||||
from abogen.epub3.exporter import build_epub3_package
|
from abogen.epub3.exporter import build_epub3_package
|
||||||
from abogen.integrations.audiobookshelf import (
|
|
||||||
AudiobookshelfClient,
|
|
||||||
AudiobookshelfConfig,
|
|
||||||
AudiobookshelfUploadError,
|
|
||||||
)
|
|
||||||
from abogen.utils import create_process
|
from abogen.utils import create_process
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -84,9 +76,14 @@ class ExportService:
|
|||||||
title = chapter.get("title")
|
title = chapter.get("title")
|
||||||
if title:
|
if title:
|
||||||
lines.append(f"title={self._escape_ffmetadata_value(title)}")
|
lines.append(f"title={self._escape_ffmetadata_value(title)}")
|
||||||
voice = chapter.get("voice")
|
voices = chapter.get("voices")
|
||||||
if voice:
|
if voices and isinstance(voices, list):
|
||||||
lines.append(f"voice={self._escape_ffmetadata_value(voice)}")
|
voice_str = ", ".join(
|
||||||
|
f"{v.get('voice', '')}@{v.get('provider', '')}"
|
||||||
|
for v in voices if v.get("voice")
|
||||||
|
)
|
||||||
|
if voice_str:
|
||||||
|
lines.append(f"voice={self._escape_ffmetadata_value(voice_str)}")
|
||||||
|
|
||||||
return "\n".join(lines) + "\n"
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
@@ -127,11 +124,16 @@ class ExportService:
|
|||||||
audio_path: Path,
|
audio_path: Path,
|
||||||
metadata: Dict[str, Any],
|
metadata: Dict[str, Any],
|
||||||
chapters: List[Dict[str, Any]],
|
chapters: List[Dict[str, Any]],
|
||||||
|
cover: "CoverConfig | None" = None,
|
||||||
cover_path: Optional[Path] = None,
|
cover_path: Optional[Path] = None,
|
||||||
cover_mime: Optional[str] = None,
|
cover_mime: Optional[str] = None,
|
||||||
log_callback: Optional[callable] = None,
|
log_callback: Optional[callable] = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Embed metadata and chapters into M4B file using FFmpeg + Mutagen."""
|
"""Embed metadata and chapters into M4B file using FFmpeg + Mutagen."""
|
||||||
|
from abogen.domain.config_types import CoverConfig
|
||||||
|
if isinstance(cover, CoverConfig):
|
||||||
|
cover_path = cover.path
|
||||||
|
cover_mime = cover.mime
|
||||||
ffmetadata_path = self.write_ffmetadata_file(audio_path, metadata, chapters)
|
ffmetadata_path = self.write_ffmetadata_file(audio_path, metadata, chapters)
|
||||||
|
|
||||||
metadata_args = self._metadata_to_ffmpeg_args(metadata)
|
metadata_args = self._metadata_to_ffmpeg_args(metadata)
|
||||||
@@ -310,132 +312,7 @@ class ExportService:
|
|||||||
cover_image_mime=cover_mime,
|
cover_image_mime=cover_mime,
|
||||||
)
|
)
|
||||||
|
|
||||||
# ----------------------------------------------------------------------
|
|
||||||
# Audiobookshelf Integration
|
|
||||||
# ----------------------------------------------------------------------
|
|
||||||
|
|
||||||
def build_audiobookshelf_metadata(self, job: Any) -> Dict[str, Any]:
|
|
||||||
"""Build Audiobookshelf metadata from job."""
|
|
||||||
filename = Path(getattr(job, "original_filename", "") or "").stem or "Audiobook"
|
|
||||||
return _build_abs_metadata(
|
|
||||||
getattr(job, "metadata_tags", {}),
|
|
||||||
language=getattr(job, "language", "") or "",
|
|
||||||
filename=filename,
|
|
||||||
)
|
|
||||||
|
|
||||||
def load_audiobookshelf_chapters(self, job: Any) -> Optional[List[Dict[str, Any]]]:
|
|
||||||
"""Load chapters from job artifacts for Audiobookshelf."""
|
|
||||||
metadata_ref = job.result.artifacts.get("metadata") if getattr(job, "result", None) else None
|
|
||||||
if not metadata_ref:
|
|
||||||
return None
|
|
||||||
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
|
|
||||||
return _load_abs_chapters(metadata_path)
|
|
||||||
|
|
||||||
def upload_audiobookshelf(
|
|
||||||
self,
|
|
||||||
job: Any,
|
|
||||||
audio_path: Path,
|
|
||||||
subtitle_paths: List[Path],
|
|
||||||
chapters: List[Dict[str, Any]],
|
|
||||||
metadata: Dict[str, Any],
|
|
||||||
cover_path: Optional[Path] = None,
|
|
||||||
config: Optional[AudiobookshelfConfig] = None,
|
|
||||||
log_callback: Optional[callable] = None,
|
|
||||||
) -> None:
|
|
||||||
"""Upload to Audiobookshelf."""
|
|
||||||
if config is None:
|
|
||||||
cfg = getattr(job, "_abs_config", None)
|
|
||||||
if cfg is None:
|
|
||||||
from abogen.utils import load_config
|
|
||||||
global_cfg = load_config() or {}
|
|
||||||
abs_cfg = global_cfg.get("audiobookshelf")
|
|
||||||
if isinstance(abs_cfg, Mapping):
|
|
||||||
config = AudiobookshelfConfig(
|
|
||||||
base_url=str(abs_cfg.get("base_url") or "").strip(),
|
|
||||||
api_token=str(abs_cfg.get("api_token") or "").strip(),
|
|
||||||
library_id=str(abs_cfg.get("library_id") or "").strip(),
|
|
||||||
collection_id=(str(abs_cfg.get("collection_id") or "").strip() or None),
|
|
||||||
folder_id=str(abs_cfg.get("folder_id") or "").strip(),
|
|
||||||
verify_ssl=self._coerce_bool(abs_cfg.get("verify_ssl"), True),
|
|
||||||
send_cover=self._coerce_bool(abs_cfg.get("send_cover"), True),
|
|
||||||
send_chapters=self._coerce_bool(abs_cfg.get("send_chapters"), True),
|
|
||||||
send_subtitles=self._coerce_bool(abs_cfg.get("send_subtitles"), False),
|
|
||||||
timeout=float(abs_cfg.get("timeout", 3600.0)),
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
if log_callback:
|
|
||||||
log_callback("Audiobookshelf upload skipped: not configured", "warning")
|
|
||||||
return
|
|
||||||
|
|
||||||
if not config.base_url or not config.api_token or not config.library_id:
|
|
||||||
if log_callback:
|
|
||||||
log_callback("Audiobookshelf upload skipped: configure base URL, API token, and library ID first", "warning")
|
|
||||||
return
|
|
||||||
if not config.folder_id:
|
|
||||||
if log_callback:
|
|
||||||
log_callback("Audiobookshelf upload skipped: enter folder name or ID in settings", "warning")
|
|
||||||
return
|
|
||||||
|
|
||||||
if not audio_path.exists():
|
|
||||||
if log_callback:
|
|
||||||
log_callback("Audiobookshelf upload skipped: audio output not found", "warning")
|
|
||||||
return
|
|
||||||
|
|
||||||
existing_subtitles = [p for p in subtitle_paths if p.exists()] if config.send_subtitles else None
|
|
||||||
chapters_to_send = chapters if config.send_chapters else None
|
|
||||||
|
|
||||||
client = AudiobookshelfClient(config)
|
|
||||||
|
|
||||||
display_title = metadata.get("title") or audio_path.stem
|
|
||||||
try:
|
|
||||||
existing_items = client.find_existing_items(display_title, folder_id=config.folder_id)
|
|
||||||
except AudiobookshelfUploadError as exc:
|
|
||||||
if log_callback:
|
|
||||||
log_callback(f"Audiobookshelf lookup failed: {exc}", "error")
|
|
||||||
return
|
|
||||||
|
|
||||||
if existing_items:
|
|
||||||
if log_callback:
|
|
||||||
log_callback(f"Removing existing Audiobookshelf item(s) for '{display_title}' before upload.", "info")
|
|
||||||
try:
|
|
||||||
client.delete_items(existing_items)
|
|
||||||
except Exception as exc:
|
|
||||||
if log_callback:
|
|
||||||
log_callback(f"Failed to remove existing item(s): {exc}", "warning")
|
|
||||||
|
|
||||||
cover_to_send = cover_path
|
|
||||||
if config.send_cover and cover_to_send:
|
|
||||||
if isinstance(cover_to_send, str):
|
|
||||||
cover_to_send = Path(cover_to_send)
|
|
||||||
if not cover_to_send.exists():
|
|
||||||
cover_to_send = None
|
|
||||||
|
|
||||||
client.upload_audiobook(
|
|
||||||
audio_path,
|
|
||||||
metadata=metadata,
|
|
||||||
cover_path=cover_to_send,
|
|
||||||
chapters=chapters_to_send,
|
|
||||||
subtitles=existing_subtitles,
|
|
||||||
)
|
|
||||||
|
|
||||||
if log_callback:
|
|
||||||
log_callback("Audiobookshelf upload queued.", "info")
|
|
||||||
|
|
||||||
# ----------------------------------------------------------------------
|
|
||||||
# Helpers
|
|
||||||
# ----------------------------------------------------------------------
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _coerce_bool(value: Any, default: bool = True) -> bool:
|
|
||||||
if isinstance(value, bool):
|
|
||||||
return value
|
|
||||||
if isinstance(value, str):
|
|
||||||
lowered = value.strip().lower()
|
|
||||||
if lowered in {"true", "1", "yes", "on"}:
|
|
||||||
return True
|
|
||||||
if lowered in {"false", "0", "no", "off"}:
|
|
||||||
return False
|
|
||||||
return default
|
|
||||||
if value is None:
|
if value is None:
|
||||||
return default
|
return default
|
||||||
return bool(value)
|
return bool(value)
|
||||||
|
|||||||
@@ -220,8 +220,10 @@ class AssWriter(SubtitleWriter):
|
|||||||
|
|
||||||
style = "Default"
|
style = "Default"
|
||||||
if self.config.mode == SubtitleMode.SENTENCE_HIGHLIGHT:
|
if self.config.mode == SubtitleMode.SENTENCE_HIGHLIGHT:
|
||||||
# Add karaoke tags for highlighting
|
# Entries from process_subtitle_tokens already carry per-word
|
||||||
text = self._add_karaoke_tags(text)
|
# {\kf...} timing; only synthesize simplified tags when absent.
|
||||||
|
if "{\\k" not in text:
|
||||||
|
text = self._add_karaoke_tags(text)
|
||||||
style = "Highlight"
|
style = "Highlight"
|
||||||
|
|
||||||
alignment_tag = r"{\an5}" if self._is_centered else ""
|
alignment_tag = r"{\an5}" if self._is_centered else ""
|
||||||
@@ -248,6 +250,19 @@ class AssWriter(SubtitleWriter):
|
|||||||
return f"{hours}:{minutes:02d}:{secs:05.2f}"
|
return f"{hours}:{minutes:02d}:{secs:05.2f}"
|
||||||
|
|
||||||
|
|
||||||
|
def _coerce_mode(mode: str) -> SubtitleMode:
|
||||||
|
"""Parse a subtitle mode, tolerating word-count strings like "5 words".
|
||||||
|
|
||||||
|
Word-count modes are grouped upstream (subtitle_generation) and the writer
|
||||||
|
only branches on SubtitleMode.SENTENCE_HIGHLIGHT, so any non-highlight
|
||||||
|
fallback is behaviorally equivalent for the writers.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return SubtitleMode(mode)
|
||||||
|
except ValueError:
|
||||||
|
return SubtitleMode.SENTENCE
|
||||||
|
|
||||||
|
|
||||||
def create_subtitle_writer(
|
def create_subtitle_writer(
|
||||||
path: Path,
|
path: Path,
|
||||||
format: str,
|
format: str,
|
||||||
@@ -257,7 +272,7 @@ def create_subtitle_writer(
|
|||||||
) -> SubtitleWriter:
|
) -> SubtitleWriter:
|
||||||
"""Factory function to create subtitle writer."""
|
"""Factory function to create subtitle writer."""
|
||||||
fmt = SubtitleFormat(format.lower())
|
fmt = SubtitleFormat(format.lower())
|
||||||
mode = SubtitleMode(mode)
|
mode = _coerce_mode(mode)
|
||||||
align = SubtitleAlignment(alignment.lower())
|
align = SubtitleAlignment(alignment.lower())
|
||||||
|
|
||||||
config = SubtitleConfig(
|
config = SubtitleConfig(
|
||||||
@@ -278,24 +293,28 @@ def create_subtitle_writer(
|
|||||||
|
|
||||||
|
|
||||||
def resolve_subtitle_format(
|
def resolve_subtitle_format(
|
||||||
subtitle_format: str | None,
|
subtitle: "SubtitleConfig | str | None",
|
||||||
subtitle_mode: str,
|
subtitle_mode: str | None = None,
|
||||||
) -> tuple[str, str]:
|
) -> tuple[str, str]:
|
||||||
"""Resolve a subtitle_format setting string to (file_extension, alignment).
|
"""Resolve a subtitle config to (file_extension, alignment).
|
||||||
|
|
||||||
Handles the PyQt convention where format strings encode alignment
|
Accepts a SubtitleConfig object or individual format/mode strings
|
||||||
(e.g. ``"ass_centered_narrow"`` → extension ``"ass"``, alignment
|
for backward compatibility.
|
||||||
``"center_narrow"``).
|
|
||||||
|
|
||||||
Also enforces that ``"Sentence + Highlighting"`` mode requires ASS.
|
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Tuple of (file_extension, alignment) suitable for
|
Tuple of (file_extension, alignment) suitable for
|
||||||
:func:`create_subtitle_writer`.
|
:func:`create_subtitle_writer`.
|
||||||
"""
|
"""
|
||||||
fmt = (subtitle_format or "srt").lower()
|
from abogen.domain.config_types import SubtitleConfig
|
||||||
|
|
||||||
if subtitle_mode == "Sentence + Highlighting" and fmt == "srt":
|
if isinstance(subtitle, SubtitleConfig):
|
||||||
|
fmt = subtitle.format.value.lower()
|
||||||
|
mode_str = subtitle.mode.value
|
||||||
|
else:
|
||||||
|
fmt = (subtitle or "srt").lower()
|
||||||
|
mode_str = subtitle_mode or "Disabled"
|
||||||
|
|
||||||
|
if mode_str == "Sentence + Highlighting" and fmt == "srt":
|
||||||
fmt = "ass"
|
fmt = "ass"
|
||||||
|
|
||||||
if "ass" in fmt:
|
if "ass" in fmt:
|
||||||
@@ -317,26 +336,39 @@ def resolve_subtitle_format(
|
|||||||
|
|
||||||
def make_subtitle_writer(
|
def make_subtitle_writer(
|
||||||
audio_path: Path,
|
audio_path: Path,
|
||||||
subtitle_format: str | None,
|
subtitle: "SubtitleConfig | str | None",
|
||||||
subtitle_mode: str,
|
subtitle_mode: str | None = None,
|
||||||
max_words: int = 50,
|
max_words: int | None = None,
|
||||||
) -> SubtitleWriter | None:
|
) -> SubtitleWriter | None:
|
||||||
"""Convenience: resolve format and create a writer, or return None if disabled.
|
"""Convenience: resolve format and create a writer, or return None if disabled.
|
||||||
|
|
||||||
Returns ``None`` when ``subtitle_mode`` is ``"Disabled"`` or the
|
Accepts a SubtitleConfig object or individual format/mode strings
|
||||||
|
for backward compatibility.
|
||||||
|
|
||||||
|
Returns ``None`` when subtitle mode is ``"Disabled"`` or the
|
||||||
format is unsupported.
|
format is unsupported.
|
||||||
"""
|
"""
|
||||||
if subtitle_mode == "Disabled":
|
from abogen.domain.config_types import SubtitleConfig
|
||||||
return None
|
|
||||||
|
|
||||||
extension, alignment = resolve_subtitle_format(subtitle_format, subtitle_mode)
|
if isinstance(subtitle, SubtitleConfig):
|
||||||
|
mode_str = subtitle.mode.value
|
||||||
|
if mode_str == "Disabled":
|
||||||
|
return None
|
||||||
|
words = subtitle.max_words
|
||||||
|
else:
|
||||||
|
mode_str = subtitle_mode or subtitle or "Disabled"
|
||||||
|
if mode_str == "Disabled":
|
||||||
|
return None
|
||||||
|
words = max_words or 50
|
||||||
|
|
||||||
|
extension, alignment = resolve_subtitle_format(subtitle, subtitle_mode)
|
||||||
try:
|
try:
|
||||||
return create_subtitle_writer(
|
return create_subtitle_writer(
|
||||||
audio_path.with_suffix(f".{extension}"),
|
audio_path.with_suffix(f".{extension}"),
|
||||||
extension,
|
extension,
|
||||||
subtitle_mode,
|
mode_str,
|
||||||
alignment=alignment,
|
alignment=alignment,
|
||||||
max_words=max_words,
|
max_words=words,
|
||||||
)
|
)
|
||||||
except (ValueError, KeyError):
|
except (ValueError, KeyError):
|
||||||
return None
|
return None
|
||||||
|
|||||||
@@ -672,6 +672,15 @@ def tokenize_with_spans(text: str) -> List[Tuple[str, int, int]]:
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
_OPENING_PUNCTUATION_CHARS = "«‹“‘([{¡¿「『"
|
||||||
|
_CLOSING_PUNCTUATION_CHARS = "»›”’)]}」』"
|
||||||
|
_STANDARD_PUNCTUATION_CHARS = ",.;:!?%"
|
||||||
|
|
||||||
|
_OPENING_PUNCT_CLASS = re.escape(_OPENING_PUNCTUATION_CHARS)
|
||||||
|
_CLOSING_PUNCT_CLASS = re.escape(_CLOSING_PUNCTUATION_CHARS)
|
||||||
|
_STANDARD_PUNCT_CLASS = re.escape(_STANDARD_PUNCTUATION_CHARS)
|
||||||
|
|
||||||
|
|
||||||
def _cleanup_spacing(text: str) -> str:
|
def _cleanup_spacing(text: str) -> str:
|
||||||
if not text:
|
if not text:
|
||||||
return text
|
return text
|
||||||
@@ -679,22 +688,39 @@ def _cleanup_spacing(text: str) -> str:
|
|||||||
for marker in ("\ufeff", "\u200b", "\u200c", "\u200d", "\u2060"):
|
for marker in ("\ufeff", "\u200b", "\u200c", "\u200d", "\u2060"):
|
||||||
text = text.replace(marker, "")
|
text = text.replace(marker, "")
|
||||||
|
|
||||||
# Collapse spaces before closing punctuation.
|
# Collapse spaces before standard punctuation and unambiguous closing quotes/brackets.
|
||||||
text = re.sub(r"\s+([,.;:!?%])", r"\1", text)
|
text = re.sub(rf"\s+([{_STANDARD_PUNCT_CLASS}])", r"\1", text)
|
||||||
text = re.sub(r"\s+([’\"”»›)\]\}])", r"\1", text)
|
text = re.sub(rf"\s+([{_CLOSING_PUNCT_CLASS}])", r"\1", text)
|
||||||
|
|
||||||
# Remove spaces directly after opening punctuation/quotes.
|
# Remove spaces directly after unambiguous opening punctuation/quotes.
|
||||||
text = re.sub(r"([«‹“‘\"'(\[\{])\s+", r"\1", text)
|
text = re.sub(rf"([{_OPENING_PUNCT_CLASS}])\s+", r"\1", text)
|
||||||
|
|
||||||
|
# Handle ambiguous straight quotes (\", ')
|
||||||
|
# 1. Remove spaces directly after opening straight quotes:
|
||||||
|
# e.g. ' \" word' -> ' \"word', '^\" word' -> '\"word', '(\" word' -> '(\"word'
|
||||||
|
text = re.sub(rf"(^|[\s{_OPENING_PUNCT_CLASS}])([\"\'])\s+", r"\1\2", text)
|
||||||
|
# 2. Collapse spaces directly before closing straight quotes:
|
||||||
|
# e.g. 'word \" ' -> 'word\" ', 'word \".' -> 'word\".'
|
||||||
|
text = re.sub(rf"\s+([\"\'])([\s{_STANDARD_PUNCT_CLASS}{_CLOSING_PUNCT_CLASS}]|$)", r"\1\2", text)
|
||||||
|
|
||||||
# Ensure spaces exist after sentence punctuation when followed by a word/quote.
|
# Ensure spaces exist after sentence punctuation when followed by a word/quote.
|
||||||
text = re.sub(r"([,.;:!?%])(?![\s”'\"’»›)])", r"\1 ", text)
|
# Runs of punctuation ("...", "?!?", "!!") must stay together: no space
|
||||||
text = re.sub(r"([”\"’])(?![\s.,;:!?\"”’»›)])", r"\1 ", text)
|
# inside the run, only after it ("a...b" -> "a... b").
|
||||||
|
text = re.sub(rf"([{_STANDARD_PUNCT_CLASS}])(?![\s{_STANDARD_PUNCT_CLASS}{_CLOSING_PUNCT_CLASS}\"\'”’»›)])", r"\1 ", text)
|
||||||
|
# Ensure space after unambiguous closing quote when followed by a word (e.g. '”Next' -> '” Next')
|
||||||
|
text = re.sub(rf"([{_CLOSING_PUNCT_CLASS}])(?![\s{_STANDARD_PUNCT_CLASS}{_CLOSING_PUNCT_CLASS}\"\'”’»›)])", r"\1 ", text)
|
||||||
|
# Straight double quote closing (preceded by non-whitespace) followed directly by a word/number/opening
|
||||||
|
text = re.sub(rf"(\S\")([A-Za-z0-9{_OPENING_PUNCT_CLASS}])", r"\1 \2", text)
|
||||||
|
# Straight single quote closing (preceded by punctuation, not internal word apostrophe) followed by a word
|
||||||
|
text = re.sub(rf"([{_STANDARD_PUNCT_CLASS}{_CLOSING_PUNCT_CLASS}]\')([A-Za-z0-9{_OPENING_PUNCT_CLASS}])", r"\1 \2", text)
|
||||||
|
|
||||||
# Tighten hyphen/em dash spacing between word characters.
|
# Tighten hyphen/em dash spacing between word characters.
|
||||||
text = re.sub(r"(?<=\w)\s*([-–—])\s*(?=\w)", r"\1", text)
|
text = re.sub(r"(?<=\w)\s*([-–—])\s*(?=\w)", r"\1", text)
|
||||||
|
|
||||||
# Normalize multiple spaces.
|
# Normalize multiple spaces, preserving paragraph breaks (double
|
||||||
text = re.sub(r"\s{2,}", " ", text)
|
# newlines must survive so the TTS engine can split on them).
|
||||||
|
text = re.sub(r"[^\S\n]{2,}", " ", text)
|
||||||
|
text = re.sub(r"\n{3,}", "\n\n", text)
|
||||||
return text.strip()
|
return text.strip()
|
||||||
|
|
||||||
|
|
||||||
@@ -1622,8 +1648,18 @@ def normalize_apostrophes(
|
|||||||
results.append((tok, category, norm))
|
results.append((tok, category, norm))
|
||||||
normalized_tokens.append(norm)
|
normalized_tokens.append(norm)
|
||||||
|
|
||||||
filtered = [token for token in normalized_tokens if token]
|
out_pieces: List[str] = []
|
||||||
normalized_text = _cleanup_spacing(" ".join(filtered))
|
last_end = 0
|
||||||
|
for (tok, start, end), norm in zip(token_entries, normalized_tokens):
|
||||||
|
if start > last_end:
|
||||||
|
out_pieces.append(text[last_end:start])
|
||||||
|
out_pieces.append(norm)
|
||||||
|
last_end = end
|
||||||
|
if last_end < len(text):
|
||||||
|
out_pieces.append(text[last_end:])
|
||||||
|
|
||||||
|
reconstructed = "".join(out_pieces)
|
||||||
|
normalized_text = _cleanup_spacing(reconstructed)
|
||||||
return normalized_text, results
|
return normalized_text, results
|
||||||
|
|
||||||
|
|
||||||
@@ -1824,7 +1860,10 @@ def _normalize_grouped_numbers(text: str, cfg: ApostropheConfig) -> str:
|
|||||||
for digit in trimmed_fraction:
|
for digit in trimmed_fraction:
|
||||||
if not digit.isdigit():
|
if not digit.isdigit():
|
||||||
return token
|
return token
|
||||||
digit_words.append(_DIGIT_WORDS[int(digit)])
|
try:
|
||||||
|
digit_words.append(_DIGIT_WORDS[int(digit)])
|
||||||
|
except (ValueError, IndexError):
|
||||||
|
return token
|
||||||
|
|
||||||
spoken = f"{integer_words} point {' '.join(digit_words)}"
|
spoken = f"{integer_words} point {' '.join(digit_words)}"
|
||||||
return f"minus {spoken}" if is_negative else spoken
|
return f"minus {spoken}" if is_negative else spoken
|
||||||
@@ -1846,18 +1885,27 @@ def _normalize_grouped_numbers(text: str, cfg: ApostropheConfig) -> str:
|
|||||||
# Magnitude case: $2.5 million -> two point five million dollars
|
# Magnitude case: $2.5 million -> two point five million dollars
|
||||||
if "." in amount_str:
|
if "." in amount_str:
|
||||||
integer_part, fraction_part = amount_str.split(".", 1)
|
integer_part, fraction_part = amount_str.split(".", 1)
|
||||||
integer_val = int(integer_part)
|
try:
|
||||||
|
integer_val = int(integer_part)
|
||||||
|
except ValueError:
|
||||||
|
return match.group(0)
|
||||||
integer_words = _int_to_words(integer_val, language)
|
integer_words = _int_to_words(integer_val, language)
|
||||||
|
|
||||||
# Spell out fraction digits
|
# Spell out fraction digits
|
||||||
digit_words = []
|
digit_words = []
|
||||||
for digit in fraction_part:
|
for digit in fraction_part:
|
||||||
if digit.isdigit():
|
if digit.isdigit():
|
||||||
digit_words.append(_DIGIT_WORDS[int(digit)])
|
try:
|
||||||
|
digit_words.append(_DIGIT_WORDS[int(digit)])
|
||||||
|
except (ValueError, IndexError):
|
||||||
|
return match.group(0)
|
||||||
|
|
||||||
amount_spoken = f"{integer_words} point {' '.join(digit_words)}"
|
amount_spoken = f"{integer_words} point {' '.join(digit_words)}"
|
||||||
else:
|
else:
|
||||||
amount_spoken = _int_to_words(int(amount), language)
|
try:
|
||||||
|
amount_spoken = _int_to_words(int(amount), language)
|
||||||
|
except (ValueError, OverflowError):
|
||||||
|
return match.group(0)
|
||||||
|
|
||||||
currency_names = {
|
currency_names = {
|
||||||
"$": "dollars",
|
"$": "dollars",
|
||||||
|
|||||||
@@ -36,10 +36,8 @@ from abogen.domain.metadata_extraction import (
|
|||||||
format_metadata_tags,
|
format_metadata_tags,
|
||||||
)
|
)
|
||||||
|
|
||||||
from abogen.subtitle_utils import (
|
from abogen.subtitle_utils import clean_text
|
||||||
clean_text,
|
from abogen.domain.text_utils import calculate_text_length
|
||||||
calculate_text_length,
|
|
||||||
)
|
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import logging
|
import logging
|
||||||
@@ -47,9 +45,9 @@ import urllib.parse
|
|||||||
import textwrap
|
import textwrap
|
||||||
|
|
||||||
# Setup logging
|
# Setup logging
|
||||||
logging.basicConfig(
|
from abogen.utils import setup_console_logging
|
||||||
level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s"
|
|
||||||
)
|
setup_console_logging()
|
||||||
|
|
||||||
_HTML_TAG_PATTERN = re.compile(r"<[^>]+>")
|
_HTML_TAG_PATTERN = re.compile(r"<[^>]+>")
|
||||||
_LEADING_DASH_PATTERN = re.compile(r"^\s*[-–—]\s*")
|
_LEADING_DASH_PATTERN = re.compile(r"^\s*[-–—]\s*")
|
||||||
|
|||||||
+46
-79
@@ -1,6 +1,6 @@
|
|||||||
import os
|
import os
|
||||||
import re
|
|
||||||
import time
|
import time
|
||||||
|
import logging
|
||||||
import hashlib # For generating unique cache filenames
|
import hashlib # For generating unique cache filenames
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from platformdirs import user_desktop_dir
|
from platformdirs import user_desktop_dir
|
||||||
@@ -10,7 +10,6 @@ from contextlib import ExitStack, contextmanager
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
import soundfile as sf
|
import soundfile as sf
|
||||||
from abogen.utils import (
|
from abogen.utils import (
|
||||||
create_process,
|
|
||||||
get_user_cache_path,
|
get_user_cache_path,
|
||||||
detect_encoding,
|
detect_encoding,
|
||||||
)
|
)
|
||||||
@@ -24,46 +23,35 @@ from abogen.constants import (
|
|||||||
)
|
)
|
||||||
from abogen.infrastructure.subtitle_writer import make_subtitle_writer, resolve_subtitle_format
|
from abogen.infrastructure.subtitle_writer import make_subtitle_writer, resolve_subtitle_format
|
||||||
from abogen.domain.split_pattern import get_split_pattern
|
from abogen.domain.split_pattern import get_split_pattern
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.domain.subtitle_processor import (
|
from abogen.domain.subtitle_processor import (
|
||||||
parse_subtitle_file,
|
parse_subtitle_file,
|
||||||
process_subtitle_entries,
|
process_subtitle_entries,
|
||||||
)
|
)
|
||||||
from abogen.domain.output_paths import (
|
from abogen.domain.output_paths import (
|
||||||
resolve_output_directory,
|
resolve_output_directory,
|
||||||
build_output_path,
|
|
||||||
sanitize_output_stem,
|
|
||||||
sanitize_filename_for_chapter,
|
sanitize_filename_for_chapter,
|
||||||
resolve_unique_path,
|
resolve_unique_path,
|
||||||
)
|
)
|
||||||
from abogen.domain.audio_helpers import build_ffmpeg_command, to_float32
|
from abogen.domain.audio_helpers import build_ffmpeg_command, to_float32
|
||||||
from abogen.domain.audio_sink import AudioSink, open_audio_sink
|
from abogen.domain.audio_sink import open_audio_sink
|
||||||
from abogen.domain.conversion_engine import run_tts_segment_loop, synthesize_text, SynthParams, SegmentStats, SegmentInfo
|
from abogen.domain.conversion_engine import run_tts_segment_loop, synthesize_text, SynthParams, SegmentStats, SegmentInfo
|
||||||
from abogen.domain.intro_outro import resolve_intro, resolve_outro
|
from abogen.domain.intro_outro import resolve_intro, resolve_outro
|
||||||
from abogen.domain.audio_buffer import (
|
from abogen.domain.audio_buffer import (
|
||||||
create_silence,
|
create_silence,
|
||||||
mix_audio,
|
|
||||||
normalize_audio,
|
|
||||||
SAMPLE_RATE,
|
|
||||||
)
|
)
|
||||||
from abogen.domain.subtitle_generation import process_subtitle_tokens
|
from abogen.domain.subtitle_generation import process_subtitle_tokens
|
||||||
from abogen.domain.voice_loader import VoiceCache, load_voice_cached, resolve_voice
|
from abogen.domain.voice_loader import VoiceCache, load_voice_cached, resolve_voice
|
||||||
from abogen.domain.progress import calc_etr_str
|
|
||||||
from abogen.domain.normalization import TTSContext
|
|
||||||
from abogen.domain.pronunciation import (
|
|
||||||
compile_pronunciation_rules,
|
|
||||||
compile_heteronym_sentence_rules,
|
|
||||||
merge_pronunciation_overrides,
|
|
||||||
)
|
|
||||||
from abogen.domain.metadata_extraction import (
|
from abogen.domain.metadata_extraction import (
|
||||||
extract_metadata_and_build_args,
|
extract_metadata_from_text,
|
||||||
extract_metadata_for_file,
|
|
||||||
)
|
)
|
||||||
from abogen.domain.text_chapters import parse_chapters_from_text
|
from abogen.domain.text_chapters import parse_chapters_from_text
|
||||||
from abogen.infrastructure.exporters import ExportService
|
from abogen.infrastructure.exporters import ExportService
|
||||||
import abogen.hf_tracker as hf_tracker
|
import abogen.hf_tracker as hf_tracker
|
||||||
import static_ffmpeg
|
import static_ffmpeg
|
||||||
import threading # for efficient waiting
|
import threading # for efficient waiting
|
||||||
import subprocess
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -242,11 +230,6 @@ class ConversionThread(QThread):
|
|||||||
log_updated = pyqtSignal(object) # Updated signal for log updates
|
log_updated = pyqtSignal(object) # Updated signal for log updates
|
||||||
chapters_detected = pyqtSignal(int) # Signal for chapter detection
|
chapters_detected = pyqtSignal(int) # Signal for chapter detection
|
||||||
|
|
||||||
# Punctuation constants for unified handling across languages
|
|
||||||
PUNCTUATION_SENTENCE = ".!?।。!?"
|
|
||||||
PUNCTUATION_SENTENCE_COMMA = ".!?,।。!?、,"
|
|
||||||
PUNCTUATION_COMMAS = ",,、"
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
file_name,
|
file_name,
|
||||||
@@ -367,7 +350,7 @@ class ConversionThread(QThread):
|
|||||||
return samples_processed
|
return samples_processed
|
||||||
|
|
||||||
def run(self):
|
def run(self):
|
||||||
print(
|
logger.info(
|
||||||
f"\nVoice: {self.voice}\nLanguage: {self.lang_code}\nSpeed: {self.speed}\nGPU: {self.use_gpu}\nFile: {self.file_name}\nSubtitle mode: {self.subtitle_mode}\nOutput format: {self.output_format}\nSave option: {self.save_option}\n"
|
f"\nVoice: {self.voice}\nLanguage: {self.lang_code}\nSpeed: {self.speed}\nGPU: {self.use_gpu}\nFile: {self.file_name}\nSubtitle mode: {self.subtitle_mode}\nOutput format: {self.output_format}\nSave option: {self.save_option}\n"
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
@@ -525,6 +508,9 @@ class ConversionThread(QThread):
|
|||||||
) as file:
|
) as file:
|
||||||
text = file.read()
|
text = file.read()
|
||||||
|
|
||||||
|
# Extract metadata BEFORE clean_text strips the tags
|
||||||
|
self._extracted_metadata = extract_metadata_from_text(text)
|
||||||
|
|
||||||
# Clean up text using utility function
|
# Clean up text using utility function
|
||||||
text = clean_text(text)
|
text = clean_text(text)
|
||||||
|
|
||||||
@@ -550,22 +536,18 @@ class ConversionThread(QThread):
|
|||||||
)
|
)
|
||||||
|
|
||||||
# --- Compile normalization rules (heteronym + pronunciation) ---
|
# --- Compile normalization rules (heteronym + pronunciation) ---
|
||||||
from abogen.domain.normalization import TTSContext
|
from abogen.domain.config_types import PronunciationConfig
|
||||||
|
from abogen.domain.normalization import build_tts_context
|
||||||
class _MergeJob:
|
self._tts_context = build_tts_context(
|
||||||
pronunciation_overrides = getattr(self, "pronunciation_overrides", None)
|
language=self.lang_code,
|
||||||
manual_overrides = getattr(self, "manual_overrides", None)
|
subtitle=self.subtitle_mode,
|
||||||
heteronym_overrides = getattr(self, "heteronym_overrides", None)
|
pronunciation=PronunciationConfig(
|
||||||
language = self.lang_code
|
pronunciation_overrides=getattr(self, "pronunciation_overrides", None) or [],
|
||||||
|
manual_overrides=getattr(self, "manual_overrides", None) or [],
|
||||||
pronunciation_overrides = merge_pronunciation_overrides(_MergeJob())
|
heteronym_overrides=getattr(self, "heteronym_overrides", None) or [],
|
||||||
self._tts_context = TTSContext(
|
normalization_overrides=getattr(self, "normalization_overrides", None),
|
||||||
split_pattern=self.split_pattern,
|
|
||||||
pronunciation_rules=compile_pronunciation_rules(pronunciation_overrides),
|
|
||||||
heteronym_rules=compile_heteronym_sentence_rules(
|
|
||||||
getattr(self, "heteronym_overrides", None)
|
|
||||||
),
|
),
|
||||||
normalization_overrides=getattr(self, "normalization_overrides", None),
|
log_callback=lambda level, msg: self.log_updated.emit((msg, "grey" if level == "info" else "orange")),
|
||||||
)
|
)
|
||||||
|
|
||||||
# --- Chapter splitting logic ---
|
# --- Chapter splitting logic ---
|
||||||
@@ -764,7 +746,7 @@ class ConversionThread(QThread):
|
|||||||
intro_emitted = False
|
intro_emitted = False
|
||||||
if merge_chapters_at_end:
|
if merge_chapters_at_end:
|
||||||
intro_spec = resolve_intro(
|
intro_spec = resolve_intro(
|
||||||
extract_metadata_for_file(self.file_name, self.is_direct_text),
|
self._extracted_metadata,
|
||||||
os.path.basename(self.file_name) if self.file_name else "",
|
os.path.basename(self.file_name) if self.file_name else "",
|
||||||
getattr(self, "read_title_intro", False),
|
getattr(self, "read_title_intro", False),
|
||||||
self.voice, self.voice, list(self.voice_cache._cache.keys()),
|
self.voice, self.voice, list(self.voice_cache._cache.keys()),
|
||||||
@@ -893,12 +875,11 @@ class ConversionThread(QThread):
|
|||||||
)
|
)
|
||||||
spacy_sentences = None
|
spacy_sentences = None
|
||||||
active_split_pattern = self.split_pattern
|
active_split_pattern = self.split_pattern
|
||||||
spacing_pattern = r"\s*" if self.lang_code in ["z", "j"] else r"\s+"
|
|
||||||
|
|
||||||
# Pre-load spaCy model for English if it will be needed for subtitle generation
|
# Pre-load spaCy model for English if it will be needed for subtitle generation
|
||||||
if (
|
if (
|
||||||
use_spacy
|
use_spacy
|
||||||
and self.lang_code in ["a", "b"]
|
and self.lang_code in (Language.EN_US, Language.EN_GB)
|
||||||
and self.subtitle_mode in ["Sentence", "Sentence + Comma"]
|
and self.subtitle_mode in ["Sentence", "Sentence + Comma"]
|
||||||
):
|
):
|
||||||
from abogen.spacy_utils import get_spacy_model
|
from abogen.spacy_utils import get_spacy_model
|
||||||
@@ -915,7 +896,7 @@ class ConversionThread(QThread):
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
if use_spacy and self.lang_code not in ["a", "b"]:
|
if use_spacy and self.lang_code not in (Language.EN_US, Language.EN_GB):
|
||||||
# Non-English: use spaCy for pre-TTS segmentation
|
# Non-English: use spaCy for pre-TTS segmentation
|
||||||
self.log_updated.emit(
|
self.log_updated.emit(
|
||||||
("\nUsing spaCy for sentence segmentation (pre-TTS)...", "grey")
|
("\nUsing spaCy for sentence segmentation (pre-TTS)...", "grey")
|
||||||
@@ -934,15 +915,11 @@ class ConversionThread(QThread):
|
|||||||
"grey",
|
"grey",
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
# For Sentence + Comma mode, still split on commas within spaCy sentences
|
# spaCy already split at sentence boundaries; the
|
||||||
if self.subtitle_mode == "Sentence + Comma":
|
# engine only splits on newlines. Commas are never
|
||||||
active_split_pattern = r"(?<=[{}]){}|\n+".format(
|
# used in the engine split pattern (Sentence +
|
||||||
self.PUNCTUATION_COMMAS, spacing_pattern
|
# Comma splits at commas only at subtitle time).
|
||||||
)
|
active_split_pattern = "\n"
|
||||||
else:
|
|
||||||
active_split_pattern = (
|
|
||||||
"\n" # Use newline splitting for Sentence mode
|
|
||||||
)
|
|
||||||
else:
|
else:
|
||||||
self.log_updated.emit(
|
self.log_updated.emit(
|
||||||
("\nspaCy: Fallback to default segmentation...", "grey")
|
("\nspaCy: Fallback to default segmentation...", "grey")
|
||||||
@@ -953,10 +930,10 @@ class ConversionThread(QThread):
|
|||||||
|
|
||||||
# Print active split pattern used by the TTS engine once for this batch
|
# Print active split pattern used by the TTS engine once for this batch
|
||||||
try:
|
try:
|
||||||
print(f"Using split pattern: {active_split_pattern!r}")
|
logger.info(f"Using split pattern: {active_split_pattern!r}")
|
||||||
except Exception:
|
except Exception:
|
||||||
# Print must never break processing
|
# Logging must never break processing
|
||||||
print("Using split pattern: (unprintable)")
|
logger.warning("Using split pattern: (unprintable)")
|
||||||
|
|
||||||
for text_segment in text_segments:
|
for text_segment in text_segments:
|
||||||
def _qt_check_cancel() -> bool:
|
def _qt_check_cancel() -> bool:
|
||||||
@@ -1032,7 +1009,7 @@ class ConversionThread(QThread):
|
|||||||
audio_sink=merged_sink if merge_chapters_at_end else None,
|
audio_sink=merged_sink if merge_chapters_at_end else None,
|
||||||
subtitle_mode=self.subtitle_mode,
|
subtitle_mode=self.subtitle_mode,
|
||||||
max_subtitle_words=self.max_subtitle_words,
|
max_subtitle_words=self.max_subtitle_words,
|
||||||
lang_code=self.lang_code,
|
language=self.lang_code,
|
||||||
use_spacy_segmentation=getattr(self, "use_spacy_segmentation", False),
|
use_spacy_segmentation=getattr(self, "use_spacy_segmentation", False),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1100,7 +1077,7 @@ class ConversionThread(QThread):
|
|||||||
# --- Outro synthesis ---
|
# --- Outro synthesis ---
|
||||||
if merge_chapters_at_end:
|
if merge_chapters_at_end:
|
||||||
outro_spec = resolve_outro(
|
outro_spec = resolve_outro(
|
||||||
extract_metadata_for_file(self.file_name, self.is_direct_text),
|
self._extracted_metadata,
|
||||||
os.path.basename(self.file_name) if self.file_name else "",
|
os.path.basename(self.file_name) if self.file_name else "",
|
||||||
getattr(self, "read_closing_outro", True),
|
getattr(self, "read_closing_outro", True),
|
||||||
self.voice, self.voice, list(self.voice_cache._cache.keys()),
|
self.voice, self.voice, list(self.voice_cache._cache.keys()),
|
||||||
@@ -1141,12 +1118,7 @@ class ConversionThread(QThread):
|
|||||||
# Add chapters via ExportService (unified with WebUI)
|
# Add chapters via ExportService (unified with WebUI)
|
||||||
if total_chapters > 1:
|
if total_chapters > 1:
|
||||||
export_svc = ExportService()
|
export_svc = ExportService()
|
||||||
metadata_text = read_text_for_metadata(
|
metadata = dict(getattr(self, "_extracted_metadata", {}))
|
||||||
file_path=self.file_name,
|
|
||||||
is_direct_text=self.is_direct_text,
|
|
||||||
direct_text=self.file_name if self.is_direct_text else None,
|
|
||||||
)
|
|
||||||
metadata = extract_metadata_from_text(metadata_text) if metadata_text else {}
|
|
||||||
# Convert cover_path from metadata to Path if present
|
# Convert cover_path from metadata to Path if present
|
||||||
cover_path_raw = metadata.pop("cover_path", None)
|
cover_path_raw = metadata.pop("cover_path", None)
|
||||||
cover_path = Path(cover_path_raw) if cover_path_raw and os.path.exists(cover_path_raw) else None
|
cover_path = Path(cover_path_raw) if cover_path_raw and os.path.exists(cover_path_raw) else None
|
||||||
@@ -1385,33 +1357,28 @@ class ConversionThread(QThread):
|
|||||||
raise ValueError(f"Unsupported output format: {self.output_format}")
|
raise ValueError(f"Unsupported output format: {self.output_format}")
|
||||||
|
|
||||||
def _extract_and_add_metadata_tags_to_ffmpeg_cmd(self):
|
def _extract_and_add_metadata_tags_to_ffmpeg_cmd(self):
|
||||||
"""Extract metadata tags from text content and add them to ffmpeg command"""
|
"""Build ffmpeg metadata args from previously extracted metadata."""
|
||||||
# Read text for metadata extraction
|
metadata = getattr(self, "_extracted_metadata", None)
|
||||||
text = read_text_for_metadata(
|
if not metadata or not any(metadata.values()):
|
||||||
file_path=self.file_name,
|
|
||||||
is_direct_text=self.is_direct_text,
|
|
||||||
direct_text=self.file_name if self.is_direct_text else None,
|
|
||||||
)
|
|
||||||
|
|
||||||
if not text:
|
|
||||||
self.log_updated.emit(
|
self.log_updated.emit(
|
||||||
("Warning: Could not read file for metadata extraction", "orange")
|
("Warning: No metadata tags found in text", "orange")
|
||||||
)
|
)
|
||||||
return [], None
|
return [], None
|
||||||
|
|
||||||
# Extract metadata and build ffmpeg args
|
|
||||||
filename = self.file_name if self.is_direct_text else (
|
filename = self.file_name if self.is_direct_text else (
|
||||||
self.display_path if self.display_path else self.file_name
|
self.display_path if self.display_path else self.file_name
|
||||||
)
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
metadata_options, cover_path = extract_metadata_and_build_args(
|
from abogen.domain.metadata_extraction import build_ffmpeg_metadata_args, get_filename_from_path
|
||||||
text=text,
|
actual_filename = get_filename_from_path(
|
||||||
filename=filename,
|
file_path=filename,
|
||||||
display_path=getattr(self, "display_path", None),
|
display_path=getattr(self, "display_path", None),
|
||||||
from_queue=getattr(self, "from_queue", False),
|
from_queue=getattr(self, "from_queue", False),
|
||||||
)
|
)
|
||||||
return metadata_options, cover_path
|
args = build_ffmpeg_metadata_args(metadata, actual_filename)
|
||||||
|
cover_path = metadata.get("cover_path")
|
||||||
|
return args, cover_path
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
self.log_updated.emit(
|
self.log_updated.emit(
|
||||||
(f"Warning: Metadata extraction error: {e}", "orange")
|
(f"Warning: Metadata extraction error: {e}", "orange")
|
||||||
@@ -1475,7 +1442,7 @@ class VoicePreviewThread(QThread):
|
|||||||
return os.path.join(self.cache_dir, filename)
|
return os.path.join(self.cache_dir, filename)
|
||||||
|
|
||||||
def run(self):
|
def run(self):
|
||||||
print(
|
logger.info(
|
||||||
f"\nVoice: {self.voice}\nLanguage: {self.lang_code}\nSpeed: {self.speed}\nGPU: {self.use_gpu}\n"
|
f"\nVoice: {self.voice}\nLanguage: {self.lang_code}\nSpeed: {self.speed}\nGPU: {self.use_gpu}\n"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -17,6 +17,12 @@ import os
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Callable, Dict, List, Optional
|
from typing import Any, Callable, Dict, List, Optional
|
||||||
|
|
||||||
|
from abogen.application.conversion_config import (
|
||||||
|
ChapterChunkConfig,
|
||||||
|
Epub3ExportConfig,
|
||||||
|
PronunciationConfig,
|
||||||
|
WordSubstitutionConfig,
|
||||||
|
)
|
||||||
from abogen.application.conversion_request import ConversionRequest
|
from abogen.application.conversion_request import ConversionRequest
|
||||||
from abogen.application.conversion_ports import ConversionCancelled, ResolvedVoice
|
from abogen.application.conversion_ports import ConversionCancelled, ResolvedVoice
|
||||||
|
|
||||||
@@ -54,6 +60,25 @@ def build_conversion_request_from_thread(thread: Any) -> ConversionRequest:
|
|||||||
if thread.output_folder:
|
if thread.output_folder:
|
||||||
output_folder = Path(thread.output_folder)
|
output_folder = Path(thread.output_folder)
|
||||||
|
|
||||||
|
# Build pronunciation config
|
||||||
|
pronunciation = None
|
||||||
|
pron_overrides = getattr(thread, "pronunciation_overrides", []) or []
|
||||||
|
manual_overrides = getattr(thread, "manual_overrides", []) or []
|
||||||
|
heteronym_overrides = getattr(thread, "heteronym_overrides", []) or []
|
||||||
|
norm_overrides = getattr(thread, "normalization_overrides", None)
|
||||||
|
if pron_overrides or manual_overrides or heteronym_overrides or norm_overrides:
|
||||||
|
pronunciation = PronunciationConfig(
|
||||||
|
pronunciation_overrides=pron_overrides,
|
||||||
|
manual_overrides=manual_overrides,
|
||||||
|
heteronym_overrides=heteronym_overrides,
|
||||||
|
normalization_overrides=norm_overrides,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Build epub3 config
|
||||||
|
epub3_export = None
|
||||||
|
if getattr(thread, "generate_epub3", False):
|
||||||
|
epub3_export = Epub3ExportConfig()
|
||||||
|
|
||||||
return ConversionRequest(
|
return ConversionRequest(
|
||||||
# Source
|
# Source
|
||||||
source_path=source_path,
|
source_path=source_path,
|
||||||
@@ -87,24 +112,16 @@ def build_conversion_request_from_thread(thread: Any) -> ConversionRequest:
|
|||||||
read_title_intro=getattr(thread, "read_title_intro", False),
|
read_title_intro=getattr(thread, "read_title_intro", False),
|
||||||
read_closing_outro=getattr(thread, "read_closing_outro", True),
|
read_closing_outro=getattr(thread, "read_closing_outro", True),
|
||||||
auto_prefix_chapter_titles=getattr(thread, "auto_prefix_chapter_titles", True),
|
auto_prefix_chapter_titles=getattr(thread, "auto_prefix_chapter_titles", True),
|
||||||
normalize_chapter_opening_caps=getattr(thread, "normalize_chapter_opening_caps", False),
|
normalize_chapter_opening_caps=thread.normalize_chapter_opening_caps,
|
||||||
# Pronunciation / Normalization
|
|
||||||
pronunciation_overrides=getattr(thread, "pronunciation_overrides", []) or [],
|
|
||||||
manual_overrides=getattr(thread, "manual_overrides", []) or [],
|
|
||||||
heteronym_overrides=getattr(thread, "heteronym_overrides", []) or [],
|
|
||||||
normalization_overrides=getattr(thread, "normalization_overrides", None),
|
|
||||||
# Chapter/Chunk Configuration
|
|
||||||
chapter_overrides=[], # PyQt doesn't use chapter overrides from GUI
|
|
||||||
chunks=[], # PyQt doesn't use chunks from GUI
|
|
||||||
chunk_level="paragraph",
|
|
||||||
speaker_mode="single",
|
|
||||||
speakers={},
|
|
||||||
# Metadata
|
# Metadata
|
||||||
metadata_tags=getattr(thread, "metadata_tags", {}) or {},
|
metadata_tags=getattr(thread, "metadata_tags", {}) or {},
|
||||||
# Artifacts
|
# Artifacts
|
||||||
cover_image_path=getattr(thread, "cover_image_path", None),
|
cover_image_path=getattr(thread, "cover_image_path", None),
|
||||||
cover_image_mime=getattr(thread, "cover_image_mime", None),
|
cover_image_mime=getattr(thread, "cover_image_mime", None),
|
||||||
generate_epub3=getattr(thread, "generate_epub3", False),
|
# Feature configs
|
||||||
|
pronunciation=pronunciation,
|
||||||
|
epub3_export=epub3_export,
|
||||||
|
chapter_chunk=ChapterChunkConfig(), # PyQt doesn't use chapter overrides from GUI
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+174
-116
@@ -5,9 +5,12 @@ import tempfile
|
|||||||
import platform
|
import platform
|
||||||
import base64
|
import base64
|
||||||
import re
|
import re
|
||||||
|
import logging
|
||||||
from abogen.pyqt.queue_manager_gui import QueueManager
|
from abogen.pyqt.queue_manager_gui import QueueManager
|
||||||
from abogen.pyqt.queued_item import QueuedItem
|
from abogen.pyqt.queued_item import QueuedItem
|
||||||
|
|
||||||
|
_log = logging.getLogger("abogen.gui")
|
||||||
|
|
||||||
import abogen.hf_tracker as hf_tracker
|
import abogen.hf_tracker as hf_tracker
|
||||||
import hashlib # Added for cache path generation
|
import hashlib # Added for cache path generation
|
||||||
from PyQt6.QtWidgets import (
|
from PyQt6.QtWidgets import (
|
||||||
@@ -70,13 +73,12 @@ from abogen.utils import (
|
|||||||
LoadPipelineThread,
|
LoadPipelineThread,
|
||||||
)
|
)
|
||||||
|
|
||||||
from abogen.subtitle_utils import (
|
from abogen.subtitle_utils import clean_text
|
||||||
clean_text,
|
from abogen.domain.text_utils import calculate_text_length
|
||||||
calculate_text_length,
|
|
||||||
)
|
|
||||||
|
|
||||||
from abogen.pyqt.conversion import ConversionThread, VoicePreviewThread, PlayAudioThread, ChapterOptionsDialog, TimestampDetectionDialog
|
from abogen.pyqt.conversion import ConversionThread, VoicePreviewThread, PlayAudioThread, ChapterOptionsDialog, TimestampDetectionDialog
|
||||||
from abogen.pyqt.book_handler import HandlerDialog
|
from abogen.pyqt.book_handler import HandlerDialog
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.constants import (
|
from abogen.constants import (
|
||||||
PROGRAM_NAME,
|
PROGRAM_NAME,
|
||||||
VERSION,
|
VERSION,
|
||||||
@@ -90,8 +92,9 @@ from abogen.constants import (
|
|||||||
from abogen.tts_plugin.utils import get_voices
|
from abogen.tts_plugin.utils import get_voices
|
||||||
import threading
|
import threading
|
||||||
from abogen.pyqt.voice_formula_gui import VoiceFormulaDialog
|
from abogen.pyqt.voice_formula_gui import VoiceFormulaDialog
|
||||||
from abogen.voice_profiles import load_profiles
|
from abogen.voice_profiles import load_profiles, resolve_profile_language
|
||||||
from abogen.domain.settings_core import all_settings_defaults
|
from abogen.domain.settings_core import all_settings_defaults
|
||||||
|
from plugins.kokoro.engine import language_for_code, language_for_voice_id
|
||||||
|
|
||||||
# Module-level default cache for use outside __init__
|
# Module-level default cache for use outside __init__
|
||||||
_DEFAULTS = all_settings_defaults()
|
_DEFAULTS = all_settings_defaults()
|
||||||
@@ -134,6 +137,28 @@ class ThreadSafeLogSignal(QObject):
|
|||||||
self.log_signal.emit(message)
|
self.log_signal.emit(message)
|
||||||
|
|
||||||
|
|
||||||
|
_UPDATE_CHECK_URL = "https://raw.githubusercontent.com/denizsafak/abogen/refs/heads/main/abogen/VERSION"
|
||||||
|
_UPDATE_CHECK_TIMEOUT = 8 # seconds; bounds offline/DNS hangs so the GUI never blocks
|
||||||
|
|
||||||
|
|
||||||
|
class _UpdateCheckThread(QThread):
|
||||||
|
"""Fetch the remote VERSION file off the GUI thread."""
|
||||||
|
|
||||||
|
succeeded = pyqtSignal(str)
|
||||||
|
failed = pyqtSignal(str)
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(
|
||||||
|
_UPDATE_CHECK_URL, timeout=_UPDATE_CHECK_TIMEOUT
|
||||||
|
) as response:
|
||||||
|
self.succeeded.emit(response.read().decode().strip())
|
||||||
|
except Exception as exc: # offline, DNS hang, HTTP error, ...
|
||||||
|
self.failed.emit(str(exc))
|
||||||
|
|
||||||
|
|
||||||
class IconProvider(QFileIconProvider):
|
class IconProvider(QFileIconProvider):
|
||||||
def icon(self, fileInfo):
|
def icon(self, fileInfo):
|
||||||
return super().icon(fileInfo)
|
return super().icon(fileInfo)
|
||||||
@@ -399,11 +424,7 @@ class InputBox(QLabel):
|
|||||||
# Re-enable subtitle and replace newlines controls when cleared
|
# Re-enable subtitle and replace newlines controls when cleared
|
||||||
window = self.window()
|
window = self.window()
|
||||||
if hasattr(window, "subtitle_combo"):
|
if hasattr(window, "subtitle_combo"):
|
||||||
# Only enable if language supports it
|
window.subtitle_combo.setEnabled(True)
|
||||||
current_lang = getattr(window, "selected_lang", "a")
|
|
||||||
window.subtitle_combo.setEnabled(
|
|
||||||
current_lang in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION
|
|
||||||
)
|
|
||||||
if hasattr(window, "replace_newlines_combo"):
|
if hasattr(window, "replace_newlines_combo"):
|
||||||
window.replace_newlines_combo.setEnabled(True)
|
window.replace_newlines_combo.setEnabled(True)
|
||||||
|
|
||||||
@@ -943,7 +964,7 @@ class abogen(QWidget):
|
|||||||
self.selected_lang = None
|
self.selected_lang = None
|
||||||
else:
|
else:
|
||||||
self.selected_voice = self.config.get("selected_voice", _d["selected_voice"])
|
self.selected_voice = self.config.get("selected_voice", _d["selected_voice"])
|
||||||
self.selected_lang = self.selected_voice[0] if self.selected_voice else None
|
self.selected_lang = language_for_voice_id(self.selected_voice)
|
||||||
self.is_converting = False
|
self.is_converting = False
|
||||||
self.subtitle_mode = self.config.get("subtitle_mode", _d["subtitle_mode"])
|
self.subtitle_mode = self.config.get("subtitle_mode", _d["subtitle_mode"])
|
||||||
self.max_subtitle_words = self.config.get("max_subtitle_words", _d["max_subtitle_words"])
|
self.max_subtitle_words = self.config.get("max_subtitle_words", _d["max_subtitle_words"])
|
||||||
@@ -990,7 +1011,12 @@ class abogen(QWidget):
|
|||||||
self.queued_items = []
|
self.queued_items = []
|
||||||
self.current_queue_index = 0
|
self.current_queue_index = 0
|
||||||
|
|
||||||
self.initUI()
|
from abogen.utils import timed_log
|
||||||
|
import logging
|
||||||
|
_startup_log = logging.getLogger("abogen.startup")
|
||||||
|
|
||||||
|
with timed_log("GUI initUI (widget building)", logger=_startup_log):
|
||||||
|
self.initUI()
|
||||||
self.speed_slider.setValue(int(self.config.get("speed", _d["speed"]) * 100))
|
self.speed_slider.setValue(int(self.config.get("speed", _d["speed"]) * 100))
|
||||||
self.update_speed_label()
|
self.update_speed_label()
|
||||||
# Set initial selection: prefer profile, else voice
|
# Set initial selection: prefer profile, else voice
|
||||||
@@ -1005,13 +1031,17 @@ class abogen(QWidget):
|
|||||||
if self.selected_profile_name:
|
if self.selected_profile_name:
|
||||||
from abogen.voice_profiles import load_profiles
|
from abogen.voice_profiles import load_profiles
|
||||||
|
|
||||||
entry = load_profiles().get(self.selected_profile_name, {})
|
with timed_log("voice profile load", logger=_startup_log):
|
||||||
|
entry = load_profiles().get(self.selected_profile_name, {})
|
||||||
if isinstance(entry, dict):
|
if isinstance(entry, dict):
|
||||||
self.mixed_voice_state = entry.get("voices", [])
|
self.mixed_voice_state = entry.get("voices", [])
|
||||||
self.selected_lang = entry.get("language")
|
self.selected_lang = resolve_profile_language(entry)
|
||||||
else:
|
else:
|
||||||
self.mixed_voice_state = entry
|
self.mixed_voice_state = entry
|
||||||
self.selected_lang = entry[0][0] if entry and entry[0] else None
|
self.selected_lang = (
|
||||||
|
language_for_voice_id(entry[0]) if entry and entry[0] else Language.EN_US
|
||||||
|
)
|
||||||
|
self.update_subtitle_options_availability()
|
||||||
if self.save_option == "Choose output folder" and self.selected_output_folder:
|
if self.save_option == "Choose output folder" and self.selected_output_folder:
|
||||||
self.save_path_label.setText(self.selected_output_folder)
|
self.save_path_label.setText(self.selected_output_folder)
|
||||||
self.save_path_row_widget.show()
|
self.save_path_row_widget.show()
|
||||||
@@ -1177,6 +1207,7 @@ class abogen(QWidget):
|
|||||||
"Sentence + Comma: Subtitles will be generated for each sentence and comma.\n"
|
"Sentence + Comma: Subtitles will be generated for each sentence and comma.\n"
|
||||||
"Sentence + Highlighting: Subtitles with word-by-word karaoke highlighting.\n"
|
"Sentence + Highlighting: Subtitles with word-by-word karaoke highlighting.\n"
|
||||||
"1+ word: Subtitles will be generated for each word(s).\n\n"
|
"1+ word: Subtitles will be generated for each word(s).\n\n"
|
||||||
|
"Word-count and highlighting modes are only available for English.\n"
|
||||||
"Supported languages for subtitle generation:\n"
|
"Supported languages for subtitle generation:\n"
|
||||||
+ "\n".join(
|
+ "\n".join(
|
||||||
f'"{lang}" => {LANGUAGE_DESCRIPTIONS.get(lang, lang)}'
|
f'"{lang}" => {LANGUAGE_DESCRIPTIONS.get(lang, lang)}'
|
||||||
@@ -1755,8 +1786,9 @@ class abogen(QWidget):
|
|||||||
|
|
||||||
def update_subtitle_options_availability(self):
|
def update_subtitle_options_availability(self):
|
||||||
"""
|
"""
|
||||||
Update the enabled state of subtitle options based on the selected language.
|
Update the enabled state of subtitle options based on the selected
|
||||||
For non-English languages, only sentence-based and line-based modes are supported.
|
language and input type. Subtitle generation works for every language,
|
||||||
|
but word-count and highlighting modes are only available for English.
|
||||||
"""
|
"""
|
||||||
# Check if current file is a subtitle file
|
# Check if current file is a subtitle file
|
||||||
is_subtitle_input = False
|
is_subtitle_input = False
|
||||||
@@ -1765,16 +1797,14 @@ class abogen(QWidget):
|
|||||||
):
|
):
|
||||||
is_subtitle_input = True
|
is_subtitle_input = True
|
||||||
|
|
||||||
if self.selected_lang not in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION:
|
|
||||||
self.subtitle_combo.setEnabled(False)
|
|
||||||
self.subtitle_format_combo.setEnabled(False)
|
|
||||||
return
|
|
||||||
|
|
||||||
# Only enable subtitle_combo if it's NOT a subtitle input
|
# Only enable subtitle_combo if it's NOT a subtitle input
|
||||||
self.subtitle_combo.setEnabled(not is_subtitle_input)
|
self.subtitle_combo.setEnabled(not is_subtitle_input)
|
||||||
self.subtitle_format_combo.setEnabled(True)
|
self.subtitle_format_combo.setEnabled(True)
|
||||||
|
|
||||||
is_english = self.selected_lang in ["a", "b"]
|
is_english = self.selected_lang in (
|
||||||
|
Language.EN_US,
|
||||||
|
Language.EN_GB,
|
||||||
|
)
|
||||||
|
|
||||||
# Items to keep enabled for non-English
|
# Items to keep enabled for non-English
|
||||||
allowed_modes = ["Disabled", "Line", "Sentence", "Sentence + Comma"]
|
allowed_modes = ["Disabled", "Line", "Sentence", "Sentence + Comma"]
|
||||||
@@ -1789,10 +1819,7 @@ class abogen(QWidget):
|
|||||||
if is_english:
|
if is_english:
|
||||||
item.setEnabled(True)
|
item.setEnabled(True)
|
||||||
else:
|
else:
|
||||||
if text in allowed_modes:
|
item.setEnabled(text in allowed_modes)
|
||||||
item.setEnabled(True)
|
|
||||||
else:
|
|
||||||
item.setEnabled(False)
|
|
||||||
|
|
||||||
# If current selection is disabled, switch to a valid one
|
# If current selection is disabled, switch to a valid one
|
||||||
current_text = self.subtitle_combo.currentText()
|
current_text = self.subtitle_combo.currentText()
|
||||||
@@ -1811,7 +1838,7 @@ class abogen(QWidget):
|
|||||||
|
|
||||||
def on_voice_changed(self, index):
|
def on_voice_changed(self, index):
|
||||||
voice = self.voice_combo.itemData(index)
|
voice = self.voice_combo.itemData(index)
|
||||||
self.selected_voice, self.selected_lang = voice, voice[0]
|
self.selected_voice, self.selected_lang = voice, language_for_voice_id(voice)
|
||||||
self.config["selected_voice"] = voice
|
self.config["selected_voice"] = voice
|
||||||
save_config(self.config)
|
save_config(self.config)
|
||||||
# Enable/disable subtitle options based on language
|
# Enable/disable subtitle options based on language
|
||||||
@@ -1828,10 +1855,12 @@ class abogen(QWidget):
|
|||||||
# set mixed voices and language
|
# set mixed voices and language
|
||||||
if isinstance(entry, dict):
|
if isinstance(entry, dict):
|
||||||
self.mixed_voice_state = entry.get("voices", [])
|
self.mixed_voice_state = entry.get("voices", [])
|
||||||
self.selected_lang = entry.get("language")
|
self.selected_lang = resolve_profile_language(entry)
|
||||||
else:
|
else:
|
||||||
self.mixed_voice_state = entry
|
self.mixed_voice_state = entry
|
||||||
self.selected_lang = entry[0][0] if entry and entry[0] else None
|
self.selected_lang = (
|
||||||
|
language_for_voice_id(entry[0]) if entry and entry[0] else Language.EN_US
|
||||||
|
)
|
||||||
self.selected_voice = None
|
self.selected_voice = None
|
||||||
self.config["selected_profile_name"] = pname
|
self.config["selected_profile_name"] = pname
|
||||||
self.config.pop("selected_voice", None)
|
self.config.pop("selected_voice", None)
|
||||||
@@ -1841,7 +1870,7 @@ class abogen(QWidget):
|
|||||||
else:
|
else:
|
||||||
self.mixed_voice_state = None
|
self.mixed_voice_state = None
|
||||||
self.selected_profile_name = None
|
self.selected_profile_name = None
|
||||||
self.selected_voice, self.selected_lang = data, data[0]
|
self.selected_voice, self.selected_lang = data, language_for_voice_id(data)
|
||||||
self.config["selected_voice"] = data
|
self.config["selected_voice"] = data
|
||||||
if "selected_profile_name" in self.config:
|
if "selected_profile_name" in self.config:
|
||||||
del self.config["selected_profile_name"]
|
del self.config["selected_profile_name"]
|
||||||
@@ -1852,8 +1881,9 @@ class abogen(QWidget):
|
|||||||
from abogen.voice_profiles import load_profiles
|
from abogen.voice_profiles import load_profiles
|
||||||
|
|
||||||
entry = load_profiles().get(profile_name, {})
|
entry = load_profiles().get(profile_name, {})
|
||||||
lang = entry.get("language") if isinstance(entry, dict) else None
|
enable = (
|
||||||
enable = lang in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION
|
resolve_profile_language(entry) in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION
|
||||||
|
)
|
||||||
self.subtitle_combo.setEnabled(enable)
|
self.subtitle_combo.setEnabled(enable)
|
||||||
self.subtitle_format_combo.setEnabled(enable)
|
self.subtitle_format_combo.setEnabled(enable)
|
||||||
|
|
||||||
@@ -2235,18 +2265,18 @@ class abogen(QWidget):
|
|||||||
else:
|
else:
|
||||||
return self.selected_voice
|
return self.selected_voice
|
||||||
|
|
||||||
def get_selected_lang(self, voice_formula) -> str:
|
def get_selected_lang(self, voice_formula) -> Language:
|
||||||
if self.selected_profile_name:
|
if self.selected_profile_name:
|
||||||
from abogen.voice_profiles import load_profiles
|
from abogen.voice_profiles import load_profiles
|
||||||
|
|
||||||
entry = load_profiles().get(self.selected_profile_name, {})
|
entry = load_profiles().get(self.selected_profile_name, {})
|
||||||
selected_lang = entry.get("language")
|
selected_lang = resolve_profile_language(entry)
|
||||||
else:
|
else:
|
||||||
selected_lang = self.selected_voice[0] if self.selected_voice else None
|
selected_lang = language_for_voice_id(self.selected_voice)
|
||||||
# fallback: extract from formula if missing
|
# fallback: extract from formula if missing
|
||||||
if not selected_lang:
|
if not selected_lang:
|
||||||
m = re.search(r"\b([a-z])", voice_formula)
|
m = re.search(r"\b([a-z])", voice_formula)
|
||||||
selected_lang = m.group(1) if m else None
|
selected_lang = language_for_code(m.group(1)) if m else Language.EN_US
|
||||||
return selected_lang
|
return selected_lang
|
||||||
|
|
||||||
def get_actual_subtitle_mode(self) -> str:
|
def get_actual_subtitle_mode(self) -> str:
|
||||||
@@ -2422,7 +2452,7 @@ class abogen(QWidget):
|
|||||||
self.update_log((gpu_msg, gpu_ok))
|
self.update_log((gpu_msg, gpu_ok))
|
||||||
self.update_log("Loading modules...")
|
self.update_log("Loading modules...")
|
||||||
|
|
||||||
lang_code = self.selected_lang or "a"
|
lang_code = self.selected_lang or Language.EN_US
|
||||||
load_thread = LoadPipelineThread(
|
load_thread = LoadPipelineThread(
|
||||||
pipeline_loaded_callback, lang_code=lang_code, use_gpu=gpu_ok
|
pipeline_loaded_callback, lang_code=lang_code, use_gpu=gpu_ok
|
||||||
)
|
)
|
||||||
@@ -2753,12 +2783,12 @@ class abogen(QWidget):
|
|||||||
from abogen.voice_profiles import load_profiles
|
from abogen.voice_profiles import load_profiles
|
||||||
|
|
||||||
entry = load_profiles().get(self.selected_profile_name, {})
|
entry = load_profiles().get(self.selected_profile_name, {})
|
||||||
lang_to_cache = entry.get("language")
|
lang_to_cache = resolve_profile_language(entry)
|
||||||
else:
|
else:
|
||||||
lang_to_cache = self.selected_lang
|
lang_to_cache = self.selected_lang
|
||||||
if not lang_to_cache and self.mixed_voice_state:
|
if not lang_to_cache and self.mixed_voice_state:
|
||||||
lang_to_cache = (
|
lang_to_cache = (
|
||||||
self.mixed_voice_state[0][0][0]
|
language_for_voice_id(self.mixed_voice_state[0][0])
|
||||||
if self.mixed_voice_state and self.mixed_voice_state[0][0]
|
if self.mixed_voice_state and self.mixed_voice_state[0][0]
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
@@ -2862,7 +2892,7 @@ class abogen(QWidget):
|
|||||||
)
|
)
|
||||||
self.loading_movie.start()
|
self.loading_movie.start()
|
||||||
|
|
||||||
lang = self.selected_lang or "a"
|
lang = self.selected_lang or Language.EN_US
|
||||||
load_thread = LoadPipelineThread(
|
load_thread = LoadPipelineThread(
|
||||||
self._on_pipeline_loaded_for_preview, lang_code=lang, use_gpu=self.gpu_ok
|
self._on_pipeline_loaded_for_preview, lang_code=lang, use_gpu=self.gpu_ok
|
||||||
)
|
)
|
||||||
@@ -2894,17 +2924,17 @@ class abogen(QWidget):
|
|||||||
from abogen.voice_profiles import load_profiles
|
from abogen.voice_profiles import load_profiles
|
||||||
|
|
||||||
entry = load_profiles().get(self.selected_profile_name, {})
|
entry = load_profiles().get(self.selected_profile_name, {})
|
||||||
lang = entry.get("language")
|
lang = resolve_profile_language(entry)
|
||||||
else:
|
else:
|
||||||
lang = self.selected_lang
|
lang = self.selected_lang
|
||||||
if not lang and self.mixed_voice_state:
|
if not lang and self.mixed_voice_state:
|
||||||
lang = (
|
lang = (
|
||||||
self.mixed_voice_state[0][0][0]
|
language_for_voice_id(self.mixed_voice_state[0][0])
|
||||||
if self.mixed_voice_state and self.mixed_voice_state[0][0]
|
if self.mixed_voice_state and self.mixed_voice_state[0][0]
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
lang = self.selected_voice[0]
|
lang = language_for_voice_id(self.selected_voice)
|
||||||
voice = self.selected_voice
|
voice = self.selected_voice
|
||||||
|
|
||||||
# use same gpu/cpu logic as in conversion
|
# use same gpu/cpu logic as in conversion
|
||||||
@@ -3165,14 +3195,25 @@ class abogen(QWidget):
|
|||||||
save_config(self.config)
|
save_config(self.config)
|
||||||
|
|
||||||
def cleanup_conversion_thread(self):
|
def cleanup_conversion_thread(self):
|
||||||
# Stop conversion thread
|
# Stop conversion thread (bounded wait so closing never hangs)
|
||||||
if (
|
if (
|
||||||
hasattr(self, "conversion_thread")
|
hasattr(self, "conversion_thread")
|
||||||
and self.conversion_thread is not None
|
and self.conversion_thread is not None
|
||||||
and self.conversion_thread.isRunning()
|
and self.conversion_thread.isRunning()
|
||||||
):
|
):
|
||||||
|
_log.info("Close: stopping conversion thread")
|
||||||
|
start = time.perf_counter()
|
||||||
self.conversion_thread.cancel()
|
self.conversion_thread.cancel()
|
||||||
self.conversion_thread.wait()
|
if not self.conversion_thread.wait(2000):
|
||||||
|
_log.warning("Close: conversion thread did not stop in 2s, terminating")
|
||||||
|
self.conversion_thread.terminate()
|
||||||
|
self.conversion_thread.wait(1000)
|
||||||
|
_log.info(
|
||||||
|
"Close: conversion thread stopped in %.2fs",
|
||||||
|
time.perf_counter() - start,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
_log.info("Close: no running conversion thread")
|
||||||
|
|
||||||
def cleanup_preview_threads(self):
|
def cleanup_preview_threads(self):
|
||||||
# Stop preview generation thread
|
# Stop preview generation thread
|
||||||
@@ -3181,8 +3222,13 @@ class abogen(QWidget):
|
|||||||
and self.preview_thread is not None
|
and self.preview_thread is not None
|
||||||
and self.preview_thread.isRunning()
|
and self.preview_thread.isRunning()
|
||||||
):
|
):
|
||||||
|
_log.info("Close: terminating preview thread")
|
||||||
|
start = time.perf_counter()
|
||||||
self.preview_thread.terminate()
|
self.preview_thread.terminate()
|
||||||
self.preview_thread.wait()
|
self.preview_thread.wait(1000)
|
||||||
|
_log.info(
|
||||||
|
"Close: preview thread stopped in %.2fs", time.perf_counter() - start
|
||||||
|
)
|
||||||
|
|
||||||
# Stop audio playback thread
|
# Stop audio playback thread
|
||||||
if (
|
if (
|
||||||
@@ -3190,8 +3236,13 @@ class abogen(QWidget):
|
|||||||
and self.play_audio_thread is not None
|
and self.play_audio_thread is not None
|
||||||
and self.play_audio_thread.isRunning()
|
and self.play_audio_thread.isRunning()
|
||||||
):
|
):
|
||||||
|
_log.info("Close: stopping audio playback thread")
|
||||||
|
start = time.perf_counter()
|
||||||
self.play_audio_thread.stop()
|
self.play_audio_thread.stop()
|
||||||
self.play_audio_thread.wait()
|
self.play_audio_thread.wait(1000)
|
||||||
|
_log.info(
|
||||||
|
"Close: audio thread stopped in %.2fs", time.perf_counter() - start
|
||||||
|
)
|
||||||
|
|
||||||
# Cleanup pygame mixer if initialized
|
# Cleanup pygame mixer if initialized
|
||||||
try:
|
try:
|
||||||
@@ -3202,6 +3253,7 @@ class abogen(QWidget):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
def closeEvent(self, event):
|
def closeEvent(self, event):
|
||||||
|
_log.info("Close: window close requested (converting=%s)", self.is_converting)
|
||||||
if self.is_converting:
|
if self.is_converting:
|
||||||
box = QMessageBox(self)
|
box = QMessageBox(self)
|
||||||
box.setIcon(QMessageBox.Icon.Warning)
|
box.setIcon(QMessageBox.Icon.Warning)
|
||||||
@@ -3214,16 +3266,14 @@ class abogen(QWidget):
|
|||||||
)
|
)
|
||||||
box.setDefaultButton(QMessageBox.StandardButton.No)
|
box.setDefaultButton(QMessageBox.StandardButton.No)
|
||||||
if box.exec() == QMessageBox.StandardButton.Yes:
|
if box.exec() == QMessageBox.StandardButton.Yes:
|
||||||
from abogen import shutdown
|
_log.info("Close: user confirmed exit during conversion")
|
||||||
shutdown.request_shutdown()
|
|
||||||
self.cleanup_conversion_thread()
|
self.cleanup_conversion_thread()
|
||||||
self.cleanup_preview_threads()
|
self.cleanup_preview_threads()
|
||||||
event.accept()
|
event.accept()
|
||||||
else:
|
else:
|
||||||
|
_log.info("Close: user cancelled exit")
|
||||||
event.ignore()
|
event.ignore()
|
||||||
else:
|
else:
|
||||||
from abogen import shutdown
|
|
||||||
shutdown.request_shutdown()
|
|
||||||
self.cleanup_conversion_thread()
|
self.cleanup_conversion_thread()
|
||||||
self.cleanup_preview_threads()
|
self.cleanup_preview_threads()
|
||||||
event.accept()
|
event.accept()
|
||||||
@@ -3950,7 +4000,9 @@ Categories=AudioVideo;Audio;Utility;
|
|||||||
initial_state = entry.get("voices", [])
|
initial_state = entry.get("voices", [])
|
||||||
else:
|
else:
|
||||||
initial_state = entry
|
initial_state = entry
|
||||||
self.selected_lang = entry[0][0] if entry and entry[0] else None
|
self.selected_lang = (
|
||||||
|
language_for_voice_id(entry[0]) if entry and entry[0] else Language.EN_US
|
||||||
|
)
|
||||||
dialog = VoiceFormulaDialog(
|
dialog = VoiceFormulaDialog(
|
||||||
self, initial_state=initial_state, selected_profile=selected_profile
|
self, initial_state=initial_state, selected_profile=selected_profile
|
||||||
)
|
)
|
||||||
@@ -4047,75 +4099,85 @@ Categories=AudioVideo;Audio;Utility;
|
|||||||
self.check_for_updates_startup()
|
self.check_for_updates_startup()
|
||||||
|
|
||||||
def check_for_updates_startup(self):
|
def check_for_updates_startup(self):
|
||||||
import urllib.request
|
# Network I/O runs in a worker thread: urlopen without a timeout on
|
||||||
|
# the GUI thread froze the whole app when offline (DNS/connect can
|
||||||
def show_update_message(remote_version, local_version):
|
# hang for minutes). Results return via signals on the GUI thread.
|
||||||
msg_box = QMessageBox(self)
|
thread = getattr(self, "_update_check_thread", None)
|
||||||
msg_box.setIcon(QMessageBox.Icon.Information)
|
if thread is not None:
|
||||||
msg_box.setWindowTitle("Update Available")
|
try:
|
||||||
msg_box.setText(
|
if thread.isRunning():
|
||||||
f"A new version of {PROGRAM_NAME} is available! ({local_version} > {remote_version})"
|
return
|
||||||
)
|
except RuntimeError:
|
||||||
msg_box.setInformativeText(
|
pass # previous thread already finished/deleted
|
||||||
f"If you installed via pip, update by running:\n"
|
|
||||||
f"pip install --upgrade {PROGRAM_NAME}\n\n"
|
|
||||||
f"If you're using the Windows portable version, run 'WINDOWS_INSTALL.bat' again.\n\n"
|
|
||||||
"Alternatively, visit the GitHub repository for more information. "
|
|
||||||
"Would you like to view the changelog?"
|
|
||||||
)
|
|
||||||
msg_box.setStandardButtons(
|
|
||||||
QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No
|
|
||||||
)
|
|
||||||
msg_box.setDefaultButton(QMessageBox.StandardButton.Yes)
|
|
||||||
if msg_box.exec() == QMessageBox.StandardButton.Yes:
|
|
||||||
try:
|
|
||||||
QDesktopServices.openUrl(QUrl(GITHUB_URL + "/releases/latest"))
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
# Reset flag to track if we should show "no updates" message
|
|
||||||
show_result = (
|
show_result = (
|
||||||
hasattr(self, "_show_update_check_result")
|
hasattr(self, "_show_update_check_result")
|
||||||
and self._show_update_check_result
|
and self._show_update_check_result
|
||||||
)
|
)
|
||||||
self._show_update_check_result = False
|
self._show_update_check_result = False
|
||||||
|
self._update_check_thread = _UpdateCheckThread(self)
|
||||||
|
self._update_check_thread.succeeded.connect(
|
||||||
|
lambda remote_raw: self._on_update_check_done(remote_raw, show_result)
|
||||||
|
)
|
||||||
|
self._update_check_thread.failed.connect(
|
||||||
|
lambda err: self._on_update_check_failed(err, show_result)
|
||||||
|
)
|
||||||
|
self._update_check_thread.finished.connect(
|
||||||
|
self._update_check_thread.deleteLater
|
||||||
|
)
|
||||||
|
self._update_check_thread.start()
|
||||||
|
|
||||||
|
def _on_update_check_done(self, remote_raw, show_result):
|
||||||
|
remote_version = remote_raw.strip()
|
||||||
|
local_version = VERSION
|
||||||
try:
|
try:
|
||||||
update_url = "https://raw.githubusercontent.com/denizsafak/abogen/refs/heads/main/abogen/VERSION"
|
remote_num = int("".join(remote_version.split(".")))
|
||||||
with urllib.request.urlopen(update_url) as response:
|
local_num = int("".join(local_version.split(".")))
|
||||||
remote_raw = response.read().decode().strip()
|
except ValueError:
|
||||||
local_raw = VERSION
|
return
|
||||||
|
if remote_num > local_num:
|
||||||
|
# Use QTimer to ensure UI is ready, then show update message.
|
||||||
|
QTimer.singleShot(
|
||||||
|
1000,
|
||||||
|
lambda: self._show_update_message(remote_version, local_version),
|
||||||
|
)
|
||||||
|
elif show_result:
|
||||||
|
QMessageBox.information(
|
||||||
|
self,
|
||||||
|
"Up to Date",
|
||||||
|
f"You are running the latest version of {PROGRAM_NAME} ({local_version}).",
|
||||||
|
)
|
||||||
|
|
||||||
# Parse version numbers
|
def _on_update_check_failed(self, err, show_result):
|
||||||
remote_version = remote_raw
|
if show_result:
|
||||||
local_version = local_raw
|
QMessageBox.warning(
|
||||||
|
self,
|
||||||
|
"Update Check Failed",
|
||||||
|
f"Could not check for updates:\n{err}",
|
||||||
|
)
|
||||||
|
|
||||||
|
def _show_update_message(self, remote_version, local_version):
|
||||||
|
msg_box = QMessageBox(self)
|
||||||
|
msg_box.setIcon(QMessageBox.Icon.Information)
|
||||||
|
msg_box.setWindowTitle("Update Available")
|
||||||
|
msg_box.setText(
|
||||||
|
f"A new version of {PROGRAM_NAME} is available! ({local_version} > {remote_version})"
|
||||||
|
)
|
||||||
|
msg_box.setInformativeText(
|
||||||
|
f"If you installed via pip, update by running:\n"
|
||||||
|
f"pip install --upgrade {PROGRAM_NAME}\n\n"
|
||||||
|
f"If you're using the Windows portable version, run 'WINDOWS_INSTALL.bat' again.\n\n"
|
||||||
|
"Alternatively, visit the GitHub repository for more information. "
|
||||||
|
"Would you like to view the changelog?"
|
||||||
|
)
|
||||||
|
msg_box.setStandardButtons(
|
||||||
|
QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No
|
||||||
|
)
|
||||||
|
msg_box.setDefaultButton(QMessageBox.StandardButton.Yes)
|
||||||
|
if msg_box.exec() == QMessageBox.StandardButton.Yes:
|
||||||
try:
|
try:
|
||||||
remote_num = int("".join(remote_version.split(".")))
|
QDesktopServices.openUrl(QUrl(GITHUB_URL + "/releases/latest"))
|
||||||
local_num = int("".join(local_version.split(".")))
|
except Exception:
|
||||||
except ValueError as ve:
|
pass
|
||||||
return
|
|
||||||
|
|
||||||
if remote_num > local_num:
|
|
||||||
# Use QTimer to ensure UI is ready, then show update message.
|
|
||||||
QTimer.singleShot(
|
|
||||||
1000, lambda: show_update_message(remote_version, local_version)
|
|
||||||
)
|
|
||||||
elif show_result:
|
|
||||||
# Show "no updates" message if manually checking
|
|
||||||
QMessageBox.information(
|
|
||||||
self,
|
|
||||||
"Up to Date",
|
|
||||||
f"You are running the latest version of {PROGRAM_NAME} ({local_version}).",
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
if show_result:
|
|
||||||
QMessageBox.warning(
|
|
||||||
self,
|
|
||||||
"Update Check Failed",
|
|
||||||
f"Could not check for updates:\n{str(e)}",
|
|
||||||
)
|
|
||||||
pass
|
|
||||||
|
|
||||||
def clear_cache_files(self):
|
def clear_cache_files(self):
|
||||||
"""Clear cache files created by the program."""
|
"""Clear cache files created by the program."""
|
||||||
@@ -4218,8 +4280,6 @@ Categories=AudioVideo;Audio;Utility;
|
|||||||
|
|
||||||
def set_max_log_lines(self):
|
def set_max_log_lines(self):
|
||||||
"""Open a dialog to set the maximum lines in the log window."""
|
"""Open a dialog to set the maximum lines in the log window."""
|
||||||
from PyQt6.QtWidgets import QInputDialog
|
|
||||||
|
|
||||||
value, ok = QInputDialog.getInt(
|
value, ok = QInputDialog.getInt(
|
||||||
self,
|
self,
|
||||||
"Max Lines in Log Window",
|
"Max Lines in Log Window",
|
||||||
@@ -4241,8 +4301,6 @@ Categories=AudioVideo;Audio;Utility;
|
|||||||
|
|
||||||
def set_max_subtitle_words(self):
|
def set_max_subtitle_words(self):
|
||||||
"""Open a dialog to set the maximum words per subtitle"""
|
"""Open a dialog to set the maximum words per subtitle"""
|
||||||
from PyQt6.QtWidgets import QInputDialog
|
|
||||||
|
|
||||||
current_value = self.config.get("max_subtitle_words", _DEFAULTS["max_subtitle_words"])
|
current_value = self.config.get("max_subtitle_words", _DEFAULTS["max_subtitle_words"])
|
||||||
|
|
||||||
value, ok = QInputDialog.getInt(
|
value, ok = QInputDialog.getInt(
|
||||||
|
|||||||
+97
-75
@@ -1,3 +1,4 @@
|
|||||||
|
import logging
|
||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
import platform
|
import platform
|
||||||
@@ -6,103 +7,113 @@ import platform
|
|||||||
from abogen import shutdown # noqa: F401
|
from abogen import shutdown # noqa: F401
|
||||||
shutdown.register_shutdown()
|
shutdown.register_shutdown()
|
||||||
|
|
||||||
|
from abogen.utils import get_resource_path, setup_console_logging, timed_log # noqa: E402
|
||||||
|
|
||||||
|
_log = logging.getLogger("abogen.startup")
|
||||||
|
setup_console_logging()
|
||||||
|
|
||||||
# Fix PyTorch DLL loading issue ([WinError 1114]) on Windows before importing PyQt6
|
# Fix PyTorch DLL loading issue ([WinError 1114]) on Windows before importing PyQt6
|
||||||
if platform.system() == "Windows":
|
if platform.system() == "Windows":
|
||||||
import ctypes
|
with timed_log("PyTorch DLLs (Windows)", logger=_log):
|
||||||
from importlib.util import find_spec
|
import ctypes
|
||||||
|
from importlib.util import find_spec
|
||||||
|
|
||||||
try:
|
try:
|
||||||
if (
|
if (
|
||||||
(spec := find_spec("torch"))
|
(spec := find_spec("torch"))
|
||||||
and spec.origin
|
and spec.origin
|
||||||
and os.path.exists(
|
and os.path.exists(
|
||||||
dll_path := os.path.join(os.path.dirname(spec.origin), "lib", "c10.dll")
|
dll_path := os.path.join(os.path.dirname(spec.origin), "lib", "c10.dll")
|
||||||
)
|
)
|
||||||
):
|
):
|
||||||
ctypes.CDLL(os.path.normpath(dll_path))
|
ctypes.CDLL(os.path.normpath(dll_path))
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
# Qt platform plugin detection (fixes #59)
|
# Qt platform plugin detection (fixes #59)
|
||||||
try:
|
with timed_log("Qt platform plugin detection", logger=_log):
|
||||||
from PyQt6.QtCore import QLibraryInfo
|
try:
|
||||||
|
from PyQt6.QtCore import QLibraryInfo
|
||||||
|
|
||||||
# Get the path to the plugins directory
|
# Get the path to the plugins directory
|
||||||
plugins = QLibraryInfo.path(QLibraryInfo.LibraryPath.PluginsPath)
|
plugins = QLibraryInfo.path(QLibraryInfo.LibraryPath.PluginsPath)
|
||||||
|
|
||||||
# Normalize path to use the OS-native separators and absolute path
|
# Normalize path to use the OS-native separators and absolute path
|
||||||
platform_dir = os.path.normpath(os.path.join(plugins, "platforms"))
|
platform_dir = os.path.normpath(os.path.join(plugins, "platforms"))
|
||||||
|
|
||||||
# Ensure we work with an absolute path for clarity
|
# Ensure we work with an absolute path for clarity
|
||||||
platform_dir = os.path.abspath(platform_dir)
|
platform_dir = os.path.abspath(platform_dir)
|
||||||
|
|
||||||
if os.path.isdir(platform_dir):
|
if os.path.isdir(platform_dir):
|
||||||
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = platform_dir
|
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = platform_dir
|
||||||
print("QT_QPA_PLATFORM_PLUGIN_PATH set to:", platform_dir)
|
_log.info("QT_QPA_PLATFORM_PLUGIN_PATH set to: %s", platform_dir)
|
||||||
else:
|
else:
|
||||||
print("PyQt6 platform plugins not found at", platform_dir)
|
_log.warning("PyQt6 platform plugins not found at %s", platform_dir)
|
||||||
except ImportError:
|
except ImportError:
|
||||||
print("PyQt6 not installed.")
|
_log.warning("PyQt6 not installed.")
|
||||||
|
|
||||||
|
|
||||||
from abogen.utils import get_resource_path
|
|
||||||
|
|
||||||
# Pre-load "libxcb-cursor" on Linux (fixes #101)
|
# Pre-load "libxcb-cursor" on Linux (fixes #101)
|
||||||
if platform.system() == "Linux":
|
if platform.system() == "Linux":
|
||||||
arch = platform.machine().lower()
|
with timed_log("libxcb-cursor preload (Linux)", logger=_log):
|
||||||
lib_filename = {"x86_64": "libxcb-cursor-amd64.so.0", "amd64": "libxcb-cursor-amd64.so.0", "aarch64": "libxcb-cursor-arm64.so.0", "arm64": "libxcb-cursor-arm64.so.0"}.get(arch)
|
arch = platform.machine().lower()
|
||||||
if lib_filename:
|
lib_filename = {"x86_64": "libxcb-cursor-amd64.so.0", "amd64": "libxcb-cursor-amd64.so.0", "aarch64": "libxcb-cursor-arm64.so.0", "arm64": "libxcb-cursor-arm64.so.0"}.get(arch)
|
||||||
import ctypes
|
if lib_filename:
|
||||||
try:
|
import ctypes
|
||||||
# Try to load the system libxcb-cursor.so.0 first
|
try:
|
||||||
ctypes.CDLL('libxcb-cursor.so.0', mode=ctypes.RTLD_GLOBAL)
|
# Try to load the system libxcb-cursor.so.0 first
|
||||||
except OSError:
|
ctypes.CDLL('libxcb-cursor.so.0', mode=ctypes.RTLD_GLOBAL)
|
||||||
# System lib not available, load the bundled version
|
except OSError:
|
||||||
lib_path = get_resource_path('abogen.libs', lib_filename)
|
# System lib not available, load the bundled version
|
||||||
if lib_path:
|
lib_path = get_resource_path('abogen.libs', lib_filename)
|
||||||
try:
|
if lib_path:
|
||||||
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
|
try:
|
||||||
except OSError:
|
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
|
||||||
# If it fails (e.g. wrong glibc version on very old systems),
|
except OSError:
|
||||||
# we simply ignore it and hope the system has the library.
|
# If it fails (e.g. wrong glibc version on very old systems),
|
||||||
pass
|
# we simply ignore it and hope the system has the library.
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
# Set application ID for Windows taskbar icon
|
# Set application ID for Windows taskbar icon
|
||||||
if platform.system() == "Windows":
|
if platform.system() == "Windows":
|
||||||
try:
|
with timed_log("Windows AppUserModelID", logger=_log):
|
||||||
from abogen.constants import PROGRAM_NAME, VERSION
|
try:
|
||||||
import ctypes
|
from abogen.constants import PROGRAM_NAME, VERSION
|
||||||
|
import ctypes
|
||||||
|
|
||||||
app_id = f"{PROGRAM_NAME}.{VERSION}"
|
app_id = f"{PROGRAM_NAME}.{VERSION}"
|
||||||
ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID(app_id)
|
ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID(app_id)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print("Warning: failed to set AppUserModelID:", e)
|
_log.warning("Failed to set AppUserModelID: %s", e)
|
||||||
|
|
||||||
from PyQt6.QtWidgets import QApplication
|
with timed_log("PyQt6 imports", logger=_log):
|
||||||
from PyQt6.QtGui import QIcon
|
from PyQt6.QtWidgets import QApplication
|
||||||
from PyQt6.QtCore import (
|
from PyQt6.QtGui import QIcon
|
||||||
QLibraryInfo,
|
from PyQt6.QtCore import (
|
||||||
qInstallMessageHandler,
|
QLibraryInfo,
|
||||||
QtMsgType,
|
qInstallMessageHandler,
|
||||||
)
|
QtMsgType,
|
||||||
|
)
|
||||||
|
|
||||||
# Add the directory to Python path
|
# Add the directory to Python path
|
||||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__)))
|
sys.path.insert(0, os.path.join(os.path.dirname(__file__)))
|
||||||
|
|
||||||
# Set Hugging Face Hub environment variables
|
# Set Hugging Face Hub environment variables
|
||||||
os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1" # Disable Hugging Face telemetry
|
with timed_log("config load + HF env setup", logger=_log):
|
||||||
os.environ["HF_HUB_ETAG_TIMEOUT"] = "10" # Metadata request timeout (seconds)
|
os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1" # Disable Hugging Face telemetry
|
||||||
os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = "10" # File download timeout (seconds)
|
os.environ["HF_HUB_ETAG_TIMEOUT"] = "10" # Metadata request timeout (seconds)
|
||||||
os.environ["HF_HUB_DISABLE_SYMLINKS_WARNING"] = "1" # Disable symlinks warning
|
os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = "10" # File download timeout (seconds)
|
||||||
from abogen.utils import load_config
|
os.environ["HF_HUB_DISABLE_SYMLINKS_WARNING"] = "1" # Disable symlinks warning
|
||||||
if load_config().get("disable_kokoro_internet", False):
|
from abogen.utils import load_config
|
||||||
print("INFO: Kokoro's internet access is disabled.")
|
if load_config().get("disable_kokoro_internet", False):
|
||||||
os.environ["HF_HUB_OFFLINE"] = "1" # Disable Hugging Face Hub internet access
|
_log.info("Kokoro's internet access is disabled.")
|
||||||
|
os.environ["HF_HUB_OFFLINE"] = "1" # Disable Hugging Face Hub internet access
|
||||||
|
|
||||||
from abogen.pyqt.gui import abogen
|
with timed_log("GUI module import (abogen.pyqt.gui)", logger=_log):
|
||||||
from abogen.constants import PROGRAM_NAME, VERSION
|
from abogen.pyqt.gui import abogen
|
||||||
|
from abogen.constants import PROGRAM_NAME, VERSION
|
||||||
|
|
||||||
# Set environment variables for AMD ROCm
|
# Set environment variables for AMD ROCm
|
||||||
os.environ["MIOPEN_FIND_MODE"] = "FAST"
|
os.environ["MIOPEN_FIND_MODE"] = "FAST"
|
||||||
@@ -150,7 +161,11 @@ if platform.system() == "Linux":
|
|||||||
|
|
||||||
def main():
|
def main():
|
||||||
"""Main entry point for console usage."""
|
"""Main entry point for console usage."""
|
||||||
app = QApplication(sys.argv)
|
with timed_log("QApplication creation", logger=_log):
|
||||||
|
app = QApplication(sys.argv)
|
||||||
|
|
||||||
|
# Qt shutdown hook must be connected AFTER QApplication exists
|
||||||
|
shutdown.install_qt_hook()
|
||||||
|
|
||||||
# Set application icon using get_resource_path from utils
|
# Set application icon using get_resource_path from utils
|
||||||
icon_path = get_resource_path("abogen.assets", "icon.ico")
|
icon_path = get_resource_path("abogen.assets", "icon.ico")
|
||||||
@@ -164,9 +179,16 @@ def main():
|
|||||||
except AttributeError:
|
except AttributeError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
ex = abogen()
|
with timed_log("main window construction", logger=_log):
|
||||||
ex.show()
|
ex = abogen()
|
||||||
sys.exit(app.exec())
|
with timed_log("window show", logger=_log):
|
||||||
|
ex.show()
|
||||||
|
_log.info("App startup complete. Showing window.")
|
||||||
|
rc = app.exec()
|
||||||
|
# Restore the default Qt message handler BEFORE interpreter shutdown.
|
||||||
|
# A Python message handler invoked during Qt teardown segfaults (SIGSEGV).
|
||||||
|
qInstallMessageHandler(None)
|
||||||
|
sys.exit(rc)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
@@ -523,7 +523,7 @@ class QueueManager(QDialog):
|
|||||||
return attrs
|
return attrs
|
||||||
|
|
||||||
def add_files_from_paths(self, file_paths):
|
def add_files_from_paths(self, file_paths):
|
||||||
from abogen.subtitle_utils import calculate_text_length
|
from abogen.domain.text_utils import calculate_text_length
|
||||||
from PyQt6.QtWidgets import QMessageBox
|
from PyQt6.QtWidgets import QMessageBox
|
||||||
import os
|
import os
|
||||||
|
|
||||||
|
|||||||
@@ -28,7 +28,6 @@ from PyQt6.QtWidgets import (
|
|||||||
from PyQt6.QtCore import Qt, QTimer, QPoint, QRect, QSize
|
from PyQt6.QtCore import Qt, QTimer, QPoint, QRect, QSize
|
||||||
from PyQt6.QtGui import QPixmap, QIcon, QAction
|
from PyQt6.QtGui import QPixmap, QIcon, QAction
|
||||||
from abogen.constants import (
|
from abogen.constants import (
|
||||||
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
|
|
||||||
LANGUAGE_DESCRIPTIONS,
|
LANGUAGE_DESCRIPTIONS,
|
||||||
COLORS,
|
COLORS,
|
||||||
)
|
)
|
||||||
@@ -949,7 +948,9 @@ class VoiceFormulaDialog(QDialog):
|
|||||||
lang = state.get("language") if isinstance(state, dict) else None
|
lang = state.get("language") if isinstance(state, dict) else None
|
||||||
# apply language selection
|
# apply language selection
|
||||||
if lang:
|
if lang:
|
||||||
i = self.language_combo.findData(lang)
|
from abogen.voice_profiles import resolve_profile_language
|
||||||
|
|
||||||
|
i = self.language_combo.findData(resolve_profile_language(state))
|
||||||
if i >= 0:
|
if i >= 0:
|
||||||
self.language_combo.blockSignals(True)
|
self.language_combo.blockSignals(True)
|
||||||
self.language_combo.setCurrentIndex(i)
|
self.language_combo.setCurrentIndex(i)
|
||||||
@@ -1571,9 +1572,10 @@ class VoiceFormulaDialog(QDialog):
|
|||||||
parent.selected_profile_name = None
|
parent.selected_profile_name = None
|
||||||
lang = self.language_combo.currentData()
|
lang = self.language_combo.currentData()
|
||||||
parent.selected_lang = lang
|
parent.selected_lang = lang
|
||||||
parent.subtitle_combo.setEnabled(
|
if hasattr(parent, "update_subtitle_options_availability"):
|
||||||
lang in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION
|
parent.update_subtitle_options_availability()
|
||||||
)
|
else:
|
||||||
|
parent.subtitle_combo.setEnabled(True)
|
||||||
# Reset start flag and trigger preview
|
# Reset start flag and trigger preview
|
||||||
self._started = False
|
self._started = False
|
||||||
parent.preview_voice()
|
parent.preview_voice()
|
||||||
|
|||||||
+56
-54
@@ -1,12 +1,27 @@
|
|||||||
"""Graceful shutdown - single module, no over-engineering."""
|
"""Graceful shutdown — process-level hooks and orchestration.
|
||||||
|
|
||||||
|
Responsibilities:
|
||||||
|
- Install atexit/signal/Qt hooks
|
||||||
|
- Stop WebUI ConversionService (worker thread)
|
||||||
|
- Restore sleep prevention
|
||||||
|
- Terminate child processes (ffmpeg, etc.)
|
||||||
|
- Delegate GPU/engine/UI cleanup to application.cleanup
|
||||||
|
|
||||||
|
App-layer cleanup (GPU, engines, UI callbacks) lives in application/cleanup.py.
|
||||||
|
Per-conversion cleanup lives in run_conversion() finally block.
|
||||||
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import atexit
|
import atexit
|
||||||
import gc
|
import logging
|
||||||
import signal
|
import signal
|
||||||
import sys
|
import sys
|
||||||
|
import time
|
||||||
from typing import Callable
|
from typing import Callable
|
||||||
|
|
||||||
|
_log = logging.getLogger("abogen.shutdown")
|
||||||
|
|
||||||
_CLEANUP_FUNCS: list[Callable[[], None]] = []
|
_CLEANUP_FUNCS: list[Callable[[], None]] = []
|
||||||
_EXECUTED = False
|
_EXECUTED = False
|
||||||
|
|
||||||
@@ -21,27 +36,24 @@ def _run_cleanups() -> None:
|
|||||||
if _EXECUTED:
|
if _EXECUTED:
|
||||||
return
|
return
|
||||||
_EXECUTED = True
|
_EXECUTED = True
|
||||||
|
_log.info("Shutdown: starting %d cleanup hook(s)", len(_CLEANUP_FUNCS))
|
||||||
for fn in _CLEANUP_FUNCS:
|
for fn in _CLEANUP_FUNCS:
|
||||||
|
start = time.perf_counter()
|
||||||
try:
|
try:
|
||||||
fn()
|
fn()
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
_log.info(
|
||||||
|
"Shutdown: %s done in %.2fs", fn.__name__, time.perf_counter() - start
|
||||||
|
)
|
||||||
|
_log.info("Shutdown: all cleanups finished")
|
||||||
|
|
||||||
|
|
||||||
# ---- Register built-in cleanup functions ----
|
# ---- Process-level cleanup functions ----
|
||||||
|
|
||||||
# 1. Restore sleep prevention
|
|
||||||
def _restore_sleep() -> None:
|
|
||||||
try:
|
|
||||||
from abogen.utils import prevent_sleep_end
|
|
||||||
prevent_sleep_end()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
register_cleanup(_restore_sleep)
|
def _stop_conversion_service() -> None:
|
||||||
|
"""Stop WebUI ConversionService worker thread."""
|
||||||
# 2. Shutdown web UI ConversionService
|
|
||||||
def _shutdown_conversion_service() -> None:
|
|
||||||
try:
|
try:
|
||||||
from abogen.webui.service import get_service
|
from abogen.webui.service import get_service
|
||||||
svc = get_service()
|
svc = get_service()
|
||||||
@@ -50,50 +62,18 @@ def _shutdown_conversion_service() -> None:
|
|||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
register_cleanup(_shutdown_conversion_service)
|
|
||||||
|
|
||||||
# 3. Clear TTS pipelines and GPU memory
|
def _restore_sleep() -> None:
|
||||||
def _cleanup_tts_pipelines() -> None:
|
"""Restore system sleep prevention (caffeinate/systemd-inhibit/Windows)."""
|
||||||
# Clear web UI pipeline cache
|
|
||||||
try:
|
try:
|
||||||
from abogen.webui.conversion_runner import _PIPELINES
|
from abogen.utils import prevent_sleep_end
|
||||||
_PIPELINES.clear()
|
prevent_sleep_end()
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
# Clear PyQt conversion thread voice cache
|
|
||||||
try:
|
|
||||||
from abogen.pyqt.conversion import ConversionThread
|
|
||||||
if hasattr(ConversionThread, "voice_cache"):
|
|
||||||
ConversionThread.voice_cache.clear()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
gc.collect()
|
|
||||||
|
|
||||||
# Release CUDA cache
|
|
||||||
try:
|
|
||||||
import torch
|
|
||||||
if torch.cuda.is_available():
|
|
||||||
torch.cuda.empty_cache()
|
|
||||||
torch.cuda.ipc_collect()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
register_cleanup(_cleanup_tts_pipelines)
|
|
||||||
|
|
||||||
# 4. Clear global voice cache
|
|
||||||
def _clear_voice_cache() -> None:
|
|
||||||
try:
|
|
||||||
from abogen.voice_cache import clear_voice_cache
|
|
||||||
clear_voice_cache()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
register_cleanup(_clear_voice_cache)
|
|
||||||
|
|
||||||
# 5. Terminate child processes (ffmpeg, etc.)
|
|
||||||
def _terminate_subprocesses() -> None:
|
def _terminate_subprocesses() -> None:
|
||||||
|
"""Terminate all child processes (ffmpeg, etc.)."""
|
||||||
try:
|
try:
|
||||||
import psutil
|
import psutil
|
||||||
except Exception:
|
except Exception:
|
||||||
@@ -115,6 +95,20 @@ def _terminate_subprocesses() -> None:
|
|||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _app_cleanup() -> None:
|
||||||
|
"""Delegate to application-layer cleanup (engines, GPU, UI callbacks)."""
|
||||||
|
try:
|
||||||
|
from abogen.application.cleanup import cleanup
|
||||||
|
cleanup()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
# Register in execution order
|
||||||
|
register_cleanup(_stop_conversion_service)
|
||||||
|
register_cleanup(_app_cleanup)
|
||||||
|
register_cleanup(_restore_sleep)
|
||||||
register_cleanup(_terminate_subprocesses)
|
register_cleanup(_terminate_subprocesses)
|
||||||
|
|
||||||
|
|
||||||
@@ -133,13 +127,19 @@ def register_shutdown() -> None:
|
|||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
# Qt hook
|
install_qt_hook()
|
||||||
|
|
||||||
|
|
||||||
|
def install_qt_hook() -> None:
|
||||||
|
"""Connect Qt aboutToQuit to cleanup. Must run AFTER QApplication is created."""
|
||||||
try:
|
try:
|
||||||
from PyQt6.QtWidgets import QApplication
|
from PyQt6.QtWidgets import QApplication
|
||||||
|
|
||||||
app = QApplication.instance()
|
app = QApplication.instance()
|
||||||
if app is not None:
|
if app is not None and not getattr(app, "_abogen_cleanup_connected", False):
|
||||||
app.aboutToQuit.connect(_run_cleanups)
|
app.aboutToQuit.connect(_run_cleanups)
|
||||||
|
app._abogen_cleanup_connected = True
|
||||||
|
_log.info("Shutdown: Qt aboutToQuit hook connected")
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
@@ -148,13 +148,15 @@ register_shutdown._registered = False
|
|||||||
|
|
||||||
|
|
||||||
def _on_signal(signum: int, _frame) -> None:
|
def _on_signal(signum: int, _frame) -> None:
|
||||||
|
_log.info("Shutdown: signal %s received", signum)
|
||||||
_run_cleanups()
|
_run_cleanups()
|
||||||
sys.exit(0)
|
sys.exit(0)
|
||||||
|
|
||||||
|
|
||||||
def request_shutdown() -> None:
|
def request_shutdown() -> None:
|
||||||
"""Programmatically trigger cleanup (e.g., from GUI closeEvent)."""
|
"""Programmatically trigger cleanup (e.g., from GUI closeEvent)."""
|
||||||
|
_log.info("Shutdown: cleanup requested")
|
||||||
_run_cleanups()
|
_run_cleanups()
|
||||||
|
|
||||||
|
|
||||||
__all__ = ["register_shutdown", "request_shutdown", "register_cleanup"]
|
__all__ = ["register_shutdown", "install_qt_hook", "request_shutdown", "register_cleanup"]
|
||||||
|
|||||||
@@ -6,10 +6,9 @@ from dataclasses import dataclass
|
|||||||
from functools import lru_cache
|
from functools import lru_cache
|
||||||
from typing import Any, Dict, Optional, Tuple
|
from typing import Any, Dict, Optional, Tuple
|
||||||
|
|
||||||
try: # pragma: no cover - optional dependency
|
# spaCy is intentionally NOT imported at module level: importing it pulls in
|
||||||
import spacy
|
# thinc -> torch, which costs seconds of startup time. It is imported lazily
|
||||||
except Exception: # pragma: no cover - spaCy unavailable at runtime
|
# inside _load_spacy_model below.
|
||||||
spacy = None
|
|
||||||
|
|
||||||
# Lazy spaCy type hints to avoid a hard dependency at import time.
|
# Lazy spaCy type hints to avoid a hard dependency at import time.
|
||||||
Language = Any # type: ignore[assignment]
|
Language = Any # type: ignore[assignment]
|
||||||
@@ -37,7 +36,9 @@ _DEFAULT_MODEL = os.environ.get("ABOGEN_SPACY_MODEL", "en_core_web_sm")
|
|||||||
|
|
||||||
@lru_cache(maxsize=1)
|
@lru_cache(maxsize=1)
|
||||||
def _load_spacy_model(model: str = _DEFAULT_MODEL) -> Optional[Language]:
|
def _load_spacy_model(model: str = _DEFAULT_MODEL) -> Optional[Language]:
|
||||||
if spacy is None:
|
try: # pragma: no cover - optional dependency
|
||||||
|
import spacy
|
||||||
|
except Exception: # pragma: no cover - spaCy unavailable at runtime
|
||||||
logger.debug("spaCy is not installed; skipping contraction disambiguation")
|
logger.debug("spaCy is not installed; skipping contraction disambiguation")
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|||||||
+16
-40
@@ -21,20 +21,6 @@ SPACY_MODELS = {
|
|||||||
Language.HI: "xx_sent_ud_sm",
|
Language.HI: "xx_sent_ud_sm",
|
||||||
}
|
}
|
||||||
|
|
||||||
# Kokoro single-letter codes -> Language enum (inverse of pipeline_factory._KOKORO_LANG_MAP)
|
|
||||||
_KOKORO_TO_LANGUAGE = {
|
|
||||||
"a": Language.EN_US,
|
|
||||||
"b": Language.EN_GB,
|
|
||||||
"e": Language.ES,
|
|
||||||
"f": Language.FR,
|
|
||||||
"h": Language.HI,
|
|
||||||
"i": Language.IT,
|
|
||||||
"j": Language.JA,
|
|
||||||
"p": Language.PT_BR,
|
|
||||||
"z": Language.ZH,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _load_spacy():
|
def _load_spacy():
|
||||||
"""Lazy load spaCy module."""
|
"""Lazy load spaCy module."""
|
||||||
global _spacy
|
global _spacy
|
||||||
@@ -48,12 +34,12 @@ def _load_spacy():
|
|||||||
return _spacy
|
return _spacy
|
||||||
|
|
||||||
|
|
||||||
def get_spacy_model(lang_code, log_callback=None):
|
def get_spacy_model(language: Language, log_callback=None):
|
||||||
"""
|
"""
|
||||||
Get or load a spaCy model for the given language code.
|
Get or load a spaCy model for the given language.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
lang_code: Language code or Language enum (e.g., "a", "en-US", Language.EN_US)
|
language: Language enum value.
|
||||||
log_callback: Optional function to log messages
|
log_callback: Optional function to log messages
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
@@ -61,36 +47,26 @@ def get_spacy_model(lang_code, log_callback=None):
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
def log(msg, is_error=False):
|
def log(msg, is_error=False):
|
||||||
# Prefer GUI log callback when provided to avoid spamming stdout.
|
|
||||||
if log_callback:
|
if log_callback:
|
||||||
color = "red" if is_error else "grey"
|
color = "red" if is_error else "grey"
|
||||||
try:
|
try:
|
||||||
log_callback((msg, color))
|
log_callback((msg, color))
|
||||||
except Exception:
|
except Exception:
|
||||||
# Fallback to printing if callback misbehaves
|
|
||||||
print(msg)
|
print(msg)
|
||||||
else:
|
else:
|
||||||
print(msg)
|
print(msg)
|
||||||
|
|
||||||
# Normalize to Language enum
|
if not isinstance(language, Language):
|
||||||
if not isinstance(lang_code, Language):
|
raise TypeError(
|
||||||
if isinstance(lang_code, str) and lang_code in _KOKORO_TO_LANGUAGE:
|
f"language must be Language enum, got {type(language).__name__}: {language!r}"
|
||||||
lang_code = _KOKORO_TO_LANGUAGE[lang_code]
|
)
|
||||||
else:
|
|
||||||
try:
|
|
||||||
lang_code = Language.from_str(lang_code)
|
|
||||||
except ValueError:
|
|
||||||
log(f"\nspaCy: Unknown language '{lang_code}'...")
|
|
||||||
return None
|
|
||||||
|
|
||||||
# Check if model is cached
|
if language in _nlp_cache:
|
||||||
if lang_code in _nlp_cache:
|
return _nlp_cache[language]
|
||||||
return _nlp_cache[lang_code]
|
|
||||||
|
|
||||||
# Check if language is supported
|
model_name = SPACY_MODELS.get(language)
|
||||||
model_name = SPACY_MODELS.get(lang_code)
|
|
||||||
if not model_name:
|
if not model_name:
|
||||||
log(f"\nspaCy: No model mapping for language '{lang_code}'...")
|
log(f"\nspaCy: No model mapping for language '{language}'...")
|
||||||
return None
|
return None
|
||||||
|
|
||||||
# Lazy load spaCy
|
# Lazy load spaCy
|
||||||
@@ -114,7 +90,7 @@ def get_spacy_model(lang_code, log_callback=None):
|
|||||||
if "parser" not in nlp.pipe_names and "sentencizer" not in nlp.pipe_names:
|
if "parser" not in nlp.pipe_names and "sentencizer" not in nlp.pipe_names:
|
||||||
nlp.add_pipe("sentencizer")
|
nlp.add_pipe("sentencizer")
|
||||||
|
|
||||||
_nlp_cache[lang_code] = nlp
|
_nlp_cache[language] = nlp
|
||||||
return nlp
|
return nlp
|
||||||
except OSError:
|
except OSError:
|
||||||
# Model not found, attempt download
|
# Model not found, attempt download
|
||||||
@@ -131,7 +107,7 @@ def get_spacy_model(lang_code, log_callback=None):
|
|||||||
if "parser" not in nlp.pipe_names and "sentencizer" not in nlp.pipe_names:
|
if "parser" not in nlp.pipe_names and "sentencizer" not in nlp.pipe_names:
|
||||||
nlp.add_pipe("sentencizer")
|
nlp.add_pipe("sentencizer")
|
||||||
|
|
||||||
_nlp_cache[lang_code] = nlp
|
_nlp_cache[language] = nlp
|
||||||
log(f"spaCy model '{model_name}' downloaded and loaded")
|
log(f"spaCy model '{model_name}' downloaded and loaded")
|
||||||
return nlp
|
return nlp
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -145,19 +121,19 @@ def get_spacy_model(lang_code, log_callback=None):
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
def segment_sentences(text, lang_code, log_callback=None):
|
def segment_sentences(text, language: Language, log_callback=None):
|
||||||
"""
|
"""
|
||||||
Segment text into sentences using spaCy.
|
Segment text into sentences using spaCy.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
text: Text to segment
|
text: Text to segment
|
||||||
lang_code: Language code
|
language: Language enum value
|
||||||
log_callback: Optional function to log messages
|
log_callback: Optional function to log messages
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
List of sentence strings, or None if spaCy unavailable
|
List of sentence strings, or None if spaCy unavailable
|
||||||
"""
|
"""
|
||||||
nlp = get_spacy_model(lang_code, log_callback)
|
nlp = get_spacy_model(language, log_callback)
|
||||||
if nlp is None:
|
if nlp is None:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ import json
|
|||||||
import os
|
import os
|
||||||
from typing import Any, Dict, List, Optional
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
from abogen.constants import LANGUAGE_DESCRIPTIONS
|
from abogen.constants import KOKORO_CODE_LABELS
|
||||||
from abogen.utils import get_user_config_path
|
from abogen.utils import get_user_config_path
|
||||||
|
|
||||||
_CONFIG_WRAPPER_KEY = "abogen_speaker_configs"
|
_CONFIG_WRAPPER_KEY = "abogen_speaker_configs"
|
||||||
@@ -163,4 +163,4 @@ def list_configs() -> List[Dict[str, Any]]:
|
|||||||
|
|
||||||
def describe_language(code: str) -> str:
|
def describe_language(code: str) -> str:
|
||||||
code = (code or "a").lower()
|
code = (code or "a").lower()
|
||||||
return LANGUAGE_DESCRIPTIONS.get(code, code.upper())
|
return KOKORO_CODE_LABELS.get(code, code.upper())
|
||||||
|
|||||||
+19
-199
@@ -1,7 +1,7 @@
|
|||||||
import re
|
import re
|
||||||
import platform
|
|
||||||
from abogen.utils import detect_encoding, load_config
|
from abogen.utils import detect_encoding, load_config
|
||||||
from abogen.constants import SAMPLE_VOICE_TEXTS
|
from abogen.constants import SAMPLE_VOICE_TEXTS
|
||||||
|
from abogen.domain.enums import Language
|
||||||
|
|
||||||
# Pre-compile frequently used regex patterns for better performance
|
# Pre-compile frequently used regex patterns for better performance
|
||||||
_METADATA_TAG_PATTERN = re.compile(r"<<METADATA_[^:]+:[^>]*>>")
|
_METADATA_TAG_PATTERN = re.compile(r"<<METADATA_[^:]+:[^>]*>>")
|
||||||
@@ -23,13 +23,6 @@ _VTT_NOTE_PATTERN = re.compile(r"NOTE\s*\n.*?(?=\n\n|$)", re.DOTALL)
|
|||||||
_DOUBLE_NEWLINE_SPLIT_PATTERN = re.compile(r"\n\s*\n")
|
_DOUBLE_NEWLINE_SPLIT_PATTERN = re.compile(r"\n\s*\n")
|
||||||
_VTT_TIMESTAMP_PATTERN = re.compile(r"([\d:.]+)\s*-->\s*([\d:.]+)")
|
_VTT_TIMESTAMP_PATTERN = re.compile(r"([\d:.]+)\s*-->\s*([\d:.]+)")
|
||||||
_TIMESTAMP_ONLY_PATTERN = re.compile(r"^(\d{1,2}:\d{2}:\d{2}(?:[.,]\d{1,3})?)$")
|
_TIMESTAMP_ONLY_PATTERN = re.compile(r"^(\d{1,2}:\d{2}:\d{2}(?:[.,]\d{1,3})?)$")
|
||||||
_WINDOWS_ILLEGAL_CHARS_PATTERN = re.compile(r'[<>:"/\\|?*]')
|
|
||||||
_CONTROL_CHARS_PATTERN = re.compile(r"[\x00-\x1f]")
|
|
||||||
_LINUX_CONTROL_CHARS_PATTERN = re.compile(
|
|
||||||
r"[\x01-\x1f]"
|
|
||||||
) # Linux: exclude \x00 for separate handling
|
|
||||||
_MACOS_ILLEGAL_CHARS_PATTERN = re.compile(r"[:]")
|
|
||||||
_LINUX_ILLEGAL_CHARS_PATTERN = re.compile(r"[/\x00]")
|
|
||||||
|
|
||||||
|
|
||||||
def clean_subtitle_text(text):
|
def clean_subtitle_text(text):
|
||||||
@@ -41,17 +34,6 @@ def clean_subtitle_text(text):
|
|||||||
return text.strip()
|
return text.strip()
|
||||||
|
|
||||||
|
|
||||||
def calculate_text_length(text):
|
|
||||||
# Use pre-compiled patterns for better performance
|
|
||||||
# Ignore chapter markers, voice markers, and metadata patterns in a single pass
|
|
||||||
text = _CHAPTER_MARKER_PATTERN.sub("", text)
|
|
||||||
text = _VOICE_MARKER_PATTERN.sub("", text)
|
|
||||||
text = _METADATA_TAG_PATTERN.sub("", text)
|
|
||||||
# Ignore newlines and leading/trailing spaces
|
|
||||||
text = text.replace("\n", "").strip()
|
|
||||||
# Calculate character count
|
|
||||||
char_count = len(text)
|
|
||||||
return char_count
|
|
||||||
|
|
||||||
|
|
||||||
def clean_text(text, *args, **kwargs):
|
def clean_text(text, *args, **kwargs):
|
||||||
@@ -396,189 +378,27 @@ def parse_ass_file(file_path):
|
|||||||
return subtitles
|
return subtitles
|
||||||
|
|
||||||
|
|
||||||
def get_sample_voice_text(lang_code):
|
def get_sample_voice_text(language):
|
||||||
return SAMPLE_VOICE_TEXTS.get(lang_code, SAMPLE_VOICE_TEXTS["a"])
|
"""Get sample voice text for a language.
|
||||||
|
|
||||||
|
|
||||||
def sanitize_name_for_os(name, is_folder=True):
|
|
||||||
"""
|
|
||||||
Sanitize a filename or folder name based on the operating system.
|
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
name: The name to sanitize
|
language: Language enum value or string (for backward compatibility).
|
||||||
is_folder: Whether this is a folder name (default: True)
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Sanitized name safe for the current OS
|
|
||||||
"""
|
"""
|
||||||
if not name:
|
if isinstance(language, str):
|
||||||
return "audiobook"
|
try:
|
||||||
|
language = Language.from_str(language)
|
||||||
system = platform.system()
|
except (ValueError, AttributeError):
|
||||||
|
language = Language.EN_US
|
||||||
if system == "Windows":
|
return SAMPLE_VOICE_TEXTS.get(language, SAMPLE_VOICE_TEXTS[Language.EN_US])
|
||||||
# Windows illegal characters: < > : " / \ | ? *
|
|
||||||
# Also can't end with space or dot
|
|
||||||
# Use pre-compiled pattern for better performance
|
|
||||||
sanitized = _WINDOWS_ILLEGAL_CHARS_PATTERN.sub("_", name)
|
|
||||||
# Remove control characters (0-31)
|
|
||||||
sanitized = _CONTROL_CHARS_PATTERN.sub("_", sanitized)
|
|
||||||
# Remove trailing spaces and dots
|
|
||||||
sanitized = sanitized.rstrip(". ")
|
|
||||||
# Windows reserved names (CON, PRN, AUX, NUL, COM1-9, LPT1-9)
|
|
||||||
reserved = (
|
|
||||||
["CON", "PRN", "AUX", "NUL"]
|
|
||||||
+ [f"COM{i}" for i in range(1, 10)]
|
|
||||||
+ [f"LPT{i}" for i in range(1, 10)]
|
|
||||||
)
|
|
||||||
if sanitized.upper() in reserved or sanitized.upper().split(".")[0] in reserved:
|
|
||||||
sanitized = f"_{sanitized}"
|
|
||||||
elif system == "Darwin": # macOS
|
|
||||||
# macOS illegal characters: : (colon is converted to / by the system)
|
|
||||||
# Also can't start with dot (hidden file) for folders typically
|
|
||||||
# Use pre-compiled pattern for better performance
|
|
||||||
sanitized = _MACOS_ILLEGAL_CHARS_PATTERN.sub("_", name)
|
|
||||||
# Remove control characters
|
|
||||||
sanitized = _CONTROL_CHARS_PATTERN.sub("_", sanitized)
|
|
||||||
# Avoid leading dot for folders (creates hidden folders)
|
|
||||||
if is_folder and sanitized.startswith("."):
|
|
||||||
sanitized = "_" + sanitized[1:]
|
|
||||||
else: # Linux and others
|
|
||||||
# Linux illegal characters: / and null character
|
|
||||||
# Though / is illegal, most other chars are technically allowed
|
|
||||||
# Use pre-compiled pattern for better performance
|
|
||||||
sanitized = _LINUX_ILLEGAL_CHARS_PATTERN.sub("_", name)
|
|
||||||
# Remove other control characters for safety (excluding \x00 which is already handled)
|
|
||||||
sanitized = _LINUX_CONTROL_CHARS_PATTERN.sub("_", sanitized)
|
|
||||||
# Avoid leading dot for folders (creates hidden folders)
|
|
||||||
if is_folder and sanitized.startswith("."):
|
|
||||||
sanitized = "_" + sanitized[1:]
|
|
||||||
|
|
||||||
# Ensure the name is not empty after sanitization
|
|
||||||
if not sanitized or sanitized.strip() == "":
|
|
||||||
sanitized = "audiobook"
|
|
||||||
|
|
||||||
# Limit length to 255 characters (common limit across filesystems)
|
|
||||||
if len(sanitized) > 255:
|
|
||||||
sanitized = sanitized[:255].rstrip(". ")
|
|
||||||
|
|
||||||
return sanitized
|
|
||||||
|
|
||||||
|
|
||||||
def validate_voice_name(voice_name):
|
# Backward-compatible re-exports — canonical location is domain/output_paths.py
|
||||||
"""Validate voice name against available voices (case-insensitive).
|
from abogen.domain.output_paths import sanitize_name_for_os # noqa: E402, F401
|
||||||
Handles both single voices and formulas like 'af_heart*0.5 + am_echo*0.5'.
|
|
||||||
|
|
||||||
Args:
|
# Backward-compatible re-exports — canonical location is domain/voice_markers.py
|
||||||
voice_name: Voice name or formula string to validate
|
from abogen.domain.voice_markers import ( # noqa: E402, F401
|
||||||
|
validate_voice_name,
|
||||||
Returns:
|
split_text_by_voice_markers,
|
||||||
Tuple of (is_valid, invalid_voice_name):
|
_VOICE_MARKER_PATTERN,
|
||||||
- is_valid: True if all voices in the name/formula are valid
|
_VOICE_MARKER_SEARCH_PATTERN,
|
||||||
- invalid_voice_name: The first invalid voice found, or None if all valid
|
)
|
||||||
"""
|
|
||||||
from abogen.tts_plugin.utils import get_voices
|
|
||||||
|
|
||||||
# Create case-insensitive lookup set (done once per call)
|
|
||||||
voice_lookup_lower = {v.lower() for v in get_voices("kokoro")}
|
|
||||||
voice_name = voice_name.strip()
|
|
||||||
|
|
||||||
# Check if it's a formula (contains *)
|
|
||||||
if "*" in voice_name:
|
|
||||||
# Extract voice names from formula
|
|
||||||
voices = voice_name.split("+")
|
|
||||||
for term in voices:
|
|
||||||
if "*" in term:
|
|
||||||
base_voice = term.split("*")[0].strip()
|
|
||||||
# Case-insensitive comparison
|
|
||||||
if base_voice.lower() not in voice_lookup_lower:
|
|
||||||
return False, base_voice
|
|
||||||
return True, None
|
|
||||||
else:
|
|
||||||
# Single voice - case-insensitive comparison
|
|
||||||
if voice_name.lower() not in voice_lookup_lower:
|
|
||||||
return False, voice_name
|
|
||||||
return True, None
|
|
||||||
|
|
||||||
|
|
||||||
def split_text_by_voice_markers(text, default_voice):
|
|
||||||
"""Split text by voice markers, returning list of (voice, text) tuples.
|
|
||||||
|
|
||||||
IMPORTANT: Returns the last voice used so it can persist across chapters.
|
|
||||||
Voice names are normalized to lowercase to match canonical voice names.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
text: Text potentially containing <<VOICE:name>> markers
|
|
||||||
default_voice: Voice to use if no markers found or before first marker
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Tuple of (segments_list, last_voice_used, valid_count, invalid_count):
|
|
||||||
- segments_list: List of (voice_name, segment_text) tuples
|
|
||||||
- last_voice_used: The voice that should continue into next chapter
|
|
||||||
- valid_count: Number of valid voice markers processed
|
|
||||||
- invalid_count: Number of invalid voice markers skipped
|
|
||||||
"""
|
|
||||||
from abogen.tts_plugin.utils import get_voices
|
|
||||||
|
|
||||||
voice_splits = list(_VOICE_MARKER_SEARCH_PATTERN.finditer(text))
|
|
||||||
|
|
||||||
if not voice_splits:
|
|
||||||
# No voice markers, return entire text with default voice
|
|
||||||
return [(default_voice, text)], default_voice, 0, 0
|
|
||||||
|
|
||||||
segments = []
|
|
||||||
current_voice = default_voice
|
|
||||||
valid_markers = 0
|
|
||||||
invalid_markers = 0
|
|
||||||
|
|
||||||
# Text before first marker uses default voice
|
|
||||||
first_start = voice_splits[0].start()
|
|
||||||
if first_start > 0:
|
|
||||||
intro_text = text[:first_start].strip()
|
|
||||||
if intro_text:
|
|
||||||
segments.append((current_voice, intro_text))
|
|
||||||
|
|
||||||
# Process each voice marker
|
|
||||||
for idx, match in enumerate(voice_splits):
|
|
||||||
voice_name = match.group(1).strip()
|
|
||||||
start = match.end()
|
|
||||||
end = voice_splits[idx + 1].start() if idx + 1 < len(voice_splits) else len(text)
|
|
||||||
segment_text = text[start:end].strip()
|
|
||||||
|
|
||||||
# Validate voice name
|
|
||||||
is_valid, invalid_voice = validate_voice_name(voice_name)
|
|
||||||
if is_valid:
|
|
||||||
# Normalize to lowercase to match canonical form
|
|
||||||
# Handle both single voices and formulas
|
|
||||||
if "*" in voice_name:
|
|
||||||
# Normalize each voice in the formula
|
|
||||||
normalized_parts = []
|
|
||||||
for part in voice_name.split("+"):
|
|
||||||
part = part.strip()
|
|
||||||
if "*" in part:
|
|
||||||
voice_part, weight = part.split("*", 1)
|
|
||||||
# Find the canonical (lowercase) voice name
|
|
||||||
voice_part_lower = voice_part.strip().lower()
|
|
||||||
canonical_voice = next(
|
|
||||||
(v for v in get_voices("kokoro") if v.lower() == voice_part_lower),
|
|
||||||
voice_part.strip()
|
|
||||||
)
|
|
||||||
normalized_parts.append(f"{canonical_voice}*{weight.strip()}")
|
|
||||||
current_voice = " + ".join(normalized_parts)
|
|
||||||
else:
|
|
||||||
# Find the canonical (lowercase) voice name
|
|
||||||
voice_name_lower = voice_name.lower()
|
|
||||||
current_voice = next(
|
|
||||||
(v for v in get_voices("kokoro") if v.lower() == voice_name_lower),
|
|
||||||
voice_name
|
|
||||||
)
|
|
||||||
valid_markers += 1
|
|
||||||
else:
|
|
||||||
# Invalid voice - stay with previous voice
|
|
||||||
invalid_markers += 1
|
|
||||||
|
|
||||||
if segment_text:
|
|
||||||
segments.append((current_voice, segment_text))
|
|
||||||
|
|
||||||
# Return segments, last voice, and counts
|
|
||||||
return segments, current_voice, valid_markers, invalid_markers
|
|
||||||
|
|||||||
@@ -16,7 +16,8 @@ import markdown # type: ignore[import]
|
|||||||
from bs4 import BeautifulSoup, NavigableString # type: ignore[import]
|
from bs4 import BeautifulSoup, NavigableString # type: ignore[import]
|
||||||
from ebooklib import epub # type: ignore[import]
|
from ebooklib import epub # type: ignore[import]
|
||||||
|
|
||||||
from .utils import calculate_text_length, clean_text, detect_encoding
|
from .utils import clean_text, detect_encoding
|
||||||
|
from .domain.text_utils import calculate_text_length
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ Usage:
|
|||||||
from abogen.tts_plugin.plugin_manager import get_plugin_manager
|
from abogen.tts_plugin.plugin_manager import get_plugin_manager
|
||||||
|
|
||||||
manager = get_plugin_manager()
|
manager = get_plugin_manager()
|
||||||
engine = manager.create_engine("kokoro", lang_code="a", device="cpu")
|
engine = manager.create_engine("kokoro", language=Language.EN_US, device="cpu")
|
||||||
session = engine.create_session()
|
session = engine.create_session()
|
||||||
try:
|
try:
|
||||||
result = session.synthesize("Hello world")
|
result = session.synthesize("Hello world")
|
||||||
|
|||||||
@@ -9,6 +9,8 @@ from __future__ import annotations
|
|||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any, Mapping
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class AudioFormat:
|
class AudioFormat:
|
||||||
@@ -77,6 +79,44 @@ class SynthesisRequest:
|
|||||||
format: AudioFormat
|
format: AudioFormat
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class TokenTiming:
|
||||||
|
"""Per-token timing within a synthesized segment.
|
||||||
|
|
||||||
|
Attributes:
|
||||||
|
text: Token text.
|
||||||
|
whitespace: Whitespace following the token ("" if none).
|
||||||
|
start: Start time in seconds (relative to segment start).
|
||||||
|
end: End time in seconds (relative to segment start).
|
||||||
|
"""
|
||||||
|
|
||||||
|
text: str
|
||||||
|
whitespace: str = ""
|
||||||
|
start: float = 0.0
|
||||||
|
end: float = 0.0
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AudioSegment:
|
||||||
|
"""One contiguous synthesized segment (sentence-level chunk).
|
||||||
|
|
||||||
|
Engines that split the input text (via ``split_pattern``) expose each
|
||||||
|
chunk as its own AudioSegment so hosts can report per-sentence progress
|
||||||
|
and build subtitles from per-token timings.
|
||||||
|
|
||||||
|
Attributes:
|
||||||
|
graphemes: The text this segment was synthesized from.
|
||||||
|
audio: Raw float32 PCM audio bytes for this segment.
|
||||||
|
sample_rate: Sample rate of ``audio``.
|
||||||
|
tokens: Per-token timing details, when the engine provides them.
|
||||||
|
"""
|
||||||
|
|
||||||
|
graphemes: str
|
||||||
|
audio: bytes
|
||||||
|
sample_rate: int
|
||||||
|
tokens: tuple[TokenTiming, ...] = ()
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class SynthesizedAudio:
|
class SynthesizedAudio:
|
||||||
"""Immutable value object for synthesized audio result.
|
"""Immutable value object for synthesized audio result.
|
||||||
@@ -85,11 +125,15 @@ class SynthesizedAudio:
|
|||||||
data: Raw audio bytes.
|
data: Raw audio bytes.
|
||||||
format: Audio format of the result.
|
format: Audio format of the result.
|
||||||
duration: Duration of the audio.
|
duration: Duration of the audio.
|
||||||
|
segments: Per-segment details when the engine split the text into
|
||||||
|
sentence-level chunks (empty for engines that only produce a
|
||||||
|
single merged result).
|
||||||
"""
|
"""
|
||||||
|
|
||||||
data: bytes
|
data: bytes
|
||||||
format: AudioFormat
|
format: AudioFormat
|
||||||
duration: Duration
|
duration: Duration
|
||||||
|
segments: tuple[AudioSegment, ...] = ()
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -103,9 +147,9 @@ class EngineConfig:
|
|||||||
|
|
||||||
Attributes:
|
Attributes:
|
||||||
device: Device to use (e.g., "cpu", "cuda:0").
|
device: Device to use (e.g., "cpu", "cuda:0").
|
||||||
lang_code: Language code for the engine (e.g., "a" for Kokoro English).
|
language: Language enum value. The engine converts to its internal
|
||||||
Plugins that do not require a language code ignore this field.
|
format internally — callers never see engine-specific codes.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
device: str = "cpu"
|
device: str = "cpu"
|
||||||
lang_code: str = "a"
|
language: Language = Language.EN_US
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ from typing import Any, Iterator
|
|||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.tts_plugin.plugin_manager import get_plugin_manager
|
from abogen.tts_plugin.plugin_manager import get_plugin_manager
|
||||||
|
|
||||||
|
|
||||||
@@ -123,7 +124,7 @@ class Pipeline:
|
|||||||
|
|
||||||
Presents the same interface that old callers expect::
|
Presents the same interface that old callers expect::
|
||||||
|
|
||||||
pipeline = create_pipeline("kokoro", lang_code="a", device="cpu")
|
pipeline = create_pipeline("kokoro", language=Language.EN_US, device="cpu")
|
||||||
for segment in pipeline(text, voice="af_nova", speed=1.0):
|
for segment in pipeline(text, voice="af_nova", speed=1.0):
|
||||||
audio = segment.audio
|
audio = segment.audio
|
||||||
"""
|
"""
|
||||||
@@ -168,15 +169,38 @@ class Pipeline:
|
|||||||
)
|
)
|
||||||
|
|
||||||
result = session.synthesize(request)
|
result = session.synthesize(request)
|
||||||
audio_array = np.frombuffer(result.data, dtype=np.float32)
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Token:
|
||||||
|
text: str
|
||||||
|
whitespace: str = ""
|
||||||
|
start_ts: float = 0.0
|
||||||
|
end_ts: float = 0.0
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class Segment:
|
class Segment:
|
||||||
graphemes: str
|
graphemes: str
|
||||||
audio: np.ndarray
|
audio: np.ndarray
|
||||||
|
tokens: list[Any] = field(default_factory=list)
|
||||||
|
|
||||||
|
if result.segments:
|
||||||
|
for seg in result.segments:
|
||||||
|
audio_array = np.frombuffer(seg.audio, dtype=np.float32)
|
||||||
|
tokens = [
|
||||||
|
Token(
|
||||||
|
text=tok.text,
|
||||||
|
whitespace=tok.whitespace,
|
||||||
|
start_ts=tok.start,
|
||||||
|
end_ts=tok.end,
|
||||||
|
)
|
||||||
|
for tok in seg.tokens
|
||||||
|
]
|
||||||
|
yield Segment(graphemes=seg.graphemes, audio=audio_array, tokens=tokens)
|
||||||
|
return
|
||||||
|
|
||||||
|
audio_array = np.frombuffer(result.data, dtype=np.float32)
|
||||||
yield Segment(graphemes=text, audio=audio_array)
|
yield Segment(graphemes=text, audio=audio_array)
|
||||||
|
|
||||||
def load_single_voice(self, voice_name: str) -> Any:
|
def load_single_voice(self, voice_name: str) -> Any:
|
||||||
@@ -200,7 +224,7 @@ class Pipeline:
|
|||||||
def create_pipeline(
|
def create_pipeline(
|
||||||
plugin_id: str,
|
plugin_id: str,
|
||||||
*,
|
*,
|
||||||
lang_code: str = "a",
|
language: Language = Language.EN_US,
|
||||||
device: str = "cpu",
|
device: str = "cpu",
|
||||||
) -> Pipeline:
|
) -> Pipeline:
|
||||||
"""Create a callable TTS pipeline via the Plugin Architecture.
|
"""Create a callable TTS pipeline via the Plugin Architecture.
|
||||||
@@ -211,7 +235,7 @@ def create_pipeline(
|
|||||||
|
|
||||||
Args:
|
Args:
|
||||||
plugin_id: Plugin identifier (e.g., "kokoro", "supertonic").
|
plugin_id: Plugin identifier (e.g., "kokoro", "supertonic").
|
||||||
lang_code: Language code for the engine.
|
language: Language enum value (app-layer type, not engine-specific).
|
||||||
device: Device to use (e.g., "cpu", "cuda:0").
|
device: Device to use (e.g., "cpu", "cuda:0").
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
@@ -235,7 +259,7 @@ def create_pipeline(
|
|||||||
})(),
|
})(),
|
||||||
)
|
)
|
||||||
|
|
||||||
config = EngineConfig(device=device, lang_code=lang_code)
|
config = EngineConfig(device=device, language=language)
|
||||||
|
|
||||||
engine = manager.create_engine(plugin_id, context=ctx, model_path=None, config=config)
|
engine = manager.create_engine(plugin_id, context=ctx, model_path=None, config=config)
|
||||||
return Pipeline(engine)
|
return Pipeline(engine)
|
||||||
|
|||||||
+133
-23
@@ -6,7 +6,9 @@ import re
|
|||||||
import shutil
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
|
import time
|
||||||
import warnings
|
import warnings
|
||||||
|
from contextlib import contextmanager
|
||||||
from threading import Thread
|
from threading import Thread
|
||||||
from typing import Dict, Optional
|
from typing import Dict, Optional
|
||||||
|
|
||||||
@@ -14,6 +16,8 @@ from functools import lru_cache
|
|||||||
|
|
||||||
from dotenv import load_dotenv, find_dotenv
|
from dotenv import load_dotenv, find_dotenv
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def _load_environment() -> None:
|
def _load_environment() -> None:
|
||||||
explicit_path = os.environ.get("ABOGEN_ENV_FILE")
|
explicit_path = os.environ.get("ABOGEN_ENV_FILE")
|
||||||
@@ -29,6 +33,125 @@ _load_environment()
|
|||||||
|
|
||||||
warnings.filterwarnings("ignore")
|
warnings.filterwarnings("ignore")
|
||||||
|
|
||||||
|
# --- Console log colorization via rich (mirrors AutoSubSync's approach) ---
|
||||||
|
|
||||||
|
try: # rich is a declared dependency, but degrade gracefully if unavailable
|
||||||
|
from rich.console import Console
|
||||||
|
from rich.highlighter import NullHighlighter
|
||||||
|
from rich.logging import RichHandler
|
||||||
|
|
||||||
|
_RICH_AVAILABLE = True
|
||||||
|
except Exception: # pragma: no cover - fallback to plain logging
|
||||||
|
Console = None
|
||||||
|
NullHighlighter = None
|
||||||
|
RichHandler = None
|
||||||
|
_RICH_AVAILABLE = False
|
||||||
|
|
||||||
|
|
||||||
|
def _console_supports_color() -> bool:
|
||||||
|
if os.environ.get("NO_COLOR"):
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
return bool(sys.stderr.isatty())
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
_RICH_CONSOLE = None
|
||||||
|
if Console is not None:
|
||||||
|
try:
|
||||||
|
_RICH_CONSOLE = Console(stderr=True, no_color=not _console_supports_color())
|
||||||
|
except Exception: # pragma: no cover - defensive
|
||||||
|
_RICH_CONSOLE = None
|
||||||
|
|
||||||
|
|
||||||
|
_ANSI_ESCAPE_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
|
||||||
|
|
||||||
|
if RichHandler is not None:
|
||||||
|
|
||||||
|
class RichConsoleHandler(RichHandler):
|
||||||
|
"""RichHandler with default settings, except raw ANSI escapes are
|
||||||
|
stripped from messages first (werkzeug colorizes its own log lines
|
||||||
|
when attached to a TTY; without this they render as literal "[36m"
|
||||||
|
fragments)."""
|
||||||
|
|
||||||
|
def emit(self, record):
|
||||||
|
# Werkzeug logs its dev-server banner at INFO but hardcodes a
|
||||||
|
# "WARNING: " prefix into the message text. Promote the record so
|
||||||
|
# the level tag matches the content.
|
||||||
|
try:
|
||||||
|
message = _ANSI_ESCAPE_RE.sub("", record.getMessage())
|
||||||
|
except Exception: # pragma: no cover - defensive
|
||||||
|
message = ""
|
||||||
|
if record.levelno < logging.WARNING and message.startswith("WARNING: "):
|
||||||
|
record.levelno = logging.WARNING
|
||||||
|
record.levelname = "WARNING"
|
||||||
|
super().emit(record)
|
||||||
|
|
||||||
|
def render_message(self, record, message):
|
||||||
|
message = _ANSI_ESCAPE_RE.sub("", message)
|
||||||
|
if message.startswith("WARNING: "):
|
||||||
|
message = message[len("WARNING: ") :]
|
||||||
|
return super().render_message(record, message)
|
||||||
|
|
||||||
|
else: # pragma: no cover - rich unavailable fallback
|
||||||
|
RichConsoleHandler = None # type: ignore[assignment, misc]
|
||||||
|
|
||||||
|
|
||||||
|
def console_handler(show_level=True):
|
||||||
|
"""Build a colored console handler. Rich's RichHandler when available
|
||||||
|
(no timestamps, colored level tags), plain StreamHandler otherwise."""
|
||||||
|
if _RICH_CONSOLE is not None and RichConsoleHandler is not None:
|
||||||
|
return RichConsoleHandler(
|
||||||
|
console=_RICH_CONSOLE,
|
||||||
|
show_path=False,
|
||||||
|
show_time=False,
|
||||||
|
rich_tracebacks=True,
|
||||||
|
)
|
||||||
|
handler = logging.StreamHandler(sys.stderr)
|
||||||
|
prefix = "%(levelname)s - " if show_level else ""
|
||||||
|
handler.setFormatter(logging.Formatter(f"{prefix}%(message)s"))
|
||||||
|
return handler
|
||||||
|
|
||||||
|
|
||||||
|
def setup_console_logging(level=logging.INFO):
|
||||||
|
"""Configure the root logger once with a colored console handler."""
|
||||||
|
root = logging.getLogger()
|
||||||
|
if not root.handlers:
|
||||||
|
root.addHandler(console_handler())
|
||||||
|
root.setLevel(level)
|
||||||
|
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def timed_log(label, logger=None, level=logging.INFO):
|
||||||
|
"""Context manager that logs the wall-clock time a block of code takes.
|
||||||
|
|
||||||
|
Used to surface which load/startup steps are slow. The elapsed time is
|
||||||
|
colorized: green < 1s, yellow 1-5s, red > 5s.
|
||||||
|
"""
|
||||||
|
log = logger or logging.getLogger(__name__)
|
||||||
|
start = time.perf_counter()
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
elapsed = time.perf_counter() - start
|
||||||
|
if _RICH_AVAILABLE and _RICH_CONSOLE is not None and not _RICH_CONSOLE.no_color:
|
||||||
|
if elapsed >= 5.0:
|
||||||
|
color = "red"
|
||||||
|
elif elapsed >= 1.0:
|
||||||
|
color = "yellow"
|
||||||
|
else:
|
||||||
|
color = "green"
|
||||||
|
log.log(
|
||||||
|
level,
|
||||||
|
"Loaded %s in %s",
|
||||||
|
f"[cyan]{label}[/cyan]",
|
||||||
|
f"[{color}]{elapsed:.2f}s[/{color}]",
|
||||||
|
extra={"markup": True, "highlighter": NullHighlighter()},
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
log.log(level, "Loaded %s in %.2fs", label, elapsed)
|
||||||
|
|
||||||
|
|
||||||
def detect_encoding(file_path):
|
def detect_encoding(file_path):
|
||||||
try:
|
try:
|
||||||
@@ -320,10 +443,6 @@ default_encoding = sys.getfilesystemencoding()
|
|||||||
|
|
||||||
|
|
||||||
def create_process(cmd, stdin=None, text=True, capture_output=False):
|
def create_process(cmd, stdin=None, text=True, capture_output=False):
|
||||||
import logging
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
# Configure root logger to output to console if not already configured
|
# Configure root logger to output to console if not already configured
|
||||||
root = logging.getLogger()
|
root = logging.getLogger()
|
||||||
if not root.handlers:
|
if not root.handlers:
|
||||||
@@ -372,8 +491,8 @@ def create_process(cmd, stdin=None, text=True, capture_output=False):
|
|||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
# Print the command being executed
|
# Log the command being executed
|
||||||
print(f"Executing: {cmd if isinstance(cmd, str) else ' '.join(cmd)}")
|
logger.info(f"Executing: {cmd if isinstance(cmd, str) else ' '.join(cmd)}")
|
||||||
|
|
||||||
proc = subprocess.Popen(cmd, **kwargs)
|
proc = subprocess.Popen(cmd, **kwargs)
|
||||||
|
|
||||||
@@ -428,19 +547,6 @@ def save_config(config):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
def calculate_text_length(text):
|
|
||||||
# Ignore chapter markers
|
|
||||||
text = re.sub(r"<<CHAPTER_MARKER:.*?>>", "", text)
|
|
||||||
# Ignore metadata patterns
|
|
||||||
text = re.sub(r"<<METADATA_[^:]+:[^>]*>>", "", text)
|
|
||||||
# Ignore newlines
|
|
||||||
text = text.replace("\n", "")
|
|
||||||
# Ignore leading/trailing spaces
|
|
||||||
text = text.strip()
|
|
||||||
# Calculate character count
|
|
||||||
char_count = len(text)
|
|
||||||
return char_count
|
|
||||||
|
|
||||||
|
|
||||||
def get_gpu_acceleration(enabled):
|
def get_gpu_acceleration(enabled):
|
||||||
try:
|
try:
|
||||||
@@ -507,7 +613,7 @@ def prevent_sleep_start():
|
|||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
# Non-systemd distro or systemd tools not installed: skip inhibition rather than crash
|
# Non-systemd distro or systemd tools not installed: skip inhibition rather than crash
|
||||||
print(
|
logger.warning(
|
||||||
"systemd-inhibit not found: skipping sleep inhibition on this Linux system."
|
"systemd-inhibit not found: skipping sleep inhibition on this Linux system."
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -540,9 +646,13 @@ class LoadPipelineThread(Thread):
|
|||||||
try:
|
try:
|
||||||
from abogen.domain.pipeline_factory import create_pipeline_for_job
|
from abogen.domain.pipeline_factory import create_pipeline_for_job
|
||||||
|
|
||||||
backend = create_pipeline_for_job(
|
with timed_log(
|
||||||
"kokoro", language=self.lang_code, use_gpu=self.use_gpu
|
f"TTS pipeline (lang={self.lang_code}, gpu={self.use_gpu})",
|
||||||
)
|
logger=logging.getLogger("abogen.startup"),
|
||||||
|
):
|
||||||
|
backend = create_pipeline_for_job(
|
||||||
|
"kokoro", language=self.lang_code, use_gpu=self.use_gpu
|
||||||
|
)
|
||||||
self.callback(backend, None)
|
self.callback(backend, None)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
self.callback(None, str(e))
|
self.callback(None, str(e))
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ import json
|
|||||||
import os
|
import os
|
||||||
from typing import Any, Dict, Iterable, List, Tuple
|
from typing import Any, Dict, Iterable, List, Tuple
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.tts_plugin.utils import get_voices, is_plugin_registered
|
from abogen.tts_plugin.utils import get_voices, is_plugin_registered
|
||||||
from abogen.utils import get_user_config_path
|
from abogen.utils import get_user_config_path
|
||||||
|
|
||||||
@@ -176,13 +177,35 @@ def save_profile(name: str, *, language: str, voices: Iterable) -> None:
|
|||||||
raise ValueError("At least one voice with a weight above zero is required")
|
raise ValueError("At least one voice with a weight above zero is required")
|
||||||
|
|
||||||
if not language:
|
if not language:
|
||||||
language = "a"
|
language = Language.EN_US
|
||||||
|
|
||||||
profiles = load_profiles()
|
profiles = load_profiles()
|
||||||
profiles[name] = {"provider": "kokoro", "language": language, "voices": normalized}
|
profiles[name] = {"provider": "kokoro", "language": language, "voices": normalized}
|
||||||
save_profiles(profiles)
|
save_profiles(profiles)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_profile_language(entry: Any) -> Language:
|
||||||
|
"""Resolve a profile's stored language to a Language enum.
|
||||||
|
|
||||||
|
New profiles store ISO codes (Language enum values); legacy profiles may
|
||||||
|
store kokoro letter codes ("a", "b", ...). Unparseable values fall back
|
||||||
|
to EN_US.
|
||||||
|
"""
|
||||||
|
|
||||||
|
raw = entry.get("language") if isinstance(entry, dict) else None
|
||||||
|
if isinstance(raw, Language):
|
||||||
|
return raw
|
||||||
|
text = str(raw or "").strip()
|
||||||
|
if not text:
|
||||||
|
return Language.EN_US
|
||||||
|
try:
|
||||||
|
return Language.from_str(text)
|
||||||
|
except ValueError:
|
||||||
|
from plugins.kokoro.engine import language_for_code
|
||||||
|
|
||||||
|
return language_for_code(text)
|
||||||
|
|
||||||
|
|
||||||
def remove_profile(name: str) -> None:
|
def remove_profile(name: str) -> None:
|
||||||
delete_profile(name)
|
delete_profile(name)
|
||||||
|
|
||||||
|
|||||||
+73
-43
@@ -9,11 +9,19 @@ from flask import Flask
|
|||||||
|
|
||||||
from abogen import shutdown # noqa: F401
|
from abogen import shutdown # noqa: F401
|
||||||
shutdown.register_shutdown()
|
shutdown.register_shutdown()
|
||||||
from abogen.utils import get_user_cache_path, get_user_output_path, get_user_settings_dir
|
from abogen.utils import (
|
||||||
|
get_user_cache_path,
|
||||||
|
get_user_output_path,
|
||||||
|
get_user_settings_dir,
|
||||||
|
setup_console_logging,
|
||||||
|
timed_log,
|
||||||
|
)
|
||||||
|
|
||||||
from .conversion_runner import run_conversion_job
|
from .conversion_runner import run_conversion_job
|
||||||
from .service import build_service
|
from .service import build_service
|
||||||
|
|
||||||
|
_logger = logging.getLogger("abogen.startup")
|
||||||
|
|
||||||
|
|
||||||
class _SuppressSuccessfulAccessFilter(logging.Filter):
|
class _SuppressSuccessfulAccessFilter(logging.Filter):
|
||||||
"""Filter out successful (HTTP 200) werkzeug access logs."""
|
"""Filter out successful (HTTP 200) werkzeug access logs."""
|
||||||
@@ -29,6 +37,13 @@ class _SuppressSuccessfulAccessFilter(logging.Filter):
|
|||||||
return " 200 " not in message and " 201 " not in message and " 204 " not in message
|
return " 200 " not in message and " 201 " not in message and " 204 " not in message
|
||||||
|
|
||||||
|
|
||||||
|
class _SuppressPhonemizerWarnings(logging.Filter):
|
||||||
|
"""Suppress phonemizer word-count-mismatch warnings (normal behavior)."""
|
||||||
|
|
||||||
|
def filter(self, record: logging.LogRecord) -> bool: # pragma: no cover - small utility
|
||||||
|
return "words count mismatch" not in record.getMessage()
|
||||||
|
|
||||||
|
|
||||||
_access_log_filter_attached = False
|
_access_log_filter_attached = False
|
||||||
|
|
||||||
|
|
||||||
@@ -72,63 +87,78 @@ def _get_secret_key() -> str:
|
|||||||
|
|
||||||
|
|
||||||
def create_app(config: Optional[dict[str, Any]] = None) -> Flask:
|
def create_app(config: Optional[dict[str, Any]] = None) -> Flask:
|
||||||
uploads_dir, outputs_dir = _default_dirs()
|
with timed_log("default directories", logger=_logger):
|
||||||
|
uploads_dir, outputs_dir = _default_dirs()
|
||||||
|
|
||||||
app = Flask(
|
with timed_log("Flask app creation + config", logger=_logger):
|
||||||
__name__,
|
app = Flask(
|
||||||
static_folder="static",
|
__name__,
|
||||||
template_folder="templates",
|
static_folder="static",
|
||||||
)
|
template_folder="templates",
|
||||||
base_config = {
|
)
|
||||||
"SECRET_KEY": _get_secret_key(),
|
base_config = {
|
||||||
"UPLOAD_FOLDER": str(uploads_dir),
|
"SECRET_KEY": _get_secret_key(),
|
||||||
"OUTPUT_FOLDER": str(outputs_dir),
|
"UPLOAD_FOLDER": str(uploads_dir),
|
||||||
"MAX_CONTENT_LENGTH": 1024 * 1024 * 400, # 400 MB uploads
|
"OUTPUT_FOLDER": str(outputs_dir),
|
||||||
# Large books can submit four form fields per chapter. Werkzeug's
|
"MAX_CONTENT_LENGTH": 1024 * 1024 * 400, # 400 MB uploads
|
||||||
# defaults reject those requests before the wizard route can process
|
# Large books can submit four form fields per chapter. Werkzeug's
|
||||||
# them, even though the encoded payload is much smaller than the upload
|
# defaults reject those requests before the wizard route can process
|
||||||
# limit above.
|
# them, even though the encoded payload is much smaller than the upload
|
||||||
"MAX_FORM_MEMORY_SIZE": 10 * 1024 * 1024,
|
# limit above.
|
||||||
"MAX_FORM_PARTS": 10_000,
|
"MAX_FORM_MEMORY_SIZE": 10 * 1024 * 1024,
|
||||||
}
|
"MAX_FORM_PARTS": 10_000,
|
||||||
if config:
|
}
|
||||||
base_config.update(config)
|
if config:
|
||||||
app.config.update(base_config)
|
base_config.update(config)
|
||||||
|
app.config.update(base_config)
|
||||||
|
|
||||||
service = build_service(
|
with timed_log("conversion service (incl. queue state load)", logger=_logger):
|
||||||
runner=run_conversion_job,
|
service = build_service(
|
||||||
output_root=Path(app.config["OUTPUT_FOLDER"]),
|
runner=run_conversion_job,
|
||||||
uploads_root=Path(app.config["UPLOAD_FOLDER"]),
|
output_root=Path(app.config["OUTPUT_FOLDER"]),
|
||||||
)
|
uploads_root=Path(app.config["UPLOAD_FOLDER"]),
|
||||||
|
)
|
||||||
app.extensions["conversion_service"] = service
|
app.extensions["conversion_service"] = service
|
||||||
|
|
||||||
from abogen.webui.routes import (
|
with timed_log("blueprint registration", logger=_logger):
|
||||||
main_bp,
|
from abogen.webui.routes import (
|
||||||
jobs_bp,
|
main_bp,
|
||||||
settings_bp,
|
jobs_bp,
|
||||||
voices_bp,
|
settings_bp,
|
||||||
entities_bp,
|
voices_bp,
|
||||||
books_bp,
|
entities_bp,
|
||||||
api_bp,
|
books_bp,
|
||||||
)
|
api_bp,
|
||||||
|
)
|
||||||
|
|
||||||
app.register_blueprint(main_bp)
|
app.register_blueprint(main_bp)
|
||||||
app.register_blueprint(jobs_bp, url_prefix="/jobs")
|
app.register_blueprint(jobs_bp, url_prefix="/jobs")
|
||||||
app.register_blueprint(settings_bp, url_prefix="/settings")
|
app.register_blueprint(settings_bp, url_prefix="/settings")
|
||||||
app.register_blueprint(voices_bp, url_prefix="/voices")
|
app.register_blueprint(voices_bp, url_prefix="/voices")
|
||||||
app.register_blueprint(entities_bp, url_prefix="/overrides")
|
app.register_blueprint(entities_bp, url_prefix="/overrides")
|
||||||
app.register_blueprint(books_bp, url_prefix="/find-books")
|
app.register_blueprint(books_bp, url_prefix="/find-books")
|
||||||
app.register_blueprint(api_bp, url_prefix="/api")
|
app.register_blueprint(api_bp, url_prefix="/api")
|
||||||
|
|
||||||
global _access_log_filter_attached
|
global _access_log_filter_attached
|
||||||
if not _access_log_filter_attached:
|
if not _access_log_filter_attached:
|
||||||
logging.getLogger("werkzeug").addFilter(_SuppressSuccessfulAccessFilter())
|
logging.getLogger("werkzeug").addFilter(_SuppressSuccessfulAccessFilter())
|
||||||
|
logging.getLogger("phonemizer").addFilter(_SuppressPhonemizerWarnings())
|
||||||
_access_log_filter_attached = True
|
_access_log_filter_attached = True
|
||||||
|
|
||||||
return app
|
return app
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
def main() -> None:
|
||||||
|
setup_console_logging()
|
||||||
|
# Route Flask's dev-server banner through our logger instead of click.echo.
|
||||||
|
import flask.cli as flask_cli
|
||||||
|
|
||||||
|
def _show_server_banner(debug, app_import_path):
|
||||||
|
_logger.info(" * Serving Flask app %r", app_import_path)
|
||||||
|
_logger.info(" * Debug mode: %s", "on" if debug else "off")
|
||||||
|
|
||||||
|
flask_cli.show_server_banner = _show_server_banner
|
||||||
|
|
||||||
app = create_app()
|
app = create_app()
|
||||||
host = os.environ.get("ABOGEN_HOST", "0.0.0.0")
|
host = os.environ.get("ABOGEN_HOST", "0.0.0.0")
|
||||||
port = int(os.environ.get("ABOGEN_PORT", "8808"))
|
port = int(os.environ.get("ABOGEN_PORT", "8808"))
|
||||||
|
|||||||
@@ -1,162 +0,0 @@
|
|||||||
"""WebUI adapter: Job -> ConversionRequest.
|
|
||||||
|
|
||||||
Converts a WebUI Job into a ConversionRequest that the application layer can process.
|
|
||||||
This adapter is the bridge between the WebUI layer and the application/domain layer.
|
|
||||||
|
|
||||||
The adapter is responsible for:
|
|
||||||
- Mapping Job fields to ConversionRequest fields
|
|
||||||
- Handling UI-specific state (logs, progress, cancellation)
|
|
||||||
- Providing PipelineProvider and VoiceResolver implementations
|
|
||||||
|
|
||||||
All conversions happen through this adapter — the application layer
|
|
||||||
never accesses Job directly.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Callable, Dict, List, Optional
|
|
||||||
|
|
||||||
from abogen.application.conversion_request import ConversionRequest
|
|
||||||
from abogen.application.conversion_ports import ConversionCancelled, ResolvedVoice
|
|
||||||
|
|
||||||
|
|
||||||
def build_conversion_request_from_job(job: Any) -> ConversionRequest:
|
|
||||||
"""Convert a WebUI Job into a ConversionRequest.
|
|
||||||
|
|
||||||
This is the primary function that maps Job fields to ConversionRequest.
|
|
||||||
All fields are copied — the request is independent of the Job.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
job: WebUI Job instance
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
ConversionRequest with all Job data mapped
|
|
||||||
"""
|
|
||||||
return ConversionRequest(
|
|
||||||
# Source
|
|
||||||
source_path=Path(job.stored_path) if job.stored_path else None,
|
|
||||||
original_filename=job.original_filename,
|
|
||||||
# TTS Settings
|
|
||||||
language=job.language,
|
|
||||||
tts_provider=job.tts_provider,
|
|
||||||
voice=job.voice,
|
|
||||||
voice_profile=job.voice_profile,
|
|
||||||
speed=job.speed,
|
|
||||||
use_gpu=job.use_gpu,
|
|
||||||
supertonic_total_steps=job.supertonic_total_steps,
|
|
||||||
# Output Format
|
|
||||||
output_format=job.output_format,
|
|
||||||
subtitle_mode=job.subtitle_mode,
|
|
||||||
subtitle_format=job.subtitle_format,
|
|
||||||
max_subtitle_words=job.max_subtitle_words,
|
|
||||||
# Save Options
|
|
||||||
save_mode=job.save_mode,
|
|
||||||
output_folder=Path(job.output_folder) if job.output_folder else None,
|
|
||||||
save_chapters_separately=job.save_chapters_separately,
|
|
||||||
merge_chapters_at_end=job.merge_chapters_at_end,
|
|
||||||
separate_chapters_format=job.separate_chapters_format,
|
|
||||||
save_as_project=job.save_as_project,
|
|
||||||
# Timing
|
|
||||||
silence_between_chapters=job.silence_between_chapters,
|
|
||||||
chapter_intro_delay=job.chapter_intro_delay,
|
|
||||||
# Content Processing
|
|
||||||
replace_single_newlines=job.replace_single_newlines,
|
|
||||||
read_title_intro=job.read_title_intro,
|
|
||||||
read_closing_outro=job.read_closing_outro,
|
|
||||||
auto_prefix_chapter_titles=job.auto_prefix_chapter_titles,
|
|
||||||
normalize_chapter_opening_caps=job.normalize_chapter_opening_caps,
|
|
||||||
# Pronunciation / Normalization
|
|
||||||
pronunciation_overrides=job.pronunciation_overrides or [],
|
|
||||||
manual_overrides=job.manual_overrides or [],
|
|
||||||
heteronym_overrides=job.heteronym_overrides or [],
|
|
||||||
normalization_overrides=job.normalization_overrides or {},
|
|
||||||
# Chapter/Chunk Configuration
|
|
||||||
chapter_overrides=job.chapters or [],
|
|
||||||
chunks=job.chunks or [],
|
|
||||||
chunk_level=job.chunk_level,
|
|
||||||
speaker_mode=job.speaker_mode,
|
|
||||||
speakers=job.speakers or {},
|
|
||||||
# Metadata
|
|
||||||
metadata_tags=job.metadata_tags or {},
|
|
||||||
# Artifacts
|
|
||||||
cover_image_path=Path(job.cover_image_path) if job.cover_image_path else None,
|
|
||||||
cover_image_mime=job.cover_image_mime,
|
|
||||||
generate_epub3=job.generate_epub3,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class WebJobEvents:
|
|
||||||
"""WebUI implementation of ConversionEvents protocol.
|
|
||||||
|
|
||||||
Wraps a Job to provide logging, progress, and cancellation.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, job: Any):
|
|
||||||
self._job = job
|
|
||||||
|
|
||||||
def log(self, message: str, level: str = "info") -> None:
|
|
||||||
"""Log a message to the Job."""
|
|
||||||
self._job.add_log(message, level=level)
|
|
||||||
|
|
||||||
def progress(self, pct: int, etr: str) -> None:
|
|
||||||
"""Update progress on the Job."""
|
|
||||||
self._job.progress = pct / 100.0
|
|
||||||
self._job.etr_str = etr
|
|
||||||
|
|
||||||
def check_cancelled(self) -> None:
|
|
||||||
"""Check if the Job was cancelled.
|
|
||||||
|
|
||||||
Raises:
|
|
||||||
ConversionCancelled: If cancellation was requested
|
|
||||||
"""
|
|
||||||
if self._job.cancel_requested:
|
|
||||||
raise ConversionCancelled("Job cancelled by user")
|
|
||||||
|
|
||||||
|
|
||||||
class WebPipelineProvider:
|
|
||||||
"""WebUI implementation of PipelineProvider protocol.
|
|
||||||
|
|
||||||
Wraps PipelinePool to provide TTS backends.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, pipeline_pool: Any):
|
|
||||||
self._pool = pipeline_pool
|
|
||||||
|
|
||||||
def get(self, provider: str, language: str, use_gpu: bool) -> Any:
|
|
||||||
"""Get a TTS backend instance."""
|
|
||||||
return self._pool.get(provider, language, use_gpu)
|
|
||||||
|
|
||||||
def dispose_all(self) -> None:
|
|
||||||
"""Dispose all backend resources."""
|
|
||||||
self._pool.dispose_all()
|
|
||||||
|
|
||||||
|
|
||||||
class WebVoiceResolver:
|
|
||||||
"""WebUI implementation of VoiceResolver protocol.
|
|
||||||
|
|
||||||
Wraps the voice resolution logic from conversion_runner.py.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
resolve_fn: Callable[[str], tuple[str, str, Any, Optional[float], Optional[int]]],
|
|
||||||
):
|
|
||||||
"""Initialize with a voice resolution function.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
resolve_fn: Function that takes a voice_spec and returns
|
|
||||||
(provider, resolved_spec, voice_choice, speed, steps)
|
|
||||||
"""
|
|
||||||
self._resolve_fn = resolve_fn
|
|
||||||
|
|
||||||
def resolve(self, voice_spec: str) -> ResolvedVoice:
|
|
||||||
"""Resolve a voice spec into a loaded voice."""
|
|
||||||
provider, resolved_spec, voice, speed, steps = self._resolve_fn(voice_spec)
|
|
||||||
return ResolvedVoice(
|
|
||||||
provider=provider,
|
|
||||||
resolved_spec=resolved_spec,
|
|
||||||
voice=voice,
|
|
||||||
speed=speed or 1.0,
|
|
||||||
supertonic_steps=steps or 5,
|
|
||||||
)
|
|
||||||
+183
-1005
File diff suppressed because it is too large
Load Diff
@@ -14,9 +14,12 @@ from abogen.kokoro_text_normalization import normalize_for_pipeline
|
|||||||
from abogen.normalization_settings import build_apostrophe_config
|
from abogen.normalization_settings import build_apostrophe_config
|
||||||
from abogen.text_extractor import extract_from_path
|
from abogen.text_extractor import extract_from_path
|
||||||
from abogen.voice_cache import ensure_voice_assets
|
from abogen.voice_cache import ensure_voice_assets
|
||||||
from abogen.webui.conversion_runner import SAMPLE_RATE, _select_device, _to_float32, _spec_to_voice_ids
|
from abogen.domain.device import select_device as _select_device
|
||||||
|
from abogen.domain.audio_helpers import to_float32 as _to_float32, SAMPLE_RATE
|
||||||
|
from abogen.domain.voice_resolution import spec_to_voice_ids as _spec_to_voice_ids
|
||||||
from abogen.domain.voice_loader import resolve_voice
|
from abogen.domain.voice_loader import resolve_voice
|
||||||
from abogen.domain.split_pattern import get_split_pattern
|
from abogen.domain.split_pattern import get_split_pattern
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.tts_plugin.utils import create_pipeline
|
from abogen.tts_plugin.utils import create_pipeline
|
||||||
|
|
||||||
|
|
||||||
@@ -43,11 +46,18 @@ def _resolve_voice_setting(value: str) -> tuple[str, Optional[str], Optional[str
|
|||||||
return resolve_voice_setting(value)
|
return resolve_voice_setting(value)
|
||||||
|
|
||||||
|
|
||||||
def _load_pipeline(language: str, use_gpu: bool) -> Any:
|
def _load_pipeline(language: Language, use_gpu: bool) -> Any:
|
||||||
device = "cpu"
|
import logging
|
||||||
if use_gpu:
|
from abogen.utils import timed_log
|
||||||
device = _select_device()
|
|
||||||
return create_pipeline("kokoro", lang_code=language, device=device)
|
with timed_log(
|
||||||
|
f"TTS pipeline (lang={language}, gpu={use_gpu})",
|
||||||
|
logger=logging.getLogger("abogen.startup"),
|
||||||
|
):
|
||||||
|
device = "cpu"
|
||||||
|
if use_gpu:
|
||||||
|
device = _select_device()
|
||||||
|
return create_pipeline("kokoro", language=language, device=device)
|
||||||
|
|
||||||
|
|
||||||
def _extract_cases_from_text(text: str) -> List[Tuple[str, str]]:
|
def _extract_cases_from_text(text: str) -> List[Tuple[str, str]]:
|
||||||
@@ -127,32 +137,14 @@ def run_debug_tts_wavs(
|
|||||||
if missing:
|
if missing:
|
||||||
raise RuntimeError(f"Debug EPUB missing expected codes: {', '.join(missing)}")
|
raise RuntimeError(f"Debug EPUB missing expected codes: {', '.join(missing)}")
|
||||||
|
|
||||||
language = str(settings.get("language") or "a").strip() or "a"
|
raw_language = str(settings.get("language") or "en-US").strip() or "en-US"
|
||||||
# Kokoro's KPipeline expects short language codes like "a" (American English),
|
try:
|
||||||
# but older settings may store ISO-like values such as "en".
|
language = Language.from_str(raw_language)
|
||||||
language_aliases = {
|
except ValueError:
|
||||||
"en": "a",
|
language = Language.EN_US
|
||||||
"en-us": "a",
|
|
||||||
"en_us": "a",
|
|
||||||
"en-gb": "b",
|
|
||||||
"en_gb": "b",
|
|
||||||
"es": "e",
|
|
||||||
"es-es": "e",
|
|
||||||
"fr": "f",
|
|
||||||
"fr-fr": "f",
|
|
||||||
"hi": "h",
|
|
||||||
"it": "i",
|
|
||||||
"pt": "p",
|
|
||||||
"pt-br": "p",
|
|
||||||
"ja": "j",
|
|
||||||
"jp": "j",
|
|
||||||
"zh": "z",
|
|
||||||
"zh-cn": "z",
|
|
||||||
}
|
|
||||||
language = language_aliases.get(language.lower(), language)
|
|
||||||
voice_spec = str(settings.get("default_voice") or "").strip()
|
voice_spec = str(settings.get("default_voice") or "").strip()
|
||||||
use_gpu = bool(settings.get("use_gpu", False))
|
use_gpu = bool(settings.get("use_gpu", False))
|
||||||
speed = float(settings.get("default_speed", 1.0) or 1.0)
|
speed = float(settings.get("default_speed") or 1.0)
|
||||||
|
|
||||||
# Settings may store "profile:<name>" which is not a Kokoro voice ID.
|
# Settings may store "profile:<name>" which is not a Kokoro voice ID.
|
||||||
# Resolve it to a concrete voice formula (e.g. "af_heart*0.5+...") so Kokoro
|
# Resolve it to a concrete voice formula (e.g. "af_heart*0.5+...") so Kokoro
|
||||||
@@ -162,7 +154,10 @@ def run_debug_tts_wavs(
|
|||||||
if resolved_voice:
|
if resolved_voice:
|
||||||
voice_spec = resolved_voice
|
voice_spec = resolved_voice
|
||||||
if profile_language:
|
if profile_language:
|
||||||
language = str(profile_language).strip() or language
|
try:
|
||||||
|
language = Language.from_str(str(profile_language).strip()) or language
|
||||||
|
except (ValueError, AttributeError):
|
||||||
|
pass
|
||||||
except Exception:
|
except Exception:
|
||||||
# Voice profile resolution is best-effort; fall back to raw voice_spec.
|
# Voice profile resolution is best-effort; fall back to raw voice_spec.
|
||||||
pass
|
pass
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ from pathlib import Path
|
|||||||
from flask import Blueprint, request, jsonify, send_file, url_for, current_app
|
from flask import Blueprint, request, jsonify, send_file, url_for, current_app
|
||||||
from flask.typing import ResponseReturnValue
|
from flask.typing import ResponseReturnValue
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.webui.routes.utils.settings import (
|
from abogen.webui.routes.utils.settings import (
|
||||||
load_settings,
|
load_settings,
|
||||||
load_integration_settings,
|
load_integration_settings,
|
||||||
@@ -47,6 +48,21 @@ from werkzeug.utils import secure_filename
|
|||||||
|
|
||||||
api_bp = Blueprint("api", __name__)
|
api_bp = Blueprint("api", __name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_language(value: Any) -> Language:
|
||||||
|
"""Parse a frontend language value to Language enum.
|
||||||
|
|
||||||
|
This is the API boundary — frontend sends strings, backend parses
|
||||||
|
to Language enum. No engine-specific codes leak outside the engine.
|
||||||
|
"""
|
||||||
|
if isinstance(value, Language):
|
||||||
|
return value
|
||||||
|
try:
|
||||||
|
return Language.from_str(str(value or "").strip())
|
||||||
|
except (ValueError, AttributeError):
|
||||||
|
return Language.EN_US
|
||||||
|
|
||||||
|
|
||||||
# --- Voice Profile Routes ---
|
# --- Voice Profile Routes ---
|
||||||
|
|
||||||
@api_bp.get("/voice-profiles")
|
@api_bp.get("/voice-profiles")
|
||||||
@@ -152,7 +168,7 @@ def api_export_voice_profiles() -> ResponseReturnValue:
|
|||||||
def api_voice_profiles_preview() -> ResponseReturnValue:
|
def api_voice_profiles_preview() -> ResponseReturnValue:
|
||||||
payload = request.get_json(force=True, silent=True) or {}
|
payload = request.get_json(force=True, silent=True) or {}
|
||||||
text = str(payload.get("text") or "").strip() or "Hello world"
|
text = str(payload.get("text") or "").strip() or "Hello world"
|
||||||
language = str(payload.get("language") or "a").strip().lower() or "a"
|
language = _parse_language(payload.get("language"))
|
||||||
speed = coerce_float(payload.get("speed"), 1.0)
|
speed = coerce_float(payload.get("speed"), 1.0)
|
||||||
max_seconds = coerce_float(payload.get("max_seconds"), 8.0)
|
max_seconds = coerce_float(payload.get("max_seconds"), 8.0)
|
||||||
|
|
||||||
@@ -168,6 +184,11 @@ def api_voice_profiles_preview() -> ResponseReturnValue:
|
|||||||
voice_spec = ""
|
voice_spec = ""
|
||||||
resolved_provider = provider or "kokoro"
|
resolved_provider = provider or "kokoro"
|
||||||
|
|
||||||
|
current_app.logger.info(
|
||||||
|
"[preview] provider=%s language=%s speed=%.2f profile=%s formula=%s",
|
||||||
|
resolved_provider, language, speed, profile_name or "-", formula or "-",
|
||||||
|
)
|
||||||
|
|
||||||
profiles = load_profiles()
|
profiles = load_profiles()
|
||||||
if resolved_provider == "supertonic" and not profile_name:
|
if resolved_provider == "supertonic" and not profile_name:
|
||||||
voice_spec = str(payload.get("voice") or payload.get("supertonic_voice") or "M1").strip() or "M1"
|
voice_spec = str(payload.get("voice") or payload.get("supertonic_voice") or "M1").strip() or "M1"
|
||||||
@@ -186,7 +207,7 @@ def api_voice_profiles_preview() -> ResponseReturnValue:
|
|||||||
speed = float(normalized_entry.get("speed") or speed)
|
speed = float(normalized_entry.get("speed") or speed)
|
||||||
else:
|
else:
|
||||||
voice_spec = formula_from_profile(normalized_entry) or ""
|
voice_spec = formula_from_profile(normalized_entry) or ""
|
||||||
language = str(normalized_entry.get("language") or language)
|
language = _parse_language(normalized_entry.get("language") or language)
|
||||||
elif formula:
|
elif formula:
|
||||||
voice_spec = formula
|
voice_spec = formula
|
||||||
resolved_provider = "kokoro"
|
resolved_provider = "kokoro"
|
||||||
@@ -198,7 +219,13 @@ def api_voice_profiles_preview() -> ResponseReturnValue:
|
|||||||
voice_spec = formula_from_profile(normalized_entry) or ""
|
voice_spec = formula_from_profile(normalized_entry) or ""
|
||||||
resolved_provider = "kokoro"
|
resolved_provider = "kokoro"
|
||||||
|
|
||||||
|
current_app.logger.info(
|
||||||
|
"[preview] resolved: provider=%s voice_spec=%s",
|
||||||
|
resolved_provider, voice_spec[:80] if voice_spec else "-",
|
||||||
|
)
|
||||||
|
|
||||||
if not voice_spec:
|
if not voice_spec:
|
||||||
|
current_app.logger.warning("[preview] empty voice_spec, returning 400")
|
||||||
return jsonify({"error": "Unable to resolve preview voice"}), 400
|
return jsonify({"error": "Unable to resolve preview voice"}), 400
|
||||||
|
|
||||||
try:
|
try:
|
||||||
@@ -213,6 +240,7 @@ def api_voice_profiles_preview() -> ResponseReturnValue:
|
|||||||
max_seconds=max_seconds,
|
max_seconds=max_seconds,
|
||||||
)
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
|
current_app.logger.exception("[preview] synthesis failed: %s", exc)
|
||||||
return jsonify({"error": str(exc)}), 500
|
return jsonify({"error": str(exc)}), 500
|
||||||
|
|
||||||
@api_bp.post("/speaker-preview")
|
@api_bp.post("/speaker-preview")
|
||||||
@@ -221,7 +249,7 @@ def api_speaker_preview() -> ResponseReturnValue:
|
|||||||
pending_id = str(payload.get("pending_id") or "").strip()
|
pending_id = str(payload.get("pending_id") or "").strip()
|
||||||
text = payload.get("text", "Hello world")
|
text = payload.get("text", "Hello world")
|
||||||
voice = payload.get("voice", "af_heart")
|
voice = payload.get("voice", "af_heart")
|
||||||
language = payload.get("language", "a")
|
language = _parse_language(payload.get("language"))
|
||||||
speed_value = payload.get("speed")
|
speed_value = payload.get("speed")
|
||||||
speed = coerce_float(speed_value, 1.0)
|
speed = coerce_float(speed_value, 1.0)
|
||||||
tts_provider = str(payload.get("tts_provider") or "").strip().lower()
|
tts_provider = str(payload.get("tts_provider") or "").strip().lower()
|
||||||
@@ -576,7 +604,7 @@ def api_entity_pronunciation_preview() -> ResponseReturnValue:
|
|||||||
token = payload.get("token", "").strip()
|
token = payload.get("token", "").strip()
|
||||||
pronunciation = payload.get("pronunciation", "").strip()
|
pronunciation = payload.get("pronunciation", "").strip()
|
||||||
voice = payload.get("voice", "").strip()
|
voice = payload.get("voice", "").strip()
|
||||||
language = payload.get("language", "a").strip()
|
language = _parse_language(payload.get("language"))
|
||||||
|
|
||||||
if not token and not pronunciation:
|
if not token and not pronunciation:
|
||||||
return jsonify({"error": "Token or pronunciation required"}), 400
|
return jsonify({"error": "Token or pronunciation required"}), 400
|
||||||
|
|||||||
@@ -8,6 +8,8 @@ from flask.typing import ResponseReturnValue
|
|||||||
|
|
||||||
from abogen.webui.service import (
|
from abogen.webui.service import (
|
||||||
JobStatus,
|
JobStatus,
|
||||||
|
)
|
||||||
|
from abogen.domain.metadata_helpers import (
|
||||||
build_audiobookshelf_metadata,
|
build_audiobookshelf_metadata,
|
||||||
load_audiobookshelf_chapters,
|
load_audiobookshelf_chapters,
|
||||||
)
|
)
|
||||||
@@ -19,9 +21,9 @@ from abogen.webui.routes.utils.epub import (
|
|||||||
locate_job_epub,
|
locate_job_epub,
|
||||||
locate_job_audio,
|
locate_job_audio,
|
||||||
)
|
)
|
||||||
from abogen.webui.routes.utils.settings import (
|
from abogen.domain.settings_core import (
|
||||||
stored_integration_config,
|
|
||||||
build_audiobookshelf_config,
|
build_audiobookshelf_config,
|
||||||
|
stored_integration_config,
|
||||||
)
|
)
|
||||||
from abogen.webui.routes.utils.common import existing_paths
|
from abogen.webui.routes.utils.common import existing_paths
|
||||||
from abogen.infrastructure.exporters import ExportService
|
from abogen.infrastructure.exporters import ExportService
|
||||||
|
|||||||
@@ -1,14 +1,12 @@
|
|||||||
|
import logging
|
||||||
import time
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
|
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
|
||||||
from flask import request, render_template, jsonify
|
from flask import request, render_template, jsonify
|
||||||
from flask.typing import ResponseReturnValue
|
from flask.typing import ResponseReturnValue
|
||||||
|
|
||||||
from abogen.domain.chapter_classification import (
|
from abogen.domain.enums import Language
|
||||||
supplement_score,
|
from abogen.application.chapter_selection import build_chapter_payload
|
||||||
should_preselect_chapter,
|
|
||||||
ensure_at_least_one_chapter_enabled,
|
|
||||||
)
|
|
||||||
from abogen.webui.service import PendingJob, JobStatus
|
from abogen.webui.service import PendingJob, JobStatus
|
||||||
from abogen.webui.routes.utils.service import get_service
|
from abogen.webui.routes.utils.service import get_service
|
||||||
from abogen.tts_plugin.utils import is_plugin_registered
|
from abogen.tts_plugin.utils import is_plugin_registered
|
||||||
@@ -24,17 +22,21 @@ from abogen.webui.routes.utils.settings import (
|
|||||||
audiobookshelf_manual_available,
|
audiobookshelf_manual_available,
|
||||||
)
|
)
|
||||||
from abogen.webui.routes.utils.voice import (
|
from abogen.webui.routes.utils.voice import (
|
||||||
|
inject_recommended_voices,
|
||||||
parse_voice_formula,
|
parse_voice_formula,
|
||||||
|
template_options,
|
||||||
|
)
|
||||||
|
from abogen.domain.speaker_metadata import prepare_speaker_metadata
|
||||||
|
from abogen.domain.metadata_helpers import expand_metadata_aliases
|
||||||
|
from abogen.domain.voice_resolution import (
|
||||||
formula_from_profile,
|
formula_from_profile,
|
||||||
resolve_voice_setting,
|
resolve_voice_setting,
|
||||||
resolve_voice_choice,
|
resolve_voice_choice,
|
||||||
prepare_speaker_metadata,
|
|
||||||
template_options,
|
|
||||||
)
|
)
|
||||||
from abogen.webui.routes.utils.entity import sync_pronunciation_overrides
|
from abogen.webui.routes.utils.entity import sync_pronunciation_overrides
|
||||||
from abogen.webui.routes.utils.epub import job_download_flags
|
from abogen.webui.routes.utils.epub import job_download_flags
|
||||||
from abogen.webui.routes.utils.common import split_profile_spec, extract_checkbox
|
from abogen.webui.routes.utils.common import split_profile_spec, extract_checkbox
|
||||||
from abogen.utils import calculate_text_length
|
from abogen.domain.text_utils import calculate_text_length
|
||||||
from abogen.voice_profiles import serialize_profiles, normalize_profile_entry
|
from abogen.voice_profiles import serialize_profiles, normalize_profile_entry
|
||||||
from abogen.chunking import ChunkLevel, build_chunks_for_chapters
|
from abogen.chunking import ChunkLevel, build_chunks_for_chapters
|
||||||
from abogen.tts_plugin.utils import get_default_voice
|
from abogen.tts_plugin.utils import get_default_voice
|
||||||
@@ -346,7 +348,10 @@ def apply_book_step_form(
|
|||||||
language_fallback = pending.language or settings.get("language", "en")
|
language_fallback = pending.language or settings.get("language", "en")
|
||||||
raw_language = (form.get("language") or language_fallback or "en").strip()
|
raw_language = (form.get("language") or language_fallback or "en").strip()
|
||||||
if raw_language:
|
if raw_language:
|
||||||
pending.language = raw_language
|
try:
|
||||||
|
pending.language = Language.from_str(raw_language)
|
||||||
|
except (ValueError, AttributeError):
|
||||||
|
pending.language = Language.EN_US
|
||||||
|
|
||||||
subtitle_mode = (form.get("subtitle_mode") or pending.subtitle_mode or "Disabled").strip()
|
subtitle_mode = (form.get("subtitle_mode") or pending.subtitle_mode or "Disabled").strip()
|
||||||
if subtitle_mode:
|
if subtitle_mode:
|
||||||
@@ -513,7 +518,10 @@ def apply_book_step_form(
|
|||||||
)
|
)
|
||||||
|
|
||||||
if resolved_language:
|
if resolved_language:
|
||||||
pending.language = resolved_language
|
try:
|
||||||
|
pending.language = Language.from_str(str(resolved_language))
|
||||||
|
except (ValueError, AttributeError):
|
||||||
|
pass # keep existing language
|
||||||
|
|
||||||
if profile_selection == "__formula" and custom_formula_raw:
|
if profile_selection == "__formula" and custom_formula_raw:
|
||||||
pending.voice = custom_formula_raw
|
pending.voice = custom_formula_raw
|
||||||
@@ -535,35 +543,27 @@ def apply_book_step_form(
|
|||||||
if "meta_subtitle" in form:
|
if "meta_subtitle" in form:
|
||||||
pending.metadata_tags["subtitle"] = str(form.get("meta_subtitle", "")).strip()
|
pending.metadata_tags["subtitle"] = str(form.get("meta_subtitle", "")).strip()
|
||||||
|
|
||||||
|
# Collect user-editable metadata fields that have concept aliases
|
||||||
|
user_metadata: Dict[str, str] = {}
|
||||||
if "meta_author" in form:
|
if "meta_author" in form:
|
||||||
authors = str(form.get("meta_author", "")).strip()
|
user_metadata["author"] = str(form.get("meta_author", "")).strip()
|
||||||
pending.metadata_tags["authors"] = authors
|
|
||||||
pending.metadata_tags["author"] = authors
|
|
||||||
|
|
||||||
if "meta_series" in form:
|
if "meta_series" in form:
|
||||||
series = str(form.get("meta_series", "")).strip()
|
user_metadata["series"] = str(form.get("meta_series", "")).strip()
|
||||||
pending.metadata_tags["series"] = series
|
|
||||||
pending.metadata_tags["series_name"] = series
|
|
||||||
pending.metadata_tags["seriesname"] = series
|
|
||||||
pending.metadata_tags["series_title"] = series
|
|
||||||
pending.metadata_tags["seriestitle"] = series
|
|
||||||
# If user manually edits series, update opds_series too so it persists
|
|
||||||
if "opds_series" in pending.metadata_tags:
|
|
||||||
pending.metadata_tags["opds_series"] = series
|
|
||||||
|
|
||||||
if "meta_series_index" in form:
|
if "meta_series_index" in form:
|
||||||
idx = str(form.get("meta_series_index", "")).strip()
|
user_metadata["series_index"] = str(form.get("meta_series_index", "")).strip()
|
||||||
pending.metadata_tags["series_index"] = idx
|
if "meta_description" in form:
|
||||||
pending.metadata_tags["series_sequence"] = idx
|
user_metadata["description"] = str(form.get("meta_description", "")).strip()
|
||||||
|
|
||||||
|
if user_metadata:
|
||||||
|
expanded = expand_metadata_aliases(user_metadata)
|
||||||
|
pending.metadata_tags.update(expanded)
|
||||||
|
# If user manually edits series, update opds_series too so it persists
|
||||||
|
if "meta_series" in form and "opds_series" in pending.metadata_tags:
|
||||||
|
pending.metadata_tags["opds_series"] = expanded.get("series", "")
|
||||||
|
|
||||||
if "meta_publisher" in form:
|
if "meta_publisher" in form:
|
||||||
pending.metadata_tags["publisher"] = str(form.get("meta_publisher", "")).strip()
|
pending.metadata_tags["publisher"] = str(form.get("meta_publisher", "")).strip()
|
||||||
|
|
||||||
if "meta_description" in form:
|
|
||||||
desc = str(form.get("meta_description", "")).strip()
|
|
||||||
pending.metadata_tags["description"] = desc
|
|
||||||
pending.metadata_tags["summary"] = desc
|
|
||||||
|
|
||||||
if coerce_bool(form.get("remove_cover"), False):
|
if coerce_bool(form.get("remove_cover"), False):
|
||||||
pending.cover_image_path = None
|
pending.cover_image_path = None
|
||||||
pending.cover_image_mime = None
|
pending.cover_image_mime = None
|
||||||
@@ -637,36 +637,13 @@ def build_pending_job_from_extraction(
|
|||||||
getattr(extraction, "combined_text", "")
|
getattr(extraction, "combined_text", "")
|
||||||
)
|
)
|
||||||
chapters_source = getattr(extraction, "chapters", []) or []
|
chapters_source = getattr(extraction, "chapters", []) or []
|
||||||
total_chapter_count = len(chapters_source)
|
chapters_payload = build_chapter_payload(chapters_source, source_name=original_name)
|
||||||
chapters_payload: List[Dict[str, Any]] = []
|
|
||||||
for index, chapter in enumerate(chapters_source):
|
|
||||||
enabled = should_preselect_chapter(chapter.title, chapter.text, index, total_chapter_count)
|
|
||||||
chapters_payload.append(
|
|
||||||
{
|
|
||||||
"id": f"{index:04d}",
|
|
||||||
"index": index,
|
|
||||||
"title": chapter.title,
|
|
||||||
"text": chapter.text,
|
|
||||||
"characters": calculate_text_length(chapter.text),
|
|
||||||
"enabled": enabled,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
if not chapters_payload:
|
raw_language = str(form.get("language") or "a").strip() or "a"
|
||||||
chapters_payload.append(
|
try:
|
||||||
{
|
language = Language.from_str(raw_language)
|
||||||
"id": "0000",
|
except (ValueError, AttributeError):
|
||||||
"index": 0,
|
language = Language.EN_US
|
||||||
"title": original_name,
|
|
||||||
"text": "",
|
|
||||||
"characters": 0,
|
|
||||||
"enabled": True,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
ensure_at_least_one_chapter_enabled(chapters_payload)
|
|
||||||
|
|
||||||
language = str(form.get("language") or "a").strip() or "a"
|
|
||||||
profiles_map = dict(profiles) if isinstance(profiles, Mapping) else dict(profiles or {})
|
profiles_map = dict(profiles) if isinstance(profiles, Mapping) else dict(profiles or {})
|
||||||
default_voice_setting = settings.get("default_voice") or ""
|
default_voice_setting = settings.get("default_voice") or ""
|
||||||
resolved_default_voice, inferred_profile, inferred_language = resolve_voice_setting(
|
resolved_default_voice, inferred_profile, inferred_language = resolve_voice_setting(
|
||||||
@@ -768,6 +745,7 @@ def build_pending_job_from_extraction(
|
|||||||
run_analysis=initial_analysis,
|
run_analysis=initial_analysis,
|
||||||
speaker_config=speaker_config_payload,
|
speaker_config=speaker_config_payload,
|
||||||
apply_config=bool(speaker_config_payload),
|
apply_config=bool(speaker_config_payload),
|
||||||
|
inject_recommended=inject_recommended_voices,
|
||||||
)
|
)
|
||||||
|
|
||||||
normalization_overrides = {}
|
normalization_overrides = {}
|
||||||
@@ -783,6 +761,11 @@ def build_pending_job_from_extraction(
|
|||||||
else:
|
else:
|
||||||
normalization_overrides[key] = default_val
|
normalization_overrides[key] = default_val
|
||||||
|
|
||||||
|
logging.info(
|
||||||
|
"[form] Creating PendingJob: language=%s voice=%s speed=%.2f provider=%s",
|
||||||
|
language, voice, speed, settings.get("tts_provider", "kokoro"),
|
||||||
|
)
|
||||||
|
|
||||||
pending = PendingJob(
|
pending = PendingJob(
|
||||||
id=uuid.uuid4().hex,
|
id=uuid.uuid4().hex,
|
||||||
original_filename=original_name,
|
original_filename=original_name,
|
||||||
@@ -826,6 +809,8 @@ def build_pending_job_from_extraction(
|
|||||||
analysis_requested=initial_analysis,
|
analysis_requested=initial_analysis,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
apply_book_step_form(pending, form, settings=settings, profiles=profiles_map)
|
||||||
|
|
||||||
return PendingBuildResult(
|
return PendingBuildResult(
|
||||||
pending=pending,
|
pending=pending,
|
||||||
selected_speaker_config=selected_speaker_config or None,
|
selected_speaker_config=selected_speaker_config or None,
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ import os
|
|||||||
from typing import Any, Dict, Mapping, Optional
|
from typing import Any, Dict, Mapping, Optional
|
||||||
|
|
||||||
from abogen.integrations.calibre_opds import CalibreOPDSClient
|
from abogen.integrations.calibre_opds import CalibreOPDSClient
|
||||||
from abogen.integrations.audiobookshelf import AudiobookshelfConfig
|
|
||||||
from abogen.utils import load_config, save_config
|
from abogen.utils import load_config, save_config
|
||||||
from abogen.domain.settings_core import (
|
from abogen.domain.settings_core import (
|
||||||
CHUNK_LEVEL_OPTIONS,
|
CHUNK_LEVEL_OPTIONS,
|
||||||
@@ -11,6 +10,7 @@ from abogen.domain.settings_core import (
|
|||||||
SAVE_MODE_LABELS,
|
SAVE_MODE_LABELS,
|
||||||
_NORMALIZATION_BOOLEAN_KEYS,
|
_NORMALIZATION_BOOLEAN_KEYS,
|
||||||
_NORMALIZATION_STRING_KEYS,
|
_NORMALIZATION_STRING_KEYS,
|
||||||
|
build_audiobookshelf_config,
|
||||||
coerce_bool,
|
coerce_bool,
|
||||||
coerce_float,
|
coerce_float,
|
||||||
coerce_int,
|
coerce_int,
|
||||||
@@ -18,6 +18,7 @@ from abogen.domain.settings_core import (
|
|||||||
load_settings,
|
load_settings,
|
||||||
llm_ready,
|
llm_ready,
|
||||||
settings_defaults,
|
settings_defaults,
|
||||||
|
stored_integration_config,
|
||||||
)
|
)
|
||||||
|
|
||||||
_NORMALIZATION_GROUPS = [
|
_NORMALIZATION_GROUPS = [
|
||||||
@@ -124,20 +125,8 @@ def load_integration_settings() -> Dict[str, Dict[str, Any]]:
|
|||||||
return integrations
|
return integrations
|
||||||
|
|
||||||
|
|
||||||
def stored_integration_config(name: str) -> Dict[str, Any]:
|
# stored_integration_config and build_audiobookshelf_config are imported from
|
||||||
cfg = load_config() or {}
|
# abogen.domain.settings_core — single source of truth for integration config.
|
||||||
# Check under "integrations" first (new structure)
|
|
||||||
integrations = cfg.get("integrations")
|
|
||||||
if isinstance(integrations, Mapping):
|
|
||||||
entry = integrations.get(name)
|
|
||||||
if isinstance(entry, Mapping):
|
|
||||||
return dict(entry)
|
|
||||||
|
|
||||||
# Fallback to top-level (legacy structure)
|
|
||||||
entry = cfg.get(name)
|
|
||||||
if isinstance(entry, Mapping):
|
|
||||||
return dict(entry)
|
|
||||||
return {}
|
|
||||||
|
|
||||||
|
|
||||||
def calibre_settings_from_payload(payload: Mapping[str, Any]) -> Dict[str, Any]:
|
def calibre_settings_from_payload(payload: Mapping[str, Any]) -> Dict[str, Any]:
|
||||||
@@ -305,30 +294,6 @@ def audiobookshelf_settings_from_payload(payload: Mapping[str, Any]) -> Dict[str
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def build_audiobookshelf_config(settings: Mapping[str, Any]) -> Optional[AudiobookshelfConfig]:
|
|
||||||
base_url = str(settings.get("base_url") or "").strip()
|
|
||||||
api_token = str(settings.get("api_token") or "").strip()
|
|
||||||
library_id = str(settings.get("library_id") or "").strip()
|
|
||||||
if not (base_url and api_token and library_id):
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
timeout = float(settings.get("timeout", 3600.0))
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
timeout = 3600.0
|
|
||||||
return AudiobookshelfConfig(
|
|
||||||
base_url=base_url,
|
|
||||||
api_token=api_token,
|
|
||||||
library_id=library_id,
|
|
||||||
collection_id=(str(settings.get("collection_id") or "").strip() or None),
|
|
||||||
folder_id=(str(settings.get("folder_id") or "").strip() or None),
|
|
||||||
verify_ssl=coerce_bool(settings.get("verify_ssl"), True),
|
|
||||||
send_cover=coerce_bool(settings.get("send_cover"), True),
|
|
||||||
send_chapters=coerce_bool(settings.get("send_chapters"), True),
|
|
||||||
send_subtitles=coerce_bool(settings.get("send_subtitles"), False),
|
|
||||||
timeout=timeout,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def calibre_integration_enabled(
|
def calibre_integration_enabled(
|
||||||
integrations: Optional[Mapping[str, Any]] = None,
|
integrations: Optional[Mapping[str, Any]] = None,
|
||||||
) -> bool:
|
) -> bool:
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import io
|
import io
|
||||||
|
import logging
|
||||||
import threading
|
import threading
|
||||||
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple
|
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple
|
||||||
import numpy as np
|
import numpy as np
|
||||||
@@ -6,22 +7,15 @@ import soundfile as sf
|
|||||||
from flask import current_app, send_file
|
from flask import current_app, send_file
|
||||||
from flask.typing import ResponseReturnValue
|
from flask.typing import ResponseReturnValue
|
||||||
|
|
||||||
|
from abogen.domain.audio_helpers import to_float32
|
||||||
from abogen.domain.device import select_device as _select_device
|
from abogen.domain.device import select_device as _select_device
|
||||||
from abogen.domain.enums import Language
|
from abogen.domain.enums import Language
|
||||||
from abogen.domain.split_pattern import get_split_pattern
|
from abogen.domain.split_pattern import get_split_pattern
|
||||||
|
from abogen.domain.pronunciation import (
|
||||||
# Kokoro-specific language mapping (engine's responsibility)
|
merge_pronunciation_overrides,
|
||||||
_KOKORO_LANG_MAP = {
|
compile_pronunciation_rules,
|
||||||
Language.EN_US: "a",
|
apply_pronunciation_rules,
|
||||||
Language.EN_GB: "b",
|
)
|
||||||
Language.ES: "e",
|
|
||||||
Language.FR: "f",
|
|
||||||
Language.HI: "h",
|
|
||||||
Language.IT: "i",
|
|
||||||
Language.JA: "j",
|
|
||||||
Language.PT_BR: "p",
|
|
||||||
Language.ZH: "z",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
SAMPLE_RATE = 24000
|
SAMPLE_RATE = 24000
|
||||||
@@ -41,7 +35,7 @@ def clear_preview_pipelines() -> None:
|
|||||||
_preview_pipelines.clear()
|
_preview_pipelines.clear()
|
||||||
|
|
||||||
|
|
||||||
def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]:
|
def _resolve_pipeline(language: Language, use_gpu: bool) -> Tuple[Any, bool]:
|
||||||
devices: List[str] = ["cpu"]
|
devices: List[str] = ["cpu"]
|
||||||
if use_gpu:
|
if use_gpu:
|
||||||
preferred = _select_device()
|
preferred = _select_device()
|
||||||
@@ -51,36 +45,33 @@ def _resolve_pipeline(language: str, use_gpu: bool) -> Tuple[Any, bool]:
|
|||||||
last_error: Optional[Exception] = None
|
last_error: Optional[Exception] = None
|
||||||
for device in devices:
|
for device in devices:
|
||||||
try:
|
try:
|
||||||
|
logging.info("[preview] Trying device=%s for language=%s", device, language)
|
||||||
return get_preview_pipeline(language, device), device != "cpu"
|
return get_preview_pipeline(language, device), device != "cpu"
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
last_error = exc
|
last_error = exc
|
||||||
|
logging.warning("[preview] Device %s failed: %s", device, exc)
|
||||||
|
|
||||||
raise RuntimeError("Preview pipeline is unavailable") from last_error
|
raise RuntimeError("Preview pipeline is unavailable") from last_error
|
||||||
|
|
||||||
|
|
||||||
def get_preview_pipeline(language: str, device: str) -> Any:
|
def get_preview_pipeline(language: Language, device: str) -> Any:
|
||||||
# Convert Language enum to Kokoro single-letter code
|
key = (language, device)
|
||||||
try:
|
|
||||||
lang = Language.from_str(language) if not isinstance(language, Language) else language
|
|
||||||
except ValueError:
|
|
||||||
lang = Language.EN_US
|
|
||||||
kokoro_code = _KOKORO_LANG_MAP.get(lang, "a")
|
|
||||||
|
|
||||||
key = (kokoro_code, device)
|
|
||||||
with _preview_pipeline_lock:
|
with _preview_pipeline_lock:
|
||||||
pipeline = _preview_pipelines.get(key)
|
pipeline = _preview_pipelines.get(key)
|
||||||
if pipeline is not None:
|
if pipeline is not None:
|
||||||
|
logging.info("[preview] Using cached pipeline for %s/%s", language, device)
|
||||||
return pipeline
|
return pipeline
|
||||||
from abogen.tts_plugin.utils import create_pipeline
|
from abogen.tts_plugin.utils import create_pipeline
|
||||||
|
|
||||||
pipeline = create_pipeline("kokoro", lang_code=kokoro_code, device=device)
|
logging.info("[preview] Creating pipeline: provider=kokoro language=%s device=%s", language, device)
|
||||||
|
pipeline = create_pipeline("kokoro", language=language, device=device)
|
||||||
_preview_pipelines[key] = pipeline
|
_preview_pipelines[key] = pipeline
|
||||||
return pipeline
|
return pipeline
|
||||||
|
|
||||||
def generate_preview_audio(
|
def generate_preview_audio(
|
||||||
text: str,
|
text: str,
|
||||||
voice_spec: str,
|
voice_spec: str,
|
||||||
language: str,
|
language: Language,
|
||||||
speed: float,
|
speed: float,
|
||||||
use_gpu: bool,
|
use_gpu: bool,
|
||||||
tts_provider: str = "kokoro",
|
tts_provider: str = "kokoro",
|
||||||
@@ -100,8 +91,6 @@ def generate_preview_audio(
|
|||||||
source_text = text
|
source_text = text
|
||||||
if pronunciation_overrides or manual_overrides or speakers:
|
if pronunciation_overrides or manual_overrides or speakers:
|
||||||
try:
|
try:
|
||||||
from abogen.webui import conversion_runner as runner
|
|
||||||
|
|
||||||
class _PreviewJob:
|
class _PreviewJob:
|
||||||
def __init__(self):
|
def __init__(self):
|
||||||
self.language = language
|
self.language = language
|
||||||
@@ -111,9 +100,9 @@ def generate_preview_audio(
|
|||||||
self.pronunciation_overrides = list(pronunciation_overrides or [])
|
self.pronunciation_overrides = list(pronunciation_overrides or [])
|
||||||
|
|
||||||
job = _PreviewJob()
|
job = _PreviewJob()
|
||||||
merged = runner._merge_pronunciation_overrides(job)
|
merged = merge_pronunciation_overrides(job)
|
||||||
rules = runner._compile_pronunciation_rules(merged)
|
rules = compile_pronunciation_rules(merged)
|
||||||
source_text = runner._apply_pronunciation_rules(source_text, rules)
|
source_text = apply_pronunciation_rules(source_text, rules)
|
||||||
except Exception:
|
except Exception:
|
||||||
current_app.logger.exception("Preview override application failed; using raw text")
|
current_app.logger.exception("Preview override application failed; using raw text")
|
||||||
source_text = text
|
source_text = text
|
||||||
@@ -128,12 +117,12 @@ def generate_preview_audio(
|
|||||||
current_app.logger.exception("Preview normalization failed; using raw text")
|
current_app.logger.exception("Preview normalization failed; using raw text")
|
||||||
normalized_text = source_text
|
normalized_text = source_text
|
||||||
|
|
||||||
preview_split = get_split_pattern(str(language or "a"), "Disabled")
|
preview_split = get_split_pattern(language, "Disabled")
|
||||||
|
|
||||||
if provider == "supertonic":
|
if provider == "supertonic":
|
||||||
from abogen.tts_plugin.utils import create_pipeline
|
from abogen.tts_plugin.utils import create_pipeline
|
||||||
|
|
||||||
pipeline = create_pipeline("supertonic")
|
pipeline = create_pipeline("supertonic", language=language)
|
||||||
segments = pipeline(
|
segments = pipeline(
|
||||||
normalized_text,
|
normalized_text,
|
||||||
voice=voice_spec,
|
voice=voice_spec,
|
||||||
@@ -148,9 +137,9 @@ def generate_preview_audio(
|
|||||||
|
|
||||||
voice_choice: Any = voice_spec
|
voice_choice: Any = voice_spec
|
||||||
if voice_spec and "*" in voice_spec:
|
if voice_spec and "*" in voice_spec:
|
||||||
from abogen.voice_formulas import get_new_voice
|
from abogen.domain.voice_loader import resolve_voice
|
||||||
|
|
||||||
voice_choice = get_new_voice(pipeline, voice_spec, pipeline_uses_gpu)
|
voice_choice = resolve_voice(voice_spec, pipeline, pipeline_uses_gpu)
|
||||||
|
|
||||||
segments = pipeline(
|
segments = pipeline(
|
||||||
normalized_text,
|
normalized_text,
|
||||||
@@ -167,7 +156,7 @@ def generate_preview_audio(
|
|||||||
graphemes = getattr(segment, "graphemes", "").strip()
|
graphemes = getattr(segment, "graphemes", "").strip()
|
||||||
if not graphemes:
|
if not graphemes:
|
||||||
continue
|
continue
|
||||||
audio = _to_float32(getattr(segment, "audio", None))
|
audio = to_float32(getattr(segment, "audio", None))
|
||||||
if audio.size == 0:
|
if audio.size == 0:
|
||||||
continue
|
continue
|
||||||
remaining = max_samples - accumulated
|
remaining = max_samples - accumulated
|
||||||
@@ -191,7 +180,7 @@ def generate_preview_audio(
|
|||||||
def synthesize_preview(
|
def synthesize_preview(
|
||||||
text: str,
|
text: str,
|
||||||
voice_spec: str,
|
voice_spec: str,
|
||||||
language: str,
|
language: Language,
|
||||||
speed: float,
|
speed: float,
|
||||||
use_gpu: bool,
|
use_gpu: bool,
|
||||||
tts_provider: str = "kokoro",
|
tts_provider: str = "kokoro",
|
||||||
|
|||||||
@@ -1,9 +1,7 @@
|
|||||||
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
|
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
|
||||||
|
|
||||||
from abogen.speaker_configs import slugify_label
|
from abogen.speaker_configs import slugify_label
|
||||||
from abogen.speaker_analysis import analyze_speakers
|
|
||||||
from abogen.webui.routes.utils.settings import load_settings, settings_defaults, _DEFAULT_ANALYSIS_THRESHOLD, _CHUNK_LEVEL_OPTIONS, _APOSTROPHE_MODE_OPTIONS, _NORMALIZATION_GROUPS
|
from abogen.webui.routes.utils.settings import load_settings, settings_defaults, _DEFAULT_ANALYSIS_THRESHOLD, _CHUNK_LEVEL_OPTIONS, _APOSTROPHE_MODE_OPTIONS, _NORMALIZATION_GROUPS
|
||||||
from abogen.webui.routes.utils.common import split_profile_spec
|
|
||||||
from abogen.voice_profiles import (
|
from abogen.voice_profiles import (
|
||||||
load_profiles,
|
load_profiles,
|
||||||
serialize_profiles,
|
serialize_profiles,
|
||||||
@@ -18,282 +16,8 @@ from abogen.constants import (
|
|||||||
)
|
)
|
||||||
from abogen.tts_plugin.utils import get_voices
|
from abogen.tts_plugin.utils import get_voices
|
||||||
from abogen.speaker_configs import list_configs
|
from abogen.speaker_configs import list_configs
|
||||||
|
from abogen.domain.voice_resolution import formula_from_profile
|
||||||
|
from abogen.domain.voice_catalog import build_voice_catalog, filter_voice_catalog
|
||||||
def build_narrator_roster(
|
|
||||||
voice: str,
|
|
||||||
voice_profile: Optional[str],
|
|
||||||
existing: Optional[Mapping[str, Any]] = None,
|
|
||||||
) -> Dict[str, Any]:
|
|
||||||
roster: Dict[str, Any] = {
|
|
||||||
"narrator": {
|
|
||||||
"id": "narrator",
|
|
||||||
"label": "Narrator",
|
|
||||||
"voice": voice,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if voice_profile:
|
|
||||||
roster["narrator"]["voice_profile"] = voice_profile
|
|
||||||
existing_entry: Optional[Mapping[str, Any]] = None
|
|
||||||
if existing is not None:
|
|
||||||
existing_entry = existing.get("narrator") if isinstance(existing, Mapping) else None
|
|
||||||
if isinstance(existing_entry, Mapping):
|
|
||||||
roster_entry = roster["narrator"]
|
|
||||||
for key in ("label", "voice", "voice_profile", "voice_formula", "pronunciation"):
|
|
||||||
value = existing_entry.get(key)
|
|
||||||
if value is not None and value != "":
|
|
||||||
roster_entry[key] = value
|
|
||||||
return roster
|
|
||||||
|
|
||||||
|
|
||||||
def build_speaker_roster(
|
|
||||||
analysis: Dict[str, Any],
|
|
||||||
base_voice: str,
|
|
||||||
voice_profile: Optional[str],
|
|
||||||
existing: Optional[Mapping[str, Any]] = None,
|
|
||||||
order: Optional[Iterable[str]] = None,
|
|
||||||
) -> Dict[str, Any]:
|
|
||||||
roster = build_narrator_roster(base_voice, voice_profile, existing)
|
|
||||||
existing_map: Dict[str, Any] = dict(existing) if isinstance(existing, Mapping) else {}
|
|
||||||
speakers = analysis.get("speakers", {}) if isinstance(analysis, dict) else {}
|
|
||||||
ordered_ids: Iterable[str]
|
|
||||||
if order is not None:
|
|
||||||
ordered_ids = [sid for sid in order if sid in speakers]
|
|
||||||
else:
|
|
||||||
ordered_ids = speakers.keys()
|
|
||||||
|
|
||||||
for speaker_id in ordered_ids:
|
|
||||||
payload = speakers.get(speaker_id, {})
|
|
||||||
if speaker_id == "narrator":
|
|
||||||
continue
|
|
||||||
if isinstance(payload, Mapping) and payload.get("suppressed"):
|
|
||||||
continue
|
|
||||||
previous = existing_map.get(speaker_id)
|
|
||||||
roster[speaker_id] = {
|
|
||||||
"id": speaker_id,
|
|
||||||
"label": payload.get("label") or speaker_id.replace("_", " ").title(),
|
|
||||||
"analysis_confidence": payload.get("confidence"),
|
|
||||||
"analysis_count": payload.get("count"),
|
|
||||||
"gender": payload.get("gender", "unknown"),
|
|
||||||
}
|
|
||||||
detected_gender = payload.get("detected_gender")
|
|
||||||
if detected_gender:
|
|
||||||
roster[speaker_id]["detected_gender"] = detected_gender
|
|
||||||
samples = payload.get("sample_quotes")
|
|
||||||
if isinstance(samples, list):
|
|
||||||
roster[speaker_id]["sample_quotes"] = samples
|
|
||||||
if isinstance(previous, Mapping):
|
|
||||||
for key in ("voice", "voice_profile", "voice_formula", "resolved_voice", "pronunciation"):
|
|
||||||
value = previous.get(key)
|
|
||||||
if value is not None and value != "":
|
|
||||||
roster[speaker_id][key] = value
|
|
||||||
if "sample_quotes" not in roster[speaker_id]:
|
|
||||||
prev_samples = previous.get("sample_quotes")
|
|
||||||
if isinstance(prev_samples, list):
|
|
||||||
roster[speaker_id]["sample_quotes"] = prev_samples
|
|
||||||
if "detected_gender" not in roster[speaker_id]:
|
|
||||||
prev_detected = previous.get("detected_gender")
|
|
||||||
if isinstance(prev_detected, str) and prev_detected:
|
|
||||||
roster[speaker_id]["detected_gender"] = prev_detected
|
|
||||||
return roster
|
|
||||||
|
|
||||||
|
|
||||||
def match_configured_speaker(
|
|
||||||
config_speakers: Mapping[str, Any],
|
|
||||||
roster_id: str,
|
|
||||||
roster_label: str,
|
|
||||||
) -> Optional[Mapping[str, Any]]:
|
|
||||||
if not config_speakers:
|
|
||||||
return None
|
|
||||||
entry = config_speakers.get(roster_id)
|
|
||||||
if entry:
|
|
||||||
return cast(Mapping[str, Any], entry)
|
|
||||||
slug = slugify_label(roster_label)
|
|
||||||
if slug != roster_id and slug in config_speakers:
|
|
||||||
return cast(Mapping[str, Any], config_speakers[slug])
|
|
||||||
lower_label = roster_label.strip().lower()
|
|
||||||
for record in config_speakers.values():
|
|
||||||
if not isinstance(record, Mapping):
|
|
||||||
continue
|
|
||||||
if str(record.get("label", "")).strip().lower() == lower_label:
|
|
||||||
return record
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def apply_speaker_config_to_roster(
|
|
||||||
roster: Mapping[str, Any],
|
|
||||||
config: Optional[Mapping[str, Any]],
|
|
||||||
*,
|
|
||||||
persist_changes: bool = False,
|
|
||||||
fallback_languages: Optional[Iterable[str]] = None,
|
|
||||||
) -> Tuple[Dict[str, Any], List[str], Optional[Dict[str, Any]]]:
|
|
||||||
if not isinstance(roster, Mapping):
|
|
||||||
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
|
|
||||||
return {}, effective_languages, None
|
|
||||||
updated_roster: Dict[str, Any] = {key: dict(value) for key, value in roster.items() if isinstance(value, Mapping)}
|
|
||||||
if not config:
|
|
||||||
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
|
|
||||||
return updated_roster, effective_languages, None
|
|
||||||
|
|
||||||
speakers_map = config.get("speakers")
|
|
||||||
if not isinstance(speakers_map, Mapping):
|
|
||||||
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
|
|
||||||
return updated_roster, effective_languages, None
|
|
||||||
|
|
||||||
config_languages = config.get("languages")
|
|
||||||
if isinstance(config_languages, list):
|
|
||||||
allowed_languages = [code for code in config_languages if isinstance(code, str) and code]
|
|
||||||
else:
|
|
||||||
allowed_languages = []
|
|
||||||
if not allowed_languages and fallback_languages:
|
|
||||||
allowed_languages = [code for code in fallback_languages if isinstance(code, str) and code]
|
|
||||||
|
|
||||||
default_voice = config.get("default_voice") if isinstance(config.get("default_voice"), str) else ""
|
|
||||||
used_voices = {entry.get("resolved_voice") or entry.get("voice") for entry in updated_roster.values()} - {None}
|
|
||||||
narrator_voice = ""
|
|
||||||
narrator_entry = updated_roster.get("narrator") if isinstance(updated_roster, Mapping) else None
|
|
||||||
if isinstance(narrator_entry, Mapping):
|
|
||||||
narrator_voice = str(
|
|
||||||
narrator_entry.get("resolved_voice")
|
|
||||||
or narrator_entry.get("default_voice")
|
|
||||||
or ""
|
|
||||||
).strip()
|
|
||||||
if narrator_voice:
|
|
||||||
used_voices.add(narrator_voice)
|
|
||||||
|
|
||||||
config_changed = False
|
|
||||||
new_config_payload: Dict[str, Any] = {
|
|
||||||
"language": config.get("language", "a"),
|
|
||||||
"languages": allowed_languages,
|
|
||||||
"default_voice": default_voice,
|
|
||||||
"speakers": dict(speakers_map),
|
|
||||||
"version": config.get("version", 1),
|
|
||||||
"notes": config.get("notes", ""),
|
|
||||||
}
|
|
||||||
|
|
||||||
speakers_payload = new_config_payload["speakers"]
|
|
||||||
|
|
||||||
for speaker_id, roster_entry in updated_roster.items():
|
|
||||||
if speaker_id == "narrator":
|
|
||||||
continue
|
|
||||||
label = str(roster_entry.get("label") or speaker_id)
|
|
||||||
config_entry = match_configured_speaker(speakers_map, speaker_id, label)
|
|
||||||
if config_entry is None:
|
|
||||||
continue
|
|
||||||
voice_id = str(config_entry.get("voice") or "").strip()
|
|
||||||
voice_profile = str(config_entry.get("voice_profile") or "").strip()
|
|
||||||
voice_formula = str(config_entry.get("voice_formula") or "").strip()
|
|
||||||
resolved_voice = str(config_entry.get("resolved_voice") or "").strip()
|
|
||||||
languages = config_entry.get("languages") if isinstance(config_entry.get("languages"), list) else []
|
|
||||||
chosen_voice = resolved_voice or voice_formula or voice_id or roster_entry.get("voice")
|
|
||||||
usable_languages = languages or allowed_languages
|
|
||||||
|
|
||||||
if chosen_voice:
|
|
||||||
roster_entry["resolved_voice"] = chosen_voice
|
|
||||||
roster_entry["voice"] = chosen_voice if not voice_profile and not voice_formula else roster_entry.get("voice", chosen_voice)
|
|
||||||
if voice_profile:
|
|
||||||
roster_entry["voice_profile"] = voice_profile
|
|
||||||
if voice_formula:
|
|
||||||
roster_entry["voice_formula"] = voice_formula
|
|
||||||
roster_entry["resolved_voice"] = voice_formula
|
|
||||||
if not voice_formula and not voice_profile and resolved_voice:
|
|
||||||
roster_entry["resolved_voice"] = resolved_voice
|
|
||||||
roster_entry["config_languages"] = usable_languages or []
|
|
||||||
|
|
||||||
if chosen_voice:
|
|
||||||
used_voices.add(chosen_voice)
|
|
||||||
|
|
||||||
# persist updates back to config payload if required
|
|
||||||
if persist_changes:
|
|
||||||
slug = config_entry.get("id") or slugify_label(label)
|
|
||||||
speakers_payload[slug] = {
|
|
||||||
"id": slug,
|
|
||||||
"label": label,
|
|
||||||
"gender": config_entry.get("gender", "unknown"),
|
|
||||||
"voice": voice_id,
|
|
||||||
"voice_profile": voice_profile,
|
|
||||||
"voice_formula": voice_formula,
|
|
||||||
"resolved_voice": roster_entry.get("resolved_voice", resolved_voice or voice_id),
|
|
||||||
"languages": usable_languages,
|
|
||||||
}
|
|
||||||
|
|
||||||
new_config = new_config_payload if (persist_changes and config_changed) else None
|
|
||||||
return updated_roster, allowed_languages, new_config
|
|
||||||
|
|
||||||
|
|
||||||
def filter_voice_catalog(
|
|
||||||
catalog: Iterable[Mapping[str, Any]],
|
|
||||||
*,
|
|
||||||
gender: str,
|
|
||||||
allowed_languages: Optional[Iterable[str]] = None,
|
|
||||||
) -> List[str]:
|
|
||||||
allowed_set = {code.lower() for code in (allowed_languages or []) if isinstance(code, str) and code}
|
|
||||||
gender_normalized = (gender or "unknown").lower()
|
|
||||||
gender_code = ""
|
|
||||||
if gender_normalized == "male":
|
|
||||||
gender_code = "m"
|
|
||||||
elif gender_normalized == "female":
|
|
||||||
gender_code = "f"
|
|
||||||
|
|
||||||
matches: List[str] = []
|
|
||||||
seen: set[str] = set()
|
|
||||||
|
|
||||||
def _consider(entry: Mapping[str, Any]) -> None:
|
|
||||||
voice_id = entry.get("id")
|
|
||||||
if not isinstance(voice_id, str) or not voice_id:
|
|
||||||
return
|
|
||||||
if voice_id in seen:
|
|
||||||
return
|
|
||||||
seen.add(voice_id)
|
|
||||||
matches.append(voice_id)
|
|
||||||
|
|
||||||
primary: List[Mapping[str, Any]] = []
|
|
||||||
fallback: List[Mapping[str, Any]] = []
|
|
||||||
for entry in catalog:
|
|
||||||
if not isinstance(entry, Mapping):
|
|
||||||
continue
|
|
||||||
voice_lang = str(entry.get("language", "")).lower()
|
|
||||||
voice_gender_code = str(entry.get("gender_code", "")).lower()
|
|
||||||
if allowed_set and voice_lang not in allowed_set:
|
|
||||||
continue
|
|
||||||
if gender_code and voice_gender_code != gender_code:
|
|
||||||
fallback.append(entry)
|
|
||||||
continue
|
|
||||||
primary.append(entry)
|
|
||||||
|
|
||||||
for entry in primary:
|
|
||||||
_consider(entry)
|
|
||||||
|
|
||||||
if not matches:
|
|
||||||
for entry in fallback:
|
|
||||||
_consider(entry)
|
|
||||||
|
|
||||||
if not matches:
|
|
||||||
for entry in catalog:
|
|
||||||
if isinstance(entry, Mapping):
|
|
||||||
_consider(entry)
|
|
||||||
|
|
||||||
return matches
|
|
||||||
|
|
||||||
|
|
||||||
def build_voice_catalog() -> List[Dict[str, str]]:
|
|
||||||
catalog: List[Dict[str, str]] = []
|
|
||||||
gender_map = {"f": "Female", "m": "Male"}
|
|
||||||
for voice_id in get_voices("kokoro"):
|
|
||||||
prefix, _, rest = voice_id.partition("_")
|
|
||||||
language_code = prefix[0] if prefix else "a"
|
|
||||||
gender_code = prefix[1] if len(prefix) > 1 else ""
|
|
||||||
catalog.append(
|
|
||||||
{
|
|
||||||
"id": voice_id,
|
|
||||||
"language": language_code,
|
|
||||||
"language_label": LANGUAGE_DESCRIPTIONS.get(language_code, language_code.upper()),
|
|
||||||
"gender": gender_map.get(gender_code, "Unknown"),
|
|
||||||
"gender_code": gender_code,
|
|
||||||
"display_name": rest.replace("_", " ").title() if rest else voice_id,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return catalog
|
|
||||||
|
|
||||||
|
|
||||||
def inject_recommended_voices(
|
def inject_recommended_voices(
|
||||||
@@ -385,177 +109,6 @@ def extract_speaker_config_form(form: Mapping[str, Any]) -> Tuple[str, Dict[str,
|
|||||||
return name, payload, errors
|
return name, payload, errors
|
||||||
|
|
||||||
|
|
||||||
def prepare_speaker_metadata(
|
|
||||||
*,
|
|
||||||
chapters: List[Dict[str, Any]],
|
|
||||||
chunks: List[Dict[str, Any]],
|
|
||||||
analysis_chunks: Optional[List[Dict[str, Any]]] = None,
|
|
||||||
voice: str,
|
|
||||||
voice_profile: Optional[str],
|
|
||||||
threshold: int,
|
|
||||||
existing_roster: Optional[Mapping[str, Any]] = None,
|
|
||||||
run_analysis: bool = True,
|
|
||||||
speaker_config: Optional[Mapping[str, Any]] = None,
|
|
||||||
apply_config: bool = False,
|
|
||||||
persist_config: bool = False,
|
|
||||||
) -> tuple[List[Dict[str, Any]], Dict[str, Any], Dict[str, Any], List[str], Optional[Dict[str, Any]]]:
|
|
||||||
chunk_list = [dict(chunk) for chunk in chunks]
|
|
||||||
analysis_source = [dict(chunk) for chunk in (analysis_chunks or chunks)]
|
|
||||||
threshold_value = max(1, int(threshold))
|
|
||||||
analysis_enabled = run_analysis
|
|
||||||
settings_state = load_settings()
|
|
||||||
global_random_languages = [
|
|
||||||
code
|
|
||||||
for code in settings_state.get("speaker_random_languages", [])
|
|
||||||
if isinstance(code, str) and code
|
|
||||||
]
|
|
||||||
|
|
||||||
if not analysis_enabled:
|
|
||||||
for chunk in chunk_list:
|
|
||||||
chunk["speaker_id"] = "narrator"
|
|
||||||
chunk["speaker_label"] = "Narrator"
|
|
||||||
analysis_payload = {
|
|
||||||
"version": "1.0",
|
|
||||||
"narrator": "narrator",
|
|
||||||
"assignments": {str(chunk.get("id")): "narrator" for chunk in chunk_list},
|
|
||||||
"speakers": {
|
|
||||||
"narrator": {
|
|
||||||
"id": "narrator",
|
|
||||||
"label": "Narrator",
|
|
||||||
"count": len(chunk_list),
|
|
||||||
"confidence": "low",
|
|
||||||
"sample_quotes": [],
|
|
||||||
"suppressed": False,
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"suppressed": [],
|
|
||||||
"stats": {
|
|
||||||
"total_chunks": len(chunk_list),
|
|
||||||
"explicit_chunks": 0,
|
|
||||||
"active_speakers": 0,
|
|
||||||
"unique_speakers": 1,
|
|
||||||
"suppressed": 0,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
roster = build_narrator_roster(voice, voice_profile, existing_roster)
|
|
||||||
narrator_pron = roster["narrator"].get("pronunciation")
|
|
||||||
if narrator_pron:
|
|
||||||
analysis_payload["speakers"]["narrator"]["pronunciation"] = narrator_pron
|
|
||||||
return chunk_list, roster, analysis_payload, [], None
|
|
||||||
|
|
||||||
analysis_result = analyze_speakers(
|
|
||||||
chapters,
|
|
||||||
analysis_source,
|
|
||||||
threshold=threshold_value,
|
|
||||||
max_speakers=0,
|
|
||||||
)
|
|
||||||
analysis_payload = analysis_result.to_dict()
|
|
||||||
speakers_payload = analysis_payload.get("speakers", {})
|
|
||||||
ordered_ids = [
|
|
||||||
sid
|
|
||||||
for sid, meta in sorted(
|
|
||||||
(
|
|
||||||
(sid, meta)
|
|
||||||
for sid, meta in speakers_payload.items()
|
|
||||||
if sid != "narrator" and isinstance(meta, Mapping) and not meta.get("suppressed")
|
|
||||||
),
|
|
||||||
key=lambda item: item[1].get("count", 0),
|
|
||||||
reverse=True,
|
|
||||||
)
|
|
||||||
]
|
|
||||||
analysis_payload["ordered_speakers"] = ordered_ids
|
|
||||||
assignments = analysis_payload.get("assignments", {})
|
|
||||||
suppressed_ids = analysis_payload.get("suppressed", [])
|
|
||||||
suppressed_details: List[Dict[str, Any]] = []
|
|
||||||
speakers_payload = analysis_payload.get("speakers", {})
|
|
||||||
if isinstance(suppressed_ids, Iterable):
|
|
||||||
for suppressed_id in suppressed_ids:
|
|
||||||
speaker_meta = speakers_payload.get(suppressed_id) if isinstance(speakers_payload, dict) else None
|
|
||||||
if isinstance(speaker_meta, dict):
|
|
||||||
suppressed_details.append(
|
|
||||||
{
|
|
||||||
"id": suppressed_id,
|
|
||||||
"label": speaker_meta.get("label")
|
|
||||||
or str(suppressed_id).replace("_", " ").title(),
|
|
||||||
"pronunciation": speaker_meta.get("pronunciation"),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
suppressed_details.append(
|
|
||||||
{
|
|
||||||
"id": suppressed_id,
|
|
||||||
"label": str(suppressed_id).replace("_", " ").title(),
|
|
||||||
"pronunciation": None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
analysis_payload["suppressed_details"] = suppressed_details
|
|
||||||
roster = build_speaker_roster(
|
|
||||||
analysis_payload,
|
|
||||||
voice,
|
|
||||||
voice_profile,
|
|
||||||
existing=existing_roster,
|
|
||||||
order=analysis_payload.get("ordered_speakers"),
|
|
||||||
)
|
|
||||||
applied_languages: List[str] = []
|
|
||||||
updated_config: Optional[Dict[str, Any]] = None
|
|
||||||
if apply_config and speaker_config:
|
|
||||||
roster, applied_languages, updated_config = apply_speaker_config_to_roster(
|
|
||||||
roster,
|
|
||||||
speaker_config,
|
|
||||||
persist_changes=persist_config,
|
|
||||||
fallback_languages=global_random_languages,
|
|
||||||
)
|
|
||||||
speakers_payload = analysis_payload.get("speakers")
|
|
||||||
if isinstance(speakers_payload, dict):
|
|
||||||
for roster_id, roster_payload in roster.items():
|
|
||||||
speaker_meta = speakers_payload.get(roster_id)
|
|
||||||
if isinstance(speaker_meta, dict):
|
|
||||||
for key in ("voice", "voice_profile", "voice_formula", "resolved_voice"):
|
|
||||||
value = roster_payload.get(key)
|
|
||||||
if value:
|
|
||||||
speaker_meta[key] = value
|
|
||||||
effective_languages: List[str] = []
|
|
||||||
if applied_languages:
|
|
||||||
effective_languages = applied_languages
|
|
||||||
elif isinstance(analysis_payload.get("config_languages"), list):
|
|
||||||
effective_languages = [
|
|
||||||
code for code in analysis_payload.get("config_languages", []) if isinstance(code, str) and code
|
|
||||||
]
|
|
||||||
elif global_random_languages:
|
|
||||||
effective_languages = list(global_random_languages)
|
|
||||||
|
|
||||||
if effective_languages:
|
|
||||||
analysis_payload["config_languages"] = effective_languages
|
|
||||||
speakers_payload = analysis_payload.get("speakers")
|
|
||||||
if isinstance(speakers_payload, dict):
|
|
||||||
for roster_id, roster_payload in roster.items():
|
|
||||||
if roster_id in speakers_payload and isinstance(roster_payload, dict):
|
|
||||||
pronunciation_value = roster_payload.get("pronunciation")
|
|
||||||
if pronunciation_value:
|
|
||||||
speakers_payload[roster_id]["pronunciation"] = pronunciation_value
|
|
||||||
|
|
||||||
fallback_languages = effective_languages or []
|
|
||||||
inject_recommended_voices(roster, fallback_languages=fallback_languages)
|
|
||||||
|
|
||||||
for chunk in chunk_list:
|
|
||||||
chunk_id = str(chunk.get("id"))
|
|
||||||
speaker_id = assignments.get(chunk_id, "narrator")
|
|
||||||
chunk["speaker_id"] = speaker_id
|
|
||||||
speaker_meta = roster.get(speaker_id)
|
|
||||||
chunk["speaker_label"] = speaker_meta.get("label") if isinstance(speaker_meta, dict) else speaker_id
|
|
||||||
|
|
||||||
return chunk_list, roster, analysis_payload, applied_languages, updated_config
|
|
||||||
|
|
||||||
|
|
||||||
def formula_from_profile(entry: Dict[str, Any]) -> Optional[str]:
|
|
||||||
from abogen.voice_formulas import pairs_to_formula
|
|
||||||
|
|
||||||
voices = entry.get("voices") or []
|
|
||||||
if not voices:
|
|
||||||
return None
|
|
||||||
return pairs_to_formula(voices)
|
|
||||||
|
|
||||||
|
|
||||||
def template_options() -> Dict[str, Any]:
|
def template_options() -> Dict[str, Any]:
|
||||||
current_settings = load_settings()
|
current_settings = load_settings()
|
||||||
profiles = serialize_profiles()
|
profiles = serialize_profiles()
|
||||||
@@ -576,7 +129,7 @@ def template_options() -> Dict[str, Any]:
|
|||||||
)
|
)
|
||||||
voice_catalog = build_voice_catalog()
|
voice_catalog = build_voice_catalog()
|
||||||
return {
|
return {
|
||||||
"languages": LANGUAGE_DESCRIPTIONS,
|
"languages": {lang.value: label for lang, label in LANGUAGE_DESCRIPTIONS.items()},
|
||||||
"voices": get_voices("kokoro"),
|
"voices": get_voices("kokoro"),
|
||||||
"subtitle_formats": SUBTITLE_FORMATS,
|
"subtitle_formats": SUBTITLE_FORMATS,
|
||||||
"supported_langs_for_subs": SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
|
"supported_langs_for_subs": SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
|
||||||
@@ -601,83 +154,6 @@ def template_options() -> Dict[str, Any]:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def resolve_profile_voice(
|
|
||||||
profile_name: Optional[str],
|
|
||||||
*,
|
|
||||||
profiles: Optional[Mapping[str, Any]] = None,
|
|
||||||
) -> tuple[str, Optional[str]]:
|
|
||||||
if not profile_name:
|
|
||||||
return "", None
|
|
||||||
source = profiles if isinstance(profiles, Mapping) else None
|
|
||||||
if source is None:
|
|
||||||
source = load_profiles()
|
|
||||||
entry = source.get(profile_name) if isinstance(source, Mapping) else None
|
|
||||||
if not isinstance(entry, Mapping):
|
|
||||||
return "", None
|
|
||||||
formula = formula_from_profile(dict(entry)) or ""
|
|
||||||
language = entry.get("language") if isinstance(entry.get("language"), str) else None
|
|
||||||
if isinstance(language, str):
|
|
||||||
language = language.strip().lower() or None
|
|
||||||
return formula, language
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_voice_setting(
|
|
||||||
value: Any,
|
|
||||||
*,
|
|
||||||
profiles: Optional[Mapping[str, Any]] = None,
|
|
||||||
) -> tuple[str, Optional[str], Optional[str]]:
|
|
||||||
base_spec, profile_name = split_profile_spec(value)
|
|
||||||
if profile_name:
|
|
||||||
formula, language = resolve_profile_voice(profile_name, profiles=profiles)
|
|
||||||
return formula or "", profile_name, language
|
|
||||||
return base_spec, None, None
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_voice_choice(
|
|
||||||
language: str,
|
|
||||||
base_voice: str,
|
|
||||||
profile_name: str,
|
|
||||||
custom_formula: str,
|
|
||||||
profiles: Dict[str, Any],
|
|
||||||
) -> tuple[str, str, Optional[str]]:
|
|
||||||
resolved_voice = base_voice
|
|
||||||
resolved_language = language
|
|
||||||
selected_profile = None
|
|
||||||
|
|
||||||
if profile_name:
|
|
||||||
from abogen.voice_profiles import normalize_profile_entry
|
|
||||||
|
|
||||||
entry_raw = profiles.get(profile_name)
|
|
||||||
entry = normalize_profile_entry(entry_raw)
|
|
||||||
provider = str((entry or {}).get("provider") or "").strip().lower()
|
|
||||||
|
|
||||||
# Provider-aware behavior:
|
|
||||||
# - Kokoro profiles typically represent mixes (formula strings).
|
|
||||||
# - SuperTonic profiles represent a discrete voice id + settings.
|
|
||||||
# In that case, we return a speaker reference so downstream can
|
|
||||||
# resolve provider per-speaker and allow mixed-provider casting.
|
|
||||||
if provider == "supertonic":
|
|
||||||
resolved_voice = f"speaker:{profile_name}"
|
|
||||||
selected_profile = profile_name
|
|
||||||
profile_language = (entry or {}).get("language")
|
|
||||||
if profile_language:
|
|
||||||
resolved_language = str(profile_language)
|
|
||||||
else:
|
|
||||||
formula = formula_from_profile(entry or {}) if entry else None
|
|
||||||
if formula:
|
|
||||||
resolved_voice = formula
|
|
||||||
selected_profile = profile_name
|
|
||||||
profile_language = (entry or {}).get("language")
|
|
||||||
if profile_language:
|
|
||||||
resolved_language = profile_language
|
|
||||||
|
|
||||||
if custom_formula:
|
|
||||||
resolved_voice = custom_formula
|
|
||||||
selected_profile = None
|
|
||||||
|
|
||||||
return resolved_voice, resolved_language, selected_profile
|
|
||||||
|
|
||||||
|
|
||||||
def parse_voice_formula(formula: str) -> List[tuple[str, float]]:
|
def parse_voice_formula(formula: str) -> List[tuple[str, float]]:
|
||||||
voices = parse_formula_terms(formula)
|
voices = parse_formula_terms(formula)
|
||||||
total = sum(weight for _, weight in voices)
|
total = sum(weight for _, weight in voices)
|
||||||
|
|||||||
@@ -2,11 +2,14 @@ from typing import Any, Dict, List, Optional
|
|||||||
from flask import Blueprint, render_template, request, jsonify, abort, flash, redirect, url_for
|
from flask import Blueprint, render_template, request, jsonify, abort, flash, redirect, url_for
|
||||||
from flask.typing import ResponseReturnValue
|
from flask.typing import ResponseReturnValue
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.webui.routes.utils.voice import (
|
from abogen.webui.routes.utils.voice import (
|
||||||
template_options,
|
template_options,
|
||||||
|
parse_voice_formula,
|
||||||
|
)
|
||||||
|
from abogen.domain.voice_resolution import (
|
||||||
resolve_voice_setting,
|
resolve_voice_setting,
|
||||||
resolve_voice_choice,
|
resolve_voice_choice,
|
||||||
parse_voice_formula,
|
|
||||||
)
|
)
|
||||||
from abogen.webui.routes.utils.settings import load_settings, coerce_bool
|
from abogen.webui.routes.utils.settings import load_settings, coerce_bool
|
||||||
from abogen.webui.routes.utils.synthesize import synthesize_preview
|
from abogen.webui.routes.utils.synthesize import synthesize_preview
|
||||||
@@ -39,7 +42,7 @@ def test_voice() -> ResponseReturnValue:
|
|||||||
return synthesize_preview(
|
return synthesize_preview(
|
||||||
text=text,
|
text=text,
|
||||||
voice_spec=voice,
|
voice_spec=voice,
|
||||||
language="a", # Default language
|
language=Language.EN_US,
|
||||||
speed=speed,
|
speed=speed,
|
||||||
use_gpu=use_gpu,
|
use_gpu=use_gpu,
|
||||||
)
|
)
|
||||||
|
|||||||
+21
-177
@@ -14,24 +14,11 @@ from enum import Enum
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Callable, Dict, Iterable, List, Optional, Mapping
|
from typing import Any, Callable, Dict, Iterable, List, Optional, Mapping
|
||||||
|
|
||||||
from abogen.utils import get_internal_cache_path, get_user_settings_dir, load_config
|
from abogen.domain.metadata_helpers import normalize_metadata_map
|
||||||
from abogen.voice_cache import bootstrap_voice_cache
|
|
||||||
from abogen.integrations.audiobookshelf import (
|
from abogen.domain.enums import Language
|
||||||
AudiobookshelfClient,
|
from abogen.utils import console_handler, get_internal_cache_path, get_user_settings_dir
|
||||||
AudiobookshelfConfig,
|
|
||||||
AudiobookshelfUploadError,
|
|
||||||
)
|
|
||||||
from abogen.domain.metadata_helpers import (
|
|
||||||
normalize_metadata_casefold as _normalize_metadata_casefold,
|
|
||||||
split_people_field as _split_people_field,
|
|
||||||
split_simple_list as _split_simple_list,
|
|
||||||
first_nonempty as _first_nonempty,
|
|
||||||
extract_year as _extract_year,
|
|
||||||
normalize_series_sequence as _normalize_series_sequence,
|
|
||||||
build_audiobookshelf_metadata as _build_abs_metadata,
|
|
||||||
load_audiobookshelf_chapters as _load_abs_chapters,
|
|
||||||
_SERIES_SEQUENCE_TAG_KEYS,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _create_set_event() -> threading.Event:
|
def _create_set_event() -> threading.Event:
|
||||||
@@ -45,9 +32,7 @@ STATE_VERSION = 8
|
|||||||
|
|
||||||
_JOB_LOGGER = logging.getLogger("abogen.jobs")
|
_JOB_LOGGER = logging.getLogger("abogen.jobs")
|
||||||
if not _JOB_LOGGER.handlers:
|
if not _JOB_LOGGER.handlers:
|
||||||
handler = logging.StreamHandler(sys.stdout)
|
_JOB_LOGGER.addHandler(console_handler())
|
||||||
handler.setFormatter(logging.Formatter("%(asctime)s [%(levelname)s] %(message)s", "%Y-%m-%d %H:%M:%S"))
|
|
||||||
_JOB_LOGGER.addHandler(handler)
|
|
||||||
_JOB_LOGGER.propagate = False
|
_JOB_LOGGER.propagate = False
|
||||||
_JOB_LOGGER.setLevel(logging.DEBUG)
|
_JOB_LOGGER.setLevel(logging.DEBUG)
|
||||||
|
|
||||||
@@ -105,7 +90,7 @@ class Job:
|
|||||||
id: str
|
id: str
|
||||||
original_filename: str
|
original_filename: str
|
||||||
stored_path: Path
|
stored_path: Path
|
||||||
language: str
|
language: Language
|
||||||
voice: str
|
voice: str
|
||||||
speed: float
|
speed: float
|
||||||
use_gpu: bool
|
use_gpu: bool
|
||||||
@@ -265,23 +250,6 @@ class Job:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def build_audiobookshelf_metadata(job: Job) -> Dict[str, Any]:
|
|
||||||
filename = Path(job.original_filename or "").stem or job.original_filename or "Audiobook"
|
|
||||||
return _build_abs_metadata(
|
|
||||||
job.metadata_tags,
|
|
||||||
language=job.language or "",
|
|
||||||
filename=filename,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def load_audiobookshelf_chapters(job: Job) -> Optional[List[Dict[str, Any]]]:
|
|
||||||
metadata_ref = job.result.artifacts.get("metadata")
|
|
||||||
if not metadata_ref:
|
|
||||||
return None
|
|
||||||
metadata_path = metadata_ref if isinstance(metadata_ref, Path) else Path(str(metadata_ref))
|
|
||||||
return _load_abs_chapters(metadata_path)
|
|
||||||
|
|
||||||
|
|
||||||
def _existing_paths(paths: Iterable[Any]) -> List[Path]:
|
def _existing_paths(paths: Iterable[Any]) -> List[Path]:
|
||||||
resolved: List[Path] = []
|
resolved: List[Path] = []
|
||||||
for item in paths:
|
for item in paths:
|
||||||
@@ -296,7 +264,7 @@ class PendingJob:
|
|||||||
id: str
|
id: str
|
||||||
original_filename: str
|
original_filename: str
|
||||||
stored_path: Path
|
stored_path: Path
|
||||||
language: str
|
language: Language
|
||||||
voice: str
|
voice: str
|
||||||
speed: float
|
speed: float
|
||||||
use_gpu: bool
|
use_gpu: bool
|
||||||
@@ -367,7 +335,6 @@ class ConversionService:
|
|||||||
self._pending_jobs: Dict[str, PendingJob] = {}
|
self._pending_jobs: Dict[str, PendingJob] = {}
|
||||||
self._state_path = self._determine_state_path()
|
self._state_path = self._determine_state_path()
|
||||||
self._ensure_directories()
|
self._ensure_directories()
|
||||||
self._bootstrap_voice_cache()
|
|
||||||
self._load_state()
|
self._load_state()
|
||||||
|
|
||||||
# Public API ---------------------------------------------------------
|
# Public API ---------------------------------------------------------
|
||||||
@@ -384,7 +351,7 @@ class ConversionService:
|
|||||||
*,
|
*,
|
||||||
original_filename: str,
|
original_filename: str,
|
||||||
stored_path: Path,
|
stored_path: Path,
|
||||||
language: str,
|
language: Language,
|
||||||
voice: str,
|
voice: str,
|
||||||
speed: float,
|
speed: float,
|
||||||
tts_provider: str = "kokoro",
|
tts_provider: str = "kokoro",
|
||||||
@@ -428,7 +395,7 @@ class ConversionService:
|
|||||||
normalization_overrides: Optional[Mapping[str, Any]] = None,
|
normalization_overrides: Optional[Mapping[str, Any]] = None,
|
||||||
) -> Job:
|
) -> Job:
|
||||||
job_id = uuid.uuid4().hex
|
job_id = uuid.uuid4().hex
|
||||||
normalized_metadata = self._normalize_metadata_tags(metadata_tags)
|
normalized_metadata = normalize_metadata_map(metadata_tags)
|
||||||
normalized_chapters = self._normalize_chapters(chapters)
|
normalized_chapters = self._normalize_chapters(chapters)
|
||||||
normalized_chunks = self._normalize_chunks(chunks)
|
normalized_chunks = self._normalize_chunks(chunks)
|
||||||
if total_characters <= 0 and normalized_chapters:
|
if total_characters <= 0 and normalized_chapters:
|
||||||
@@ -697,23 +664,6 @@ class ConversionService:
|
|||||||
self._uploads_root.mkdir(parents=True, exist_ok=True)
|
self._uploads_root.mkdir(parents=True, exist_ok=True)
|
||||||
self._state_path.parent.mkdir(parents=True, exist_ok=True)
|
self._state_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
def _bootstrap_voice_cache(self) -> None:
|
|
||||||
try:
|
|
||||||
downloaded, errors = bootstrap_voice_cache(
|
|
||||||
on_progress=lambda msg: _JOB_LOGGER.debug("[voice cache] %s", msg)
|
|
||||||
)
|
|
||||||
except RuntimeError as exc:
|
|
||||||
_JOB_LOGGER.warning("Voice cache bootstrap skipped: %s", exc)
|
|
||||||
return
|
|
||||||
|
|
||||||
if downloaded:
|
|
||||||
count = len(downloaded)
|
|
||||||
suffix = "s" if count != 1 else ""
|
|
||||||
_JOB_LOGGER.info("Voice cache ready: downloaded %d new asset%s.", count, suffix)
|
|
||||||
if errors:
|
|
||||||
for voice_id, message in errors.items():
|
|
||||||
_JOB_LOGGER.warning("Voice cache failed for %s: %s", voice_id, message)
|
|
||||||
|
|
||||||
def _ensure_worker(self) -> None:
|
def _ensure_worker(self) -> None:
|
||||||
with self._lock:
|
with self._lock:
|
||||||
if self._worker_thread and self._worker_thread.is_alive():
|
if self._worker_thread and self._worker_thread.is_alive():
|
||||||
@@ -780,7 +730,6 @@ class ConversionService:
|
|||||||
elif job.status != JobStatus.FAILED:
|
elif job.status != JobStatus.FAILED:
|
||||||
job.status = JobStatus.COMPLETED
|
job.status = JobStatus.COMPLETED
|
||||||
job.add_log("Job completed", level="success")
|
job.add_log("Job completed", level="success")
|
||||||
self._post_completion_hooks(job)
|
|
||||||
job.finished_at = time.time()
|
job.finished_at = time.time()
|
||||||
finally:
|
finally:
|
||||||
job.pause_event.set()
|
job.pause_event.set()
|
||||||
@@ -801,105 +750,6 @@ class ConversionService:
|
|||||||
self._queue.remove(job_id)
|
self._queue.remove(job_id)
|
||||||
self._update_queue_positions_locked()
|
self._update_queue_positions_locked()
|
||||||
|
|
||||||
def _post_completion_hooks(self, job: Job) -> None:
|
|
||||||
try:
|
|
||||||
self._maybe_send_to_audiobookshelf(job)
|
|
||||||
except AudiobookshelfUploadError as exc:
|
|
||||||
job.add_log(f"Audiobookshelf upload failed: {exc}", level="error")
|
|
||||||
except Exception as exc: # pragma: no cover - defensive guard
|
|
||||||
job.add_log(f"Audiobookshelf integration error: {exc}", level="error")
|
|
||||||
|
|
||||||
def _maybe_send_to_audiobookshelf(self, job: Job) -> None:
|
|
||||||
cfg = load_config() or {}
|
|
||||||
integration_cfg = cfg.get("audiobookshelf")
|
|
||||||
if not isinstance(integration_cfg, Mapping):
|
|
||||||
return
|
|
||||||
enabled = self._coerce_bool(integration_cfg.get("enabled"), False)
|
|
||||||
auto_send = self._coerce_bool(integration_cfg.get("auto_send"), False)
|
|
||||||
if not (enabled and auto_send):
|
|
||||||
return
|
|
||||||
|
|
||||||
base_url = str(integration_cfg.get("base_url") or "").strip()
|
|
||||||
api_token = str(integration_cfg.get("api_token") or "").strip()
|
|
||||||
library_id = str(integration_cfg.get("library_id") or "").strip()
|
|
||||||
folder_id = str(integration_cfg.get("folder_id") or "").strip()
|
|
||||||
if not base_url or not api_token or not library_id:
|
|
||||||
job.add_log(
|
|
||||||
"Audiobookshelf upload skipped: configure base URL, API token, and library ID first.",
|
|
||||||
level="warning",
|
|
||||||
)
|
|
||||||
return
|
|
||||||
if not folder_id:
|
|
||||||
job.add_log(
|
|
||||||
"Audiobookshelf upload skipped: enter the folder name or ID in the Audiobookshelf settings.",
|
|
||||||
level="warning",
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
audio_ref = job.result.audio_path
|
|
||||||
audio_path = audio_ref if isinstance(audio_ref, Path) else Path(str(audio_ref)) if audio_ref else None
|
|
||||||
if not audio_path or not audio_path.exists():
|
|
||||||
job.add_log("Audiobookshelf upload skipped: audio output not found.", level="warning")
|
|
||||||
return
|
|
||||||
|
|
||||||
timeout_raw = integration_cfg.get("timeout", 3600.0)
|
|
||||||
try:
|
|
||||||
timeout_value = float(timeout_raw)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
timeout_value = 3600.0
|
|
||||||
|
|
||||||
config = AudiobookshelfConfig(
|
|
||||||
base_url=base_url,
|
|
||||||
api_token=api_token,
|
|
||||||
library_id=library_id,
|
|
||||||
collection_id=(str(integration_cfg.get("collection_id") or "").strip() or None),
|
|
||||||
folder_id=folder_id,
|
|
||||||
verify_ssl=self._coerce_bool(integration_cfg.get("verify_ssl"), True),
|
|
||||||
send_cover=self._coerce_bool(integration_cfg.get("send_cover"), True),
|
|
||||||
send_chapters=self._coerce_bool(integration_cfg.get("send_chapters"), True),
|
|
||||||
send_subtitles=self._coerce_bool(integration_cfg.get("send_subtitles"), False),
|
|
||||||
timeout=timeout_value,
|
|
||||||
)
|
|
||||||
|
|
||||||
cover_ref = job.cover_image_path
|
|
||||||
cover_path = None
|
|
||||||
if config.send_cover and cover_ref:
|
|
||||||
cover_candidate = cover_ref if isinstance(cover_ref, Path) else Path(str(cover_ref))
|
|
||||||
if cover_candidate.exists():
|
|
||||||
cover_path = cover_candidate
|
|
||||||
|
|
||||||
subtitles = _existing_paths(job.result.subtitle_paths) if config.send_subtitles else None
|
|
||||||
chapters = load_audiobookshelf_chapters(job) if config.send_chapters else None
|
|
||||||
metadata = build_audiobookshelf_metadata(job)
|
|
||||||
|
|
||||||
client = AudiobookshelfClient(config)
|
|
||||||
|
|
||||||
display_title = metadata.get("title") or audio_path.stem
|
|
||||||
try:
|
|
||||||
existing_items = client.find_existing_items(display_title, folder_id=config.folder_id)
|
|
||||||
except AudiobookshelfUploadError as exc:
|
|
||||||
job.add_log(f"Audiobookshelf lookup failed: {exc}", level="error")
|
|
||||||
return
|
|
||||||
|
|
||||||
if existing_items:
|
|
||||||
job.add_log(
|
|
||||||
f"Removing existing Audiobookshelf item(s) for '{display_title}' before upload.",
|
|
||||||
level="info",
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
client.delete_items(existing_items)
|
|
||||||
except Exception as exc:
|
|
||||||
job.add_log(f"Failed to remove existing item(s): {exc}", level="warning")
|
|
||||||
|
|
||||||
client.upload_audiobook(
|
|
||||||
audio_path,
|
|
||||||
metadata=metadata,
|
|
||||||
cover_path=cover_path,
|
|
||||||
chapters=chapters,
|
|
||||||
subtitles=subtitles,
|
|
||||||
)
|
|
||||||
job.add_log("Audiobookshelf upload queued.", level="info")
|
|
||||||
|
|
||||||
# Persistence ------------------------------------------------------
|
# Persistence ------------------------------------------------------
|
||||||
def _serialize_job(self, job: Job) -> Dict[str, Any]:
|
def _serialize_job(self, job: Job) -> Dict[str, Any]:
|
||||||
result_audio = str(job.result.audio_path) if job.result.audio_path else None
|
result_audio = str(job.result.audio_path) if job.result.audio_path else None
|
||||||
@@ -910,7 +760,7 @@ class ConversionService:
|
|||||||
"id": job.id,
|
"id": job.id,
|
||||||
"original_filename": job.original_filename,
|
"original_filename": job.original_filename,
|
||||||
"stored_path": str(job.stored_path),
|
"stored_path": str(job.stored_path),
|
||||||
"language": job.language,
|
"language": job.language.value if isinstance(job.language, Language) else str(job.language),
|
||||||
"tts_provider": getattr(job, "tts_provider", "kokoro"),
|
"tts_provider": getattr(job, "tts_provider", "kokoro"),
|
||||||
"voice": job.voice,
|
"voice": job.voice,
|
||||||
"speed": job.speed,
|
"speed": job.speed,
|
||||||
@@ -1026,11 +876,19 @@ class ConversionService:
|
|||||||
stored_path = Path(payload["stored_path"])
|
stored_path = Path(payload["stored_path"])
|
||||||
output_folder_raw = payload.get("output_folder")
|
output_folder_raw = payload.get("output_folder")
|
||||||
output_folder = Path(output_folder_raw) if output_folder_raw else None
|
output_folder = Path(output_folder_raw) if output_folder_raw else None
|
||||||
|
raw_lang = payload.get("language", "")
|
||||||
|
if isinstance(raw_lang, Language):
|
||||||
|
language = raw_lang
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
language = Language.from_str(str(raw_lang or "").strip())
|
||||||
|
except (ValueError, AttributeError):
|
||||||
|
language = Language.EN_US
|
||||||
job = Job(
|
job = Job(
|
||||||
id=payload["id"],
|
id=payload["id"],
|
||||||
original_filename=payload["original_filename"],
|
original_filename=payload["original_filename"],
|
||||||
stored_path=stored_path,
|
stored_path=stored_path,
|
||||||
language=payload.get("language", "a"),
|
language=language,
|
||||||
tts_provider=str(payload.get("tts_provider") or "kokoro"),
|
tts_provider=str(payload.get("tts_provider") or "kokoro"),
|
||||||
voice=payload.get("voice", ""),
|
voice=payload.get("voice", ""),
|
||||||
speed=float(payload.get("speed", 1.0)),
|
speed=float(payload.get("speed", 1.0)),
|
||||||
@@ -1177,20 +1035,6 @@ class ConversionService:
|
|||||||
except (TypeError, ValueError):
|
except (TypeError, ValueError):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _normalize_metadata_tags(values: Optional[Mapping[str, Any]]) -> Dict[str, str]:
|
|
||||||
if not values:
|
|
||||||
return {}
|
|
||||||
normalized: Dict[str, str] = {}
|
|
||||||
for key, raw_value in values.items():
|
|
||||||
if raw_value is None:
|
|
||||||
continue
|
|
||||||
key_str = str(key).strip()
|
|
||||||
if not key_str:
|
|
||||||
continue
|
|
||||||
normalized[key_str] = str(raw_value)
|
|
||||||
return normalized
|
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _normalize_chapters(cls, chapters: Optional[Iterable[Any]]) -> List[Dict[str, Any]]:
|
def _normalize_chapters(cls, chapters: Optional[Iterable[Any]]) -> List[Dict[str, Any]]:
|
||||||
if not chapters:
|
if not chapters:
|
||||||
@@ -1267,7 +1111,7 @@ class ConversionService:
|
|||||||
entry["enabled"] = enabled
|
entry["enabled"] = enabled
|
||||||
|
|
||||||
metadata_payload = raw_dict.get("metadata") or raw_dict.get("metadata_tags")
|
metadata_payload = raw_dict.get("metadata") or raw_dict.get("metadata_tags")
|
||||||
normalized_metadata = cls._normalize_metadata_tags(metadata_payload)
|
normalized_metadata = normalize_metadata_map(metadata_payload)
|
||||||
if normalized_metadata:
|
if normalized_metadata:
|
||||||
entry["metadata"] = normalized_metadata
|
entry["metadata"] = normalized_metadata
|
||||||
|
|
||||||
|
|||||||
@@ -29,8 +29,8 @@ def apply_form_to_settings(current: dict, form: Mapping[str, Any]) -> dict:
|
|||||||
DEFAULT_ANALYSIS_THRESHOLD,
|
DEFAULT_ANALYSIS_THRESHOLD,
|
||||||
_NORMALIZATION_BOOLEAN_KEYS,
|
_NORMALIZATION_BOOLEAN_KEYS,
|
||||||
_NORMALIZATION_STRING_KEYS,
|
_NORMALIZATION_STRING_KEYS,
|
||||||
|
stored_integration_config,
|
||||||
)
|
)
|
||||||
from abogen.webui.routes.utils.settings import stored_integration_config
|
|
||||||
from abogen.webui.routes.utils.common import extract_checkbox
|
from abogen.webui.routes.utils.common import extract_checkbox
|
||||||
from abogen.utils import load_config
|
from abogen.utils import load_config
|
||||||
# General settings
|
# General settings
|
||||||
|
|||||||
@@ -452,6 +452,9 @@ const initDashboard = () => {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
openUploadModal(dropzone);
|
openUploadModal(dropzone);
|
||||||
|
if (sourceFileInput) {
|
||||||
|
sourceFileInput.click();
|
||||||
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
dropzone.addEventListener("keydown", (event) => {
|
dropzone.addEventListener("keydown", (event) => {
|
||||||
|
|||||||
@@ -165,6 +165,7 @@ def create_engine(
|
|||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
KPipeline = _load_kpipeline()
|
KPipeline = _load_kpipeline()
|
||||||
|
from plugins.kokoro.engine import engine_language
|
||||||
|
|
||||||
# Determine repo_id from model_path or use default
|
# Determine repo_id from model_path or use default
|
||||||
repo_id = "hexgrad/Kokoro-82M"
|
repo_id = "hexgrad/Kokoro-82M"
|
||||||
@@ -172,8 +173,9 @@ def create_engine(
|
|||||||
# If a specific model path is provided, use it as repo_id
|
# If a specific model path is provided, use it as repo_id
|
||||||
repo_id = str(model_path)
|
repo_id = str(model_path)
|
||||||
|
|
||||||
|
kokoro_code = engine_language(config.language)
|
||||||
pipeline = KPipeline(
|
pipeline = KPipeline(
|
||||||
lang_code=config.lang_code,
|
lang_code=kokoro_code,
|
||||||
repo_id=repo_id,
|
repo_id=repo_id,
|
||||||
device=config.device,
|
device=config.device,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -2,6 +2,10 @@
|
|||||||
|
|
||||||
This module adapts the existing Kokoro backend to the new Engine/EngineSession
|
This module adapts the existing Kokoro backend to the new Engine/EngineSession
|
||||||
protocol. It wraps the KokoroBackend without modifying it.
|
protocol. It wraps the KokoroBackend without modifying it.
|
||||||
|
|
||||||
|
Language mapping: this is the engine's responsibility. The engine knows
|
||||||
|
which languages it supports and converts Language enum → internal format.
|
||||||
|
Callers outside this module never see engine-specific codes.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -11,15 +15,18 @@ from typing import Any
|
|||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.tts_plugin.capabilities import VoiceLister
|
from abogen.tts_plugin.capabilities import VoiceLister
|
||||||
from abogen.tts_plugin.engine import Engine, EngineSession
|
from abogen.tts_plugin.engine import Engine, EngineSession
|
||||||
from abogen.tts_plugin.errors import EngineError
|
from abogen.tts_plugin.errors import EngineError
|
||||||
from abogen.tts_plugin.manifest import VoiceManifest
|
from abogen.tts_plugin.manifest import VoiceManifest
|
||||||
from abogen.tts_plugin.types import (
|
from abogen.tts_plugin.types import (
|
||||||
AudioFormat,
|
AudioFormat,
|
||||||
|
AudioSegment,
|
||||||
Duration,
|
Duration,
|
||||||
SynthesisRequest,
|
SynthesisRequest,
|
||||||
SynthesizedAudio,
|
SynthesizedAudio,
|
||||||
|
TokenTiming,
|
||||||
)
|
)
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -27,6 +34,69 @@ logger = logging.getLogger(__name__)
|
|||||||
# Sample rate for Kokoro audio
|
# Sample rate for Kokoro audio
|
||||||
_KOKORO_SAMPLE_RATE = 24000
|
_KOKORO_SAMPLE_RATE = 24000
|
||||||
|
|
||||||
|
# Engine-internal language mapping: Language enum → kokoro code.
|
||||||
|
# ONLY visible inside this module — callers never see kokoro codes.
|
||||||
|
_KOKORO_LANG_MAP: dict[Language, str] = {
|
||||||
|
Language.EN_US: "a",
|
||||||
|
Language.EN_GB: "b",
|
||||||
|
Language.ES: "e",
|
||||||
|
Language.FR: "f",
|
||||||
|
Language.HI: "h",
|
||||||
|
Language.IT: "i",
|
||||||
|
Language.JA: "j",
|
||||||
|
Language.PT_BR: "p",
|
||||||
|
Language.ZH: "z",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Reverse mapping: engine-internal code → Language enum.
|
||||||
|
# Used by voice catalog and other places that need to convert
|
||||||
|
# engine codes back to Language enum (e.g. voice ID prefix extraction).
|
||||||
|
_CODE_TO_LANGUAGE: dict[str, Language] = {v: k for k, v in _KOKORO_LANG_MAP.items()}
|
||||||
|
|
||||||
|
|
||||||
|
def supported_languages() -> list[Language]:
|
||||||
|
"""Return the list of Language enum values this engine supports.
|
||||||
|
|
||||||
|
This is the engine's responsibility — the engine knows which
|
||||||
|
languages it supports and exposes them as Language enum values.
|
||||||
|
UI layers query this to populate language selectors.
|
||||||
|
"""
|
||||||
|
return list(_KOKORO_LANG_MAP.keys())
|
||||||
|
|
||||||
|
|
||||||
|
def engine_language(lang: Language) -> str:
|
||||||
|
"""Map a Language enum to the engine's internal code.
|
||||||
|
|
||||||
|
This is the engine's responsibility — the engine owns the mapping
|
||||||
|
between Language enum and its internal format. Callers pass Language
|
||||||
|
enum; the engine converts internally. The returned string is ONLY
|
||||||
|
used inside the engine implementation.
|
||||||
|
"""
|
||||||
|
return _KOKORO_LANG_MAP.get(lang, "a")
|
||||||
|
|
||||||
|
|
||||||
|
def language_for_code(code: str | None) -> Language:
|
||||||
|
"""Map a kokoro engine language code (single letter) to a Language enum.
|
||||||
|
|
||||||
|
Used to resolve legacy data such as old profile files that stored
|
||||||
|
kokoro letter codes. This is kokoro-specific knowledge that stays
|
||||||
|
inside the engine. Unparseable values fall back to EN_US.
|
||||||
|
"""
|
||||||
|
letter = str(code or "").strip()[:1].lower()
|
||||||
|
if letter in _CODE_TO_LANGUAGE:
|
||||||
|
return _CODE_TO_LANGUAGE[letter]
|
||||||
|
return Language.EN_US
|
||||||
|
|
||||||
|
|
||||||
|
def language_for_voice_id(voice_id: str | None) -> Language:
|
||||||
|
"""Determine which Language a voice belongs to from its voice ID.
|
||||||
|
|
||||||
|
Kokoro voice IDs encode language as a prefix (e.g. "af_heart" → "a" → EN_US).
|
||||||
|
This is kokoro-specific knowledge that stays inside the engine.
|
||||||
|
Callers pass a voice ID string; the engine returns a Language enum.
|
||||||
|
"""
|
||||||
|
return language_for_code(voice_id)
|
||||||
|
|
||||||
|
|
||||||
class KokoroSession:
|
class KokoroSession:
|
||||||
"""EngineSession implementation for Kokoro.
|
"""EngineSession implementation for Kokoro.
|
||||||
@@ -49,7 +119,9 @@ class KokoroSession:
|
|||||||
speed = request.parameters.values.get("speed", 1.0)
|
speed = request.parameters.values.get("speed", 1.0)
|
||||||
split_pattern = request.parameters.values.get("split_pattern", None)
|
split_pattern = request.parameters.values.get("split_pattern", None)
|
||||||
|
|
||||||
|
sample_rate = _KOKORO_SAMPLE_RATE
|
||||||
audio_parts: list[np.ndarray] = []
|
audio_parts: list[np.ndarray] = []
|
||||||
|
segments: list[AudioSegment] = []
|
||||||
for segment in self._pipeline(
|
for segment in self._pipeline(
|
||||||
request.text,
|
request.text,
|
||||||
voice=voice,
|
voice=voice,
|
||||||
@@ -59,7 +131,28 @@ class KokoroSession:
|
|||||||
audio = segment.audio
|
audio = segment.audio
|
||||||
if hasattr(audio, "numpy"):
|
if hasattr(audio, "numpy"):
|
||||||
audio = audio.numpy()
|
audio = audio.numpy()
|
||||||
audio_parts.append(np.asarray(audio, dtype="float32"))
|
audio = np.asarray(audio, dtype="float32")
|
||||||
|
if audio.size == 0:
|
||||||
|
continue
|
||||||
|
audio_parts.append(audio)
|
||||||
|
|
||||||
|
tokens = tuple(
|
||||||
|
TokenTiming(
|
||||||
|
text=str(tok.text),
|
||||||
|
whitespace=str(tok.whitespace or ""),
|
||||||
|
start=float(tok.start_ts or 0.0),
|
||||||
|
end=float(tok.end_ts or 0.0),
|
||||||
|
)
|
||||||
|
for tok in (getattr(segment, "tokens", None) or [])
|
||||||
|
)
|
||||||
|
segments.append(
|
||||||
|
AudioSegment(
|
||||||
|
graphemes=str(getattr(segment, "graphemes", "") or ""),
|
||||||
|
audio=audio.tobytes(),
|
||||||
|
sample_rate=sample_rate,
|
||||||
|
tokens=tokens,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
if not audio_parts:
|
if not audio_parts:
|
||||||
return SynthesizedAudio(
|
return SynthesizedAudio(
|
||||||
@@ -70,12 +163,13 @@ class KokoroSession:
|
|||||||
|
|
||||||
combined = np.concatenate(audio_parts).astype("float32", copy=False)
|
combined = np.concatenate(audio_parts).astype("float32", copy=False)
|
||||||
audio_bytes = combined.tobytes()
|
audio_bytes = combined.tobytes()
|
||||||
duration_seconds = len(combined) / _KOKORO_SAMPLE_RATE
|
duration_seconds = len(combined) / sample_rate
|
||||||
|
|
||||||
return SynthesizedAudio(
|
return SynthesizedAudio(
|
||||||
data=audio_bytes,
|
data=audio_bytes,
|
||||||
format=AudioFormat(mime="audio/wav", extension="wav"),
|
format=AudioFormat(mime="audio/wav", extension="wav"),
|
||||||
duration=Duration(seconds=duration_seconds),
|
duration=Duration(seconds=duration_seconds),
|
||||||
|
segments=tuple(segments),
|
||||||
)
|
)
|
||||||
except EngineError:
|
except EngineError:
|
||||||
raise
|
raise
|
||||||
|
|||||||
@@ -32,11 +32,12 @@ from abogen.tts_plugin.types import EngineConfig
|
|||||||
from .engine import SuperTonicEngine
|
from .engine import SuperTonicEngine
|
||||||
|
|
||||||
|
|
||||||
def _load_supertonic_pipeline() -> Any:
|
def _load_supertonic_pipeline(language: Any = None) -> Any:
|
||||||
"""Lazy-load SuperTonic dependencies and create pipeline."""
|
"""Lazy-load SuperTonic dependencies and create pipeline."""
|
||||||
from plugins.supertonic.pipeline import SupertonicPipeline
|
from plugins.supertonic.pipeline import SupertonicPipeline
|
||||||
|
|
||||||
return SupertonicPipeline(
|
return SupertonicPipeline(
|
||||||
|
language=language,
|
||||||
sample_rate=24000,
|
sample_rate=24000,
|
||||||
auto_download=True,
|
auto_download=True,
|
||||||
total_steps=5,
|
total_steps=5,
|
||||||
@@ -128,7 +129,7 @@ def create_engine(
|
|||||||
EngineError: On failure. Cleans up partially created resources.
|
EngineError: On failure. Cleans up partially created resources.
|
||||||
"""
|
"""
|
||||||
try:
|
try:
|
||||||
pipeline = _load_supertonic_pipeline()
|
pipeline = _load_supertonic_pipeline(language=config.language)
|
||||||
engine = SuperTonicEngine(pipeline)
|
engine = SuperTonicEngine(pipeline)
|
||||||
return engine
|
return engine
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
|
|||||||
@@ -12,12 +12,14 @@ from typing import Any
|
|||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.tts_plugin.capabilities import VoiceLister
|
from abogen.tts_plugin.capabilities import VoiceLister
|
||||||
from abogen.tts_plugin.engine import Engine, EngineSession
|
from abogen.tts_plugin.engine import Engine, EngineSession
|
||||||
from abogen.tts_plugin.errors import EngineError
|
from abogen.tts_plugin.errors import EngineError
|
||||||
from abogen.tts_plugin.manifest import VoiceManifest
|
from abogen.tts_plugin.manifest import VoiceManifest
|
||||||
from abogen.tts_plugin.types import (
|
from abogen.tts_plugin.types import (
|
||||||
AudioFormat,
|
AudioFormat,
|
||||||
|
AudioSegment,
|
||||||
Duration,
|
Duration,
|
||||||
SynthesisRequest,
|
SynthesisRequest,
|
||||||
SynthesizedAudio,
|
SynthesizedAudio,
|
||||||
@@ -28,6 +30,61 @@ logger = logging.getLogger(__name__)
|
|||||||
# Sample rate for SuperTonic audio
|
# Sample rate for SuperTonic audio
|
||||||
_SUPERTONIC_SAMPLE_RATE = 24000
|
_SUPERTONIC_SAMPLE_RATE = 24000
|
||||||
|
|
||||||
|
# Engine-internal language mapping: Language enum → Supertonic ISO 639-1 code.
|
||||||
|
_SUPERTONIC_LANG_MAP: dict[Language, str] = {
|
||||||
|
Language.EN_US: "en",
|
||||||
|
Language.EN_GB: "en",
|
||||||
|
Language.AR: "ar",
|
||||||
|
Language.BG: "bg",
|
||||||
|
Language.CS: "cs",
|
||||||
|
Language.DA: "da",
|
||||||
|
Language.DE: "de",
|
||||||
|
Language.EL: "el",
|
||||||
|
Language.ES: "es",
|
||||||
|
Language.ET: "et",
|
||||||
|
Language.FI: "fi",
|
||||||
|
Language.FR: "fr",
|
||||||
|
Language.HI: "hi",
|
||||||
|
Language.HR: "hr",
|
||||||
|
Language.HU: "hu",
|
||||||
|
Language.ID: "id",
|
||||||
|
Language.IT: "it",
|
||||||
|
Language.JA: "ja",
|
||||||
|
Language.KO: "ko",
|
||||||
|
Language.LT: "lt",
|
||||||
|
Language.LV: "lv",
|
||||||
|
Language.NL: "nl",
|
||||||
|
Language.PL: "pl",
|
||||||
|
Language.PT_BR: "pt",
|
||||||
|
Language.RO: "ro",
|
||||||
|
Language.RU: "ru",
|
||||||
|
Language.SK: "sk",
|
||||||
|
Language.SL: "sl",
|
||||||
|
Language.SV: "sv",
|
||||||
|
Language.TR: "tr",
|
||||||
|
Language.UK: "uk",
|
||||||
|
Language.VI: "vi",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def supported_languages() -> list[Language]:
|
||||||
|
"""Return the list of Language enum values this engine supports."""
|
||||||
|
return list(_SUPERTONIC_LANG_MAP.keys())
|
||||||
|
|
||||||
|
|
||||||
|
def engine_language(lang: Language) -> str:
|
||||||
|
"""Map a Language enum to the engine's internal ISO 639-1 code.
|
||||||
|
|
||||||
|
Raises ValueError for unsupported languages.
|
||||||
|
"""
|
||||||
|
result = _SUPERTONIC_LANG_MAP.get(lang)
|
||||||
|
if result is None:
|
||||||
|
raise ValueError(
|
||||||
|
f"Supertonic does not support language: {lang!r}. "
|
||||||
|
f"Supported: {supported_languages()}"
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
class SuperTonicSession:
|
class SuperTonicSession:
|
||||||
"""EngineSession implementation for SuperTonic.
|
"""EngineSession implementation for SuperTonic.
|
||||||
@@ -57,6 +114,7 @@ class SuperTonicSession:
|
|||||||
total_steps = int(total_steps)
|
total_steps = int(total_steps)
|
||||||
|
|
||||||
audio_parts: list[np.ndarray] = []
|
audio_parts: list[np.ndarray] = []
|
||||||
|
segments: list[AudioSegment] = []
|
||||||
for segment in self._pipeline(
|
for segment in self._pipeline(
|
||||||
request.text,
|
request.text,
|
||||||
voice=voice,
|
voice=voice,
|
||||||
@@ -64,7 +122,17 @@ class SuperTonicSession:
|
|||||||
split_pattern=split_pattern,
|
split_pattern=split_pattern,
|
||||||
total_steps=total_steps,
|
total_steps=total_steps,
|
||||||
):
|
):
|
||||||
audio_parts.append(segment.audio)
|
audio = np.asarray(segment.audio, dtype="float32")
|
||||||
|
if audio.size == 0:
|
||||||
|
continue
|
||||||
|
audio_parts.append(audio)
|
||||||
|
segments.append(
|
||||||
|
AudioSegment(
|
||||||
|
graphemes=str(getattr(segment, "graphemes", "") or ""),
|
||||||
|
audio=audio.tobytes(),
|
||||||
|
sample_rate=self._pipeline.sample_rate,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
if not audio_parts:
|
if not audio_parts:
|
||||||
return SynthesizedAudio(
|
return SynthesizedAudio(
|
||||||
@@ -83,6 +151,7 @@ class SuperTonicSession:
|
|||||||
data=audio_bytes,
|
data=audio_bytes,
|
||||||
format=AudioFormat(mime="audio/wav", extension="wav"),
|
format=AudioFormat(mime="audio/wav", extension="wav"),
|
||||||
duration=Duration(seconds=duration_seconds),
|
duration=Duration(seconds=duration_seconds),
|
||||||
|
segments=tuple(segments),
|
||||||
)
|
)
|
||||||
except EngineError:
|
except EngineError:
|
||||||
raise
|
raise
|
||||||
|
|||||||
@@ -158,6 +158,7 @@ class SupertonicPipeline:
|
|||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
|
language: Any = None,
|
||||||
sample_rate: int,
|
sample_rate: int,
|
||||||
auto_download: bool = True,
|
auto_download: bool = True,
|
||||||
total_steps: int = 5,
|
total_steps: int = 5,
|
||||||
@@ -167,6 +168,13 @@ class SupertonicPipeline:
|
|||||||
self.total_steps = int(total_steps)
|
self.total_steps = int(total_steps)
|
||||||
self.max_chunk_length = int(max_chunk_length)
|
self.max_chunk_length = int(max_chunk_length)
|
||||||
|
|
||||||
|
# Resolve language to ISO 639-1 code for Supertonic
|
||||||
|
if language is not None:
|
||||||
|
from plugins.supertonic.engine import engine_language
|
||||||
|
self._lang = engine_language(language)
|
||||||
|
else:
|
||||||
|
self._lang = "en"
|
||||||
|
|
||||||
_configure_supertonic_gpu()
|
_configure_supertonic_gpu()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
@@ -212,6 +220,7 @@ class SupertonicPipeline:
|
|||||||
max_chunk_length=self.max_chunk_length,
|
max_chunk_length=self.max_chunk_length,
|
||||||
silence_duration=0.0,
|
silence_duration=0.0,
|
||||||
verbose=False,
|
verbose=False,
|
||||||
|
lang=self._lang,
|
||||||
)
|
)
|
||||||
break
|
break
|
||||||
except ValueError as exc:
|
except ValueError as exc:
|
||||||
|
|||||||
@@ -50,6 +50,7 @@ dependencies = [
|
|||||||
"num2words>=0.5.13",
|
"num2words>=0.5.13",
|
||||||
"httpx>=0.27.0",
|
"httpx>=0.27.0",
|
||||||
"PyQt6>=6.5.0",
|
"PyQt6>=6.5.0",
|
||||||
|
"rich>=13.0.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
classifiers = [
|
classifiers = [
|
||||||
|
|||||||
@@ -14,6 +14,8 @@ from unittest.mock import MagicMock, patch
|
|||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
|
|
||||||
from abogen.tts_plugin.engine import Engine, EngineSession
|
from abogen.tts_plugin.engine import Engine, EngineSession
|
||||||
from abogen.tts_plugin.errors import EngineError
|
from abogen.tts_plugin.errors import EngineError
|
||||||
from abogen.tts_plugin.plugin_manager import PluginManager, get_plugin_manager, reset_plugin_manager
|
from abogen.tts_plugin.plugin_manager import PluginManager, get_plugin_manager, reset_plugin_manager
|
||||||
@@ -328,7 +330,7 @@ class TestRegression:
|
|||||||
manager._loaded = True
|
manager._loaded = True
|
||||||
|
|
||||||
with patch("abogen.tts_plugin.utils.get_plugin_manager", return_value=manager):
|
with patch("abogen.tts_plugin.utils.get_plugin_manager", return_value=manager):
|
||||||
backend = create_pipeline("mock_tts", lang_code="a", device="cpu")
|
backend = create_pipeline("mock_tts", language=Language.EN_US, device="cpu")
|
||||||
|
|
||||||
# Old interface: pipeline(text, voice=..., speed=..., split_pattern=...)
|
# Old interface: pipeline(text, voice=..., speed=..., split_pattern=...)
|
||||||
segments = list(backend(
|
segments = list(backend(
|
||||||
|
|||||||
@@ -3,6 +3,7 @@
|
|||||||
import pytest
|
import pytest
|
||||||
from unittest.mock import MagicMock, patch
|
from unittest.mock import MagicMock, patch
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.tts_plugin.plugin_manager import PluginManager, get_plugin_manager, reset_plugin_manager
|
from abogen.tts_plugin.plugin_manager import PluginManager, get_plugin_manager, reset_plugin_manager
|
||||||
from abogen.tts_plugin.utils import Pipeline, create_pipeline
|
from abogen.tts_plugin.utils import Pipeline, create_pipeline
|
||||||
from abogen.tts_plugin.engine import Engine, EngineSession
|
from abogen.tts_plugin.engine import Engine, EngineSession
|
||||||
@@ -175,7 +176,7 @@ class TestCreatePipelineCompat:
|
|||||||
mock_engine = FakeEngine()
|
mock_engine = FakeEngine()
|
||||||
mock_manager.create_engine.return_value = mock_engine
|
mock_manager.create_engine.return_value = mock_engine
|
||||||
|
|
||||||
backend = create_pipeline("kokoro", lang_code="a", device="cpu")
|
backend = create_pipeline("kokoro", language=Language.EN_US, device="cpu")
|
||||||
|
|
||||||
assert callable(backend)
|
assert callable(backend)
|
||||||
mock_manager.create_engine.assert_called_once()
|
mock_manager.create_engine.assert_called_once()
|
||||||
@@ -185,7 +186,7 @@ class TestCreatePipelineCompat:
|
|||||||
assert call_args.kwargs["model_path"] is None
|
assert call_args.kwargs["model_path"] is None
|
||||||
assert isinstance(call_args.kwargs["config"], EngineConfig)
|
assert isinstance(call_args.kwargs["config"], EngineConfig)
|
||||||
assert call_args.kwargs["config"].device == "cpu"
|
assert call_args.kwargs["config"].device == "cpu"
|
||||||
assert call_args.kwargs["config"].lang_code == "a"
|
assert call_args.kwargs["config"].language == Language.EN_US
|
||||||
|
|
||||||
def test_create_pipeline_raises_for_unknown_plugin(self):
|
def test_create_pipeline_raises_for_unknown_plugin(self):
|
||||||
"""create_pipeline raises KeyError for unknown plugins."""
|
"""create_pipeline raises KeyError for unknown plugins."""
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ These tests verify that value objects satisfy the architectural requirements:
|
|||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
|
from abogen.domain.enums import Language
|
||||||
from abogen.tts_plugin.types import (
|
from abogen.tts_plugin.types import (
|
||||||
AudioFormat,
|
AudioFormat,
|
||||||
Duration,
|
Duration,
|
||||||
@@ -192,23 +193,23 @@ class TestEngineConfigContract:
|
|||||||
config = EngineConfig(device="cuda:0")
|
config = EngineConfig(device="cuda:0")
|
||||||
assert config.device == "cuda:0"
|
assert config.device == "cuda:0"
|
||||||
|
|
||||||
def test_default_lang_code(self) -> None:
|
def test_default_language(self) -> None:
|
||||||
config = EngineConfig()
|
config = EngineConfig()
|
||||||
assert config.lang_code == "a"
|
assert config.language == Language.EN_US
|
||||||
|
|
||||||
def test_custom_lang_code(self) -> None:
|
def test_custom_language(self) -> None:
|
||||||
config = EngineConfig(lang_code="j")
|
config = EngineConfig(language=Language.JA)
|
||||||
assert config.lang_code == "j"
|
assert config.language == Language.JA
|
||||||
|
|
||||||
def test_immutability(self) -> None:
|
def test_immutability(self) -> None:
|
||||||
config = EngineConfig()
|
config = EngineConfig()
|
||||||
with pytest.raises(AttributeError):
|
with pytest.raises(AttributeError):
|
||||||
config.device = "cuda:0" # type: ignore[misc]
|
config.device = "cuda:0" # type: ignore[misc]
|
||||||
|
|
||||||
def test_immutability_lang_code(self) -> None:
|
def test_immutability_language(self) -> None:
|
||||||
config = EngineConfig()
|
config = EngineConfig()
|
||||||
with pytest.raises(AttributeError):
|
with pytest.raises(AttributeError):
|
||||||
config.lang_code = "j" # type: ignore[misc]
|
config.language = Language.JA # type: ignore[misc]
|
||||||
|
|
||||||
def test_unknown_keys_ignored_per_spec(self) -> None:
|
def test_unknown_keys_ignored_per_spec(self) -> None:
|
||||||
"""Architecture spec: Unknown keys are ignored (no error).
|
"""Architecture spec: Unknown keys are ignored (no error).
|
||||||
@@ -225,11 +226,11 @@ class TestEngineConfigContract:
|
|||||||
EngineConfig may contain fields that are not relevant to every plugin.
|
EngineConfig may contain fields that are not relevant to every plugin.
|
||||||
Plugins MUST ignore fields they do not need, not raise on them.
|
Plugins MUST ignore fields they do not need, not raise on them.
|
||||||
"""
|
"""
|
||||||
config = EngineConfig(device="cuda:0", lang_code="j")
|
config = EngineConfig(device="cuda:0", language=Language.JA)
|
||||||
assert config.device == "cuda:0"
|
assert config.device == "cuda:0"
|
||||||
assert config.lang_code == "j"
|
assert config.language == Language.JA
|
||||||
# A plugin that only needs device simply reads config.device
|
# A plugin that only needs device simply reads config.device
|
||||||
# and ignores config.lang_code — this must not raise.
|
# and ignores config.language — this must not raise.
|
||||||
|
|
||||||
def test_engine_config_contains_engine_instance_configuration(self) -> None:
|
def test_engine_config_contains_engine_instance_configuration(self) -> None:
|
||||||
"""Architecture Amendment #1: EngineConfig definition.
|
"""Architecture Amendment #1: EngineConfig definition.
|
||||||
@@ -238,7 +239,7 @@ class TestEngineConfigContract:
|
|||||||
Engine instance is created and that remain constant throughout
|
Engine instance is created and that remain constant throughout
|
||||||
the lifetime of that Engine.
|
the lifetime of that Engine.
|
||||||
"""
|
"""
|
||||||
config = EngineConfig(device="cpu", lang_code="a")
|
config = EngineConfig(device="cpu", language=Language.EN_US)
|
||||||
# Both fields are init-time, immutable, engine-scoped.
|
# Both fields are init-time, immutable, engine-scoped.
|
||||||
assert config.device == "cpu"
|
assert config.device == "cpu"
|
||||||
assert config.lang_code == "a"
|
assert config.language == Language.EN_US
|
||||||
|
|||||||
@@ -0,0 +1,102 @@
|
|||||||
|
"""Tests for application/chapter_selection.py."""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from abogen.application.chapter_selection import build_chapter_payload
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class FakeChapter:
|
||||||
|
title: str
|
||||||
|
text: str
|
||||||
|
|
||||||
|
|
||||||
|
class TestBuildChapterPayload:
|
||||||
|
def test_empty_chapters(self):
|
||||||
|
result = build_chapter_payload([], source_name="book.txt")
|
||||||
|
assert len(result) == 1
|
||||||
|
assert result[0]["id"] == "0000"
|
||||||
|
assert result[0]["title"] == "book.txt"
|
||||||
|
assert result[0]["text"] == ""
|
||||||
|
assert result[0]["characters"] == 0
|
||||||
|
assert result[0]["enabled"] is True
|
||||||
|
|
||||||
|
def test_single_chapter_always_enabled(self):
|
||||||
|
chapters = [FakeChapter("Chapter 1", "Once upon a time.")]
|
||||||
|
result = build_chapter_payload(chapters)
|
||||||
|
assert len(result) == 1
|
||||||
|
assert result[0]["title"] == "Chapter 1"
|
||||||
|
assert result[0]["enabled"] is True
|
||||||
|
assert result[0]["index"] == 0
|
||||||
|
assert result[0]["id"] == "0000"
|
||||||
|
|
||||||
|
def test_content_chapters_preselected(self):
|
||||||
|
chapters = [
|
||||||
|
FakeChapter("Chapter 1", "The story begins with a long enough text to pass the threshold."),
|
||||||
|
FakeChapter("Chapter 2", "The story continues with another substantial chunk of text."),
|
||||||
|
]
|
||||||
|
result = build_chapter_payload(chapters)
|
||||||
|
assert all(ch["enabled"] for ch in result)
|
||||||
|
|
||||||
|
def test_supplement_not_preselected(self):
|
||||||
|
chapters = [
|
||||||
|
FakeChapter("Chapter 1", "The story begins with a long enough text to pass the threshold."),
|
||||||
|
FakeChapter("Title Page", ""),
|
||||||
|
FakeChapter("Copyright", "All rights reserved."),
|
||||||
|
FakeChapter("Table of Contents", ""),
|
||||||
|
FakeChapter("Chapter 2", "The story continues with another substantial chunk of text."),
|
||||||
|
]
|
||||||
|
result = build_chapter_payload(chapters)
|
||||||
|
titles_enabled = {ch["title"]: ch["enabled"] for ch in result}
|
||||||
|
assert titles_enabled["Chapter 1"] is True
|
||||||
|
assert titles_enabled["Chapter 2"] is True
|
||||||
|
assert titles_enabled["Title Page"] is False
|
||||||
|
assert titles_enabled["Copyright"] is False
|
||||||
|
assert titles_enabled["Table of Contents"] is False
|
||||||
|
|
||||||
|
def test_at_least_one_enabled(self):
|
||||||
|
chapters = [
|
||||||
|
FakeChapter("Title Page", ""),
|
||||||
|
FakeChapter("Copyright", "All rights reserved."),
|
||||||
|
]
|
||||||
|
result = build_chapter_payload(chapters)
|
||||||
|
assert any(ch["enabled"] for ch in result)
|
||||||
|
|
||||||
|
def test_characters_calculated(self):
|
||||||
|
chapters = [FakeChapter("Ch1", "Hello world")]
|
||||||
|
result = build_chapter_payload(chapters)
|
||||||
|
assert result[0]["characters"] == 11
|
||||||
|
|
||||||
|
def test_ids_are_zero_padded(self):
|
||||||
|
chapters = [FakeChapter(f"Ch{i}", f"text {i}") for i in range(5)]
|
||||||
|
result = build_chapter_payload(chapters)
|
||||||
|
ids = [ch["id"] for ch in result]
|
||||||
|
assert ids == ["0000", "0001", "0002", "0003", "0004"]
|
||||||
|
|
||||||
|
def test_indices_are_sequential(self):
|
||||||
|
chapters = [FakeChapter(f"Ch{i}", f"text {i}") for i in range(3)]
|
||||||
|
result = build_chapter_payload(chapters)
|
||||||
|
indices = [ch["index"] for ch in result]
|
||||||
|
assert indices == [0, 1, 2]
|
||||||
|
|
||||||
|
def test_source_name_used_for_empty(self):
|
||||||
|
result = build_chapter_payload([], source_name="mybook.epub")
|
||||||
|
assert result[0]["title"] == "mybook.epub"
|
||||||
|
|
||||||
|
def test_default_source_name(self):
|
||||||
|
result = build_chapter_payload([])
|
||||||
|
assert result[0]["title"] == ""
|
||||||
|
|
||||||
|
def test_none_title_and_text(self):
|
||||||
|
class BadChapter:
|
||||||
|
def __init__(self):
|
||||||
|
self.title = None
|
||||||
|
self.text = None
|
||||||
|
|
||||||
|
result = build_chapter_payload([BadChapter()])
|
||||||
|
assert result[0]["title"] == ""
|
||||||
|
assert result[0]["text"] == ""
|
||||||
|
assert result[0]["enabled"] is True # single chapter always enabled
|
||||||
@@ -11,6 +11,12 @@ from unittest.mock import MagicMock, patch
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
|
from abogen.application.conversion_config import (
|
||||||
|
CoverConfig,
|
||||||
|
PronunciationConfig,
|
||||||
|
SaveConfig,
|
||||||
|
SubtitleConfig,
|
||||||
|
)
|
||||||
from abogen.application.conversion_request import ConversionRequest
|
from abogen.application.conversion_request import ConversionRequest
|
||||||
from abogen.application.conversion_models import (
|
from abogen.application.conversion_models import (
|
||||||
ChapterPlan,
|
ChapterPlan,
|
||||||
@@ -48,7 +54,7 @@ class FakeBackend:
|
|||||||
def __init__(self):
|
def __init__(self):
|
||||||
self.synthesized: List[str] = []
|
self.synthesized: List[str] = []
|
||||||
|
|
||||||
def __call__(self, text: str, *, voice: Any, speed: float = 1.0, split_pattern: str = "") -> List:
|
def __call__(self, text: str, *, voice: Any, speed: float = 1.0, split_pattern: str = "", **kwargs: Any) -> List:
|
||||||
self.synthesized.append(text)
|
self.synthesized.append(text)
|
||||||
|
|
||||||
class FakeSegment:
|
class FakeSegment:
|
||||||
@@ -106,6 +112,23 @@ class FakeVoiceResolver:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _mock_pool_and_resolver():
|
||||||
|
"""Mock PipelinePool and _create_voice_resolver for all service tests."""
|
||||||
|
fake_pool = FakePipelineProvider()
|
||||||
|
fake_resolver = FakeVoiceResolver()
|
||||||
|
with patch(
|
||||||
|
"abogen.domain.pipeline_factory.PipelinePool",
|
||||||
|
return_value=fake_pool,
|
||||||
|
), patch(
|
||||||
|
"abogen.domain.voice_loader.VoiceCache",
|
||||||
|
), patch(
|
||||||
|
"abogen.application.conversion_service._create_voice_resolver",
|
||||||
|
return_value=fake_resolver,
|
||||||
|
):
|
||||||
|
yield
|
||||||
|
|
||||||
|
|
||||||
# ─── Tests for conversion_service.py ───────────────────────────────
|
# ─── Tests for conversion_service.py ───────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
@@ -120,14 +143,11 @@ class TestConversionService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello world",
|
direct_text="Hello world",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
events = FakeEvents()
|
events = FakeEvents()
|
||||||
pipeline = FakePipelineProvider()
|
|
||||||
resolver = FakeVoiceResolver()
|
|
||||||
|
|
||||||
result = run_conversion(req, events, pipeline, resolver)
|
result = run_conversion(req, events)
|
||||||
|
|
||||||
assert result is not None
|
assert result is not None
|
||||||
assert result.audio_path is not None
|
assert result.audio_path is not None
|
||||||
@@ -141,14 +161,11 @@ class TestConversionService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
events = FakeEvents()
|
events = FakeEvents()
|
||||||
pipeline = FakePipelineProvider()
|
|
||||||
resolver = FakeVoiceResolver()
|
|
||||||
|
|
||||||
result = run_conversion(req, events, pipeline, resolver)
|
result = run_conversion(req, events)
|
||||||
|
|
||||||
log_messages = [msg for msg, _ in events.logs]
|
log_messages = [msg for msg, _ in events.logs]
|
||||||
assert any("Preparing conversion pipeline" in msg for msg in log_messages)
|
assert any("Preparing conversion pipeline" in msg for msg in log_messages)
|
||||||
@@ -164,16 +181,13 @@ class TestConversionService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
events = FakeEvents()
|
events = FakeEvents()
|
||||||
events.cancelled = True
|
events.cancelled = True
|
||||||
pipeline = FakePipelineProvider()
|
|
||||||
resolver = FakeVoiceResolver()
|
|
||||||
|
|
||||||
with pytest.raises(RuntimeError, match="Conversion cancelled"):
|
with pytest.raises(RuntimeError, match="Conversion cancelled"):
|
||||||
run_conversion(req, events, pipeline, resolver)
|
run_conversion(req, events)
|
||||||
|
|
||||||
def test_service_handles_empty_text(self):
|
def test_service_handles_empty_text(self):
|
||||||
"""Service raises ValueError for empty text."""
|
"""Service raises ValueError for empty text."""
|
||||||
@@ -181,11 +195,9 @@ class TestConversionService:
|
|||||||
|
|
||||||
req = ConversionRequest(direct_text="", voice="M1")
|
req = ConversionRequest(direct_text="", voice="M1")
|
||||||
events = FakeEvents()
|
events = FakeEvents()
|
||||||
pipeline = FakePipelineProvider()
|
|
||||||
resolver = FakeVoiceResolver()
|
|
||||||
|
|
||||||
with pytest.raises(ValueError, match="No text content"):
|
with pytest.raises(ValueError, match="No text content"):
|
||||||
run_conversion(req, events, pipeline, resolver)
|
run_conversion(req, events)
|
||||||
|
|
||||||
def test_service_multi_chapter(self):
|
def test_service_multi_chapter(self):
|
||||||
"""Service handles multi-chapter conversion."""
|
"""Service handles multi-chapter conversion."""
|
||||||
@@ -195,14 +207,11 @@ class TestConversionService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="<<CHAPTER_MARKER:Ch1>>\nText A\n<<CHAPTER_MARKER:Ch2>>\nText B",
|
direct_text="<<CHAPTER_MARKER:Ch1>>\nText A\n<<CHAPTER_MARKER:Ch2>>\nText B",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
events = FakeEvents()
|
events = FakeEvents()
|
||||||
pipeline = FakePipelineProvider()
|
|
||||||
resolver = FakeVoiceResolver()
|
|
||||||
|
|
||||||
result = run_conversion(req, events, pipeline, resolver)
|
result = run_conversion(req, events)
|
||||||
|
|
||||||
assert result.total_chapters == 2
|
assert result.total_chapters == 2
|
||||||
|
|
||||||
@@ -214,17 +223,14 @@ class TestConversionService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Body text",
|
direct_text="Body text",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
read_title_intro=True,
|
read_title_intro=True,
|
||||||
read_closing_outro=True,
|
read_closing_outro=True,
|
||||||
metadata_tags={"title": "Test Book", "author": "Author"},
|
metadata_tags={"title": "Test Book", "author": "Author"},
|
||||||
)
|
)
|
||||||
events = FakeEvents()
|
events = FakeEvents()
|
||||||
pipeline = FakePipelineProvider()
|
|
||||||
resolver = FakeVoiceResolver()
|
|
||||||
|
|
||||||
result = run_conversion(req, events, pipeline, resolver)
|
result = run_conversion(req, events)
|
||||||
|
|
||||||
assert result is not None
|
assert result is not None
|
||||||
|
|
||||||
@@ -234,17 +240,67 @@ class TestConversionService:
|
|||||||
|
|
||||||
req = ConversionRequest(direct_text="Hello", voice="M1")
|
req = ConversionRequest(direct_text="Hello", voice="M1")
|
||||||
events = FakeEvents()
|
events = FakeEvents()
|
||||||
pipeline = FakePipelineProvider()
|
|
||||||
resolver = FakeVoiceResolver()
|
|
||||||
|
|
||||||
# Mock build_conversion_plan to raise an error
|
# Mock build_conversion_plan to raise an error
|
||||||
with patch("abogen.application.conversion_service.build_conversion_plan", side_effect=RuntimeError("Test error")):
|
with patch("abogen.application.conversion_service.build_conversion_plan", side_effect=RuntimeError("Test error")):
|
||||||
with pytest.raises(RuntimeError, match="Test error"):
|
with pytest.raises(RuntimeError, match="Test error"):
|
||||||
run_conversion(req, events, pipeline, resolver)
|
run_conversion(req, events)
|
||||||
|
|
||||||
log_messages = [msg for msg, _ in events.logs]
|
log_messages = [msg for msg, _ in events.logs]
|
||||||
assert any("Conversion failed" in msg for msg in log_messages)
|
assert any("Conversion failed" in msg for msg in log_messages)
|
||||||
|
|
||||||
|
def test_tts_context_applies_normalization_overrides(self):
|
||||||
|
"""Service applies normalization_overrides from request to apostrophe config."""
|
||||||
|
from abogen.application.conversion_service import run_conversion
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="Hello",
|
||||||
|
voice="M1",
|
||||||
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
|
pronunciation=PronunciationConfig(
|
||||||
|
normalization_overrides={"normalization_numbers": False},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
events = FakeEvents()
|
||||||
|
|
||||||
|
result = run_conversion(req, events)
|
||||||
|
assert result is not None
|
||||||
|
|
||||||
|
def test_tts_context_rejects_unconfigured_llm_mode(self):
|
||||||
|
"""Service raises RuntimeError if LLM apostrophe mode is selected but unconfigured."""
|
||||||
|
from abogen.application.conversion_service import run_conversion
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="Hello",
|
||||||
|
voice="M1",
|
||||||
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
|
pronunciation=PronunciationConfig(
|
||||||
|
normalization_overrides={"normalization_apostrophe_mode": "llm"},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
events = FakeEvents()
|
||||||
|
|
||||||
|
with pytest.raises(RuntimeError, match="LLM.*apostrophe"):
|
||||||
|
run_conversion(req, events)
|
||||||
|
|
||||||
|
def test_usage_counter_populated_in_result(self):
|
||||||
|
"""usage_counter is created and accessible in result."""
|
||||||
|
from abogen.application.conversion_service import run_conversion
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="Hello",
|
||||||
|
voice="M1",
|
||||||
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
|
)
|
||||||
|
events = FakeEvents()
|
||||||
|
|
||||||
|
result = run_conversion(req, events)
|
||||||
|
assert hasattr(result, "usage_counter")
|
||||||
|
assert isinstance(result.usage_counter, dict)
|
||||||
|
|
||||||
|
|
||||||
# ─── Tests for output_layout_service.py ─────────────────────────────
|
# ─── Tests for output_layout_service.py ─────────────────────────────
|
||||||
|
|
||||||
@@ -260,8 +316,7 @@ class TestOutputLayoutService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
layout = resolve_output_layout(req)
|
layout = resolve_output_layout(req)
|
||||||
|
|
||||||
@@ -278,7 +333,7 @@ class TestOutputLayoutService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
source_path=source,
|
source_path=source,
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="save_next_to_input",
|
save=SaveConfig(mode="save_next_to_input"),
|
||||||
)
|
)
|
||||||
layout = resolve_output_layout(req)
|
layout = resolve_output_layout(req)
|
||||||
|
|
||||||
@@ -292,9 +347,11 @@ class TestOutputLayoutService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(
|
||||||
output_folder=Path(tmpdir),
|
mode="custom_folder",
|
||||||
save_as_project=True,
|
output_folder=Path(tmpdir),
|
||||||
|
save_as_project=True,
|
||||||
|
),
|
||||||
original_filename="test.wav",
|
original_filename="test.wav",
|
||||||
)
|
)
|
||||||
layout = resolve_output_layout(req)
|
layout = resolve_output_layout(req)
|
||||||
@@ -334,7 +391,7 @@ class TestOutputLayoutService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
separate_chapters_format="wav",
|
save=SaveConfig(separate_chapters_format="wav"),
|
||||||
)
|
)
|
||||||
path = resolve_chapter_path(layout, req, "Chapter 1", 1)
|
path = resolve_chapter_path(layout, req, "Chapter 1", 1)
|
||||||
|
|
||||||
@@ -353,7 +410,7 @@ class TestOutputLayoutService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
separate_chapters_format="wav",
|
save=SaveConfig(separate_chapters_format="wav"),
|
||||||
)
|
)
|
||||||
path = resolve_chapter_path(layout, req, "", 3)
|
path = resolve_chapter_path(layout, req, "", 3)
|
||||||
|
|
||||||
@@ -367,7 +424,7 @@ class TestOutputLayoutService:
|
|||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
output_format="m4b",
|
output_format="m4b",
|
||||||
merge_chapters_at_end=False,
|
save=SaveConfig(merge_chapters_at_end=False),
|
||||||
)
|
)
|
||||||
assert should_merge_output(req) is True
|
assert should_merge_output(req) is True
|
||||||
|
|
||||||
@@ -378,7 +435,7 @@ class TestOutputLayoutService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_chapters_separately=False,
|
save=SaveConfig(save_chapters_separately=False),
|
||||||
)
|
)
|
||||||
assert should_merge_output(req) is True
|
assert should_merge_output(req) is True
|
||||||
|
|
||||||
@@ -389,8 +446,10 @@ class TestOutputLayoutService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_chapters_separately=True,
|
save=SaveConfig(
|
||||||
merge_chapters_at_end=True,
|
save_chapters_separately=True,
|
||||||
|
merge_chapters_at_end=True,
|
||||||
|
),
|
||||||
)
|
)
|
||||||
assert should_merge_output(req) is True
|
assert should_merge_output(req) is True
|
||||||
|
|
||||||
@@ -401,8 +460,10 @@ class TestOutputLayoutService:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_chapters_separately=True,
|
save=SaveConfig(
|
||||||
merge_chapters_at_end=False,
|
save_chapters_separately=True,
|
||||||
|
merge_chapters_at_end=False,
|
||||||
|
),
|
||||||
)
|
)
|
||||||
assert should_merge_output(req) is False
|
assert should_merge_output(req) is False
|
||||||
|
|
||||||
@@ -432,19 +493,28 @@ class TestExecutorGaps:
|
|||||||
with pytest.raises(ValueError, match="output_layout"):
|
with pytest.raises(ValueError, match="output_layout"):
|
||||||
execute_conversion(plan, events, pipeline, resolver, tts_context)
|
execute_conversion(plan, events, pipeline, resolver, tts_context)
|
||||||
|
|
||||||
def test_executor_m4b_forces_merge(self):
|
@patch("subprocess.Popen")
|
||||||
|
def test_executor_m4b_forces_merge(self, mock_popen):
|
||||||
"""Executor forces merge for m4b format."""
|
"""Executor forces merge for m4b format."""
|
||||||
from abogen.application.conversion_executor import execute_conversion
|
from abogen.application.conversion_executor import execute_conversion
|
||||||
|
|
||||||
|
mock_proc = mock_popen.return_value
|
||||||
|
mock_proc.returncode = 0
|
||||||
|
mock_proc.wait.return_value = 0
|
||||||
|
mock_proc.stdin = MagicMock()
|
||||||
|
mock_proc.communicate.return_value = (b"", b"")
|
||||||
|
|
||||||
with tempfile.TemporaryDirectory() as tmpdir:
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(
|
||||||
output_folder=Path(tmpdir),
|
mode="custom_folder",
|
||||||
|
output_folder=Path(tmpdir),
|
||||||
|
save_chapters_separately=True,
|
||||||
|
merge_chapters_at_end=False,
|
||||||
|
),
|
||||||
output_format="m4b",
|
output_format="m4b",
|
||||||
save_chapters_separately=True,
|
|
||||||
merge_chapters_at_end=False,
|
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -485,10 +555,12 @@ class TestExecutorGaps:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(
|
||||||
output_folder=Path(tmpdir),
|
mode="custom_folder",
|
||||||
save_chapters_separately=True,
|
output_folder=Path(tmpdir),
|
||||||
merge_chapters_at_end=True,
|
save_chapters_separately=True,
|
||||||
|
merge_chapters_at_end=True,
|
||||||
|
),
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -538,8 +610,7 @@ class TestExecutorGaps:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -596,8 +667,7 @@ class TestExecutorGaps:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -637,8 +707,7 @@ class TestExecutorGaps:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
silence_between_chapters=1.0,
|
silence_between_chapters=1.0,
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
|
|||||||
@@ -928,10 +928,11 @@ class TestValueObjectsBehavioral:
|
|||||||
|
|
||||||
def test_engine_config_defaults(self) -> None:
|
def test_engine_config_defaults(self) -> None:
|
||||||
from abogen.tts_plugin.types import EngineConfig
|
from abogen.tts_plugin.types import EngineConfig
|
||||||
|
from abogen.domain.enums import Language
|
||||||
|
|
||||||
config = EngineConfig()
|
config = EngineConfig()
|
||||||
assert config.device == "cpu"
|
assert config.device == "cpu"
|
||||||
assert config.lang_code == "a"
|
assert config.language == Language.EN_US
|
||||||
|
|
||||||
def test_parameter_values_defaults(self) -> None:
|
def test_parameter_values_defaults(self) -> None:
|
||||||
pv = ParameterValues()
|
pv = ParameterValues()
|
||||||
|
|||||||
@@ -1,195 +0,0 @@
|
|||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import sys
|
|
||||||
import types
|
|
||||||
|
|
||||||
|
|
||||||
def _install_dependency_stubs() -> None:
|
|
||||||
if "ebooklib" not in sys.modules:
|
|
||||||
ebooklib_stub = types.ModuleType("ebooklib")
|
|
||||||
epub_stub = types.ModuleType("ebooklib.epub")
|
|
||||||
setattr(ebooklib_stub, "epub", epub_stub)
|
|
||||||
sys.modules["ebooklib"] = ebooklib_stub
|
|
||||||
sys.modules["ebooklib.epub"] = epub_stub
|
|
||||||
|
|
||||||
if "dotenv" not in sys.modules:
|
|
||||||
dotenv_stub = types.ModuleType("dotenv")
|
|
||||||
|
|
||||||
def _noop(*_, **__):
|
|
||||||
return None
|
|
||||||
|
|
||||||
setattr(dotenv_stub, "load_dotenv", _noop)
|
|
||||||
setattr(dotenv_stub, "find_dotenv", lambda *_, **__: "")
|
|
||||||
sys.modules["dotenv"] = dotenv_stub
|
|
||||||
|
|
||||||
if "numpy" not in sys.modules:
|
|
||||||
numpy_stub = types.ModuleType("numpy")
|
|
||||||
|
|
||||||
class _DummyArray(list):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def _zeros(shape, dtype=None):
|
|
||||||
size = 1
|
|
||||||
if isinstance(shape, int):
|
|
||||||
size = shape
|
|
||||||
elif shape:
|
|
||||||
size = 1
|
|
||||||
for dimension in shape:
|
|
||||||
size *= int(dimension)
|
|
||||||
return [0.0] * size
|
|
||||||
|
|
||||||
setattr(numpy_stub, "ndarray", _DummyArray)
|
|
||||||
setattr(numpy_stub, "zeros", _zeros)
|
|
||||||
setattr(numpy_stub, "float32", "float32")
|
|
||||||
setattr(numpy_stub, "array", lambda data, dtype=None: data)
|
|
||||||
setattr(numpy_stub, "asarray", lambda data, dtype=None: data)
|
|
||||||
setattr(
|
|
||||||
numpy_stub,
|
|
||||||
"concatenate",
|
|
||||||
lambda seq, axis=0: sum((list(item) for item in seq), []),
|
|
||||||
)
|
|
||||||
sys.modules["numpy"] = numpy_stub
|
|
||||||
|
|
||||||
if "soundfile" not in sys.modules:
|
|
||||||
soundfile_stub = types.ModuleType("soundfile")
|
|
||||||
|
|
||||||
class _DummySoundFile:
|
|
||||||
def __init__(self, *_, **__):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def write(self, *_args, **_kwargs):
|
|
||||||
return None
|
|
||||||
|
|
||||||
def close(self):
|
|
||||||
return None
|
|
||||||
|
|
||||||
setattr(soundfile_stub, "SoundFile", _DummySoundFile)
|
|
||||||
setattr(soundfile_stub, "write", lambda *_args, **_kwargs: None)
|
|
||||||
sys.modules["soundfile"] = soundfile_stub
|
|
||||||
|
|
||||||
if "fitz" not in sys.modules:
|
|
||||||
sys.modules["fitz"] = types.ModuleType("fitz")
|
|
||||||
|
|
||||||
if "markdown" not in sys.modules:
|
|
||||||
markdown_stub = types.ModuleType("markdown")
|
|
||||||
|
|
||||||
class _DummyMarkdown:
|
|
||||||
def __init__(self, *_, **__):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def convert(self, text: str) -> str:
|
|
||||||
return text
|
|
||||||
|
|
||||||
setattr(markdown_stub, "Markdown", _DummyMarkdown)
|
|
||||||
sys.modules["markdown"] = markdown_stub
|
|
||||||
|
|
||||||
if "bs4" not in sys.modules:
|
|
||||||
bs4_stub = types.ModuleType("bs4")
|
|
||||||
|
|
||||||
class _DummySoup:
|
|
||||||
def __init__(self, *_, **__):
|
|
||||||
pass
|
|
||||||
|
|
||||||
def select(self, *_, **__):
|
|
||||||
return []
|
|
||||||
|
|
||||||
def find_all(self, *_, **__):
|
|
||||||
return []
|
|
||||||
|
|
||||||
setattr(bs4_stub, "BeautifulSoup", _DummySoup)
|
|
||||||
setattr(bs4_stub, "NavigableString", str)
|
|
||||||
sys.modules["bs4"] = bs4_stub
|
|
||||||
|
|
||||||
|
|
||||||
_install_dependency_stubs()
|
|
||||||
|
|
||||||
from abogen.text_extractor import ExtractedChapter
|
|
||||||
from abogen.webui.conversion_runner import _apply_chapter_overrides, _merge_metadata
|
|
||||||
|
|
||||||
|
|
||||||
def _sample_chapters() -> list[ExtractedChapter]:
|
|
||||||
return [
|
|
||||||
ExtractedChapter(title="Chapter 1", text="Original one"),
|
|
||||||
ExtractedChapter(title="Chapter 2", text="Original two"),
|
|
||||||
ExtractedChapter(title="Chapter 3", text="Original three"),
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def test_apply_chapter_overrides_with_custom_text() -> None:
|
|
||||||
overrides = [
|
|
||||||
{"index": 0, "enabled": True, "title": "Intro", "text": "Hello world"},
|
|
||||||
{"index": 1, "enabled": False},
|
|
||||||
]
|
|
||||||
|
|
||||||
selected, metadata, diagnostics = _apply_chapter_overrides(
|
|
||||||
_sample_chapters(), overrides
|
|
||||||
)
|
|
||||||
|
|
||||||
assert len(selected) == 1
|
|
||||||
assert selected[0].title == "Intro"
|
|
||||||
assert selected[0].text == "Hello world"
|
|
||||||
assert overrides[0]["characters"] == len("Hello world")
|
|
||||||
assert metadata == {}
|
|
||||||
assert diagnostics == []
|
|
||||||
|
|
||||||
|
|
||||||
def test_apply_chapter_overrides_uses_original_content_when_text_missing() -> None:
|
|
||||||
overrides = [
|
|
||||||
{"index": 1, "enabled": True},
|
|
||||||
]
|
|
||||||
|
|
||||||
selected, metadata, diagnostics = _apply_chapter_overrides(
|
|
||||||
_sample_chapters(), overrides
|
|
||||||
)
|
|
||||||
|
|
||||||
assert len(selected) == 1
|
|
||||||
assert selected[0].title == "Chapter 2"
|
|
||||||
assert selected[0].text == "Original two"
|
|
||||||
assert overrides[0]["text"] == "Original two"
|
|
||||||
assert overrides[0]["characters"] == len("Original two")
|
|
||||||
assert metadata == {}
|
|
||||||
assert diagnostics == []
|
|
||||||
|
|
||||||
|
|
||||||
def test_apply_chapter_overrides_collects_metadata_updates() -> None:
|
|
||||||
overrides = [
|
|
||||||
{
|
|
||||||
"index": 2,
|
|
||||||
"enabled": True,
|
|
||||||
"metadata": {"artist": "Test Author", "year": 2024},
|
|
||||||
}
|
|
||||||
]
|
|
||||||
|
|
||||||
selected, metadata, diagnostics = _apply_chapter_overrides(
|
|
||||||
_sample_chapters(), overrides
|
|
||||||
)
|
|
||||||
|
|
||||||
assert len(selected) == 1
|
|
||||||
assert metadata == {"artist": "Test Author", "year": "2024"}
|
|
||||||
assert diagnostics == []
|
|
||||||
|
|
||||||
|
|
||||||
def test_apply_chapter_overrides_reports_diagnostics_for_invalid_payload() -> None:
|
|
||||||
overrides = [
|
|
||||||
{"enabled": True, "title": "Missing"},
|
|
||||||
]
|
|
||||||
|
|
||||||
selected, metadata, diagnostics = _apply_chapter_overrides(
|
|
||||||
_sample_chapters(), overrides
|
|
||||||
)
|
|
||||||
|
|
||||||
assert selected == []
|
|
||||||
assert metadata == {}
|
|
||||||
assert diagnostics and "Skipped chapter override" in diagnostics[0]
|
|
||||||
|
|
||||||
|
|
||||||
def test_merge_metadata_prefers_overrides_and_drops_none_values() -> None:
|
|
||||||
extracted = {"title": "Original", "artist": "Someone"}
|
|
||||||
overrides = {"artist": "Another", "genre": "Fiction", "year": None}
|
|
||||||
|
|
||||||
merged = _merge_metadata(extracted, overrides)
|
|
||||||
|
|
||||||
assert merged["title"] == "Original"
|
|
||||||
assert merged["artist"] == "Another"
|
|
||||||
assert merged["genre"] == "Fiction"
|
|
||||||
assert "year" not in merged
|
|
||||||
@@ -1,500 +0,0 @@
|
|||||||
"""Tests for conversion adapters (WebUI + PyQt).
|
|
||||||
|
|
||||||
Covers field mapping, event bridging, voice resolution, and cancellation behavior.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
from types import SimpleNamespace
|
|
||||||
from unittest.mock import MagicMock, patch
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from abogen.application.conversion_request import ConversionRequest
|
|
||||||
from abogen.application.conversion_ports import ResolvedVoice
|
|
||||||
|
|
||||||
|
|
||||||
# ─── WebUI adapter tests ───────────────────────────────────────────
|
|
||||||
|
|
||||||
|
|
||||||
class TestWebUIAdapter:
|
|
||||||
"""Test WebUI conversion adapter field mapping."""
|
|
||||||
|
|
||||||
def _make_job(self, **overrides):
|
|
||||||
"""Create a mock WebUI Job with default values."""
|
|
||||||
defaults = dict(
|
|
||||||
stored_path="/tmp/test.epub",
|
|
||||||
original_filename="test.epub",
|
|
||||||
language="a",
|
|
||||||
tts_provider="kokoro",
|
|
||||||
voice="M1",
|
|
||||||
voice_profile=None,
|
|
||||||
speed=1.0,
|
|
||||||
use_gpu=False,
|
|
||||||
supertonic_total_steps=5,
|
|
||||||
output_format="wav",
|
|
||||||
subtitle_mode="Disabled",
|
|
||||||
subtitle_format="srt",
|
|
||||||
max_subtitle_words=50,
|
|
||||||
save_mode="save_next_to_input",
|
|
||||||
output_folder=None,
|
|
||||||
save_chapters_separately=False,
|
|
||||||
merge_chapters_at_end=True,
|
|
||||||
separate_chapters_format="wav",
|
|
||||||
save_as_project=False,
|
|
||||||
silence_between_chapters=2.0,
|
|
||||||
chapter_intro_delay=0.0,
|
|
||||||
replace_single_newlines=False,
|
|
||||||
read_title_intro=False,
|
|
||||||
read_closing_outro=False,
|
|
||||||
auto_prefix_chapter_titles=True,
|
|
||||||
normalize_chapter_opening_caps=False,
|
|
||||||
pronunciation_overrides=[],
|
|
||||||
manual_overrides=[],
|
|
||||||
heteronym_overrides=[],
|
|
||||||
normalization_overrides={},
|
|
||||||
chapters=[],
|
|
||||||
chunks=[],
|
|
||||||
chunk_level="paragraph",
|
|
||||||
speaker_mode="single",
|
|
||||||
speakers={},
|
|
||||||
metadata_tags={},
|
|
||||||
cover_image_path=None,
|
|
||||||
cover_image_mime=None,
|
|
||||||
generate_epub3=False,
|
|
||||||
cancel_requested=False,
|
|
||||||
)
|
|
||||||
defaults.update(overrides)
|
|
||||||
return SimpleNamespace(**defaults)
|
|
||||||
|
|
||||||
def test_basic_field_mapping(self):
|
|
||||||
from abogen.webui.conversion_adapter import build_conversion_request_from_job
|
|
||||||
|
|
||||||
job = self._make_job()
|
|
||||||
req = build_conversion_request_from_job(job)
|
|
||||||
|
|
||||||
assert isinstance(req, ConversionRequest)
|
|
||||||
assert req.source_path == Path("/tmp/test.epub")
|
|
||||||
assert req.original_filename == "test.epub"
|
|
||||||
assert req.language == "a"
|
|
||||||
assert req.voice == "M1"
|
|
||||||
assert req.speed == 1.0
|
|
||||||
assert req.output_format == "wav"
|
|
||||||
|
|
||||||
def test_optional_fields_mapped(self):
|
|
||||||
from abogen.webui.conversion_adapter import build_conversion_request_from_job
|
|
||||||
|
|
||||||
job = self._make_job(
|
|
||||||
voice_profile="custom_profile",
|
|
||||||
output_folder="/output",
|
|
||||||
cover_image_path="/cover.jpg",
|
|
||||||
cover_image_mime="image/jpeg",
|
|
||||||
metadata_tags={"title": "Test"},
|
|
||||||
)
|
|
||||||
req = build_conversion_request_from_job(job)
|
|
||||||
|
|
||||||
assert req.voice_profile == "custom_profile"
|
|
||||||
assert req.output_folder == Path("/output")
|
|
||||||
assert req.cover_image_path == Path("/cover.jpg")
|
|
||||||
assert req.cover_image_mime == "image/jpeg"
|
|
||||||
assert req.metadata_tags == {"title": "Test"}
|
|
||||||
|
|
||||||
def test_none_paths_result_in_none(self):
|
|
||||||
from abogen.webui.conversion_adapter import build_conversion_request_from_job
|
|
||||||
|
|
||||||
job = self._make_job(
|
|
||||||
stored_path=None,
|
|
||||||
output_folder=None,
|
|
||||||
cover_image_path=None,
|
|
||||||
)
|
|
||||||
req = build_conversion_request_from_job(job)
|
|
||||||
|
|
||||||
assert req.source_path is None
|
|
||||||
assert req.output_folder is None
|
|
||||||
assert req.cover_image_path is None
|
|
||||||
|
|
||||||
def test_chapter_overrides_mapped(self):
|
|
||||||
from abogen.webui.conversion_adapter import build_conversion_request_from_job
|
|
||||||
|
|
||||||
chapters = [{"title": "Ch1", "voice": "F1"}]
|
|
||||||
job = self._make_job(chapters=chapters)
|
|
||||||
req = build_conversion_request_from_job(job)
|
|
||||||
|
|
||||||
assert req.chapter_overrides == chapters
|
|
||||||
|
|
||||||
def test_chunks_mapped(self):
|
|
||||||
from abogen.webui.conversion_adapter import build_conversion_request_from_job
|
|
||||||
|
|
||||||
chunks = [{"text": "Hello", "speaker": "A"}]
|
|
||||||
job = self._make_job(chunks=chunks)
|
|
||||||
req = build_conversion_request_from_job(job)
|
|
||||||
|
|
||||||
assert req.chunks == chunks
|
|
||||||
|
|
||||||
def test_pronunciation_overrides_mapped(self):
|
|
||||||
from abogen.webui.conversion_adapter import build_conversion_request_from_job
|
|
||||||
|
|
||||||
job = self._make_job(
|
|
||||||
pronunciation_overrides=["word=pron"],
|
|
||||||
manual_overrides=["manual=override"],
|
|
||||||
heteronym_overrides=["read=reed"],
|
|
||||||
)
|
|
||||||
req = build_conversion_request_from_job(job)
|
|
||||||
|
|
||||||
assert req.pronunciation_overrides == ["word=pron"]
|
|
||||||
assert req.manual_overrides == ["manual=override"]
|
|
||||||
assert req.heteronym_overrides == ["read=reed"]
|
|
||||||
|
|
||||||
def test_none_defaults_handled(self):
|
|
||||||
from abogen.webui.conversion_adapter import build_conversion_request_from_job
|
|
||||||
from abogen.domain.enums import Language, OutputFormat, SubtitleMode, SaveMode
|
|
||||||
|
|
||||||
job = self._make_job(
|
|
||||||
language=None,
|
|
||||||
voice=None,
|
|
||||||
speed=None,
|
|
||||||
output_format=None,
|
|
||||||
subtitle_mode=None,
|
|
||||||
save_mode=None,
|
|
||||||
silence_between_chapters=None,
|
|
||||||
chapter_intro_delay=None,
|
|
||||||
supertonic_total_steps=None,
|
|
||||||
max_subtitle_words=None,
|
|
||||||
)
|
|
||||||
req = build_conversion_request_from_job(job)
|
|
||||||
|
|
||||||
# None values pass through adapter; ConversionRequest.__post_init__
|
|
||||||
# applies defaults and clamping for numeric fields.
|
|
||||||
assert req.language == Language.EN_US
|
|
||||||
assert req.speed == 1.0
|
|
||||||
assert req.output_format == OutputFormat.WAV
|
|
||||||
assert req.subtitle_mode == SubtitleMode.DISABLED
|
|
||||||
assert req.save_mode == SaveMode.SAVE_NEXT_TO_INPUT
|
|
||||||
assert req.silence_between_chapters == 2.0
|
|
||||||
assert req.chapter_intro_delay == 0.0
|
|
||||||
assert req.supertonic_total_steps == 5
|
|
||||||
assert req.max_subtitle_words == 50
|
|
||||||
|
|
||||||
|
|
||||||
class TestWebUIEvents:
|
|
||||||
"""Test WebUI ConversionEvents implementation."""
|
|
||||||
|
|
||||||
def test_log_calls_add_log(self):
|
|
||||||
from abogen.webui.conversion_adapter import WebJobEvents
|
|
||||||
|
|
||||||
job = SimpleNamespace(add_log=MagicMock())
|
|
||||||
events = WebJobEvents(job)
|
|
||||||
events.log("test message", level="info")
|
|
||||||
|
|
||||||
job.add_log.assert_called_once_with("test message", level="info")
|
|
||||||
|
|
||||||
def test_progress_updates_job(self):
|
|
||||||
from abogen.webui.conversion_adapter import WebJobEvents
|
|
||||||
|
|
||||||
job = SimpleNamespace(progress=0.0, etr_str="")
|
|
||||||
events = WebJobEvents(job)
|
|
||||||
events.progress(50, "2m 30s")
|
|
||||||
|
|
||||||
assert job.progress == 0.5
|
|
||||||
assert job.etr_str == "2m 30s"
|
|
||||||
|
|
||||||
def test_check_cancelled_raises(self):
|
|
||||||
from abogen.webui.conversion_adapter import ConversionCancelled, WebJobEvents
|
|
||||||
|
|
||||||
job = SimpleNamespace(cancel_requested=True)
|
|
||||||
events = WebJobEvents(job)
|
|
||||||
|
|
||||||
with pytest.raises(ConversionCancelled):
|
|
||||||
events.check_cancelled()
|
|
||||||
|
|
||||||
def test_check_not_cancelled_passes(self):
|
|
||||||
from abogen.webui.conversion_adapter import WebJobEvents
|
|
||||||
|
|
||||||
job = SimpleNamespace(cancel_requested=False)
|
|
||||||
events = WebJobEvents(job)
|
|
||||||
|
|
||||||
events.check_cancelled() # Should not raise
|
|
||||||
|
|
||||||
|
|
||||||
class TestWebUIPipelineProvider:
|
|
||||||
"""Test WebUI PipelineProvider implementation."""
|
|
||||||
|
|
||||||
def test_get_returns_backend(self):
|
|
||||||
from abogen.webui.conversion_adapter import WebPipelineProvider
|
|
||||||
|
|
||||||
backend = MagicMock()
|
|
||||||
pool = SimpleNamespace(get=MagicMock(return_value=backend))
|
|
||||||
provider = WebPipelineProvider(pool)
|
|
||||||
|
|
||||||
result = provider.get("kokoro", "a", False)
|
|
||||||
|
|
||||||
assert result is backend
|
|
||||||
pool.get.assert_called_once_with("kokoro", "a", False)
|
|
||||||
|
|
||||||
|
|
||||||
class TestWebUIVoiceResolver:
|
|
||||||
"""Test WebUI VoiceResolver implementation."""
|
|
||||||
|
|
||||||
def test_resolve_returns_resolved_voice(self):
|
|
||||||
from abogen.webui.conversion_adapter import WebVoiceResolver
|
|
||||||
|
|
||||||
def resolve_fn(spec):
|
|
||||||
return ("kokoro", spec, "M1", 1.0, 5)
|
|
||||||
|
|
||||||
resolver = WebVoiceResolver(resolve_fn)
|
|
||||||
result = resolver.resolve("M1")
|
|
||||||
|
|
||||||
assert isinstance(result, ResolvedVoice)
|
|
||||||
assert result.provider == "kokoro"
|
|
||||||
assert result.voice == "M1"
|
|
||||||
assert result.speed == 1.0
|
|
||||||
assert result.supertonic_steps == 5
|
|
||||||
|
|
||||||
def test_resolve_none_speed_defaults(self):
|
|
||||||
from abogen.webui.conversion_adapter import WebVoiceResolver
|
|
||||||
|
|
||||||
def resolve_fn(spec):
|
|
||||||
return ("kokoro", spec, "M1", None, None)
|
|
||||||
|
|
||||||
resolver = WebVoiceResolver(resolve_fn)
|
|
||||||
result = resolver.resolve("M1")
|
|
||||||
|
|
||||||
assert result.speed == 1.0
|
|
||||||
assert result.supertonic_steps == 5
|
|
||||||
|
|
||||||
|
|
||||||
# ─── PyQt adapter tests ────────────────────────────────────────────
|
|
||||||
|
|
||||||
|
|
||||||
class TestPyQtAdapter:
|
|
||||||
"""Test PyQt conversion adapter field mapping."""
|
|
||||||
|
|
||||||
def _make_thread(self, **overrides):
|
|
||||||
"""Create a mock ConversionThread with default values."""
|
|
||||||
defaults = dict(
|
|
||||||
file_name="/tmp/test.epub",
|
|
||||||
lang_code="a",
|
|
||||||
voice="M1",
|
|
||||||
voice_profile=None,
|
|
||||||
speed=1.0,
|
|
||||||
use_gpu=False,
|
|
||||||
supertonic_total_steps=5,
|
|
||||||
output_format="wav",
|
|
||||||
subtitle_mode="Disabled",
|
|
||||||
subtitle_format="srt",
|
|
||||||
max_subtitle_words=50,
|
|
||||||
save_option="save_next_to_input",
|
|
||||||
output_folder=None,
|
|
||||||
save_chapters_separately=False,
|
|
||||||
merge_chapters_at_end=True,
|
|
||||||
separate_chapters_format="wav",
|
|
||||||
save_as_project=False,
|
|
||||||
silence_duration=2.0,
|
|
||||||
chapter_intro_delay=0.0,
|
|
||||||
replace_single_newlines=False,
|
|
||||||
read_title_intro=False,
|
|
||||||
read_closing_outro=True,
|
|
||||||
auto_prefix_chapter_titles=True,
|
|
||||||
normalize_chapter_opening_caps=False,
|
|
||||||
pronunciation_overrides=[],
|
|
||||||
manual_overrides=[],
|
|
||||||
heteronym_overrides=[],
|
|
||||||
normalization_overrides=None,
|
|
||||||
metadata_tags={},
|
|
||||||
cover_image_path=None,
|
|
||||||
cover_image_mime=None,
|
|
||||||
generate_epub3=False,
|
|
||||||
is_direct_text=False,
|
|
||||||
from_queue=False,
|
|
||||||
display_path=None,
|
|
||||||
save_base_path=None,
|
|
||||||
cancel_requested=False,
|
|
||||||
)
|
|
||||||
defaults.update(overrides)
|
|
||||||
return SimpleNamespace(**defaults)
|
|
||||||
|
|
||||||
def test_basic_field_mapping(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
|
|
||||||
|
|
||||||
thread = self._make_thread()
|
|
||||||
req = build_conversion_request_from_thread(thread)
|
|
||||||
|
|
||||||
assert isinstance(req, ConversionRequest)
|
|
||||||
assert req.source_path == Path("/tmp/test.epub")
|
|
||||||
assert req.language == "a"
|
|
||||||
assert req.voice == "M1"
|
|
||||||
assert req.speed == 1.0
|
|
||||||
assert req.output_format == "wav"
|
|
||||||
|
|
||||||
def test_direct_text_mode(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
|
|
||||||
|
|
||||||
thread = self._make_thread(
|
|
||||||
is_direct_text=True,
|
|
||||||
file_name="Hello world",
|
|
||||||
)
|
|
||||||
req = build_conversion_request_from_thread(thread)
|
|
||||||
|
|
||||||
assert req.source_path is None
|
|
||||||
assert req.direct_text == "Hello world"
|
|
||||||
|
|
||||||
def test_from_queue_uses_save_base_path(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
|
|
||||||
|
|
||||||
thread = self._make_thread(
|
|
||||||
from_queue=True,
|
|
||||||
save_base_path="/queue/book.epub",
|
|
||||||
display_path="/display/book.epub",
|
|
||||||
)
|
|
||||||
req = build_conversion_request_from_thread(thread)
|
|
||||||
|
|
||||||
assert req.original_filename == "book.epub"
|
|
||||||
|
|
||||||
def test_display_path_used_when_not_from_queue(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
|
|
||||||
|
|
||||||
thread = self._make_thread(
|
|
||||||
from_queue=False,
|
|
||||||
display_path="/display/book.epub",
|
|
||||||
save_base_path="/queue/book.epub",
|
|
||||||
)
|
|
||||||
req = build_conversion_request_from_thread(thread)
|
|
||||||
|
|
||||||
assert req.original_filename == "book.epub"
|
|
||||||
|
|
||||||
def test_output_folder_mapped(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
|
|
||||||
|
|
||||||
thread = self._make_thread(output_folder="/output")
|
|
||||||
req = build_conversion_request_from_thread(thread)
|
|
||||||
|
|
||||||
assert req.output_folder == Path("/output")
|
|
||||||
|
|
||||||
def test_none_defaults_handled(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
|
|
||||||
from abogen.domain.enums import Language, OutputFormat, SubtitleMode, SaveMode
|
|
||||||
|
|
||||||
thread = self._make_thread(
|
|
||||||
lang_code=None,
|
|
||||||
voice=None,
|
|
||||||
speed=None,
|
|
||||||
output_format=None,
|
|
||||||
subtitle_mode=None,
|
|
||||||
save_option=None,
|
|
||||||
silence_duration=None,
|
|
||||||
chapter_intro_delay=None,
|
|
||||||
supertonic_total_steps=None,
|
|
||||||
max_subtitle_words=None,
|
|
||||||
)
|
|
||||||
req = build_conversion_request_from_thread(thread)
|
|
||||||
|
|
||||||
# None values pass through adapter; ConversionRequest.__post_init__
|
|
||||||
# applies defaults and clamping for numeric fields.
|
|
||||||
assert req.language == Language.EN_US
|
|
||||||
assert req.speed == 1.0
|
|
||||||
assert req.output_format == OutputFormat.WAV
|
|
||||||
assert req.subtitle_mode == SubtitleMode.DISABLED
|
|
||||||
assert req.save_mode == SaveMode.SAVE_NEXT_TO_INPUT
|
|
||||||
assert req.silence_between_chapters == 2.0
|
|
||||||
assert req.chapter_intro_delay == 0.0
|
|
||||||
assert req.supertonic_total_steps == 5
|
|
||||||
assert req.max_subtitle_words == 50
|
|
||||||
|
|
||||||
def test_chapter_chunks_not_mapped(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import build_conversion_request_from_thread
|
|
||||||
|
|
||||||
thread = self._make_thread()
|
|
||||||
req = build_conversion_request_from_thread(thread)
|
|
||||||
|
|
||||||
assert req.chapter_overrides == []
|
|
||||||
assert req.chunks == []
|
|
||||||
assert req.chunk_level == "paragraph"
|
|
||||||
assert req.speaker_mode == "single"
|
|
||||||
assert req.speakers == {}
|
|
||||||
|
|
||||||
|
|
||||||
class TestPyQtEvents:
|
|
||||||
"""Test PyQt ConversionEvents implementation."""
|
|
||||||
|
|
||||||
def test_log_emits_signal(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import PyQtEvents
|
|
||||||
|
|
||||||
thread = SimpleNamespace(
|
|
||||||
log_updated=MagicMock(),
|
|
||||||
)
|
|
||||||
events = PyQtEvents(thread)
|
|
||||||
events.log("test message", level="info")
|
|
||||||
|
|
||||||
thread.log_updated.emit.assert_called_once()
|
|
||||||
|
|
||||||
def test_progress_emits_signal(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import PyQtEvents
|
|
||||||
|
|
||||||
thread = SimpleNamespace(
|
|
||||||
progress_updated=MagicMock(),
|
|
||||||
)
|
|
||||||
events = PyQtEvents(thread)
|
|
||||||
events.progress(50, "2m 30s")
|
|
||||||
|
|
||||||
thread.progress_updated.emit.assert_called_once_with(50, "2m 30s")
|
|
||||||
|
|
||||||
def test_check_cancelled_raises(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import ConversionCancelled, PyQtEvents
|
|
||||||
|
|
||||||
thread = SimpleNamespace(cancel_requested=True)
|
|
||||||
events = PyQtEvents(thread)
|
|
||||||
|
|
||||||
with pytest.raises(ConversionCancelled):
|
|
||||||
events.check_cancelled()
|
|
||||||
|
|
||||||
def test_check_not_cancelled_passes(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import PyQtEvents
|
|
||||||
|
|
||||||
thread = SimpleNamespace(cancel_requested=False)
|
|
||||||
events = PyQtEvents(thread)
|
|
||||||
|
|
||||||
events.check_cancelled() # Should not raise
|
|
||||||
|
|
||||||
|
|
||||||
class TestPyQtPipelineProvider:
|
|
||||||
"""Test PyQt PipelineProvider implementation."""
|
|
||||||
|
|
||||||
def test_get_returns_backend(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import PyQtPipelineProvider
|
|
||||||
|
|
||||||
backend = MagicMock()
|
|
||||||
provider = PyQtPipelineProvider(backend)
|
|
||||||
|
|
||||||
result = provider.get("kokoro", "a", False)
|
|
||||||
|
|
||||||
assert result is backend
|
|
||||||
|
|
||||||
def test_dispose_all_noop(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import PyQtPipelineProvider
|
|
||||||
|
|
||||||
backend = MagicMock()
|
|
||||||
provider = PyQtPipelineProvider(backend)
|
|
||||||
|
|
||||||
provider.dispose_all() # Should not raise
|
|
||||||
|
|
||||||
|
|
||||||
class TestPyQtVoiceResolver:
|
|
||||||
"""Test PyQt VoiceResolver implementation."""
|
|
||||||
|
|
||||||
def test_resolve_returns_resolved_voice(self):
|
|
||||||
from abogen.pyqt.conversion_adapter import PyQtVoiceResolver
|
|
||||||
|
|
||||||
loaded_voice = MagicMock()
|
|
||||||
thread = SimpleNamespace(
|
|
||||||
load_voice_cached=MagicMock(return_value=loaded_voice),
|
|
||||||
backend=MagicMock(),
|
|
||||||
speed=1.0,
|
|
||||||
supertonic_total_steps=5,
|
|
||||||
)
|
|
||||||
resolver = PyQtVoiceResolver(thread)
|
|
||||||
result = resolver.resolve("M1")
|
|
||||||
|
|
||||||
assert isinstance(result, ResolvedVoice)
|
|
||||||
assert result.provider == "kokoro"
|
|
||||||
assert result.voice is loaded_voice
|
|
||||||
assert result.speed == 1.0
|
|
||||||
assert result.supertonic_steps == 5
|
|
||||||
@@ -1,240 +0,0 @@
|
|||||||
import sys
|
|
||||||
import types
|
|
||||||
|
|
||||||
if "soundfile" not in sys.modules:
|
|
||||||
soundfile_stub = types.ModuleType("soundfile")
|
|
||||||
|
|
||||||
class _SoundFileStub: # pragma: no cover - placeholder to satisfy imports
|
|
||||||
def __init__(self, *args: object, **kwargs: object) -> None:
|
|
||||||
raise RuntimeError("soundfile is not installed in the test environment")
|
|
||||||
|
|
||||||
soundfile_stub.SoundFile = _SoundFileStub # type: ignore[attr-defined]
|
|
||||||
sys.modules["soundfile"] = soundfile_stub
|
|
||||||
|
|
||||||
if "static_ffmpeg" not in sys.modules:
|
|
||||||
sys.modules["static_ffmpeg"] = types.ModuleType("static_ffmpeg")
|
|
||||||
|
|
||||||
if "ebooklib" not in sys.modules:
|
|
||||||
ebooklib_stub = types.ModuleType("ebooklib")
|
|
||||||
ebooklib_epub_stub = types.ModuleType("ebooklib.epub")
|
|
||||||
ebooklib_stub.epub = ebooklib_epub_stub # type: ignore[attr-defined]
|
|
||||||
sys.modules["ebooklib"] = ebooklib_stub
|
|
||||||
sys.modules["ebooklib.epub"] = ebooklib_epub_stub
|
|
||||||
|
|
||||||
if "fitz" not in sys.modules:
|
|
||||||
sys.modules["fitz"] = types.ModuleType("fitz")
|
|
||||||
|
|
||||||
if "markdown" not in sys.modules:
|
|
||||||
markdown_stub = types.ModuleType("markdown")
|
|
||||||
|
|
||||||
class _MarkdownStub:
|
|
||||||
def __init__(self, *args: object, **kwargs: object) -> None:
|
|
||||||
self.toc_tokens = []
|
|
||||||
|
|
||||||
def convert(self, text: str) -> str:
|
|
||||||
return text
|
|
||||||
|
|
||||||
markdown_stub.Markdown = _MarkdownStub # type: ignore[attr-defined]
|
|
||||||
sys.modules["markdown"] = markdown_stub
|
|
||||||
|
|
||||||
if "bs4" not in sys.modules:
|
|
||||||
bs4_stub = types.ModuleType("bs4")
|
|
||||||
|
|
||||||
class _BeautifulSoupStub:
|
|
||||||
def __init__(self, *args: object, **kwargs: object) -> None:
|
|
||||||
self._text = ""
|
|
||||||
|
|
||||||
def find(self, *args: object, **kwargs: object) -> None:
|
|
||||||
return None
|
|
||||||
|
|
||||||
def get_text(self) -> str:
|
|
||||||
return self._text
|
|
||||||
|
|
||||||
def decompose(self) -> None: # pragma: no cover - compatibility shim
|
|
||||||
return None
|
|
||||||
|
|
||||||
class _NavigableStringStub(str):
|
|
||||||
pass
|
|
||||||
|
|
||||||
bs4_stub.BeautifulSoup = _BeautifulSoupStub # type: ignore[attr-defined]
|
|
||||||
bs4_stub.NavigableString = _NavigableStringStub # type: ignore[attr-defined]
|
|
||||||
sys.modules["bs4"] = bs4_stub
|
|
||||||
|
|
||||||
|
|
||||||
from abogen.webui.conversion_runner import (
|
|
||||||
_format_spoken_chapter_title,
|
|
||||||
_headings_equivalent,
|
|
||||||
_normalize_chapter_opening_caps,
|
|
||||||
_strip_duplicate_heading_line,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_format_spoken_chapter_title_adds_prefix() -> None:
|
|
||||||
assert _format_spoken_chapter_title("1: A Tale", 1, True) == "Chapter 1. A Tale"
|
|
||||||
|
|
||||||
|
|
||||||
def test_format_spoken_chapter_title_respects_existing_prefix() -> None:
|
|
||||||
assert (
|
|
||||||
_format_spoken_chapter_title("Chapter 2: Story", 2, True) == "Chapter 2: Story"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_format_spoken_chapter_title_handles_empty_title() -> None:
|
|
||||||
assert _format_spoken_chapter_title("", 4, True) == "Chapter 4"
|
|
||||||
|
|
||||||
|
|
||||||
def test_format_spoken_chapter_title_trims_delimiters() -> None:
|
|
||||||
assert (
|
|
||||||
_format_spoken_chapter_title("7 - Into the Wild", 7, True)
|
|
||||||
== "Chapter 7. Into the Wild"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_headings_equivalent_ignores_case_and_prefix() -> None:
|
|
||||||
assert _headings_equivalent("1: The House", "Chapter 1: The House")
|
|
||||||
|
|
||||||
|
|
||||||
def test_strip_duplicate_heading_line_removes_first_match() -> None:
|
|
||||||
text, removed = _strip_duplicate_heading_line(
|
|
||||||
"Chapter 3: Intro\nBody text", "Chapter 3: Intro"
|
|
||||||
)
|
|
||||||
assert removed is True
|
|
||||||
assert text.strip() == "Body text"
|
|
||||||
|
|
||||||
|
|
||||||
def test_normalize_chapter_opening_caps_basic_title() -> None:
|
|
||||||
normalized, changed = _normalize_chapter_opening_caps("ALL CAPS TITLE")
|
|
||||||
assert normalized == "All Caps Title"
|
|
||||||
assert changed is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_normalize_chapter_opening_caps_respects_acronyms() -> None:
|
|
||||||
normalized, changed = _normalize_chapter_opening_caps("NASA MISSION LOG")
|
|
||||||
assert normalized == "NASA Mission Log"
|
|
||||||
assert changed is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_normalize_chapter_opening_caps_handles_roman_numerals() -> None:
|
|
||||||
normalized, changed = _normalize_chapter_opening_caps("IV. THE RETURN")
|
|
||||||
assert normalized == "IV. The Return"
|
|
||||||
assert changed is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_normalize_chapter_opening_caps_keeps_mixed_case() -> None:
|
|
||||||
normalized, changed = _normalize_chapter_opening_caps("Already Mixed Case")
|
|
||||||
assert normalized == "Already Mixed Case"
|
|
||||||
assert changed is False
|
|
||||||
|
|
||||||
|
|
||||||
class TestApplyChapterTextTransforms:
|
|
||||||
"""Tests for the combined heading-strip + opening-caps helper."""
|
|
||||||
|
|
||||||
def test_both_enabled_heading_matches(self) -> None:
|
|
||||||
from abogen.domain.chapter_titles import apply_chapter_text_transforms
|
|
||||||
|
|
||||||
text, heading_removed, caps_changed = apply_chapter_text_transforms(
|
|
||||||
"Chapter 1: The Beginning\nBody text here",
|
|
||||||
heading_text="Chapter 1: The Beginning",
|
|
||||||
raw_title="Chapter 1: The Beginning",
|
|
||||||
strip_heading=True,
|
|
||||||
normalize_caps=True,
|
|
||||||
)
|
|
||||||
assert heading_removed is True
|
|
||||||
assert "Body text here" in text
|
|
||||||
assert "Chapter 1" not in text
|
|
||||||
|
|
||||||
def test_heading_fallback_to_number(self) -> None:
|
|
||||||
from abogen.domain.chapter_titles import apply_chapter_text_transforms
|
|
||||||
|
|
||||||
text, heading_removed, caps_changed = apply_chapter_text_transforms(
|
|
||||||
"1. The Beginning\nBody text",
|
|
||||||
heading_text="Chapter 1: The Beginning",
|
|
||||||
raw_title="1: The Beginning",
|
|
||||||
strip_heading=True,
|
|
||||||
normalize_caps=False,
|
|
||||||
)
|
|
||||||
assert heading_removed is True
|
|
||||||
assert "Body text" in text
|
|
||||||
|
|
||||||
def test_only_heading_strip(self) -> None:
|
|
||||||
from abogen.domain.chapter_titles import apply_chapter_text_transforms
|
|
||||||
|
|
||||||
text, heading_removed, caps_changed = apply_chapter_text_transforms(
|
|
||||||
"Chapter 1: Title\nBody text",
|
|
||||||
heading_text="Chapter 1: Title",
|
|
||||||
raw_title="",
|
|
||||||
strip_heading=True,
|
|
||||||
normalize_caps=False,
|
|
||||||
)
|
|
||||||
assert heading_removed is True
|
|
||||||
assert caps_changed is False
|
|
||||||
|
|
||||||
def test_only_opening_caps(self) -> None:
|
|
||||||
from abogen.domain.chapter_titles import apply_chapter_text_transforms
|
|
||||||
|
|
||||||
text, heading_removed, caps_changed = apply_chapter_text_transforms(
|
|
||||||
"ALL CAPS START OF CHAPTER",
|
|
||||||
heading_text="Chapter 1",
|
|
||||||
raw_title="",
|
|
||||||
strip_heading=False,
|
|
||||||
normalize_caps=True,
|
|
||||||
)
|
|
||||||
assert heading_removed is False
|
|
||||||
assert caps_changed is True
|
|
||||||
assert text == "All Caps Start Of Chapter"
|
|
||||||
|
|
||||||
def test_both_disabled_no_change(self) -> None:
|
|
||||||
from abogen.domain.chapter_titles import apply_chapter_text_transforms
|
|
||||||
|
|
||||||
original = "Some text here"
|
|
||||||
text, heading_removed, caps_changed = apply_chapter_text_transforms(
|
|
||||||
original,
|
|
||||||
heading_text="Chapter 1",
|
|
||||||
raw_title="",
|
|
||||||
strip_heading=False,
|
|
||||||
normalize_caps=False,
|
|
||||||
)
|
|
||||||
assert text == original
|
|
||||||
assert heading_removed is False
|
|
||||||
assert caps_changed is False
|
|
||||||
|
|
||||||
def test_heading_not_matching(self) -> None:
|
|
||||||
from abogen.domain.chapter_titles import apply_chapter_text_transforms
|
|
||||||
|
|
||||||
text, heading_removed, caps_changed = apply_chapter_text_transforms(
|
|
||||||
"Completely different text",
|
|
||||||
heading_text="Chapter 1: Title",
|
|
||||||
raw_title="",
|
|
||||||
strip_heading=True,
|
|
||||||
normalize_caps=False,
|
|
||||||
)
|
|
||||||
assert heading_removed is False
|
|
||||||
assert text == "Completely different text"
|
|
||||||
|
|
||||||
def test_empty_text(self) -> None:
|
|
||||||
from abogen.domain.chapter_titles import apply_chapter_text_transforms
|
|
||||||
|
|
||||||
text, heading_removed, caps_changed = apply_chapter_text_transforms(
|
|
||||||
"",
|
|
||||||
heading_text="Chapter 1",
|
|
||||||
raw_title="",
|
|
||||||
strip_heading=True,
|
|
||||||
normalize_caps=True,
|
|
||||||
)
|
|
||||||
assert text == ""
|
|
||||||
assert heading_removed is False
|
|
||||||
assert caps_changed is False
|
|
||||||
|
|
||||||
def test_both_enabled_text_only_has_caps(self) -> None:
|
|
||||||
from abogen.domain.chapter_titles import apply_chapter_text_transforms
|
|
||||||
|
|
||||||
text, heading_removed, caps_changed = apply_chapter_text_transforms(
|
|
||||||
"NASA MISSION LOG",
|
|
||||||
heading_text="Chapter 1",
|
|
||||||
raw_title="",
|
|
||||||
strip_heading=True,
|
|
||||||
normalize_caps=True,
|
|
||||||
)
|
|
||||||
assert heading_removed is False
|
|
||||||
assert caps_changed is True
|
|
||||||
assert text == "NASA Mission Log"
|
|
||||||
@@ -14,6 +14,8 @@ from unittest.mock import MagicMock, patch
|
|||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
from abogen.domain.config_types import SubtitleConfig
|
||||||
|
from abogen.domain.enums import Language, SubtitleMode
|
||||||
from abogen.domain.conversion_engine import (
|
from abogen.domain.conversion_engine import (
|
||||||
synthesize_text,
|
synthesize_text,
|
||||||
SynthParams,
|
SynthParams,
|
||||||
@@ -56,7 +58,7 @@ class FakeBackend:
|
|||||||
self.segment_duration = segment_duration
|
self.segment_duration = segment_duration
|
||||||
self.call_count = 0
|
self.call_count = 0
|
||||||
|
|
||||||
def __call__(self, text: str, voice: Any, speed: float = 1.0, split_pattern: str = ""):
|
def __call__(self, text: str, voice: Any, speed: float = 1.0, split_pattern: str = "", **kwargs: Any):
|
||||||
self.call_count += 1
|
self.call_count += 1
|
||||||
# Return fake segment objects with required attributes
|
# Return fake segment objects with required attributes
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -252,9 +254,8 @@ class TestProcessAndWriteSubtitles:
|
|||||||
process_and_write_subtitles(
|
process_and_write_subtitles(
|
||||||
[],
|
[],
|
||||||
writer,
|
writer,
|
||||||
subtitle_mode="Sentence",
|
subtitle=SubtitleConfig(mode=SubtitleMode.SENTENCE, max_words=5),
|
||||||
max_subtitle_words=5,
|
language=Language.EN_US,
|
||||||
lang_code="a",
|
|
||||||
use_spacy_segmentation=False,
|
use_spacy_segmentation=False,
|
||||||
fallback_end_time=10.0,
|
fallback_end_time=10.0,
|
||||||
)
|
)
|
||||||
@@ -269,9 +270,8 @@ class TestProcessAndWriteSubtitles:
|
|||||||
process_and_write_subtitles(
|
process_and_write_subtitles(
|
||||||
tokens,
|
tokens,
|
||||||
writer,
|
writer,
|
||||||
subtitle_mode="Sentence",
|
subtitle=SubtitleConfig(mode=SubtitleMode.SENTENCE, max_words=5),
|
||||||
max_subtitle_words=5,
|
language=Language.EN_US,
|
||||||
lang_code="a",
|
|
||||||
use_spacy_segmentation=False,
|
use_spacy_segmentation=False,
|
||||||
fallback_end_time=2.0,
|
fallback_end_time=2.0,
|
||||||
)
|
)
|
||||||
@@ -291,9 +291,8 @@ class TestProcessAndWriteSubtitles:
|
|||||||
process_and_write_subtitles(
|
process_and_write_subtitles(
|
||||||
tokens,
|
tokens,
|
||||||
writer,
|
writer,
|
||||||
subtitle_mode="Line",
|
subtitle=SubtitleConfig(mode=SubtitleMode.LINE, max_words=5),
|
||||||
max_subtitle_words=5,
|
language=Language.EN_US,
|
||||||
lang_code="a",
|
|
||||||
use_spacy_segmentation=False,
|
use_spacy_segmentation=False,
|
||||||
fallback_end_time=3.0,
|
fallback_end_time=3.0,
|
||||||
)
|
)
|
||||||
@@ -310,9 +309,8 @@ class TestProcessAndWriteSubtitles:
|
|||||||
process_and_write_subtitles(
|
process_and_write_subtitles(
|
||||||
tokens,
|
tokens,
|
||||||
writer,
|
writer,
|
||||||
subtitle_mode="Disabled",
|
subtitle=SubtitleConfig(mode=SubtitleMode.DISABLED, max_words=5),
|
||||||
max_subtitle_words=5,
|
language=Language.EN_US,
|
||||||
lang_code="a",
|
|
||||||
use_spacy_segmentation=False,
|
use_spacy_segmentation=False,
|
||||||
fallback_end_time=2.0,
|
fallback_end_time=2.0,
|
||||||
)
|
)
|
||||||
@@ -348,7 +346,7 @@ class TestFullPipeline:
|
|||||||
audio_sink=merged_sink,
|
audio_sink=merged_sink,
|
||||||
subtitle_mode="Sentence",
|
subtitle_mode="Sentence",
|
||||||
max_subtitle_words=5,
|
max_subtitle_words=5,
|
||||||
lang_code="a",
|
language=Language.EN_US,
|
||||||
use_spacy_segmentation=False,
|
use_spacy_segmentation=False,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -365,9 +363,8 @@ class TestFullPipeline:
|
|||||||
process_and_write_subtitles(
|
process_and_write_subtitles(
|
||||||
tokens,
|
tokens,
|
||||||
subtitle_writer,
|
subtitle_writer,
|
||||||
subtitle_mode="Sentence",
|
subtitle=SubtitleConfig(mode=SubtitleMode.SENTENCE, max_words=5),
|
||||||
max_subtitle_words=5,
|
language=Language.EN_US,
|
||||||
lang_code="a",
|
|
||||||
use_spacy_segmentation=False,
|
use_spacy_segmentation=False,
|
||||||
fallback_end_time=stats.current_time,
|
fallback_end_time=stats.current_time,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ from unittest.mock import MagicMock
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
|
from abogen.application.conversion_config import SaveConfig
|
||||||
from abogen.application.conversion_executor import execute_conversion
|
from abogen.application.conversion_executor import execute_conversion
|
||||||
from abogen.application.conversion_models import (
|
from abogen.application.conversion_models import (
|
||||||
ChapterPlan,
|
ChapterPlan,
|
||||||
@@ -72,7 +73,7 @@ class FakeBackend:
|
|||||||
def __init__(self):
|
def __init__(self):
|
||||||
self.synthesized: List[str] = []
|
self.synthesized: List[str] = []
|
||||||
|
|
||||||
def __call__(self, text: str, *, voice: Any, speed: float = 1.0, split_pattern: str = "") -> List:
|
def __call__(self, text: str, *, voice: Any, speed: float = 1.0, split_pattern: str = "", **kwargs: Any) -> List:
|
||||||
"""Return fake TTS segments."""
|
"""Return fake TTS segments."""
|
||||||
self.synthesized.append(text)
|
self.synthesized.append(text)
|
||||||
|
|
||||||
@@ -150,8 +151,7 @@ class TestExecuteConversion:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello world",
|
direct_text="Hello world",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -198,10 +198,7 @@ class TestExecuteConversion:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Text",
|
direct_text="Text",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir), save_chapters_separately=True, merge_chapters_at_end=True),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
save_chapters_separately=True,
|
|
||||||
merge_chapters_at_end=True,
|
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -262,8 +259,7 @@ class TestExecuteConversion:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Text",
|
direct_text="Text",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -315,8 +311,7 @@ class TestExecuteConversion:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Text",
|
direct_text="Text",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -377,8 +372,7 @@ class TestExecuteConversion:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Text",
|
direct_text="Text",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -424,8 +418,7 @@ class TestExecuteConversion:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello world",
|
direct_text="Hello world",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -471,8 +464,7 @@ class TestExecuteConversion:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Text",
|
direct_text="Text",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
plan = ConversionPlan(
|
plan = ConversionPlan(
|
||||||
request=req,
|
request=req,
|
||||||
@@ -511,3 +503,387 @@ class TestExecuteConversion:
|
|||||||
|
|
||||||
assert result.metadata["title"] == "Test Book"
|
assert result.metadata["title"] == "Test Book"
|
||||||
assert result.metadata["author"] == "Author"
|
assert result.metadata["author"] == "Author"
|
||||||
|
|
||||||
|
|
||||||
|
class TestHeadingDedup:
|
||||||
|
"""Tests for heading dedup in executor."""
|
||||||
|
|
||||||
|
def test_heading_dedup_strips_matching_first_line(self):
|
||||||
|
"""When first segment matches heading, it should be stripped."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="Hello",
|
||||||
|
voice="M1",
|
||||||
|
auto_prefix_chapter_titles=True,
|
||||||
|
)
|
||||||
|
# Simulate: heading = "Chapter 1", first segment = "Chapter 1: The Beginning"
|
||||||
|
# headings_equivalent should match these
|
||||||
|
plan = ConversionPlan(
|
||||||
|
request=req,
|
||||||
|
metadata={},
|
||||||
|
chapters=[
|
||||||
|
ChapterPlan(
|
||||||
|
index=1,
|
||||||
|
title="Chapter 1",
|
||||||
|
original_title="Chapter 1",
|
||||||
|
body_text="Chapter 1: The Beginning\nBody text here",
|
||||||
|
segments=[
|
||||||
|
SegmentPlan(
|
||||||
|
text="Chapter 1: The Beginning",
|
||||||
|
voice_spec="M1",
|
||||||
|
kind="body",
|
||||||
|
source="chapter",
|
||||||
|
),
|
||||||
|
SegmentPlan(
|
||||||
|
text="Body text here",
|
||||||
|
voice_spec="M1",
|
||||||
|
kind="body",
|
||||||
|
source="chapter",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
voice_spec="M1",
|
||||||
|
)
|
||||||
|
],
|
||||||
|
output_layout=OutputLayout(
|
||||||
|
parent_dir=Path(tmpdir),
|
||||||
|
audio_dir=Path(tmpdir),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
events = FakeEvents()
|
||||||
|
pipeline = FakePipelineProvider()
|
||||||
|
resolver = FakeVoiceResolver()
|
||||||
|
tts_context = TTSContext()
|
||||||
|
|
||||||
|
result = execute_conversion(
|
||||||
|
plan, events, pipeline, resolver, tts_context
|
||||||
|
)
|
||||||
|
|
||||||
|
# The executor should have logged the heading
|
||||||
|
log_messages = [m for m, _ in events.logs if "Title:" in m]
|
||||||
|
assert len(log_messages) >= 1
|
||||||
|
|
||||||
|
def test_heading_dedup_no_match_preserves_all(self):
|
||||||
|
"""When first segment doesn't match heading, nothing is stripped."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="Hello",
|
||||||
|
voice="M1",
|
||||||
|
auto_prefix_chapter_titles=True,
|
||||||
|
)
|
||||||
|
plan = ConversionPlan(
|
||||||
|
request=req,
|
||||||
|
metadata={},
|
||||||
|
chapters=[
|
||||||
|
ChapterPlan(
|
||||||
|
index=1,
|
||||||
|
title="Chapter 1",
|
||||||
|
original_title="Chapter 1",
|
||||||
|
body_text="Completely different text\nMore text",
|
||||||
|
segments=[
|
||||||
|
SegmentPlan(
|
||||||
|
text="Completely different text",
|
||||||
|
voice_spec="M1",
|
||||||
|
kind="body",
|
||||||
|
source="chapter",
|
||||||
|
),
|
||||||
|
SegmentPlan(
|
||||||
|
text="More text",
|
||||||
|
voice_spec="M1",
|
||||||
|
kind="body",
|
||||||
|
source="chapter",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
voice_spec="M1",
|
||||||
|
)
|
||||||
|
],
|
||||||
|
output_layout=OutputLayout(
|
||||||
|
parent_dir=Path(tmpdir),
|
||||||
|
audio_dir=Path(tmpdir),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
events = FakeEvents()
|
||||||
|
pipeline = FakePipelineProvider()
|
||||||
|
resolver = FakeVoiceResolver()
|
||||||
|
tts_context = TTSContext()
|
||||||
|
|
||||||
|
result = execute_conversion(
|
||||||
|
plan, events, pipeline, resolver, tts_context
|
||||||
|
)
|
||||||
|
|
||||||
|
# Both segments should be synthesized (heading + 2 body segments)
|
||||||
|
assert result.total_segments >= 2
|
||||||
|
|
||||||
|
|
||||||
|
class TestMarkerCollector:
|
||||||
|
"""Tests for MarkerCollector."""
|
||||||
|
|
||||||
|
def test_chapter_marker_has_voices_list(self):
|
||||||
|
"""Chapter markers should have 'voices' as list of dicts."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
req = ConversionRequest(direct_text="Hello", voice="M1")
|
||||||
|
plan = ConversionPlan(
|
||||||
|
request=req,
|
||||||
|
metadata={},
|
||||||
|
chapters=[
|
||||||
|
ChapterPlan(
|
||||||
|
index=1,
|
||||||
|
title="Chapter 1",
|
||||||
|
original_title="Chapter 1",
|
||||||
|
body_text="Body text",
|
||||||
|
segments=[
|
||||||
|
SegmentPlan(
|
||||||
|
text="Body text",
|
||||||
|
voice_spec="M1",
|
||||||
|
kind="body",
|
||||||
|
source="chapter",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
voice_spec="M1",
|
||||||
|
)
|
||||||
|
],
|
||||||
|
output_layout=OutputLayout(
|
||||||
|
parent_dir=Path(tmpdir),
|
||||||
|
audio_dir=Path(tmpdir),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
events = FakeEvents()
|
||||||
|
pipeline = FakePipelineProvider()
|
||||||
|
resolver = FakeVoiceResolver()
|
||||||
|
tts_context = TTSContext()
|
||||||
|
|
||||||
|
result = execute_conversion(plan, events, pipeline, resolver, tts_context)
|
||||||
|
|
||||||
|
assert len(result.chapter_markers) == 1
|
||||||
|
marker = result.chapter_markers[0]
|
||||||
|
assert "voices" in marker
|
||||||
|
assert isinstance(marker["voices"], list)
|
||||||
|
assert len(marker["voices"]) == 1
|
||||||
|
assert marker["voices"][0]["provider"] == "kokoro"
|
||||||
|
assert marker["voices"][0]["voice"] == "M1"
|
||||||
|
|
||||||
|
def test_outro_marker_recorded(self):
|
||||||
|
"""Outro should be recorded as a chapter marker."""
|
||||||
|
from abogen.application.conversion_models import IntroOutroSpec
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
req = ConversionRequest(direct_text="Hello", voice="M1")
|
||||||
|
plan = ConversionPlan(
|
||||||
|
request=req,
|
||||||
|
metadata={},
|
||||||
|
chapters=[
|
||||||
|
ChapterPlan(
|
||||||
|
index=1,
|
||||||
|
title="Chapter 1",
|
||||||
|
original_title="Chapter 1",
|
||||||
|
body_text="Body text",
|
||||||
|
segments=[
|
||||||
|
SegmentPlan(
|
||||||
|
text="Body text",
|
||||||
|
voice_spec="M1",
|
||||||
|
kind="body",
|
||||||
|
source="chapter",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
voice_spec="M1",
|
||||||
|
)
|
||||||
|
],
|
||||||
|
outro=IntroOutroSpec(
|
||||||
|
enabled=True,
|
||||||
|
text="Thanks for listening",
|
||||||
|
voice_spec="M1",
|
||||||
|
kind="outro",
|
||||||
|
),
|
||||||
|
output_layout=OutputLayout(
|
||||||
|
parent_dir=Path(tmpdir),
|
||||||
|
audio_dir=Path(tmpdir),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
events = FakeEvents()
|
||||||
|
pipeline = FakePipelineProvider()
|
||||||
|
resolver = FakeVoiceResolver()
|
||||||
|
tts_context = TTSContext()
|
||||||
|
|
||||||
|
result = execute_conversion(plan, events, pipeline, resolver, tts_context)
|
||||||
|
|
||||||
|
# Should have chapter marker + outro marker
|
||||||
|
assert len(result.chapter_markers) == 2
|
||||||
|
outro_marker = result.chapter_markers[1]
|
||||||
|
assert outro_marker["title"] == "Outro"
|
||||||
|
assert "start" in outro_marker
|
||||||
|
assert "end" in outro_marker
|
||||||
|
assert outro_marker["end"] > outro_marker["start"]
|
||||||
|
|
||||||
|
def test_chunk_marker_voice_is_dict(self):
|
||||||
|
"""Chunk markers should have 'voice' as dict with provider."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
req = ConversionRequest(direct_text="Hello", voice="M1")
|
||||||
|
plan = ConversionPlan(
|
||||||
|
request=req,
|
||||||
|
metadata={},
|
||||||
|
chapters=[
|
||||||
|
ChapterPlan(
|
||||||
|
index=1,
|
||||||
|
title="Chapter 1",
|
||||||
|
original_title="Chapter 1",
|
||||||
|
body_text="Body text",
|
||||||
|
segments=[
|
||||||
|
SegmentPlan(
|
||||||
|
text="Body text",
|
||||||
|
voice_spec="M1",
|
||||||
|
kind="body",
|
||||||
|
source="chunk",
|
||||||
|
chunk_id="chunk_001",
|
||||||
|
chunk_index=0,
|
||||||
|
speaker_id="narrator",
|
||||||
|
level="paragraph",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
voice_spec="M1",
|
||||||
|
)
|
||||||
|
],
|
||||||
|
output_layout=OutputLayout(
|
||||||
|
parent_dir=Path(tmpdir),
|
||||||
|
audio_dir=Path(tmpdir),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
events = FakeEvents()
|
||||||
|
pipeline = FakePipelineProvider()
|
||||||
|
resolver = FakeVoiceResolver()
|
||||||
|
tts_context = TTSContext()
|
||||||
|
|
||||||
|
result = execute_conversion(plan, events, pipeline, resolver, tts_context)
|
||||||
|
|
||||||
|
assert len(result.chunk_markers) == 1
|
||||||
|
chunk = result.chunk_markers[0]
|
||||||
|
assert isinstance(chunk["voice"], dict)
|
||||||
|
assert chunk["voice"]["provider"] == "kokoro"
|
||||||
|
assert chunk["voice"]["voice"] == "M1"
|
||||||
|
|
||||||
|
def test_multi_speaker_collects_unique_voices(self):
|
||||||
|
"""Multi-speaker chapters should collect all unique voices."""
|
||||||
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
|
req = ConversionRequest(direct_text="Hello", voice="M1")
|
||||||
|
plan = ConversionPlan(
|
||||||
|
request=req,
|
||||||
|
metadata={},
|
||||||
|
chapters=[
|
||||||
|
ChapterPlan(
|
||||||
|
index=1,
|
||||||
|
title="Chapter 1",
|
||||||
|
original_title="Chapter 1",
|
||||||
|
body_text="Body text",
|
||||||
|
segments=[
|
||||||
|
SegmentPlan(
|
||||||
|
text="Narrator speaks",
|
||||||
|
voice_spec="M1",
|
||||||
|
kind="body",
|
||||||
|
source="chapter",
|
||||||
|
),
|
||||||
|
SegmentPlan(
|
||||||
|
text="Character speaks",
|
||||||
|
voice_spec="F1",
|
||||||
|
kind="body",
|
||||||
|
source="chapter",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
voice_spec="M1",
|
||||||
|
)
|
||||||
|
],
|
||||||
|
output_layout=OutputLayout(
|
||||||
|
parent_dir=Path(tmpdir),
|
||||||
|
audio_dir=Path(tmpdir),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
events = FakeEvents()
|
||||||
|
pipeline = FakePipelineProvider()
|
||||||
|
resolver = FakeVoiceResolver()
|
||||||
|
tts_context = TTSContext()
|
||||||
|
|
||||||
|
result = execute_conversion(plan, events, pipeline, resolver, tts_context)
|
||||||
|
|
||||||
|
marker = result.chapter_markers[0]
|
||||||
|
assert len(marker["voices"]) == 2
|
||||||
|
voice_specs = {v["voice"] for v in marker["voices"]}
|
||||||
|
assert "M1" in voice_specs
|
||||||
|
assert "F1" in voice_specs
|
||||||
|
|
||||||
|
|
||||||
|
class TestFfmetadataVoiceFormat:
|
||||||
|
"""Tests for ffmetadata rendering with new voice format."""
|
||||||
|
|
||||||
|
def test_render_ffmetadata_with_voices_list(self):
|
||||||
|
"""ffmetadata should render voices list as comma-separated string."""
|
||||||
|
from abogen.infrastructure.exporters import ExportService
|
||||||
|
|
||||||
|
svc = ExportService()
|
||||||
|
chapters = [
|
||||||
|
{
|
||||||
|
"title": "Chapter 1",
|
||||||
|
"start": 0.0,
|
||||||
|
"end": 60.0,
|
||||||
|
"voices": [
|
||||||
|
{"provider": "kokoro", "voice": "M1"},
|
||||||
|
{"provider": "kokoro", "voice": "F1"},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
]
|
||||||
|
content = svc.render_ffmetadata({}, chapters)
|
||||||
|
assert "voice=M1@kokoro, F1@kokoro" in content
|
||||||
|
|
||||||
|
def test_render_ffmetadata_with_empty_voices(self):
|
||||||
|
"""ffmetadata should handle empty voices list."""
|
||||||
|
from abogen.infrastructure.exporters import ExportService
|
||||||
|
|
||||||
|
svc = ExportService()
|
||||||
|
chapters = [
|
||||||
|
{
|
||||||
|
"title": "Chapter 1",
|
||||||
|
"start": 0.0,
|
||||||
|
"end": 60.0,
|
||||||
|
"voices": [],
|
||||||
|
}
|
||||||
|
]
|
||||||
|
content = svc.render_ffmetadata({}, chapters)
|
||||||
|
assert "voice=" not in content
|
||||||
|
|
||||||
|
|
||||||
|
class TestEpub3VoiceFormat:
|
||||||
|
"""Tests for EPUB3 voice handling with new format."""
|
||||||
|
|
||||||
|
def test_chunk_overlay_voice_is_dict(self):
|
||||||
|
"""ChunkOverlay should accept voice as dict."""
|
||||||
|
from abogen.epub3.exporter import ChunkOverlay
|
||||||
|
|
||||||
|
overlay = ChunkOverlay(
|
||||||
|
id="test",
|
||||||
|
text="hello",
|
||||||
|
original_text=None,
|
||||||
|
start=0.0,
|
||||||
|
end=1.0,
|
||||||
|
speaker_id="narrator",
|
||||||
|
voice={"provider": "kokoro", "voice": "M1"},
|
||||||
|
)
|
||||||
|
assert isinstance(overlay.voice, dict)
|
||||||
|
assert overlay.voice["provider"] == "kokoro"
|
||||||
|
|
||||||
|
def test_render_chunk_inline_with_voice_dict(self):
|
||||||
|
"""_render_chunk_inline should render voice dict as data-voice attribute."""
|
||||||
|
from abogen.epub3.exporter import ChunkOverlay, _render_chunk_inline
|
||||||
|
|
||||||
|
overlay = ChunkOverlay(
|
||||||
|
id="chunk_001",
|
||||||
|
text="Hello world",
|
||||||
|
original_text=None,
|
||||||
|
start=0.0,
|
||||||
|
end=1.0,
|
||||||
|
speaker_id="narrator",
|
||||||
|
voice={"provider": "kokoro", "voice": "M1"},
|
||||||
|
)
|
||||||
|
html = _render_chunk_inline(overlay)
|
||||||
|
assert 'data-voice="M1@kokoro"' in html
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ from abogen.application.conversion_models import (
|
|||||||
OutputLayout,
|
OutputLayout,
|
||||||
SegmentPlan,
|
SegmentPlan,
|
||||||
)
|
)
|
||||||
|
from abogen.application.conversion_config import ChapterChunkConfig, WordSubstitutionConfig
|
||||||
from abogen.application.conversion_planner import build_conversion_plan
|
from abogen.application.conversion_planner import build_conversion_plan
|
||||||
from abogen.application.conversion_request import ConversionRequest
|
from abogen.application.conversion_request import ConversionRequest
|
||||||
|
|
||||||
@@ -74,10 +75,12 @@ class TestBuildConversionPlan:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Some text",
|
direct_text="Some text",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
chunks=[
|
chapter_chunk=ChapterChunkConfig(
|
||||||
{"text": "Chunk 1", "speaker_id": "narrator"},
|
chunks=[
|
||||||
{"text": "Chunk 2", "speaker_id": "narrator"},
|
{"text": "Chunk 1", "speaker_id": "narrator"},
|
||||||
],
|
{"text": "Chunk 2", "speaker_id": "narrator"},
|
||||||
|
],
|
||||||
|
),
|
||||||
)
|
)
|
||||||
plan = build_conversion_plan(req)
|
plan = build_conversion_plan(req)
|
||||||
|
|
||||||
@@ -92,11 +95,13 @@ class TestBuildConversionPlan:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Text",
|
direct_text="Text",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
chunks=[
|
chapter_chunk=ChapterChunkConfig(
|
||||||
{"text": "Narrator speaks", "speaker_id": "narrator"},
|
chunks=[
|
||||||
{"text": "Character speaks", "speaker_id": "alice", "voice": "F1"},
|
{"text": "Narrator speaks", "speaker_id": "narrator"},
|
||||||
],
|
{"text": "Character speaks", "speaker_id": "alice", "voice": "F1"},
|
||||||
speakers={"alice": {"voice": "F1"}},
|
],
|
||||||
|
speakers={"alice": {"voice": "F1"}},
|
||||||
|
),
|
||||||
)
|
)
|
||||||
plan = build_conversion_plan(req)
|
plan = build_conversion_plan(req)
|
||||||
|
|
||||||
@@ -120,12 +125,12 @@ class TestBuildConversionPlan:
|
|||||||
|
|
||||||
def test_output_layout(self):
|
def test_output_layout(self):
|
||||||
"""Output layout is resolved from request."""
|
"""Output layout is resolved from request."""
|
||||||
|
from abogen.application.conversion_config import SaveConfig
|
||||||
with tempfile.TemporaryDirectory() as tmpdir:
|
with tempfile.TemporaryDirectory() as tmpdir:
|
||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
direct_text="Hello",
|
direct_text="Hello",
|
||||||
voice="M1",
|
voice="M1",
|
||||||
save_mode="custom_folder",
|
save=SaveConfig(mode="custom_folder", output_folder=Path(tmpdir)),
|
||||||
output_folder=Path(tmpdir),
|
|
||||||
)
|
)
|
||||||
plan = build_conversion_plan(req)
|
plan = build_conversion_plan(req)
|
||||||
|
|
||||||
@@ -198,6 +203,84 @@ class TestBuildConversionPlan:
|
|||||||
assert plan.chapters[0].segments[0].kind == "body"
|
assert plan.chapters[0].segments[0].kind == "body"
|
||||||
|
|
||||||
|
|
||||||
|
class TestWordSubstitution:
|
||||||
|
"""Tests for word substitution in the planner."""
|
||||||
|
|
||||||
|
def test_basic_substitution(self):
|
||||||
|
"""Single word substitution is applied."""
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="The quick brown fox",
|
||||||
|
voice="M1",
|
||||||
|
word_substitution=WordSubstitutionConfig(
|
||||||
|
substitutions_list="fox|cat",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
plan = build_conversion_plan(req)
|
||||||
|
assert "cat" in plan.chapters[0].body_text
|
||||||
|
assert "fox" not in plan.chapters[0].body_text
|
||||||
|
|
||||||
|
def test_multiple_substitutions(self):
|
||||||
|
"""Multiple word substitutions are applied."""
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="The quick brown fox jumps",
|
||||||
|
voice="M1",
|
||||||
|
word_substitution=WordSubstitutionConfig(
|
||||||
|
substitutions_list="fox|cat\nquick|slow",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
plan = build_conversion_plan(req)
|
||||||
|
text = plan.chapters[0].body_text
|
||||||
|
assert "cat" in text
|
||||||
|
assert "slow" in text
|
||||||
|
|
||||||
|
def test_substitution_preserves_chapter_markers(self):
|
||||||
|
"""Chapter markers are preserved during substitution."""
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="<<CHAPTER_MARKER:Ch1>>\nThe quick brown fox",
|
||||||
|
voice="M1",
|
||||||
|
word_substitution=WordSubstitutionConfig(
|
||||||
|
substitutions_list="fox|cat",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
plan = build_conversion_plan(req)
|
||||||
|
assert len(plan.chapters) >= 1
|
||||||
|
assert "cat" in plan.chapters[0].body_text
|
||||||
|
|
||||||
|
def test_chunks_assigned_to_correct_chapter(self):
|
||||||
|
"""Chunks are grouped by chapter_index and only assigned to matching chapters."""
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="<<CHAPTER_MARKER:Ch1>>\nText A\n<<CHAPTER_MARKER:Ch2>>\nText B",
|
||||||
|
voice="M1",
|
||||||
|
chapter_chunk=ChapterChunkConfig(
|
||||||
|
chunks=[
|
||||||
|
{"text": "Ch1 chunk", "chapter_index": 0},
|
||||||
|
{"text": "Ch2 chunk", "chapter_index": 1},
|
||||||
|
],
|
||||||
|
),
|
||||||
|
)
|
||||||
|
plan = build_conversion_plan(req)
|
||||||
|
|
||||||
|
assert len(plan.chapters) == 2
|
||||||
|
# Ch1 should have only its chunk
|
||||||
|
ch1_texts = [s.text for s in plan.chapters[0].segments]
|
||||||
|
assert "Ch1 chunk" in ch1_texts
|
||||||
|
assert "Ch2 chunk" not in ch1_texts
|
||||||
|
# Ch2 should have only its chunk
|
||||||
|
ch2_texts = [s.text for s in plan.chapters[1].segments]
|
||||||
|
assert "Ch2 chunk" in ch2_texts
|
||||||
|
assert "Ch1 chunk" not in ch2_texts
|
||||||
|
|
||||||
|
def test_no_substitution_when_disabled(self):
|
||||||
|
"""No substitution when word_substitution is None."""
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="The quick brown fox",
|
||||||
|
voice="M1",
|
||||||
|
word_substitution=None,
|
||||||
|
)
|
||||||
|
plan = build_conversion_plan(req)
|
||||||
|
assert "fox" in plan.chapters[0].body_text
|
||||||
|
|
||||||
|
|
||||||
class TestPlannerWithFileSource:
|
class TestPlannerWithFileSource:
|
||||||
"""Tests using actual file sources (not direct_text)."""
|
"""Tests using actual file sources (not direct_text)."""
|
||||||
|
|
||||||
@@ -558,3 +641,31 @@ class TestFeatureParity:
|
|||||||
if output_format.lower() == "m4b":
|
if output_format.lower() == "m4b":
|
||||||
merge_chapters_at_end = True
|
merge_chapters_at_end = True
|
||||||
assert merge_chapters_at_end is True
|
assert merge_chapters_at_end is True
|
||||||
|
|
||||||
|
|
||||||
|
class TestCapsNormalization:
|
||||||
|
"""Tests for caps normalization in planner."""
|
||||||
|
|
||||||
|
def test_caps_normalization_applied_when_enabled(self):
|
||||||
|
"""When normalize_chapter_opening_caps=True, body text is normalized."""
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="<<CHAPTER_MARKER:Chapter 1>>\nALL CAPS OPENING TEXT here",
|
||||||
|
voice="M1",
|
||||||
|
normalize_chapter_opening_caps=True,
|
||||||
|
)
|
||||||
|
plan = build_conversion_plan(req)
|
||||||
|
body = plan.chapters[0].body_text
|
||||||
|
# ALL CAPS should be normalized to Title Case
|
||||||
|
assert body != "ALL CAPS OPENING TEXT here"
|
||||||
|
assert "ALL CAPS" not in body
|
||||||
|
|
||||||
|
def test_caps_normalization_skipped_when_disabled(self):
|
||||||
|
"""When normalize_chapter_opening_caps=False, body text is unchanged."""
|
||||||
|
req = ConversionRequest(
|
||||||
|
direct_text="<<CHAPTER_MARKER:Chapter 1>>\nALL CAPS OPENING TEXT here",
|
||||||
|
voice="M1",
|
||||||
|
normalize_chapter_opening_caps=False,
|
||||||
|
)
|
||||||
|
plan = build_conversion_plan(req)
|
||||||
|
body = plan.chapters[0].body_text
|
||||||
|
assert "ALL CAPS OPENING TEXT" in body
|
||||||
|
|||||||
@@ -16,6 +16,14 @@ from pathlib import Path
|
|||||||
from unittest.mock import MagicMock
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
from abogen.application.conversion_request import ConversionRequest, ConversionRequestError
|
from abogen.application.conversion_request import ConversionRequest, ConversionRequestError
|
||||||
|
from abogen.application.conversion_config import (
|
||||||
|
ChapterChunkConfig,
|
||||||
|
CoverConfig,
|
||||||
|
PronunciationConfig,
|
||||||
|
SaveConfig,
|
||||||
|
SubtitleConfig,
|
||||||
|
WordSubstitutionConfig,
|
||||||
|
)
|
||||||
from abogen.domain.enums import Language, OutputFormat, SaveMode, SubtitleFormat, SubtitleMode
|
from abogen.domain.enums import Language, OutputFormat, SaveMode, SubtitleFormat, SubtitleMode
|
||||||
from abogen.domain.normalization import TTSContext
|
from abogen.domain.normalization import TTSContext
|
||||||
from abogen.domain.settings_core import settings_defaults
|
from abogen.domain.settings_core import settings_defaults
|
||||||
@@ -36,13 +44,13 @@ class TestConversionRequestBasics:
|
|||||||
assert "merge_chapters_at_end" in defaults
|
assert "merge_chapters_at_end" in defaults
|
||||||
|
|
||||||
def test_split_pattern_computation(self):
|
def test_split_pattern_computation(self):
|
||||||
pattern = get_split_pattern("a", "Disabled")
|
pattern = get_split_pattern(Language.EN_US, "Disabled")
|
||||||
assert isinstance(pattern, str)
|
assert isinstance(pattern, str)
|
||||||
assert len(pattern) > 0
|
assert len(pattern) > 0
|
||||||
|
|
||||||
def test_split_pattern_varies_by_subtitle_mode(self):
|
def test_split_pattern_varies_by_subtitle_mode(self):
|
||||||
pattern_disabled = get_split_pattern("a", "Disabled")
|
pattern_disabled = get_split_pattern(Language.EN_US, "Disabled")
|
||||||
pattern_sentence = get_split_pattern("a", "Sentence")
|
pattern_sentence = get_split_pattern(Language.EN_US, "Sentence")
|
||||||
# Different modes should produce different patterns
|
# Different modes should produce different patterns
|
||||||
assert isinstance(pattern_disabled, str)
|
assert isinstance(pattern_disabled, str)
|
||||||
assert isinstance(pattern_sentence, str)
|
assert isinstance(pattern_sentence, str)
|
||||||
@@ -201,23 +209,11 @@ class TestConversionRequestValidation:
|
|||||||
|
|
||||||
def test_defaults_are_valid(self):
|
def test_defaults_are_valid(self):
|
||||||
req = ConversionRequest()
|
req = ConversionRequest()
|
||||||
assert req.max_subtitle_words == 50
|
assert req.subtitle.max_words == 50
|
||||||
assert req.speed == 1.0
|
assert req.speed == 1.0
|
||||||
assert req.supertonic_total_steps == 5
|
assert req.supertonic_total_steps == 5
|
||||||
assert req.output_format == OutputFormat.WAV
|
assert req.output_format == OutputFormat.WAV
|
||||||
assert req.subtitle_mode == SubtitleMode.DISABLED
|
assert req.subtitle.mode == SubtitleMode.DISABLED
|
||||||
|
|
||||||
def test_max_subtitle_words_clamped_below_min(self):
|
|
||||||
req = ConversionRequest(max_subtitle_words=0)
|
|
||||||
assert req.max_subtitle_words == 1
|
|
||||||
|
|
||||||
def test_max_subtitle_words_clamped_above_max(self):
|
|
||||||
req = ConversionRequest(max_subtitle_words=999)
|
|
||||||
assert req.max_subtitle_words == 500
|
|
||||||
|
|
||||||
def test_max_subtitle_words_valid(self):
|
|
||||||
req = ConversionRequest(max_subtitle_words=100)
|
|
||||||
assert req.max_subtitle_words == 100
|
|
||||||
|
|
||||||
def test_speed_clamped_below_min(self):
|
def test_speed_clamped_below_min(self):
|
||||||
req = ConversionRequest(speed=0.1)
|
req = ConversionRequest(speed=0.1)
|
||||||
@@ -248,16 +244,12 @@ class TestConversionRequestValidation:
|
|||||||
assert req.chapter_intro_delay == 0.0
|
assert req.chapter_intro_delay == 0.0
|
||||||
|
|
||||||
def test_invalid_chunk_level_raises(self):
|
def test_invalid_chunk_level_raises(self):
|
||||||
with pytest.raises(ConversionRequestError, match="chunk_level"):
|
with pytest.raises(ValueError, match="chunk_level"):
|
||||||
ConversionRequest(chunk_level="invalid")
|
ConversionRequest(chapter_chunk=ChapterChunkConfig(chunk_level="invalid"))
|
||||||
|
|
||||||
def test_invalid_speaker_mode_raises(self):
|
def test_invalid_speaker_mode_raises(self):
|
||||||
with pytest.raises(ConversionRequestError, match="speaker_mode"):
|
with pytest.raises(ValueError, match="speaker_mode"):
|
||||||
ConversionRequest(speaker_mode="invalid")
|
ConversionRequest(chapter_chunk=ChapterChunkConfig(speaker_mode="invalid"))
|
||||||
|
|
||||||
def test_invalid_max_subtitle_words_type_raises(self):
|
|
||||||
with pytest.raises(ConversionRequestError, match="max_subtitle_words"):
|
|
||||||
ConversionRequest(max_subtitle_words="not_a_number")
|
|
||||||
|
|
||||||
def test_invalid_speed_type_raises(self):
|
def test_invalid_speed_type_raises(self):
|
||||||
with pytest.raises(ConversionRequestError, match="speed"):
|
with pytest.raises(ConversionRequestError, match="speed"):
|
||||||
@@ -275,12 +267,27 @@ class TestConversionRequestValidation:
|
|||||||
req = ConversionRequest(
|
req = ConversionRequest(
|
||||||
language=Language.FR,
|
language=Language.FR,
|
||||||
output_format=OutputFormat.MP3,
|
output_format=OutputFormat.MP3,
|
||||||
subtitle_mode=SubtitleMode.SENTENCE,
|
subtitle=SubtitleConfig(
|
||||||
subtitle_format=SubtitleFormat.ASS,
|
mode=SubtitleMode.SENTENCE,
|
||||||
save_mode=SaveMode.CUSTOM_FOLDER,
|
format=SubtitleFormat.ASS,
|
||||||
|
),
|
||||||
|
save=SaveConfig(mode=SaveMode.CUSTOM_FOLDER),
|
||||||
)
|
)
|
||||||
assert req.language == Language.FR
|
assert req.language == Language.FR
|
||||||
assert req.output_format == OutputFormat.MP3
|
assert req.output_format == OutputFormat.MP3
|
||||||
assert req.subtitle_mode == SubtitleMode.SENTENCE
|
assert req.subtitle.mode == SubtitleMode.SENTENCE
|
||||||
assert req.subtitle_format == SubtitleFormat.ASS
|
assert req.subtitle.format == SubtitleFormat.ASS
|
||||||
assert req.save_mode == SaveMode.CUSTOM_FOLDER
|
assert req.save.mode == SaveMode.CUSTOM_FOLDER
|
||||||
|
|
||||||
|
def test_config_objects_constructed(self):
|
||||||
|
req = ConversionRequest(
|
||||||
|
subtitle=SubtitleConfig(max_words=100),
|
||||||
|
cover=CoverConfig(path=Path("/tmp/cover.jpg"), mime="image/jpeg"),
|
||||||
|
pronunciation=PronunciationConfig(normalization_overrides={"key": "val"}),
|
||||||
|
save=SaveConfig(save_as_project=True),
|
||||||
|
)
|
||||||
|
assert req.subtitle.max_words == 100
|
||||||
|
assert req.cover.path == Path("/tmp/cover.jpg")
|
||||||
|
assert req.cover.mime == "image/jpeg"
|
||||||
|
assert req.pronunciation.normalization_overrides == {"key": "val"}
|
||||||
|
assert req.save.save_as_project is True
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user