Author SHA1 Message Date
Deniz Şafak 6bd2c109d8 flet ui (experimental) 2026-05-24 14:58:41 +03:00
246 changed files with 15143 additions and 33895 deletions
-15
View File
@@ -1,15 +0,0 @@
*.py text eol=lf
*.md text eol=lf
*.yml text eol=lf
*.yaml text eol=lf
*.toml text eol=lf
*.json text eol=lf
*.txt text eol=lf
*.html text eol=lf
*.css text eol=lf
*.js text eol=lf
*.sh text eol=lf
*.cfg text eol=lf
*.ini text eol=lf
*.svg text eol=lf
*.j2 text eol=lf
+1 -1
View File
@@ -1,6 +1,6 @@
# These are supported funding model platforms
github: [jborza, jeremiahsb, mohangk, k0sm0naft]
github: [jborza, jeremiahsb, mohangk]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
+12 -32
View File
@@ -1,9 +1,7 @@
name: CI
run-name: CI
on:
name: pip install
run-name: pip install
on:
push:
branches: [main]
paths:
- '**.py'
- 'pyproject.toml'
@@ -13,41 +11,23 @@ on:
- 'pyproject.toml'
- '.github/workflows/**'
workflow_dispatch:
jobs:
test:
install-and-run:
strategy:
matrix:
os: [ubuntu-latest, macos-14, windows-latest]
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ['3.12']
fail-fast: false
continue-on-error: true
runs-on: ${{ matrix.os }}
steps:
- name: Checkout repository
uses: actions/checkout@v7
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v6
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install uv
uses: astral-sh/setup-uv@v8.3.1
with:
enable-cache: true
prune-cache: false
cache-dependency-glob: pyproject.toml
- name: Install system dependencies (Ubuntu)
if: runner.os == 'Linux'
run: sudo apt-get update && sudo apt-get install -y libegl1
- name: Install dependencies
run: uv pip install --system .[dev]
env:
UV_LINK_MODE: copy
- name: Run tests
env:
QT_QPA_PLATFORM: offscreen
run: pytest tests/ -v --tb=short
- name: Install from repository
run: python -m pip install .
#- name: Run abogen
# run: abogen
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v4
- name: Login to Github Container Registry
# Only if we need to push an image
-4
View File
@@ -39,7 +39,3 @@ dist/
test_assets/
dev_notes/
.claude/
.coverage
# CodeGraph index (local, machine-specific)
.codegraph/
-82
View File
@@ -1,82 +0,0 @@
# AGENTS.md — Segmentation & Subtitle System Contract
This document is the source of truth for how text is split for **voice
processing** (TTS engine segmentation) and **subtitle processing**, across
languages, TTS engines, and subtitle modes. It was written after a bug where
sentence modes "processed all text as a whole" (one merged engine segment →
one giant subtitle). **Do not change this behavior without updating this
table.**
## Voice processing — split pattern passed to the TTS engine
`get_split_pattern(language, mode)` in `abogen/domain/split_pattern.py` is the
default; the spaCy pre-TTS path overrides it. Both UIs must stay in sync:
`spacy_pre_tts_segmentation` (`abogen/domain/conversion_pipeline.py`, WebUI)
and the inline branch in `abogen/pyqt/conversion.py` (~line 860, PyQt).
| Subtitle mode | English (en-US/en-GB) | Non-English, spaCy ON | Non-English, spaCy OFF | CJK (ja/zh) |
|---|---|---|---|---|
| Disabled | `\n` | spaCy pre-split, engine `\n` | `\n+` | `(?<=[.!?؟。!?।])\s*\|\n+` |
| Line | `\n` | spaCy pre-split, engine `\n` | `\n` | `(?<=[.!?؟。!?।])\s*\|\n+` |
| Sentence | `\n` | spaCy pre-split, engine `\n` | `(?<=[.!?؟。!?।])\s+\|\n+` | `(?<=[.!?؟。!?।])\s*\|\n+` |
| Sentence + Comma | `\n` | spaCy pre-split, engine `\n` | `(?<=[.!?,؟。!?،،、।])\s+\|\n+` (commas kept) | `(?<=[.!?,؟。!?،،、।])\s*\|\n+` |
| Sentence + Highlighting | `\n+` | `\n+` | `\n+` | `\n+` |
| N words ("5 words") | `\n` (→ Disabled) | `\n+` | `\n+` | Disabled CJK pattern |
Rules baked into this table:
- **English voice splitting is ALWAYS newline-only** for Disabled, Line,
Sentence, and Sentence + Comma. English sentence/comma boundaries are
produced ONLY at subtitle time (spaCy post-TTS / regex fallback). Never add
punctuation to the English engine pattern.
- **Non-English + spaCy ON**: spaCy pre-segments the text (pre-TTS); the
engine pattern is `\n` for Sentence AND Sentence + Comma — **never commas**.
spaCy is skipped when the toggle is off, mode is Disabled/Line, or input is
a subtitle file.
- **Non-English + spaCy OFF** (toggle off, spaCy failure, subtitle input): the
default pattern is used — Sentence + Comma KEEPS its commas here. This is
the intentional fallback, not a bug.
- CJK: punctuation-based patterns for Disabled/Line (historical); spacing is
`\s*` (no spaces needed between CJK chars).
- Engine-level extra chunking (applies after the pattern): kokoro English
re-chunks at ~510 phonemes; kokoro non-English at ~400 chars; supertonic
caps each part at 300 chars.
## Subtitle processing — post-TTS, from tokens
| Mode | Behavior |
|---|---|
| Disabled | no subtitles |
| Line | one entry per TTS segment (line) |
| Sentence | sentence boundaries: English → spaCy; others → regex on `[.!?…]` |
| Sentence + Comma | sentence + comma boundaries at subtitle time (both languages) — commas never affect voice |
| Sentence + Highlighting | karaoke `{\kf…}` per word, grouped by sentence |
| N words | groups of N words by whitespace counting |
Token granularity (timing quality): kokoro English emits **per-word tokens**
with timestamps; kokoro non-English and supertonic emit **no tokens** → each
engine segment becomes one FakeToken, split by regex with proportional timing
when it contains multiple sentences.
## Hard invariants (breaking these reintroduces the original bug)
1. `Pipeline.__call__` (`abogen/tts_plugin/utils.py`) must yield ONE `Segment`
per engine segment (with tokens) — never merge segments back into the
whole text. `SynthesizedAudio.segments` carries the per-segment data;
engines expose it in `plugins/kokoro/engine.py` and
`plugins/supertonic/engine.py`.
2. `tts_segments` (`abogen/domain/conversion_pipeline.py`) restores trailing
whitespace on segment-boundary tokens ONLY for real per-word tokens, never
for FakeToken fallbacks.
3. `_to_language_enum` must return `lang_code` as-is when it is already a
`Language` enum (`str(Language.ES)` is `"Language.ES"`, which silently
resolved to EN_US and disabled spaCy pre-TTS for every language in WebUI).
4. English must never use spaCy for PRE-TTS segmentation — only for subtitles.
## Guarded by tests
- `tests/test_split_pattern.py` — English newline-only; non-English sentence
patterns; CJK behavior.
- `tests/test_domain_conversion_pipeline.py``tts_segments` / spaCy
segmentation helpers.
- Full suite: `python -m pytest tests/ -q` (expect 1566+ passing).
+540 -438
View File
File diff suppressed because it is too large Load Diff
-8
View File
@@ -1,8 +0,0 @@
"""Application layer for conversion flow unification.
This package contains the application-level orchestration logic
that bridges UI adapters (PyQt, WebUI) with domain functions.
The main entry point is ConversionService.run() which coordinates
planning, execution, and finalization of a conversion job.
"""
-62
View File
@@ -1,62 +0,0 @@
"""Chapter selection helpers for the application layer.
Builds chapter payloads with smart defaults (preselection based on
supplement score) and character counts. Used by both WebUI and PyQt.
"""
from __future__ import annotations
from typing import Any, Dict, List
from abogen.domain.chapter_classification import (
ensure_at_least_one_chapter_enabled,
should_preselect_chapter,
)
from abogen.domain.text_utils import calculate_text_length
def build_chapter_payload(
chapters: List[Any],
source_name: str = "",
) -> List[Dict[str, Any]]:
"""Build a chapter payload with preselection and character counts.
Args:
chapters: List of chapter-like objects with ``title`` and ``text`` attributes.
source_name: Fallback title for the placeholder chapter when *chapters* is empty.
Returns:
List of chapter dicts ready for ``PendingJob.chapters`` or ``ChapterChunkConfig``.
"""
total = len(chapters)
payload: List[Dict[str, Any]] = []
for index, chapter in enumerate(chapters):
title = getattr(chapter, "title", "") or ""
text = getattr(chapter, "text", "") or ""
enabled = should_preselect_chapter(title, text, index, total)
payload.append(
{
"id": f"{index:04d}",
"index": index,
"title": title,
"text": text,
"characters": calculate_text_length(text),
"enabled": enabled,
}
)
if not payload:
payload.append(
{
"id": "0000",
"index": 0,
"title": source_name,
"text": "",
"characters": 0,
"enabled": True,
}
)
ensure_at_least_one_chapter_enabled(payload)
return payload
-78
View File
@@ -1,78 +0,0 @@
"""Application-layer cleanup — global resource disposal.
Handles:
- GPU/CUDA memory flush
- TTS engine disposal (PluginManager)
- UI-specific cleanup callbacks (registered by entry points)
Called by shutdown.py at process exit and by run_conversion() per-conversion.
"""
from __future__ import annotations
import gc
import sys
from typing import Callable
_UI_CLEANUPS: list[Callable[[], None]] = []
def flush_cuda() -> None:
"""Run GC and release CUDA cache. Safe to call multiple times."""
gc.collect()
# Skip entirely if torch was never imported — importing it here just to
# check would add several seconds to shutdown with nothing to flush.
if "torch" not in sys.modules:
return
try:
torch = sys.modules["torch"]
if torch.cuda.is_available():
torch.cuda.empty_cache()
torch.cuda.ipc_collect()
except Exception:
pass
def dispose_engines() -> None:
"""Dispose all cached TTS engines via PluginManager."""
try:
from abogen.tts_plugin.plugin_manager import get_plugin_manager
get_plugin_manager().dispose_all()
except Exception:
pass
def _clear_global_voice_cache() -> None:
"""Reset the global voice download cache state."""
try:
from abogen.voice_cache import clear_voice_cache
clear_voice_cache()
except Exception:
pass
def register_ui_cleanup(fn: Callable[[], None]) -> None:
"""Register a UI-specific cleanup callback (e.g. preview threads, temp files)."""
_UI_CLEANUPS.append(fn)
def cleanup() -> None:
"""Run all application-level cleanups. Idempotent."""
dispose_engines()
flush_cuda()
_clear_global_voice_cache()
for fn in _UI_CLEANUPS:
try:
fn()
except Exception:
pass
_UI_CLEANUPS.clear()
__all__ = [
"flush_cuda",
"dispose_engines",
"register_ui_cleanup",
"cleanup",
]
-95
View File
@@ -1,95 +0,0 @@
"""Feature config objects for ConversionRequest.
Each config object groups parameters for a specific feature.
If the object is None, the feature is disabled.
This keeps ConversionRequest clean: no boolean flags for feature toggles,
no scattered parameters across unrelated fields.
Domain config types (PronunciationConfig, SubtitleConfig) live in
domain/config_types.py — domain defines the contract, app fills them.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional
from abogen.domain.config_types import CoverConfig, PronunciationConfig, SubtitleConfig
from abogen.domain.enums import OutputFormat, SaveMode
@dataclass(frozen=True)
class WordSubstitutionConfig:
"""Word substitution settings.
When present on ConversionRequest, word substitution is applied
to the source text before chapter parsing.
"""
substitutions_list: str = ""
case_sensitive: bool = False
replace_caps: bool = False
replace_numerals: bool = False
fix_punctuation: bool = False
@dataclass(frozen=True)
class SubtitleInputConfig:
"""Subtitle file input settings.
When present on ConversionRequest, the source is treated as a
subtitle file (.srt/.ass/.vtt) or timestamp text, and the
subtitle-to-audio pipeline is used instead of normal text conversion.
"""
is_timestamp_text: bool = False
@dataclass(frozen=True)
class Epub3ExportConfig:
"""EPUB3 export settings.
When present on ConversionRequest, an EPUB3 package with
synchronized audio narration is generated after conversion.
"""
book_id: str = ""
@dataclass(frozen=True)
class ChapterChunkConfig:
"""Chapter and chunk configuration.
Groups chapter overrides, chunk data, and speaker settings
used by the planner to build segments.
"""
chapter_overrides: List[Dict[str, Any]] = field(default_factory=list)
chunks: List[Dict[str, Any]] = field(default_factory=list)
chunk_level: str = "paragraph"
speaker_mode: str = "single"
speakers: Dict[str, Any] = field(default_factory=dict)
def __post_init__(self) -> None:
_VALID_CHUNK_LEVELS = ("paragraph", "sentence")
_VALID_SPEAKER_MODES = ("single", "multi")
if self.chunk_level not in _VALID_CHUNK_LEVELS:
raise ValueError(
f"chunk_level must be one of {_VALID_CHUNK_LEVELS}, got {self.chunk_level!r}"
)
if self.speaker_mode not in _VALID_SPEAKER_MODES:
raise ValueError(
f"speaker_mode must be one of {_VALID_SPEAKER_MODES}, got {self.speaker_mode!r}"
)
@dataclass(frozen=True)
class SaveConfig:
"""Save/output settings.
Groups save mode, output folder, chapter splitting, and merge options.
"""
mode: SaveMode = SaveMode.SAVE_NEXT_TO_INPUT
output_folder: Optional[Path] = None
save_chapters_separately: bool = False
merge_chapters_at_end: bool = True
separate_chapters_format: OutputFormat = OutputFormat.WAV
save_as_project: bool = False
-660
View File
@@ -1,660 +0,0 @@
"""Unified conversion executor.
Takes a ConversionPlan and ports, executes the TTS conversion,
and returns a ConversionResult. No UI imports allowed.
This is Stage 6 of the conversion flow unification plan.
"""
from __future__ import annotations
import logging
import time
from contextlib import ExitStack
from typing import Any, Callable, Dict, List, Optional, Set, Tuple
from abogen.application.conversion_models import (
ConversionPlan,
)
from abogen.application.conversion_ports import (
AudioSink,
ConversionEvents,
PipelineProvider,
SubtitleWriter,
VoiceResolver,
)
from abogen.application.conversion_result import ConversionResult
from abogen.domain.audio_sink import open_audio_sink
from abogen.domain.conversion_engine import (
SegmentStats,
SynthParams,
process_and_write_subtitles,
synthesize_text,
)
from abogen.domain.enums import OutputFormat, SubtitleMode
from abogen.domain.normalization import TTSContext
from abogen.domain.chapter_titles import (
apply_chapter_text_transforms,
headings_equivalent as _headings_equivalent,
)
from abogen.domain.output_paths import sanitize_filename_for_chapter
from abogen.infrastructure.subtitle_writer import make_subtitle_writer
# ─── MarkerCollector ───
class MarkerCollector:
"""Observes execution events and accumulates chapter/chunk markers.
Separates marker collection from synthesis logic.
"""
def __init__(self) -> None:
self._chapter_markers: List[Dict[str, Any]] = []
self._chunk_markers: List[Dict[str, Any]] = []
self._current_chapter_voices: Set[Tuple[str, str]] = set()
self._current_chapter_index: int = 0
self._current_chapter_title: str = ""
self._current_chapter_start: float = 0.0
def on_chapter_start(
self, index: int, title: str, start_time: float
) -> None:
"""Record chapter start."""
self._current_chapter_index = index
self._current_chapter_title = title
self._current_chapter_start = start_time
self._current_chapter_voices.clear()
def on_segment(
self,
provider: str,
voice: Any,
voice_spec: str,
speaker_id: str = "narrator",
) -> None:
"""Record a voice used in this chapter (for multi-speaker tracking)."""
self._current_chapter_voices.add((provider, voice_spec))
def on_chunk(
self,
chunk_id: str,
chapter_index: int,
chunk_index: int,
start: float,
end: float,
speaker_id: str,
provider: str,
voice_spec: str,
level: str,
characters: int,
) -> None:
"""Record a chunk marker."""
self._chunk_markers.append({
"id": chunk_id,
"chapter_index": chapter_index,
"chunk_index": chunk_index,
"start": start,
"end": end,
"speaker_id": speaker_id,
"voice": {"provider": provider, "voice": voice_spec},
"level": level,
"characters": characters,
})
def on_chapter_end(self, end_time: float) -> None:
"""Record chapter end and build chapter marker."""
voices = [
{"provider": p, "voice": v}
for p, v in sorted(self._current_chapter_voices)
]
self._chapter_markers.append({
"chapter_index": self._current_chapter_index,
"index": self._current_chapter_index + 1,
"title": self._current_chapter_title,
"start": self._current_chapter_start,
"end": end_time,
"voices": voices,
})
def on_outro(
self,
start_time: float,
end_time: float,
provider: str,
voice_spec: str,
) -> None:
"""Record outro chapter marker."""
self._chapter_markers.append({
"chapter_index": len(self._chapter_markers),
"index": len(self._chapter_markers) + 1,
"title": "Outro",
"start": start_time,
"end": end_time,
"voices": [{"provider": provider, "voice": voice_spec}],
})
@property
def chapter_markers(self) -> List[Dict[str, Any]]:
return self._chapter_markers
@property
def chunk_markers(self) -> List[Dict[str, Any]]:
return self._chunk_markers
def execute_conversion(
plan: ConversionPlan,
events: ConversionEvents,
pipeline_provider: PipelineProvider,
voice_resolver: VoiceResolver,
tts_context: TTSContext,
*,
check_cancelled: Optional[Callable[[], None]] = None,
) -> ConversionResult:
"""Execute a conversion plan and return the result.
Args:
plan: The conversion plan from build_conversion_plan()
events: UI-specific callbacks (log, progress, check_cancelled)
pipeline_provider: Provides TTS backends
voice_resolver: Resolves voice specs into loaded voices
tts_context: Normalization context for text processing
check_cancelled: Optional cancellation checker (overrides events.check_cancelled)
Returns:
ConversionResult with paths and markers
Raises:
ConversionCancelled: If conversion is cancelled
"""
request = plan.request
result = ConversionResult(metadata=plan.metadata)
collector = MarkerCollector()
logging.info(
"[executor] Starting: chapters=%d intro=%s outro=%s merge=%s",
len(plan.chapters),
bool(plan.intro and plan.intro.enabled),
bool(plan.outro and plan.outro.enabled),
request.save.merge_chapters_at_end,
)
# Determine cancellation checker
if check_cancelled is None:
check_cancelled = lambda: events.check_cancelled()
# Stats for progress tracking
total_characters = sum(
len(ch.body_text) for ch in plan.chapters
)
if plan.intro and plan.intro.enabled:
total_characters += len(plan.intro.text)
if plan.outro and plan.outro.enabled:
total_characters += len(plan.outro.text)
stats = SegmentStats(
processed_chars=0,
current_time=0.0,
etr_start_time=time.time(),
total_characters=total_characters,
)
# Compute subtitle flag once (used in every synthesize_text call)
use_spacy = request.subtitle.mode not in (SubtitleMode.DISABLED, SubtitleMode.LINE)
# Output paths
output_layout = plan.output_layout
if not output_layout:
raise ValueError("ConversionPlan must have an output_layout")
# Determine if merged output is needed
merge_chapters = request.save.merge_chapters_at_end or not request.save.save_chapters_separately
if request.output_format == OutputFormat.M4B:
merge_chapters = True
# Resolve voices
base_voice_spec = request.voice or "M1"
logging.info("[executor] Resolving base voice: spec=%s", base_voice_spec)
base_provider, base_voice_choice, base_speed, base_steps = _resolve_voice(
voice_resolver, base_voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
)
logging.info("[executor] Base voice resolved: provider=%s voice=%s speed=%.2f", base_provider, base_voice_choice, base_speed)
# Use ExitStack for resource management
with ExitStack() as stack:
# Open merged audio sink
audio_sink: Optional[AudioSink] = None
audio_path = None
if merge_chapters:
audio_path = output_layout.audio_dir / f"{_base_name(request)}{request.output_format.dot_ext}"
meta = plan.metadata if plan.metadata else None
audio_sink = stack.enter_context(
open_audio_sink(
audio_path,
request.output_format,
metadata=meta,
cancel_check=check_cancelled,
)
)
result.audio_path = audio_path
# Open subtitle writer if needed
subtitle_writer: Optional[SubtitleWriter] = None
if request.subtitle.mode != SubtitleMode.DISABLED and audio_sink:
subtitle_writer = make_subtitle_writer(
audio_path,
request.subtitle,
)
if subtitle_writer:
subtitle_writer.open()
stack.callback(subtitle_writer.close)
result.subtitle_paths.append(subtitle_writer.path)
effective_subtitle_mode = request.subtitle.mode if subtitle_writer else SubtitleMode.DISABLED
synth = SynthParams(
tts_context=tts_context,
stats=stats,
check_cancel=check_cancelled,
on_progress=lambda pct, etr: events.progress(pct, etr),
audio_sink=audio_sink,
subtitle_mode=effective_subtitle_mode,
max_subtitle_words=request.subtitle.max_words,
language=request.language,
use_spacy_segmentation=use_spacy,
)
# Chapter directory
chapter_dir = None
if request.save.save_chapters_separately and len(plan.chapters) > 1:
chapter_dir = output_layout.audio_dir / "chapters"
chapter_dir.mkdir(parents=True, exist_ok=True)
# Process intro
intro_emitted = False
if plan.intro and plan.intro.enabled and merge_chapters:
events.log(f"Title intro: {plan.intro.text[:80]}")
intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice(
voice_resolver, plan.intro.voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
)
intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu)
synthesize_text(
text=plan.intro.text,
params=synth,
backend=intro_backend,
voice=intro_voice,
speed=intro_speed or request.speed,
total_steps=intro_steps,
chapter_sink=None,
preview_callback=lambda text: events.log(f" {text[:80]}"),
)
intro_emitted = True
events.log("Intro synthesized.")
# Chapter loop
for chapter_idx, chapter in enumerate(plan.chapters, 1):
check_cancelled()
chapter_display = f"Chapter {chapter_idx}/{len(plan.chapters)}: {chapter.title}"
events.log(f"Processing {chapter_display}")
logging.info("[executor] Chapter %d/%d: %s", chapter_idx, len(plan.chapters), chapter.title)
# Resolve chapter voice
chapter_provider, chapter_voice, chapter_speed, chapter_steps = _resolve_voice(
voice_resolver, chapter.voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
)
logging.info("[executor] Chapter %d voice: provider=%s voice=%s speed=%.2f", chapter_idx, chapter_provider, chapter_voice, chapter_speed)
chapter_backend = pipeline_provider.get(chapter_provider, request.language, request.use_gpu)
# Record chapter start for markers
collector.on_chapter_start(chapter_idx - 1, chapter.title, stats.current_time)
# Per-chapter sink
chapter_sink: Optional[AudioSink] = None
chapter_path = None
if chapter_dir:
chapter_filename = sanitize_filename_for_chapter(chapter.title, chapter_idx)
chapter_path = chapter_dir / f"{chapter_filename}.{request.save.separate_chapters_format}"
chapter_sink = stack.enter_context(
open_audio_sink(
chapter_path,
request.save.separate_chapters_format,
cancel_check=check_cancelled,
)
)
result.chapter_paths.append(chapter_path)
# Per-chapter subtitle writer
chapter_subtitle_writer: Optional[SubtitleWriter] = None
if chapter_dir and request.subtitle.mode != SubtitleMode.DISABLED and chapter_sink:
from abogen.infrastructure.subtitle_writer import resolve_subtitle_format
chapter_filename = sanitize_filename_for_chapter(chapter.title, chapter_idx)
subtitle_ext, _ = resolve_subtitle_format(
request.subtitle
)
chapter_subtitle_path = chapter_dir / f"{chapter_filename}.{subtitle_ext}"
chapter_subtitle_writer = make_subtitle_writer(
chapter_subtitle_path,
request.subtitle,
)
if chapter_subtitle_writer:
chapter_subtitle_writer.open()
result.subtitle_paths.append(chapter_subtitle_writer.path)
# Intro delay before first chapter
if not intro_emitted and plan.intro and plan.intro.enabled:
# Intro will be emitted with first chapter
intro_provider, intro_voice, intro_speed, intro_steps = _resolve_voice(
voice_resolver, plan.intro.voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
)
intro_backend = pipeline_provider.get(intro_provider, request.language, request.use_gpu)
synthesize_text(
text=plan.intro.text,
params=synth,
backend=intro_backend,
voice=intro_voice,
speed=intro_speed or request.speed,
total_steps=intro_steps,
chapter_sink=chapter_sink,
preview_callback=lambda text: events.log(f" Intro: {text[:80]}"),
)
intro_emitted = True
if request.chapter_intro_delay > 0:
_append_silence(
request.chapter_intro_delay,
chapter_sink=chapter_sink,
audio_sink=audio_sink,
stats=stats,
)
# Process heading
heading_text = ""
if chapter.title:
heading_text = _format_heading(chapter.title, chapter_idx, request)
if heading_text:
synthesize_text(
text=heading_text,
params=synth,
backend=chapter_backend,
voice=chapter_voice,
speed=chapter_speed or request.speed,
chapter_sink=chapter_sink,
preview_callback=lambda text: events.log(f" Title: {text[:80]}"),
)
if request.chapter_intro_delay > 0:
_append_silence(
request.chapter_intro_delay,
chapter_sink=chapter_sink,
audio_sink=audio_sink,
stats=stats,
)
# Heading dedup: check if first line of body matches heading
pending_heading_strip = False
if heading_text and chapter.body_text:
first_line = next(
(line.strip() for line in chapter.body_text.splitlines() if line.strip()),
"",
)
if first_line and _headings_equivalent(first_line, heading_text):
pending_heading_strip = True
# Process body segments
for seg_idx, segment in enumerate(chapter.segments):
check_cancelled()
# Apply heading dedup to first segment (consume-once)
seg_text = segment.text
if pending_heading_strip and seg_text.strip():
seg_text, heading_removed, _ = apply_chapter_text_transforms(
seg_text,
heading_text=heading_text,
raw_title=chapter.title,
strip_heading=True,
normalize_caps=False,
)
if heading_removed:
pending_heading_strip = False
if not seg_text.strip():
continue
# Resolve segment voice (may differ from chapter voice)
if segment.voice_spec != chapter.voice_spec:
seg_provider, seg_voice, seg_speed, seg_steps = _resolve_voice(
voice_resolver, segment.voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
)
seg_backend = pipeline_provider.get(seg_provider, request.language, request.use_gpu)
else:
seg_provider = chapter_provider
seg_voice = chapter_voice
seg_speed = chapter_speed
seg_steps = chapter_steps
seg_backend = chapter_backend
# Track voice for chapter marker
collector.on_segment(seg_provider, seg_voice, segment.voice_spec)
# spaCy pre-TTS segmentation
from abogen.domain.conversion_pipeline import spacy_pre_tts_segmentation
is_subtitle_input = bool(
request.subtitle_input
)
spacy_segments, active_split = spacy_pre_tts_segmentation(
seg_text,
request.language,
request.subtitle.mode,
is_subtitle_input=is_subtitle_input,
use_spacy_segmentation=use_spacy,
log_callback=lambda msg: events.log(msg),
)
seg_start_time = stats.current_time
accumulated_tokens: List[Dict[str, Any]] = []
for spacy_seg in spacy_segments:
if not spacy_seg.strip():
continue
_, seg_tokens = synthesize_text(
text=spacy_seg,
params=synth,
backend=seg_backend,
voice=seg_voice,
speed=seg_speed or request.speed,
total_steps=seg_steps,
chapter_sink=chapter_sink,
preview_callback=lambda text: events.log(f" {text[:80]}"),
split_pattern_override=active_split,
)
accumulated_tokens.extend(seg_tokens)
# Process subtitles
if audio_sink and accumulated_tokens:
if subtitle_writer:
process_and_write_subtitles(
accumulated_tokens,
subtitle_writer,
subtitle=request.subtitle,
language=request.language,
use_spacy_segmentation=use_spacy,
fallback_end_time=stats.current_time,
)
if chapter_subtitle_writer:
process_and_write_subtitles(
accumulated_tokens,
chapter_subtitle_writer,
subtitle=request.subtitle,
language=request.language,
use_spacy_segmentation=use_spacy,
fallback_end_time=stats.current_time,
)
# Record chunk marker
if segment.source in ("chunk", "voice_marker"):
collector.on_chunk(
chunk_id=segment.chunk_id or "",
chapter_index=chapter_idx - 1,
chunk_index=segment.chunk_index or seg_idx,
start=seg_start_time,
end=stats.current_time,
speaker_id=segment.speaker_id or "narrator",
provider=seg_provider,
voice_spec=segment.voice_spec,
level=segment.level or (request.chapter_chunk.chunk_level if request.chapter_chunk else "paragraph"),
characters=len(segment.text),
)
# Silence between chapters
if chapter_idx < len(plan.chapters) and request.silence_between_chapters > 0:
_append_silence(
request.silence_between_chapters,
chapter_sink=chapter_sink,
audio_sink=audio_sink,
stats=stats,
)
# Close chapter sink
if chapter_sink:
chapter_sink.close()
# Close chapter subtitle writer
if chapter_subtitle_writer:
chapter_subtitle_writer.close()
# Record chapter end for markers
collector.on_chapter_end(stats.current_time)
logging.info("[executor] Chapter %d/%d done: time=%.1fs", chapter_idx, len(plan.chapters), stats.current_time)
logging.info("[executor] All chapters done: total=%.1fs", stats.current_time)
# Process outro
if plan.outro and plan.outro.enabled and merge_chapters:
events.log(f"Closing outro: {plan.outro.text[:80]}")
outro_provider, outro_voice, outro_speed, outro_steps = _resolve_voice(
voice_resolver, plan.outro.voice_spec, request,
log_callback=lambda msg: events.log(msg, level="warning"),
)
outro_backend = pipeline_provider.get(outro_provider, request.language, request.use_gpu)
# Silence before outro
if request.silence_between_chapters > 0:
_append_silence(
request.silence_between_chapters,
chapter_sink=None,
audio_sink=audio_sink,
stats=stats,
)
outro_start = stats.current_time
synthesize_text(
text=plan.outro.text,
params=synth,
backend=outro_backend,
voice=outro_voice,
speed=outro_speed or request.speed,
total_steps=outro_steps,
chapter_sink=None,
preview_callback=lambda text: events.log(f" {text[:80]}"),
)
# Record outro marker
collector.on_outro(outro_start, stats.current_time, outro_provider, plan.outro.voice_spec)
events.log("Outro synthesized.")
# Set result metadata
result.chapter_markers = collector.chapter_markers
result.chunk_markers = collector.chunk_markers
result.total_chapters = len(plan.chapters)
result.total_segments = sum(len(ch.segments) for ch in plan.chapters)
result.total_characters = total_characters
if output_layout.project_root:
result.project_root = output_layout.project_root
return result
# ─── Helpers ────────────────────────────────────────────────────────
def _resolve_voice(
resolver: VoiceResolver,
voice_spec: str,
request: Any,
*,
log_callback: Optional[Callable[[str], None]] = None,
) -> Tuple[str, Any, Optional[float], Optional[int]]:
"""Resolve a voice spec and return (provider, voice, speed, steps)."""
try:
resolved = resolver.resolve(voice_spec)
return (
resolved.provider,
resolved.voice,
resolved.speed,
resolved.supertonic_steps,
)
except Exception as exc:
# Fallback to base voice
base_spec = request.voice or "M1"
if log_callback:
log_callback(
f"Voice '{voice_spec}' failed to resolve: {exc}. "
f"Falling back to '{base_spec}'."
)
try:
resolved = resolver.resolve(base_spec)
except Exception as fallback_exc:
raise RuntimeError(
f"Both voice '{voice_spec}' and fallback '{base_spec}' failed to resolve. "
f"Primary error: {exc}; Fallback error: {fallback_exc}"
) from fallback_exc
return (
resolved.provider,
resolved.voice,
resolved.speed,
resolved.supertonic_steps,
)
def _base_name(request: Any) -> str:
"""Get base name for output file."""
from abogen.domain.output_paths import sanitize_output_stem
if request.original_filename:
return sanitize_output_stem(request.original_filename)
return "output"
def _format_heading(title: str, index: int, request: Any) -> str:
"""Format chapter heading for TTS."""
from abogen.domain.chapter_titles import format_spoken_chapter_title
if request.auto_prefix_chapter_titles:
return format_spoken_chapter_title(title, index, apply_prefix=True)
return title
def _append_silence(
duration: float,
*,
chapter_sink: Optional[AudioSink],
audio_sink: Optional[AudioSink],
stats: SegmentStats,
) -> None:
"""Append silence to sinks."""
from abogen.domain.audio_buffer import create_silence
silence = create_silence(duration)
if silence.size == 0:
return
if chapter_sink:
chapter_sink.write(silence)
if audio_sink:
audio_sink.write(silence)
stats.current_time += duration
-97
View File
@@ -1,97 +0,0 @@
"""Core models for conversion planning.
These dataclasses represent the structured plan for a conversion job.
They are UI-agnostic and describe WHAT to convert, not HOW to do it.
The planning flow:
ConversionRequest -> ConversionPlan -> ConversionResult
ConversionPlan contains:
- ChapterPlan[]: chapters with their segments
- SegmentPlan[]: individual text segments with voice specs
- OutputLayout: where to write outputs
- IntroOutroSpec: optional intro/outro
"""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING, Any, Dict, List, Optional
if TYPE_CHECKING:
from abogen.application.conversion_request import ConversionRequest
from abogen.text_extractor import ExtractionResult
@dataclass
class SegmentPlan:
"""A single text segment with its voice specification.
This is the unified model for:
- Regular chapter body text
- PyQt voice markers (<<VOICE:F1>>)
- WebUI chunks with per-chunk voice/speaker
- Intro/outro text
- Chapter headings
"""
text: str
voice_spec: str
kind: str = "body" # intro, heading, body, outro
speaker_id: str = "narrator"
chunk_id: Optional[str] = None
chunk_index: Optional[int] = None
level: Optional[str] = None # chunk level (paragraph, sentence, etc.)
source: str = "chapter" # chapter, voice_marker, chunk
@dataclass
class ChapterPlan:
"""A chapter with its metadata and segments."""
index: int
title: str
original_title: str
body_text: str
segments: List[SegmentPlan]
voice_spec: str # default voice for this chapter
@dataclass
class OutputLayout:
"""Resolved output paths for a conversion job."""
parent_dir: Path
merged_path: Optional[Path] = None
chapter_dir: Optional[Path] = None
project_root: Optional[Path] = None
audio_dir: Optional[Path] = None
subtitle_dir: Optional[Path] = None
metadata_dir: Optional[Path] = None
@dataclass
class IntroOutroSpec:
"""Intro/outro specification with resolved text and voice."""
enabled: bool = False
text: str = ""
voice_spec: str = ""
kind: str = "intro" # intro or outro
@dataclass
class ConversionPlan:
"""Complete plan for a conversion job.
This is the output of the planning phase and input to the executor.
"""
request: ConversionRequest
metadata: Dict[str, Any]
chapters: List[ChapterPlan]
intro: Optional[IntroOutroSpec] = None
outro: Optional[IntroOutroSpec] = None
output_layout: Optional[OutputLayout] = None
extraction: Optional[ExtractionResult] = None
-393
View File
@@ -1,393 +0,0 @@
"""Unified conversion planner.
Pure functions that take a ConversionRequest and produce a ConversionPlan.
No side effects, no I/O — all complexity from both UIs in one place.
This is Stage 2 of the conversion flow unification plan.
"""
from __future__ import annotations
import logging
from typing import Any, Dict, List, Optional, Tuple
from abogen.application.conversion_models import (
ChapterPlan,
ConversionPlan,
IntroOutroSpec,
SegmentPlan,
)
from abogen.application.conversion_request import ConversionRequest
from abogen.application.output_layout_service import resolve_output_layout
from abogen.domain.chapter_overrides import apply_chapter_overrides
from abogen.domain.file_type import auto_select_relevant_chapters
from abogen.domain.intro_outro import resolve_intro, resolve_outro
from abogen.domain.metadata_extraction import extract_metadata_for_file
from abogen.domain.metadata_merge import merge_metadata
from abogen.domain.voice_markers import split_text_by_voice_markers
def build_conversion_plan(request: ConversionRequest) -> ConversionPlan:
"""Build a complete conversion plan from a request.
This is the single entry point that both UIs will call.
It handles all the planning logic that was previously duplicated
in both PyQt and WebUI conversion runners.
Args:
request: Normalized conversion request
Returns:
ConversionPlan with all chapters, segments, and output layout
Raises:
ValueError: If request is invalid (no source, no chapters, etc.)
"""
# 1. Extract and validate source
source_text = _extract_source_text(request)
if not source_text or not source_text.strip():
raise ValueError("No text content to convert")
# 2. Extract metadata
metadata, extraction = _extract_metadata(request)
# 3. Parse chapters
raw_chapters = _parse_chapters(source_text, request)
# 4. Apply chapter selection/overrides
selected_chapters = _apply_selection(raw_chapters, request)
# 5. Build segments for each chapter
chapters = _build_chapters(selected_chapters, request)
# 6. Build intro/outro
intro, outro = _build_intro_outro(metadata, request)
# 7. Resolve output layout
output_layout = resolve_output_layout(request)
logging.info(
"[planner] Plan built: chapters=%d intro=%s outro=%s",
len(chapters),
bool(intro and intro.enabled),
bool(outro and outro.enabled),
)
return ConversionPlan(
request=request,
metadata=metadata,
chapters=chapters,
intro=intro,
outro=outro,
output_layout=output_layout,
extraction=extraction,
)
def _extract_source_text(request: ConversionRequest) -> Optional[str]:
"""Extract text from request source."""
from abogen.subtitle_utils import clean_text
if request.direct_text:
text = clean_text(request.direct_text)
elif request.source_path and request.source_path.exists():
encoding = "utf-8"
try:
with open(request.source_path, "r", encoding=encoding, errors="replace") as f:
text = f.read()
except Exception:
return None
text = clean_text(text)
else:
return None
# Apply word substitutions if configured
if request.word_substitution:
from abogen.word_substitution import apply_word_substitutions
ws = request.word_substitution
text = apply_word_substitutions(
text,
ws.substitutions_list,
ws.case_sensitive,
ws.replace_caps,
ws.replace_numerals,
ws.fix_punctuation,
)
return text
def _extract_metadata(
request: ConversionRequest,
) -> Tuple[Dict[str, Any], Optional[Any]]:
"""Extract metadata from source file.
Returns (metadata, extraction) tuple.
"""
if request.direct_text:
return dict(request.metadata_tags), None
if request.source_path and request.source_path.exists():
try:
extraction = extract_metadata_for_file(
str(request.source_path), is_direct_text=False
)
metadata = dict(extraction.metadata) if extraction.metadata else {}
except Exception:
extraction = None
metadata = {}
metadata = merge_metadata(metadata, request.metadata_tags)
return metadata, extraction
return dict(request.metadata_tags), None
def _parse_chapters(
source_text: str, request: ConversionRequest
) -> List[Tuple[str, str, str]]:
"""Parse source text into raw chapters.
Returns list of (title, body_text, default_voice) tuples.
"""
from abogen.domain.text_chapters import parse_chapters_from_text
# Text is already cleaned in _extract_source_text, so clean=False here
chapters = parse_chapters_from_text(source_text, default_title="text", clean=False)
# Default voice from request
default_voice = request.voice or "M1"
return [(title, text, default_voice) for title, text in chapters]
def _apply_selection(
raw_chapters: List[Tuple[str, str, str]], request: ConversionRequest
) -> List[Tuple[str, str, str]]:
"""Apply chapter selection and overrides."""
from abogen.text_extractor import ExtractedChapter
# Convert to ExtractedChapter objects for auto_select_relevant_chapters
extracted = [
ExtractedChapter(title=title, text=text)
for title, text, _ in raw_chapters
]
# If user specified chapters, apply overrides
chapter_chunk = request.chapter_chunk
if chapter_chunk and chapter_chunk.chapter_overrides:
selected, _, diagnostics = apply_chapter_overrides(extracted, chapter_chunk.chapter_overrides)
if selected:
# Map back to (title, text, voice) tuples
result = []
for ch in selected:
# Find matching original chapter to get voice
voice = request.voice or "M1"
for orig_title, orig_text, orig_voice in raw_chapters:
if orig_title == ch.title:
voice = orig_voice
break
result.append((ch.title, ch.text or "", voice))
return result
# If no chapters selected, fall through to auto-selection
# Auto-select relevant chapters
from abogen.domain.file_type import infer_file_type
file_type = infer_file_type(request.source_path) if request.source_path else "text"
result = auto_select_relevant_chapters(extracted, file_type)
filtered = result.kept
if filtered:
# Map back to (title, text, voice) tuples
result = []
for ch in filtered:
voice = request.voice or "M1"
for orig_title, orig_text, orig_voice in raw_chapters:
if orig_title == ch.title:
voice = orig_voice
break
result.append((ch.title, ch.text or "", voice))
return result
# Fall back to all chapters
return raw_chapters
def _build_chapters(
selected_chapters: List[Tuple[str, str, str]], request: ConversionRequest
) -> List[ChapterPlan]:
"""Build ChapterPlan with SegmentPlan for each chapter."""
from abogen.domain.chapter_titles import normalize_chapter_opening_caps
chapters = []
for idx, (title, body_text, default_voice) in enumerate(selected_chapters, 1):
# Apply caps normalization to body text if enabled
if request.normalize_chapter_opening_caps and body_text:
body_text, _ = normalize_chapter_opening_caps(body_text)
# Build segments for this chapter (idx is 1-based, chunks use 0-based)
segments = _build_segments(body_text, default_voice, request, chapter_index=idx - 1)
chapter = ChapterPlan(
index=idx,
title=title,
original_title=title,
body_text=body_text,
segments=segments,
voice_spec=default_voice,
)
chapters.append(chapter)
return chapters
def _build_segments(
body_text: str, default_voice: str, request: ConversionRequest,
chapter_index: int = 0,
) -> List[SegmentPlan]:
"""Build SegmentPlan list for a chapter's body text.
Handles voice markers (PyQt) and chunks (WebUI).
"""
segments = []
# Check for chunks (WebUI style)
chapter_chunk = request.chapter_chunk
if chapter_chunk and chapter_chunk.chunks:
# Group chunks by chapter index
from abogen.domain.chunk_utils import group_chunks_by_chapter
chunk_groups = group_chunks_by_chapter(chapter_chunk.chunks)
chunks_for_chapter = chunk_groups.get(chapter_index, [])
for chunk_idx, chunk in enumerate(chunks_for_chapter):
chunk_text = chunk.get("normalized_text") or chunk.get("text", "")
if not chunk_text or not chunk_text.strip():
continue
chunk_voice = _resolve_chunk_voice(chunk, default_voice, request)
speaker_id = chunk.get("speaker_id", "narrator")
segments.append(
SegmentPlan(
text=chunk_text.strip(),
voice_spec=chunk_voice,
kind="body",
speaker_id=speaker_id,
chunk_id=chunk.get("id"),
chunk_index=chunk.get("chunk_index", chunk_idx),
level=chunk.get("level", chapter_chunk.chunk_level),
source="chunk",
)
)
return segments
# Check for voice markers (PyQt style)
# Detect markers even if validation fails (voice names may not be loaded yet)
from abogen.domain.voice_markers import _VOICE_MARKER_SEARCH_PATTERN
has_voice_markers = bool(_VOICE_MARKER_SEARCH_PATTERN.search(body_text))
voice_segments, last_voice, valid_count, invalid_count = split_text_by_voice_markers(
body_text, default_voice
)
if has_voice_markers or (len(voice_segments) > 1):
# Voice markers were used
for voice_name, segment_text in voice_segments:
if not segment_text or not segment_text.strip():
continue
segments.append(
SegmentPlan(
text=segment_text.strip(),
voice_spec=voice_name,
kind="body",
source="voice_marker",
)
)
return segments
# No voice markers — single segment for entire body
if body_text and body_text.strip():
segments.append(
SegmentPlan(
text=body_text.strip(),
voice_spec=default_voice,
kind="body",
source="chapter",
)
)
return segments
def _resolve_chunk_voice(
chunk: Dict[str, Any], default_voice: str, request: ConversionRequest
) -> str:
"""Resolve voice for a chunk."""
# Check for speaker-based voice
speaker_id = chunk.get("speaker_id", "narrator")
speakers = request.chapter_chunk.speakers if request.chapter_chunk else {}
if speaker_id and speaker_id != "narrator" and speakers:
speaker_config = speakers.get(speaker_id, {})
if isinstance(speaker_config, dict):
voice = speaker_config.get("voice")
if voice:
return voice
# Check for direct voice field
voice = chunk.get("voice")
if voice:
return voice
return default_voice
def _build_intro_outro(
metadata: Dict[str, Any], request: ConversionRequest
) -> Tuple[Optional[IntroOutroSpec], Optional[IntroOutroSpec]]:
"""Build intro and outro specs."""
intro_spec = None
outro_spec = None
# Intro
if request.read_title_intro:
resolved = resolve_intro(
metadata,
request.original_filename,
True,
request.voice or "M1",
request.voice or "M1",
[],
)
if resolved.enabled:
intro_spec = IntroOutroSpec(
enabled=True,
text=resolved.text,
voice_spec=resolved.voice_spec,
kind="intro",
)
# Outro
if request.read_closing_outro:
resolved = resolve_outro(
metadata,
request.original_filename,
True,
request.voice or "M1",
request.voice or "M1",
[],
)
if resolved.enabled:
outro_spec = IntroOutroSpec(
enabled=True,
text=resolved.text,
voice_spec=resolved.voice_spec,
kind="outro",
)
return intro_spec, outro_spec
# Output layout resolution is now in application/output_layout_service.py
-112
View File
@@ -1,112 +0,0 @@
"""Ports / interfaces for the conversion service.
These protocols define how the conversion service communicates with
the outside world (UI, TTS backends, voice resolvers).
The service ONLY depends on these interfaces, never on concrete
implementations (PyQt signals, Flask Job, etc.).
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Protocol
class ConversionCancelled(Exception):
"""Raised when conversion is cancelled by user."""
pass
class ConversionEvents(Protocol):
"""UI-specific actions the conversion service delegates back to the caller.
Implementations:
- PyQt: emits signals (log_updated, progress_updated, etc.)
- WebUI: updates Job attributes (job.add_log, job.progress, etc.)
"""
def log(self, message: str, level: str = "info") -> None:
"""Log a message to the UI."""
...
def progress(self, processed: int, total: int, etr: str) -> None:
"""Update progress display."""
...
def check_cancelled(self) -> None:
"""Check if conversion was cancelled.
Should raise ConversionCancelled (or UI-specific exception)
if cancellation is requested. Normal return means "continue".
"""
...
class PipelineProvider(Protocol):
"""Provides access to TTS backends (Kokoro, SuperTonic, etc.).
Implementations:
- PyQt: wraps self.backend (single pipeline)
- WebUI: wraps PipelinePool (multi-provider)
"""
def get(self, provider: str, language: str, use_gpu: bool) -> Any:
"""Get a TTS backend instance."""
...
def dispose_all(self) -> None:
"""Dispose all backend resources."""
...
@dataclass
class ResolvedVoice:
"""A resolved voice ready for TTS synthesis."""
provider: str
resolved_spec: str
voice: Any # loaded voice tensor or name
speed: float
supertonic_steps: int
class VoiceResolver(Protocol):
"""Resolves voice specs into loaded voice objects.
Implementations:
- PyQt: wraps load_voice_cached + VoiceCache
- WebUI: wraps resolve_voice_choice + PipelinePool + VoiceCache
"""
def resolve(self, voice_spec: str) -> ResolvedVoice:
"""Resolve a voice spec into a loaded voice."""
...
class SubtitleWriter(Protocol):
"""Writes subtitle entries to a file."""
def open(self) -> None:
"""Open the subtitle file for writing."""
...
def write_entry(self, start: float, end: float, text: str) -> None:
"""Write a single subtitle entry."""
...
def close(self) -> None:
"""Close the subtitle file."""
...
class AudioSink(Protocol):
"""Writes audio data to a file."""
def write(self, audio: Any) -> None:
"""Write audio samples to the sink."""
...
def close(self) -> None:
"""Close the audio file."""
...
-155
View File
@@ -1,155 +0,0 @@
"""ConversionRequest — normalized input for a conversion job.
This is NOT a WebUI Job and NOT a PyQt ConversionThread state.
It describes the TASK, not the UI.
UI adapters are responsible for converting their respective state
into a ConversionRequest before calling ConversionService.run().
"""
from __future__ import annotations
import dataclasses
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional
from abogen.application.conversion_config import (
ChapterChunkConfig,
CoverConfig,
Epub3ExportConfig,
PronunciationConfig,
SaveConfig,
SubtitleConfig,
SubtitleInputConfig,
WordSubstitutionConfig,
)
from abogen.domain.enums import Language, OutputFormat
class ConversionRequestError(ValueError):
"""Raised when ConversionRequest has invalid field values."""
# Numeric field constraints: attr -> (min, max)
_NUMERIC_CONSTRAINTS: dict[str, tuple[float, float | None]] = {
"speed": (0.5, 3.0),
"supertonic_total_steps": (2, 15),
"silence_between_chapters": (0.0, None),
"chapter_intro_delay": (0.0, None),
}
@dataclass
class ConversionRequest:
"""Normalized request for a conversion job.
Only contains fields that describe the conversion task itself.
UI-only fields (display, logging, user prompts) stay in adapters.
Feature toggles use config objects (None = disabled):
- word_substitution, subtitle_input, chapter_chunk, epub3_export
- pronunciation (raw data, compiled by app layer)
- subtitle, save, cover (grouped parameters)
Validation runs on creation via __post_init__:
- None values → replaced with field default (from declaration)
- Numeric fields → clamped to valid range
"""
# --- Source ---
source_path: Optional[Path] = None
direct_text: Optional[str] = None
original_filename: str = ""
# --- TTS Settings ---
language: Language = Language.EN_US
tts_provider: str = "kokoro"
voice: str = "M1"
voice_profile: Optional[str] = None
speed: float = 1.0
use_gpu: bool = True
supertonic_total_steps: int = 5
# --- Output Format ---
output_format: OutputFormat = OutputFormat.WAV
# --- Timing ---
silence_between_chapters: float = 2.0
chapter_intro_delay: float = 0.0
# --- Content Processing ---
replace_single_newlines: bool = False
read_title_intro: bool = False
read_closing_outro: bool = True
auto_prefix_chapter_titles: bool = True
normalize_chapter_opening_caps: bool = False
# --- Metadata ---
metadata_tags: Dict[str, Any] = field(default_factory=dict)
# --- Grouped configs ---
subtitle: SubtitleConfig = field(default_factory=SubtitleConfig)
save: SaveConfig = field(default_factory=SaveConfig)
cover: CoverConfig = field(default_factory=CoverConfig)
pronunciation: PronunciationConfig = field(default_factory=PronunciationConfig)
# --- Feature configs (None = disabled) ---
epub3_export: Optional[Epub3ExportConfig] = None
word_substitution: Optional[WordSubstitutionConfig] = None
subtitle_input: Optional[SubtitleInputConfig] = None
chapter_chunk: Optional[ChapterChunkConfig] = None
def __post_init__(self) -> None:
"""Resolve None → default, then validate and clamp."""
_apply_none_defaults(self)
if not self.tts_provider:
self.tts_provider = "kokoro"
_coerce_enums(self)
_clamp_numerics(self)
def _apply_none_defaults(obj: ConversionRequest) -> None:
"""Replace None values with field defaults from dataclass declaration."""
for f in dataclasses.fields(obj):
if getattr(obj, f.name) is not None:
continue
if f.default is not dataclasses.MISSING:
setattr(obj, f.name, f.default)
elif f.default_factory is not dataclasses.MISSING:
setattr(obj, f.name, f.default_factory())
# Enum fields that accept string coercion: attr -> (enum_class, fallback)
_ENUM_COERCIONS: dict[str, tuple[type, Any]] = {
"language": (Language, Language.EN_US),
"output_format": (OutputFormat, OutputFormat.WAV),
}
def _coerce_enums(obj: ConversionRequest) -> None:
"""Coerce string values to their expected enum types."""
for attr, (enum_cls, fallback) in _ENUM_COERCIONS.items():
val = getattr(obj, attr)
if isinstance(val, enum_cls):
continue
try:
setattr(obj, attr, enum_cls.from_str(str(val)))
except (ValueError, AttributeError):
setattr(obj, attr, fallback)
def _clamp_numerics(obj: ConversionRequest) -> None:
"""Clamp numeric fields to valid ranges."""
for attr, (min_v, max_v) in _NUMERIC_CONSTRAINTS.items():
val = getattr(obj, attr)
if val is None:
continue
if not isinstance(val, (int, float)):
raise ConversionRequestError(
f"{attr} must be a number, got {type(val).__name__}"
)
clamped = max(min_v, float(val))
if max_v is not None:
clamped = min(max_v, clamped)
setattr(obj, attr, clamped)
-50
View File
@@ -1,50 +0,0 @@
"""ConversionResult — output of a successful conversion.
Returned by ConversionService.run() after all synthesis and finalization.
UI adapters consume this to update their respective state (Job, signals, etc.).
"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional
@dataclass
class ConversionResult:
"""Output of a successful conversion job."""
# --- Primary outputs ---
audio_path: Optional[Path] = None
subtitle_paths: List[Path] = field(default_factory=list)
chapter_paths: List[Path] = field(default_factory=list)
# --- Markers (for metadata/audiobookshelf) ---
chapter_markers: List[Dict[str, Any]] = field(default_factory=list)
chunk_markers: List[Dict[str, Any]] = field(default_factory=list)
# --- Metadata ---
metadata: Dict[str, Any] = field(default_factory=dict)
# --- Artifacts ---
artifacts: Dict[str, Path] = field(default_factory=dict)
project_root: Optional[Path] = None
epub_path: Optional[Path] = None
# --- Stats ---
total_chapters: int = 0
total_segments: int = 0
total_characters: int = 0
# --- Override usage tracking ---
usage_counter: Dict[str, int] = field(default_factory=dict)
@dataclass
class ConversionError:
"""Error information when conversion fails."""
message: str
details: Optional[str] = None
is_cancelled: bool = False
-250
View File
@@ -1,250 +0,0 @@
"""ConversionService — main orchestrator for the conversion flow.
Ties together planner, executor, and finalizers into a single entry point.
Both UIs (PyQt, WebUI) call ConversionService.run() to execute a conversion.
Responsibilities:
- Prepare TTSContext (normalization settings, pronunciation rules)
- Build ConversionPlan via planner
- Execute conversion via executor
- Handle lifecycle (cleanup, error handling)
- Return ConversionResult
The service NEVER imports from PyQt or WebUI.
"""
from __future__ import annotations
import logging
from collections import defaultdict
from typing import Any, Dict
from abogen.application.conversion_executor import execute_conversion
from abogen.application.conversion_models import ConversionPlan
from abogen.application.conversion_planner import build_conversion_plan
from abogen.application.conversion_ports import ConversionEvents
from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_result import ConversionResult
from abogen.domain.normalization import build_tts_context
def run_conversion(
request: ConversionRequest,
events: ConversionEvents,
) -> ConversionResult:
"""Execute a conversion request and return the result.
This is the single entry point for both UIs. It orchestrates:
1. Voice infrastructure setup (pool, cache, resolver)
2. TTS context preparation
3. Conversion planning
4. Conversion execution
5. Resource cleanup
Args:
request: Normalized conversion request
events: UI-specific callbacks (log, progress, check_cancelled)
Returns:
ConversionResult with paths and markers
Raises:
ConversionCancelled: If conversion was cancelled
ValueError: If request is invalid
Exception: On TTS or I/O errors
"""
from abogen.domain.pipeline_factory import PipelinePool
from abogen.domain.voice_loader import VoiceCache
pool = PipelinePool()
voice_cache = VoiceCache()
try:
# Stage 0: Create voice resolver
events.log("Preparing conversion pipeline")
logging.info(
"[app] run_conversion: provider=%s language=%s voice=%s speed=%.2f",
request.tts_provider, request.language, request.voice, request.speed,
)
resolver = _create_voice_resolver(request, pool, voice_cache)
# Stage 1: Prepare TTS context
usage_counter: Dict[str, int] = defaultdict(int)
tts_context = build_tts_context(
language=request.language,
subtitle=request.subtitle,
pronunciation=request.pronunciation,
usage_counter=usage_counter,
log_callback=lambda level, msg: events.log(msg, level=level),
)
# Stage 2: Build conversion plan
events.log("Building conversion plan")
plan = build_conversion_plan(request)
# Stage 3: Execute conversion
events.log("Starting conversion")
result = execute_conversion(
plan=plan,
events=events,
pipeline_provider=pool,
voice_resolver=resolver,
tts_context=tts_context,
)
# Propagate usage counter to result
result.usage_counter = dict(usage_counter)
# Stage 4: Finalize (m4b metadata embedding, EPUB3 generation)
_finalize(request, result, plan, events)
events.log("Conversion complete")
logging.info("[app] run_conversion completed successfully")
return result
except Exception as e:
events.log(f"Conversion failed: {e}", level="error")
logging.exception("[app] run_conversion failed: %s", e)
raise
finally:
pool.dispose_all()
voice_cache.clear()
from abogen.application.cleanup import flush_cuda
flush_cuda()
def _create_voice_resolver(
request: ConversionRequest,
pool: Any,
cache: Any,
) -> Any:
"""Create AppVoiceResolver with loaded profiles.
Loads voice profiles from disk, normalizes them, and creates
an AppVoiceResolver that can resolve voice specs into loaded voices.
"""
from abogen.application.voice_resolver import AppVoiceResolver
from abogen.voice_profiles import load_profiles, normalize_profile_entry
try:
profiles = load_profiles()
except Exception:
profiles = {}
normalized_profiles: Dict[str, Dict[str, Any]] = {}
for name, entry in (profiles or {}).items():
normalized = normalize_profile_entry(entry)
if normalized:
normalized_profiles[str(name)] = normalized
return AppVoiceResolver(request, normalized_profiles, pool, cache)
def _finalize(
request: ConversionRequest,
result: ConversionResult,
plan: ConversionPlan,
events: ConversionEvents,
) -> None:
"""Post-conversion finalization (m4b metadata embedding, EPUB3 generation, etc.)."""
from abogen.domain.enums import OutputFormat
# m4b metadata embedding
if (
result.audio_path
and request.output_format == OutputFormat.M4B
):
from abogen.infrastructure.exporters import ExportService
export_svc = ExportService()
try:
export_svc.embed_m4b_metadata(
audio_path=result.audio_path,
metadata=result.metadata or {},
chapters=result.chapter_markers or [],
cover=request.cover,
log_callback=lambda msg, level="info": events.log(msg, level=level),
)
except Exception as exc:
events.log(f"Failed to embed m4b metadata: {exc}", level="error")
raise RuntimeError(f"Failed to embed m4b metadata: {exc}") from exc
# EPUB3 generation
if request.epub3_export and plan.extraction:
audio_asset = result.audio_path
if not audio_asset and result.chapter_paths:
audio_asset = result.chapter_paths[0]
if audio_asset:
try:
from abogen.epub3.exporter import build_epub3_package
epub_root = result.project_root or plan.output_layout.parent_dir
from abogen.domain.output_paths import build_output_path
epub_output_path = build_output_path(epub_root, request.original_filename, "epub")
events.log("Generating EPUB 3 package...")
epub_path = build_epub3_package(
output_path=epub_output_path,
book_id=request.epub3_export.book_id,
extraction=plan.extraction,
metadata_tags=result.metadata or {},
chapter_markers=result.chapter_markers or [],
chunk_markers=result.chunk_markers or [],
chunks=request.chapter_chunk.chunks if request.chapter_chunk else [],
audio_path=audio_asset,
speaker_mode=request.chapter_chunk.speaker_mode if request.chapter_chunk else "single",
cover=request.cover,
)
result.epub_path = epub_path
result.artifacts["epub3"] = epub_path
events.log(f"EPUB 3 package created at {epub_path}")
except Exception as exc:
events.log(f"Failed to generate EPUB 3: {exc}", level="error")
else:
events.log("Skipped EPUB 3 generation: audio output unavailable.", level="warning")
# Build metadata payload and write metadata.json
if plan.output_layout and plan.output_layout.metadata_dir:
from abogen.domain.metadata_helpers import build_metadata_payload
metadata_payload = build_metadata_payload(
metadata=result.metadata,
chapter_markers=result.chapter_markers,
chunk_markers=result.chunk_markers,
chunk_level=request.chapter_chunk.chunk_level if request.chapter_chunk else None,
speaker_mode=request.chapter_chunk.speaker_mode if request.chapter_chunk else None,
speakers=request.chapter_chunk.speakers if request.chapter_chunk else None,
generate_epub3=bool(request.epub3_export),
)
metadata_dir = plan.output_layout.metadata_dir
metadata_dir.mkdir(parents=True, exist_ok=True)
metadata_file = metadata_dir / "metadata.json"
import json
metadata_file.write_text(json.dumps(metadata_payload, indent=2), encoding="utf-8")
result.artifacts["metadata"] = metadata_file
events.log(f"Metadata written to {metadata_file}")
# Record override usage
if result.usage_counter:
try:
from abogen.normalization_settings import record_override_usage
record_override_usage(result.usage_counter)
except Exception as exc:
events.log(f"Failed to record override usage: {exc}", level="debug")
# Post-conversion hooks (Audiobookshelf, etc.)
from abogen.application.integration_hooks import PostConversionHooks
hooks = PostConversionHooks()
hooks.run(request, result, events)
-164
View File
@@ -1,164 +0,0 @@
"""Post-conversion integration hooks.
Called by ConversionService after finalization.
Each integration is a method on PostConversionHooks — isolated, testable,
and easy to extend with new hooks (Plex, Navidrome, etc.).
The service NEVER imports from PyQt or WebUI.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any, Mapping, Optional
from abogen.application.conversion_ports import ConversionEvents
from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_result import ConversionResult
from abogen.domain.metadata_helpers import (
build_audiobookshelf_metadata as _build_abs_metadata,
load_audiobookshelf_chapters as _load_abs_chapters,
)
from abogen.domain.settings_core import (
build_audiobookshelf_config,
coerce_bool,
load_audiobookshelf_config,
stored_integration_config,
)
from abogen.integrations.audiobookshelf import (
AudiobookshelfClient,
AudiobookshelfUploadError,
)
logger = logging.getLogger(__name__)
class PostConversionHooks:
"""Runs post-conversion integrations (Audiobookshelf, etc.).
Usage::
hooks = PostConversionHooks()
hooks.run(request, result, events)
"""
def run(
self,
request: ConversionRequest,
result: ConversionResult,
events: ConversionEvents,
) -> None:
"""Run all registered post-conversion hooks."""
self._maybe_send_to_audiobookshelf(request, result, events)
# ------------------------------------------------------------------
# Audiobookshelf
# ------------------------------------------------------------------
def _maybe_send_to_audiobookshelf(
self,
request: ConversionRequest,
result: ConversionResult,
events: ConversionEvents,
) -> None:
"""Upload finished audiobook to Audiobookshelf if enabled."""
abs_settings = stored_integration_config("audiobookshelf")
if not abs_settings:
return
enabled = coerce_bool(abs_settings.get("enabled"), False)
auto_send = coerce_bool(abs_settings.get("auto_send"), False)
if not (enabled and auto_send):
return
config = build_audiobookshelf_config(abs_settings)
if config is None:
events.log(
"Audiobookshelf upload skipped: configure base URL, API token, "
"library ID, and folder ID first.",
level="warning",
)
return
audio_path = result.audio_path
if not audio_path or not audio_path.exists():
events.log(
"Audiobookshelf upload skipped: audio output not found.",
level="warning",
)
return
# Build metadata
filename = request.original_filename or "Audiobook"
lang = request.language.value if hasattr(request.language, "value") else str(request.language)
metadata = _build_abs_metadata(
result.metadata or {},
language=lang,
filename=Path(filename).stem,
)
# Load chapters from metadata artifact
chapters = None
if config.send_chapters:
metadata_artifact = result.artifacts.get("metadata")
if metadata_artifact:
metadata_path = (
metadata_artifact
if isinstance(metadata_artifact, Path)
else Path(str(metadata_artifact))
)
chapters = _load_abs_chapters(metadata_path)
# Resolve cover
cover_path = None
if config.send_cover and request.cover and request.cover.path:
candidate = request.cover.path
if isinstance(candidate, Path) and candidate.exists():
cover_path = candidate
# Resolve subtitles
subtitles = None
if config.send_subtitles and result.subtitle_paths:
subtitles = [
p for p in result.subtitle_paths
if isinstance(p, Path) and p.exists()
]
# Upload
client = AudiobookshelfClient(config)
display_title = metadata.get("title") or audio_path.stem
try:
existing_items = client.find_existing_items(
display_title, folder_id=config.folder_id,
)
except AudiobookshelfUploadError as exc:
events.log(f"Audiobookshelf lookup failed: {exc}", level="error")
return
if existing_items:
events.log(
f"Removing existing Audiobookshelf item(s) for '{display_title}'.",
level="info",
)
try:
client.delete_items(existing_items)
except Exception as exc:
events.log(
f"Failed to remove existing item(s): {exc}", level="warning",
)
try:
client.upload_audiobook(
audio_path,
metadata=metadata,
cover_path=cover_path,
chapters=chapters,
subtitles=subtitles,
)
events.log("Audiobookshelf upload queued.", level="info")
except AudiobookshelfUploadError as exc:
events.log(f"Audiobookshelf upload failed: {exc}", level="error")
except Exception as exc:
events.log(f"Audiobookshelf integration error: {exc}", level="error")
-149
View File
@@ -1,149 +0,0 @@
"""Output layout resolution service.
Determines where conversion outputs (audio, subtitles, metadata) should be written.
Extracted from conversion_planner.py as a standalone service per plan Stage 5.
Responsibilities:
- Resolve base output directory from save_mode and source_path
- Determine base filename from original_filename
- Find unique output path to avoid overwrites
- Resolve project layout (audio_dir, subtitle_dir, metadata_dir)
- Force merged output for m4b format
- Return OutputLayout dataclass
"""
from __future__ import annotations
from pathlib import Path
from abogen.application.conversion_models import OutputLayout
from abogen.application.conversion_request import ConversionRequest
from abogen.domain.enums import OutputFormat, SaveMode, SubtitleFormat
from abogen.domain.output_paths import (
resolve_project_layout,
resolve_unique_path,
sanitize_output_stem,
)
def resolve_output_layout(request: ConversionRequest) -> OutputLayout:
"""Resolve output paths for a conversion request.
This is the single entry point for output path resolution,
used by both UIs and the conversion service.
Args:
request: Normalized conversion request
Returns:
OutputLayout with resolved paths
"""
# Determine base output directory
if request.save.mode == SaveMode.CUSTOM_FOLDER and request.save.output_folder:
parent_dir = Path(request.save.output_folder)
elif request.source_path:
parent_dir = request.source_path.parent
else:
parent_dir = Path.cwd()
# Determine base name
if request.original_filename:
base_name = sanitize_output_stem(request.original_filename)
elif request.source_path:
base_name = sanitize_output_stem(request.source_path.stem)
else:
base_name = "output"
# Find unique output path
allowed_exts = {request.output_format, SubtitleFormat.SRT, SubtitleFormat.ASS, "vtt", "mp4", OutputFormat.M4B}
unique_base = resolve_unique_path(
parent_dir, base_name, "", allowed_extensions=allowed_exts
)
# Resolve project layout
project_root = None
audio_dir = parent_dir
subtitle_dir = None
metadata_dir = None
if request.save.save_as_project:
project_root, audio_dir, subtitle_dir, metadata_dir = resolve_project_layout(
original_filename=request.original_filename,
save_as_project=True,
base_dir=parent_dir,
)
return OutputLayout(
parent_dir=parent_dir,
project_root=project_root,
audio_dir=audio_dir,
subtitle_dir=subtitle_dir,
metadata_dir=metadata_dir,
)
def resolve_merged_path(
layout: OutputLayout,
request: ConversionRequest,
) -> Path:
"""Resolve the merged output audio file path.
Args:
layout: Resolved output layout
request: Conversion request
Returns:
Path to the merged output file
"""
base_name = sanitize_output_stem(
request.original_filename or "output"
)
return layout.audio_dir / f"{base_name}{request.output_format.dot_ext}"
def resolve_chapter_path(
layout: OutputLayout,
request: ConversionRequest,
chapter_title: str,
chapter_index: int,
) -> Path:
"""Resolve the output path for a separate chapter file.
Args:
layout: Resolved output layout
request: Conversion request
chapter_title: Chapter title for filename
chapter_index: Chapter number (1-based)
Returns:
Path to the chapter output file
"""
import re
slug = re.sub(r'[^\w\s-]', '', chapter_title.lower())
slug = re.sub(r'[\s_]+', '_', slug).strip('_')
if not slug:
slug = f"chapter_{chapter_index}"
filename = f"{chapter_index:02d}_{slug}.{request.save.separate_chapters_format}"
return layout.audio_dir / "chapters" / filename
def should_merge_output(request: ConversionRequest) -> bool:
"""Determine if merged output is required.
Rules:
- m4b format always forces merged output
- If save_chapters_separately is False, merged is required
- Otherwise, use merge_chapters_at_end setting
Args:
request: Conversion request
Returns:
True if merged output should be created
"""
if request.output_format == OutputFormat.M4B:
return True
if not request.save.save_chapters_separately:
return True
return request.save.merge_chapters_at_end
-81
View File
@@ -1,81 +0,0 @@
"""AppVoiceResolver — voice resolution inside the application layer.
Resolves voice specs into loaded voices using profiles, pipeline pool,
and voice cache. Replaces UI-specific resolvers (WebUIVoiceResolver,
PyQtVoiceResolver) with a single app-layer implementation.
"""
from __future__ import annotations
import logging
from typing import Any, Dict, Optional
from abogen.application.conversion_ports import ResolvedVoice, VoiceResolver
from abogen.application.conversion_request import ConversionRequest
from abogen.domain.pipeline_factory import PipelinePool
from abogen.domain.voice_loader import VoiceCache, resolve_voice
from abogen.domain.voice_utils import resolve_voice_target
class AppVoiceResolver:
"""App-layer implementation of VoiceResolver protocol.
Uses ConversionRequest instead of Job. Loads profiles, creates
resolver internally — UIs don't need to manage this.
"""
def __init__(
self,
request: ConversionRequest,
normalized_profiles: Dict[str, Dict[str, Any]],
pool: PipelinePool,
cache: VoiceCache,
):
self._request = request
self._profiles = normalized_profiles
self._cache = cache
self._pool = pool
def resolve(self, voice_spec: str) -> ResolvedVoice:
"""Resolve a voice spec into a loaded voice."""
provider, resolved, speed, steps = resolve_voice_target(
voice_spec,
self._profiles,
job_voice=self._request.voice,
job_tts_provider=self._request.tts_provider,
job_supertonic_total_steps=self._request.supertonic_total_steps,
job_speed=self._request.speed,
)
cache_key = f"{provider}:{resolved}" if resolved else provider
cached = self._cache.get(cache_key)
if cached is not None:
logging.info("[resolver] Cache hit: spec=%s -> provider=%s resolved=%s", voice_spec, provider, resolved)
return ResolvedVoice(
provider=provider,
resolved_spec=resolved,
voice=cached,
speed=speed,
supertonic_steps=steps or 0,
)
if provider == "kokoro":
kokoro_backend = self._pool.get(
"kokoro", self._request.language, self._request.use_gpu,
)
loaded = resolve_voice(
resolved, kokoro_backend, self._request.use_gpu, cache=self._cache,
)
else:
loaded = resolved
self._cache.set(cache_key, loaded)
logging.info("[resolver] Resolved: spec=%s -> provider=%s resolved=%s speed=%.2f steps=%s",
voice_spec, provider, resolved, speed, steps)
return ResolvedVoice(
provider=provider,
resolved_spec=resolved,
voice=loaded,
speed=speed,
supertonic_steps=steps or 0,
)
+30 -30
View File
@@ -1,31 +1,31 @@
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
<svg height="800px" width="800px" version="1.1" id="_x32_" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"
viewBox="0 0 512 512" xml:space="preserve">
<style type="text/css">
.st0{fill:#808080;}
</style>
<g>
<path class="st0" d="M502.325,307.303l-39.006-30.805c-6.215-4.908-9.665-12.429-9.668-20.348c0-0.084,0-0.168,0-0.252
c-0.014-7.936,3.44-15.478,9.667-20.396l39.007-30.806c8.933-7.055,12.093-19.185,7.737-29.701l-17.134-41.366
c-4.356-10.516-15.167-16.86-26.472-15.532l-49.366,5.8c-7.881,0.926-15.656-1.966-21.258-7.586
c-0.059-0.06-0.118-0.119-0.177-0.178c-5.597-5.602-8.476-13.36-7.552-21.225l5.799-49.363
c1.328-11.305-5.015-22.116-15.531-26.472L337.004,1.939c-10.516-4.356-22.646-1.196-29.701,7.736l-30.805,39.005
c-4.908,6.215-12.43,9.665-20.349,9.668c-0.084,0-0.168,0-0.252,0c-7.935,0.014-15.477-3.44-20.395-9.667L204.697,9.675
c-7.055-8.933-19.185-12.092-29.702-7.736L133.63,19.072c-10.516,4.356-16.86,15.167-15.532,26.473l5.799,49.366
c0.926,7.881-1.964,15.656-7.585,21.257c-0.059,0.059-0.118,0.118-0.178,0.178c-5.602,5.598-13.36,8.477-21.226,7.552
l-49.363-5.799c-11.305-1.328-22.116,5.015-26.472,15.531L1.939,174.996c-4.356,10.516-1.196,22.646,7.736,29.701l39.006,30.805
c6.215,4.908,9.665,12.429,9.668,20.348c0,0.084,0,0.167,0,0.251c0.014,7.935-3.44,15.477-9.667,20.395L9.675,307.303
c-8.933,7.055-12.092,19.185-7.736,29.701l17.134,41.365c4.356,10.516,15.168,16.86,26.472,15.532l49.366-5.799
c7.882-0.926,15.656,1.965,21.258,7.586c0.059,0.059,0.118,0.119,0.178,0.178c5.597,5.603,8.476,13.36,7.552,21.226l-5.799,49.364
c-1.328,11.305,5.015,22.116,15.532,26.472l41.366,17.134c10.516,4.356,22.646,1.196,29.701-7.736l30.804-39.005
c4.908-6.215,12.43-9.665,20.348-9.669c0.084,0,0.168,0,0.251,0c7.936-0.014,15.478,3.44,20.396,9.667l30.806,39.007
c7.055,8.933,19.185,12.093,29.701,7.736l41.366-17.134c10.516-4.356,16.86-15.168,15.532-26.472l-5.8-49.366
c-0.926-7.881,1.965-15.656,7.586-21.257c0.059-0.059,0.119-0.119,0.178-0.178c5.602-5.597,13.36-8.476,21.225-7.552l49.364,5.799
c11.305,1.328,22.117-5.015,26.472-15.531l17.134-41.365C514.418,326.488,511.258,314.358,502.325,307.303z M281.292,329.698
c-39.68,16.436-85.172-2.407-101.607-42.087c-16.436-39.68,2.407-85.171,42.087-101.608c39.68-16.436,85.172,2.407,101.608,42.088
C339.815,267.771,320.972,313.262,281.292,329.698z"/>
</g>
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
<svg height="800px" width="800px" version="1.1" id="_x32_" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"
viewBox="0 0 512 512" xml:space="preserve">
<style type="text/css">
.st0{fill:#808080;}
</style>
<g>
<path class="st0" d="M502.325,307.303l-39.006-30.805c-6.215-4.908-9.665-12.429-9.668-20.348c0-0.084,0-0.168,0-0.252
c-0.014-7.936,3.44-15.478,9.667-20.396l39.007-30.806c8.933-7.055,12.093-19.185,7.737-29.701l-17.134-41.366
c-4.356-10.516-15.167-16.86-26.472-15.532l-49.366,5.8c-7.881,0.926-15.656-1.966-21.258-7.586
c-0.059-0.06-0.118-0.119-0.177-0.178c-5.597-5.602-8.476-13.36-7.552-21.225l5.799-49.363
c1.328-11.305-5.015-22.116-15.531-26.472L337.004,1.939c-10.516-4.356-22.646-1.196-29.701,7.736l-30.805,39.005
c-4.908,6.215-12.43,9.665-20.349,9.668c-0.084,0-0.168,0-0.252,0c-7.935,0.014-15.477-3.44-20.395-9.667L204.697,9.675
c-7.055-8.933-19.185-12.092-29.702-7.736L133.63,19.072c-10.516,4.356-16.86,15.167-15.532,26.473l5.799,49.366
c0.926,7.881-1.964,15.656-7.585,21.257c-0.059,0.059-0.118,0.118-0.178,0.178c-5.602,5.598-13.36,8.477-21.226,7.552
l-49.363-5.799c-11.305-1.328-22.116,5.015-26.472,15.531L1.939,174.996c-4.356,10.516-1.196,22.646,7.736,29.701l39.006,30.805
c6.215,4.908,9.665,12.429,9.668,20.348c0,0.084,0,0.167,0,0.251c0.014,7.935-3.44,15.477-9.667,20.395L9.675,307.303
c-8.933,7.055-12.092,19.185-7.736,29.701l17.134,41.365c4.356,10.516,15.168,16.86,26.472,15.532l49.366-5.799
c7.882-0.926,15.656,1.965,21.258,7.586c0.059,0.059,0.118,0.119,0.178,0.178c5.597,5.603,8.476,13.36,7.552,21.226l-5.799,49.364
c-1.328,11.305,5.015,22.116,15.532,26.472l41.366,17.134c10.516,4.356,22.646,1.196,29.701-7.736l30.804-39.005
c4.908-6.215,12.43-9.665,20.348-9.669c0.084,0,0.168,0,0.251,0c7.936-0.014,15.478,3.44,20.396,9.667l30.806,39.007
c7.055,8.933,19.185,12.093,29.701,7.736l41.366-17.134c10.516-4.356,16.86-15.168,15.532-26.472l-5.8-49.366
c-0.926-7.881,1.965-15.656,7.586-21.257c0.059-0.059,0.119-0.119,0.178-0.178c5.602-5.597,13.36-8.476,21.225-7.552l49.364,5.799
c11.305,1.328,22.117-5.015,26.472-15.531l17.134-41.365C514.418,326.488,511.258,314.358,502.325,307.303z M281.292,329.698
c-39.68,16.436-85.172-2.407-101.607-42.087c-16.436-39.68,2.407-85.171,42.087-101.608c39.68-16.436,85.172,2.407,101.608,42.088
C339.815,267.771,320.972,313.262,281.292,329.698z"/>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 2.5 KiB

After

Width:  |  Height:  |  Size: 2.6 KiB

+1 -2
View File
@@ -12,8 +12,7 @@ import fitz # PyMuPDF
import markdown
from abogen.utils import detect_encoding
from abogen.subtitle_utils import clean_text
from abogen.domain.text_utils import calculate_text_length
from abogen.subtitle_utils import clean_text, calculate_text_length
# Pre-compile frequently used regex patterns
_BRACKETED_NUMBERS_PATTERN = re.compile(r"\[\s*\d+\s*\]")
+75 -31
View File
@@ -1,5 +1,4 @@
from abogen.utils import get_version
from abogen.domain.enums import Language
# Program Information
PROGRAM_NAME = "abogen"
@@ -17,22 +16,8 @@ SUBTITLE_FORMATS = [
("ass_centered_narrow", "ASS (centered narrow)"),
]
# Language description mapping (Language enum → human-readable label).
# Language description mapping
LANGUAGE_DESCRIPTIONS = {
Language.EN_US: "American English",
Language.EN_GB: "British English",
Language.ES: "Spanish",
Language.FR: "French",
Language.HI: "Hindi",
Language.IT: "Italian",
Language.JA: "Japanese",
Language.PT_BR: "Brazilian Portuguese",
Language.ZH: "Mandarin Chinese",
}
# Display-only mapping for kokoro codes → labels.
# Used by voice catalog and PyQt (legacy) where kokoro codes are still present.
KOKORO_CODE_LABELS = {
"a": "American English",
"b": "British English",
"e": "Spanish",
@@ -70,24 +55,83 @@ SUPPORTED_INPUT_FORMATS = [
"vtt",
]
# Supported languages for subtitle generation.
# All languages are supported: only English emits per-word timestamped tokens
# in the Kokoro pipeline, but other languages fall back to segment-level fake
# tokens (see abogen.domain.tokens.FakeToken), so subtitles are still
# generated at segment granularity.
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION = list(Language)
# Supported languages for subtitle generation
# Currently, only 'a (American English)' and 'b (British English)' are supported for subtitle generation.
# This is because tokens that contain timestamps are not generated for other languages in the Kokoro pipeline.
# Please refer to: https://github.com/hexgrad/kokoro/blob/6d87f4ae7abc2d14dbc4b3ef2e5f19852e861ac2/kokoro/pipeline.py
# 383 English processing (unchanged)
# 384 if self.lang_code in 'ab':
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION = list(LANGUAGE_DESCRIPTIONS.keys())
# Voice and sample text constants
VOICES_INTERNAL = [
"af_alloy",
"af_aoede",
"af_bella",
"af_heart",
"af_jessica",
"af_kore",
"af_nicole",
"af_nova",
"af_river",
"af_sarah",
"af_sky",
"am_adam",
"am_echo",
"am_eric",
"am_fenrir",
"am_liam",
"am_michael",
"am_onyx",
"am_puck",
"am_santa",
"bf_alice",
"bf_emma",
"bf_isabella",
"bf_lily",
"bm_daniel",
"bm_fable",
"bm_george",
"bm_lewis",
"ef_dora",
"em_alex",
"em_santa",
"ff_siwis",
"hf_alpha",
"hf_beta",
"hm_omega",
"hm_psi",
"if_sara",
"im_nicola",
"jf_alpha",
"jf_gongitsune",
"jf_nezumi",
"jf_tebukuro",
"jm_kumo",
"pf_dora",
"pm_alex",
"pm_santa",
"zf_xiaobei",
"zf_xiaoni",
"zf_xiaoxiao",
"zf_xiaoyi",
"zm_yunjian",
"zm_yunxi",
"zm_yunxia",
"zm_yunyang",
]
# Voice and sample text mapping
SAMPLE_VOICE_TEXTS = {
Language.EN_US: "This is a sample of the selected voice.",
Language.EN_GB: "This is a sample of the selected voice.",
Language.ES: "Este es una muestra de la voz seleccionada.",
Language.FR: "Ceci est un exemple de la voix sélectionnée.",
Language.HI: "यह चयनित आवाज़ का एक नमूना है।",
Language.IT: "Questo è un esempio della voce selezionata.",
Language.JA: "これは選択した声のサンプルです。",
Language.PT_BR: "Este é um exemplo da voz selecionada.",
Language.ZH: "这是所选语音的示例。",
"a": "This is a sample of the selected voice.",
"b": "This is a sample of the selected voice.",
"e": "Este es una muestra de la voz seleccionada.",
"f": "Ceci est un exemple de la voix sélectionnée.",
"h": "यह चयनित आवाज़ का एक नमूना है।",
"i": "Questo è un esempio della voce selezionata.",
"j": "これは選択した声のサンプルです。",
"p": "Este é um exemplo da voz selecionada.",
"z": "这是所选语音的示例。",
}
COLORS = {
-239
View File
@@ -1,239 +0,0 @@
"""Audio buffer operations for audiobook generation.
This module provides core audio buffer manipulation functions including:
- Silence generation
- Audio mixing
- Audio normalization
- Audio buffer resizing
"""
from __future__ import annotations
from typing import Optional
import numpy as np
# Standard sample rate used throughout the application
SAMPLE_RATE = 24000
def create_silence(duration_seconds: float) -> np.ndarray:
"""Create a silence audio buffer.
Args:
duration_seconds: Duration of silence in seconds.
Returns:
Numpy array of float32 zeros with length = duration_seconds * SAMPLE_RATE.
Returns empty array if duration is <= 0.
"""
if duration_seconds <= 0:
return np.array([], dtype="float32")
samples = int(round(duration_seconds * SAMPLE_RATE))
if samples <= 0:
return np.array([], dtype="float32")
return np.zeros(samples, dtype="float32")
def mix_audio(
target: np.ndarray,
source: np.ndarray,
start_sample: int,
end_sample: Optional[int] = None,
) -> np.ndarray:
"""Mix source audio into target buffer at specified position.
This performs additive mixing (target += source). The target buffer
is extended if necessary to accommodate the source audio.
Args:
target: The target audio buffer to mix into.
source: The source audio buffer to mix.
start_sample: Starting sample index in target buffer.
end_sample: Optional end sample index. If None, calculated from source length.
Returns:
The target buffer (possibly extended). If target was extended, returns new array.
"""
if source.size == 0:
return target
if end_sample is None:
end_sample = start_sample + len(source)
# Extend target buffer if needed
if end_sample > len(target):
new_length = end_sample
new_target = np.concatenate([
target,
np.zeros(new_length - len(target), dtype="float32")
])
target = new_target
# Perform the mix (additive)
target[start_sample:end_sample] += source
return target
def normalize_audio(
audio: np.ndarray,
target_peak: float = 1.0,
) -> np.ndarray:
"""Normalize audio buffer to prevent clipping.
If the audio exceeds the target peak (default 1.0), it is scaled down
proportionally to prevent distortion.
Args:
audio: Input audio buffer.
target_peak: Target maximum amplitude (default 1.0).
Returns:
Normalized audio buffer (new array, original is not modified).
"""
if audio.size == 0:
return audio.copy()
max_amplitude = float(np.abs(audio).max())
if max_amplitude <= target_peak:
return audio.copy()
# Scale down to prevent clipping
scale_factor = target_peak / max_amplitude
return (audio * scale_factor).astype("float32")
def ensure_buffer_size(
buffer: np.ndarray,
min_samples: int,
) -> np.ndarray:
"""Ensure audio buffer is at least min_samples long.
If buffer is shorter, it is extended with zeros.
Args:
buffer: Input audio buffer.
min_samples: Minimum required length in samples.
Returns:
Buffer of at least min_samples length (new array if extended).
"""
if len(buffer) >= min_samples:
return buffer
new_buffer = np.zeros(min_samples, dtype="float32")
new_buffer[:len(buffer)] = buffer
return new_buffer
def concatenate_audio(*buffers: np.ndarray) -> np.ndarray:
"""Concatenate multiple audio buffers.
Args:
*buffers: Audio buffers to concatenate.
Returns:
Single concatenated audio buffer.
"""
non_empty = [b for b in buffers if b.size > 0]
if not non_empty:
return np.array([], dtype="float32")
return np.concatenate(non_empty)
def audio_duration(audio: np.ndarray, sample_rate: int = SAMPLE_RATE) -> float:
"""Calculate duration of audio buffer in seconds.
Args:
audio: Audio buffer.
sample_rate: Sample rate in Hz (default SAMPLE_RATE).
Returns:
Duration in seconds.
"""
return len(audio) / sample_rate
def samples_for_duration(duration_seconds: float, sample_rate: int = SAMPLE_RATE) -> int:
"""Calculate number of samples for a given duration.
Args:
duration_seconds: Duration in seconds.
sample_rate: Sample rate in Hz (default SAMPLE_RATE).
Returns:
Number of samples (rounded to nearest integer), or 0 if duration is <= 0.
"""
if duration_seconds <= 0:
return 0
return int(round(duration_seconds * sample_rate))
def fit_audio_to_duration(
audio: np.ndarray,
target_duration: float,
sample_rate: int = SAMPLE_RATE,
) -> np.ndarray:
"""Pad or trim audio to match target duration.
Args:
audio: Input audio buffer.
target_duration: Desired duration in seconds.
sample_rate: Sample rate in Hz.
Returns:
Audio buffer of exact length target_duration * sample_rate.
"""
target_samples = int(target_duration * sample_rate)
if len(audio) < target_samples:
padding = np.zeros(target_samples - len(audio), dtype="float32")
return np.concatenate([audio, padding])
return audio[:target_samples]
def ffmpeg_time_stretch(
audio: np.ndarray,
speed_factor: float,
sample_rate: int = SAMPLE_RATE,
) -> np.ndarray:
"""Time-stretch audio using FFmpeg's atempo filter.
Args:
audio: Input audio buffer (float32).
speed_factor: Speed multiplier (>1.0 = faster).
sample_rate: Sample rate in Hz.
Returns:
Time-stretched audio buffer.
"""
import math
import subprocess
import static_ffmpeg
if speed_factor <= 1.0 or audio.size == 0:
return audio
static_ffmpeg.add_paths()
num_stages = max(1, int(math.ceil(math.log(speed_factor) / math.log(2.0))))
tempo = speed_factor ** (1.0 / num_stages)
filter_str = ",".join([f"atempo={tempo:.6f}"] * num_stages)
proc = subprocess.Popen(
[
"ffmpeg", "-y",
"-f", "f32le", "-ar", str(sample_rate), "-ac", "1",
"-i", "pipe:0",
"-filter:a", filter_str,
"-f", "f32le", "-ar", str(sample_rate), "-ac", "1",
"pipe:1",
],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
out, _ = proc.communicate(input=audio.tobytes())
return np.frombuffer(out, dtype="float32")
-118
View File
@@ -1,118 +0,0 @@
"""Audio helper utilities.
Functions for building ffmpeg commands, converting audio formats,
and applying chapter metadata to MP4 files.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Dict, List, Optional
import numpy as np
SAMPLE_RATE = 24000
def build_ffmpeg_command(path: Path, fmt: str, metadata: Optional[Dict[str, str]] = None) -> list[str]:
from abogen.infrastructure.exporters import ExportService
base = [
"ffmpeg",
"-y",
"-f",
"f32le",
"-ar",
str(SAMPLE_RATE),
"-ac",
"1",
"-i",
"pipe:0",
]
if fmt == "mp3":
base += ["-c:a", "libmp3lame", "-qscale:a", "2"]
elif fmt == "opus":
base += ["-c:a", "libopus", "-b:a", "128000"]
elif fmt == "m4b":
base += ["-c:a", "aac", "-q:a", "2", "-movflags", "+faststart+use_metadata_tags"]
else:
base += ["-c:a", "copy"]
if metadata:
svc = ExportService()
base.extend(svc._metadata_to_ffmpeg_args(metadata))
base.append(str(path))
return base
def to_float32(audio_segment) -> np.ndarray:
if audio_segment is None:
return np.zeros(0, dtype="float32")
tensor = audio_segment
if hasattr(tensor, "detach"):
tensor = tensor.detach()
if hasattr(tensor, "cpu"):
try:
tensor = tensor.cpu()
except Exception:
pass
if hasattr(tensor, "numpy"):
return np.asarray(tensor.numpy(), dtype="float32").reshape(-1)
return np.asarray(tensor, dtype="float32").reshape(-1)
def apply_m4b_chapters_with_mutagen(
audio_path: Path,
chapters: List[Dict[str, Any]],
) -> bool:
"""Apply chapter atoms to an MP4/M4B file using mutagen.
Returns True if chapters were written, False otherwise.
Raises ImportError if mutagen is not installed.
"""
if not chapters:
return False
from fractions import Fraction
from mutagen.mp4 import MP4, MP4Chapter # type: ignore[import]
mp4 = MP4(str(audio_path))
chapter_objects: List[MP4Chapter] = []
for index, entry in enumerate(sorted(chapters, key=lambda item: float(item.get("start") or 0.0))):
start_raw = entry.get("start")
if start_raw is None:
continue
try:
start_seconds = max(0.0, float(start_raw))
except (TypeError, ValueError):
continue
title_value = entry.get("title")
title_text = str(title_value) if title_value else f"Chapter {index + 1}"
start_fraction = Fraction(int(round(start_seconds * 1000)), 1000)
chapter_atom = MP4Chapter(start_fraction, title_text)
end_raw = entry.get("end")
if end_raw is not None:
try:
end_seconds = float(end_raw)
except (TypeError, ValueError):
end_seconds = None
if end_seconds is not None and end_seconds > start_seconds:
chapter_atom.end = Fraction(int(round(end_seconds * 1000)), 1000)
chapter_objects.append(chapter_atom)
if not chapter_objects:
return False
from typing import cast
mp4.chapters = cast(Any, chapter_objects)
mp4.save()
return True
-131
View File
@@ -1,131 +0,0 @@
"""Audio sink abstraction for unified audio output.
Provides a context-manager-based abstraction for writing audio data
to various output formats (WAV, FLAC via soundfile; compressed via ffmpeg).
Usage:
with open_audio_sink(path, "wav") as sink:
sink.write(audio_data)
"""
from __future__ import annotations
import os
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Optional
import numpy as np
from abogen.domain.audio_buffer import SAMPLE_RATE
from abogen.domain.audio_helpers import build_ffmpeg_command
@dataclass(frozen=True)
class AudioSink:
"""Represents an open audio output target."""
write: Callable[[np.ndarray], None]
close: Callable[[], None]
def __enter__(self) -> AudioSink:
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.close()
def _ensure_ffmpeg() -> None:
"""Ensure static ffmpeg binaries are on PATH."""
import static_ffmpeg # type: ignore
ffmpeg_cache_root = _get_ffmpeg_cache_root()
platform_cache = os.path.join(ffmpeg_cache_root, sys.platform)
os.makedirs(platform_cache, exist_ok=True)
try:
import static_ffmpeg.run as static_ffmpeg_run # type: ignore
static_ffmpeg_run.LOCK_FILE = os.path.join(ffmpeg_cache_root, "lock.file")
except Exception:
pass
static_ffmpeg.add_paths(weak=True, download_dir=platform_cache)
def _get_ffmpeg_cache_root() -> str:
from abogen.utils import get_internal_cache_path
return get_internal_cache_path("ffmpeg")
def open_audio_sink(
path: Path,
fmt: str,
*,
metadata: Optional[dict[str, str]] = None,
cancel_check: Optional[Callable[[], bool]] = None,
extra_ffmpeg_args: Optional[list[str]] = None,
ffmpeg_cmd: Optional[list[str]] = None,
) -> AudioSink:
"""Open an audio output sink for writing raw float32 PCM samples.
Args:
path: Output file path.
fmt: Output format ("wav", "flac", "mp3", "opus", "m4b").
metadata: Optional metadata dict (ignored when ffmpeg_cmd is provided).
cancel_check: Optional callable; if it returns True, writes are silently skipped.
extra_ffmpeg_args: Optional extra args inserted after ffmpeg header (ignored when ffmpeg_cmd is provided).
ffmpeg_cmd: Optional pre-built ffmpeg command list (for m4b with cover art etc.).
Returns:
AudioSink with write() and close() methods.
"""
fmt = fmt.lower()
if fmt in {"wav", "flac"}:
import soundfile as sf
soundfile_obj = sf.SoundFile(
path,
mode="w",
samplerate=SAMPLE_RATE,
channels=1,
format=fmt.upper(),
)
def _write_wav(data: np.ndarray) -> None:
if cancel_check and cancel_check():
return
soundfile_obj.write(data)
def _close_wav() -> None:
soundfile_obj.close()
return AudioSink(write=_write_wav, close=_close_wav)
# Compressed formats: pipe through ffmpeg
_ensure_ffmpeg()
if ffmpeg_cmd is not None:
cmd = list(ffmpeg_cmd)
else:
cmd = build_ffmpeg_command(path, fmt, metadata=metadata)
if extra_ffmpeg_args:
cmd[2:2] = extra_ffmpeg_args
process = subprocess.Popen(
cmd, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL
)
def _write_compressed(data: np.ndarray) -> None:
if (cancel_check and cancel_check()) or process.stdin is None or process.stdin.closed:
return
process.stdin.write(data.tobytes())
def _close_compressed() -> None:
if process.stdin and not process.stdin.closed:
process.stdin.close()
process.wait()
return AudioSink(write=_write_compressed, close=_close_compressed)
-131
View File
@@ -1,131 +0,0 @@
"""Heuristics for classifying chapters as content vs. supplements.
A 'supplement' is any non-story material that a listener would typically
skip: title page, copyright, table of contents, acknowledgements, etc.
The scoring functions return a float; higher ⇒ more likely to be a
supplement. ``should_preselect_chapter`` turns that score into a
boolean suitable for a web form default.
"""
from __future__ import annotations
import re
from typing import Any, Dict, List, Tuple
# Compiled once at module load these are immutable.
_SUPPLEMENT_TITLE_PATTERNS: List[Tuple[re.Pattern[str], float]] = [
(re.compile(r"\btitle\s+page\b"), 3.0),
(re.compile(r"\bcopyright\b"), 2.4),
(re.compile(r"\btable\s+of\s+contents\b"), 2.8),
(re.compile(r"\bcontents\b"), 2.0),
(re.compile(r"\backnowledg(e)?ments?\b"), 2.0),
(re.compile(r"\bdedication\b"), 2.0),
(re.compile(r"\babout\s+the\s+author(s)?\b"), 2.4),
(re.compile(r"\balso\s+by\b"), 2.0),
(re.compile(r"\bpraise\s+for\b"), 2.0),
(re.compile(r"\bcolophon\b"), 2.2),
(re.compile(r"\bpublication\s+data\b"), 2.2),
(re.compile(r"\btranscriber'?s?\s+note\b"), 2.2),
(re.compile(r"\bglossary\b"), 2.2),
(re.compile(r"\bindex\b"), 2.0),
(re.compile(r"\bbibliograph(y|ies)\b"), 2.0),
(re.compile(r"\breferences\b"), 1.8),
(re.compile(r"\bappendix\b"), 1.9),
]
_CONTENT_TITLE_PATTERNS: List[re.Pattern[str]] = [
re.compile(r"\bchapter\b"),
re.compile(r"\bbook\b"),
re.compile(r"\bpart\b"),
re.compile(r"\bsection\b"),
re.compile(r"\bscene\b"),
re.compile(r"\bprologue\b"),
re.compile(r"\bepilogue\b"),
re.compile(r"\bintroduction\b"),
re.compile(r"\bstory\b"),
]
_SUPPLEMENT_TEXT_KEYWORDS: List[Tuple[str, float]] = [
("copyright", 1.2),
("all rights reserved", 1.1),
("isbn", 0.9),
("library of congress", 1.0),
("table of contents", 1.0),
("dedicated to", 0.8),
("acknowledg", 0.8),
("printed in", 0.6),
("permission", 0.6),
("publisher", 0.5),
("praise for", 0.9),
("also by", 0.9),
("glossary", 0.8),
("index", 0.8),
("newsletter", 3.2),
("mailing list", 2.6),
("sign-up", 2.2),
]
def supplement_score(title: str, text: str, index: int) -> float:
"""Return a score indicating how likely *title*/*text* is a supplement.
Higher values ⇒ more likely to be non-story material (title page,
copyright, acknowledgements, etc.).
"""
normalized_title = (title or "").lower()
score = 0.0
for pattern, weight in _SUPPLEMENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score += weight
for pattern in _CONTENT_TITLE_PATTERNS:
if pattern.search(normalized_title):
score -= 2.0
stripped_text = (text or "").strip()
length = len(stripped_text)
if length <= 150:
score += 0.9
elif length <= 400:
score += 0.6
elif length <= 800:
score += 0.35
lowercase_text = stripped_text.lower()
for keyword, weight in _SUPPLEMENT_TEXT_KEYWORDS:
if keyword in lowercase_text:
score += weight
if index == 0 and score > 0:
score += 0.25
return score
def should_preselect_chapter(
title: str,
text: str,
index: int,
total_count: int,
) -> bool:
"""Return True if the chapter should be *enabled* by default in the form.
A single chapter is always preselected. For multi-chapter books, the
chapter is preselected when its supplement score is below 1.9.
"""
if total_count <= 1:
return True
score = supplement_score(title, text, index)
return score < 1.9
def ensure_at_least_one_chapter_enabled(chapters: List[Dict[str, Any]]) -> None:
"""Mutate *chapters* in-place so that at least one has ``enabled=True``."""
if not chapters:
return
if any(chapter.get("enabled") for chapter in chapters):
return
best_index = max(range(len(chapters)), key=lambda idx: chapters[idx].get("characters", 0))
chapters[best_index]["enabled"] = True
-92
View File
@@ -1,92 +0,0 @@
from __future__ import annotations
from typing import Any, Dict, List, Optional, Tuple
from abogen.text_extractor import ExtractedChapter
from abogen.domain.voice_utils import coerce_truthy
def apply_chapter_overrides(
extracted: List[ExtractedChapter],
overrides: List[Dict[str, Any]],
) -> Tuple[List[ExtractedChapter], Dict[str, str], List[str]]:
if not overrides:
return [], {}, []
selected: List[ExtractedChapter] = []
metadata_updates: Dict[str, str] = {}
diagnostics: List[str] = []
for position, payload in enumerate(overrides):
if not isinstance(payload, dict):
diagnostics.append(
f"Skipped chapter override at position {position + 1}: unsupported payload type {type(payload).__name__}."
)
continue
enabled = coerce_truthy(payload.get("enabled", True))
payload["enabled"] = enabled
if not enabled:
continue
metadata_payload = payload.get("metadata") or {}
if isinstance(metadata_payload, dict):
for key, value in metadata_payload.items():
if value is None:
continue
metadata_updates[str(key)] = str(value)
base: Optional[ExtractedChapter] = None
idx_candidate = payload.get("index")
idx_normalized: Optional[int] = None
if isinstance(idx_candidate, int):
idx_normalized = idx_candidate
elif isinstance(idx_candidate, str):
try:
idx_normalized = int(idx_candidate)
except ValueError:
idx_normalized = None
if idx_normalized is not None and 0 <= idx_normalized < len(extracted):
base = extracted[idx_normalized]
payload["index"] = idx_normalized
if base is None:
source_title = payload.get("source_title")
if isinstance(source_title, str):
base = next((chapter for chapter in extracted if chapter.title == source_title), None)
if base is None:
candidate_title = payload.get("title")
if isinstance(candidate_title, str):
base = next((chapter for chapter in extracted if chapter.title == candidate_title), None)
text_override = payload.get("text")
if text_override is not None:
text_value = str(text_override)
elif base is not None:
text_value = base.text
else:
diagnostics.append(
f"Skipped chapter override at position {position + 1}: no text provided and no matching source chapter found."
)
continue
title_override = payload.get("title")
if title_override is not None:
title_value = str(title_override)
elif base is not None:
title_value = base.title
else:
title_value = f"Chapter {position + 1}"
if base and not payload.get("source_title"):
payload["source_title"] = base.title
payload["title"] = title_value
payload["text"] = text_value
payload["characters"] = len(text_value)
payload.setdefault("order", payload.get("order", position))
selected.append(ExtractedChapter(title=title_value, text=text_value))
return selected, metadata_updates, diagnostics
-204
View File
@@ -1,204 +0,0 @@
from __future__ import annotations
import re
from typing import List, Tuple
_HEADING_SANITIZE_RE = re.compile(r"[^a-z0-9]+")
_HEADING_NUMBER_PREFIX_RE = re.compile(
r"^\s*(?P<number>(?:\d+|[ivxlcdm]+))(?P<suffix>(?:[\s.:;-].*)?)$",
re.IGNORECASE,
)
_ACRONYM_ALLOWLIST = {
"AI", "API", "CPU", "DIY", "GPU", "HTML", "HTTP", "HTTPS", "ID",
"JSON", "MP3", "MP4", "M4B", "NASA", "OCR", "PDF", "SQL", "TV",
"TTS", "UK", "UN", "UFO", "OK", "URL", "USA", "US", "VR",
}
_ROMAN_NUMERAL_CHARS = frozenset("IVXLCDM")
_CAPS_WORD_RE = re.compile(r"[A-Z][A-Z0-9'\u2019-]*")
def simplify_heading_text(text: str) -> str:
raw = str(text or "").strip().lower()
if not raw:
return ""
simplified = _HEADING_SANITIZE_RE.sub("", raw)
if simplified.startswith("chapter"):
simplified = simplified[7:]
return simplified
def headings_equivalent(left: str, right: str) -> bool:
simple_left = simplify_heading_text(left)
simple_right = simplify_heading_text(right)
if not simple_left or not simple_right:
return False
if simple_left == simple_right:
return True
if simple_right.startswith(simple_left):
return True
if simple_left.startswith(simple_right):
return True
if len(simple_left) > 5 and simple_left in simple_right:
return True
return False
def strip_duplicate_heading_line(text: str, heading: str) -> Tuple[str, bool]:
source_text = str(text or "")
if not source_text:
return source_text, False
normalized_heading = simplify_heading_text(heading)
if not normalized_heading:
return source_text, False
lines = source_text.splitlines()
new_lines: List[str] = []
removed = False
for line in lines:
stripped = line.strip()
if not removed and stripped:
if headings_equivalent(stripped, heading):
removed = True
continue
new_lines.append(line)
if not removed:
return source_text, False
while new_lines and not new_lines[0].strip():
new_lines.pop(0)
return "\n".join(new_lines), True
def normalize_caps_word(word: str) -> str:
upper = word.upper()
letters = [char for char in upper if char.isalpha()]
if not letters:
return word
if upper in _ACRONYM_ALLOWLIST:
return word
if len(letters) <= 1:
return word
if all(char in _ROMAN_NUMERAL_CHARS for char in letters) and len(letters) <= 7:
return word
parts = re.split(r"(['\-\u2019])", word)
normalized_parts: List[str] = []
for part in parts:
if part in {"'", "-", "\u2019"}:
normalized_parts.append(part)
continue
if not part:
continue
normalized_parts.append(part[0].upper() + part[1:].lower())
return "".join(normalized_parts) or word
def normalize_chapter_opening_caps(text: str) -> Tuple[str, bool]:
if not text:
return text, False
leading_len = len(text) - len(text.lstrip())
leading = text[:leading_len]
working = text[leading_len:]
if not working:
return text, False
builder: List[str] = []
pos = 0
changed = False
while pos < len(working):
char = working[pos]
if char in "\r\n":
builder.append(working[pos:])
pos = len(working)
break
if char.isspace():
builder.append(char)
pos += 1
continue
if char.islower():
builder.append(working[pos:])
pos = len(working)
break
if not char.isalpha():
builder.append(char)
pos += 1
continue
match = _CAPS_WORD_RE.match(working, pos)
if not match:
builder.append(char)
pos += 1
continue
word = match.group(0)
if any(ch.islower() for ch in word):
builder.append(working[pos:])
pos = len(working)
break
normalized = normalize_caps_word(word)
if normalized != word:
changed = True
builder.append(normalized)
pos = match.end()
if pos < len(working):
builder.append(working[pos:])
if not changed:
return text, False
return leading + "".join(builder), True
def format_spoken_chapter_title(title: str, index: int, apply_prefix: bool) -> str:
base = str(title or "").strip()
if not base:
return f"Chapter {index}" if apply_prefix else ""
if not apply_prefix:
return base
lowered = base.lower()
if lowered.startswith("chapter") and (len(lowered) == 7 or not lowered[7].isalpha()):
return base
match = _HEADING_NUMBER_PREFIX_RE.match(base)
if match:
number = match.group("number") or ""
suffix = match.group("suffix") or ""
cleaned_suffix = suffix.lstrip(" .,:;-_ \t\u2013\u2014\u00b7\u2022")
if cleaned_suffix:
return f"Chapter {number}. {cleaned_suffix}"
return f"Chapter {number}"
return base
def apply_chapter_text_transforms(
text: str,
*,
heading_text: str,
raw_title: str,
strip_heading: bool,
normalize_caps: bool,
) -> Tuple[str, bool, bool]:
"""Strip duplicate heading and normalize opening caps.
Returns ``(text, heading_removed, caps_changed)``.
The caller is responsible for state updates (pending flags, logging,
dict mutation, ``continue``).
"""
heading_removed = False
caps_changed = False
if strip_heading and heading_text:
text, heading_removed = strip_duplicate_heading_line(text, heading_text)
if not heading_removed and raw_title:
match = _HEADING_NUMBER_PREFIX_RE.match(raw_title)
if match:
number = match.group("number")
if number:
text, heading_removed = strip_duplicate_heading_line(text, number)
if normalize_caps and text:
text, caps_changed = normalize_chapter_opening_caps(text)
return text, heading_removed, caps_changed
-76
View File
@@ -1,76 +0,0 @@
"""Chunk processing utilities.
Functions for grouping chunks, recording override usage, and selecting
text for TTS synthesis.
"""
from __future__ import annotations
from collections import defaultdict
from typing import Any, Dict, Iterable, Mapping
from abogen.domain.enums import Language
from abogen.pronunciation_store import increment_usage
def safe_int(value: Any, default: int = 0) -> int:
try:
return int(value)
except (TypeError, ValueError):
return default
def group_chunks_by_chapter(chunks: Iterable[Dict[str, Any]]) -> Dict[int, List[Dict[str, Any]]]:
grouped: Dict[int, List[Dict[str, Any]]] = defaultdict(list)
for entry in chunks or []:
if not isinstance(entry, dict):
continue
try:
chapter_index = int(entry.get("chapter_index", 0))
except (TypeError, ValueError):
chapter_index = 0
grouped[chapter_index].append(dict(entry))
for chapter_index, items in grouped.items():
items.sort(key=lambda payload: safe_int(payload.get("chunk_index")))
return grouped
def record_override_usage(
job: Any,
usage_counter: Mapping[str, int],
token_map: Mapping[str, str],
) -> None:
if not usage_counter:
return
language = getattr(job, "language", Language.EN_US) or Language.EN_US
for normalized, amount in usage_counter.items():
if amount <= 0:
continue
token_value = token_map.get(normalized, normalized)
try:
increment_usage(language=language, token=token_value, amount=int(amount))
except Exception: # pragma: no cover - defensive logging
job.add_log(f"Failed to record usage for override {token_value}", level="warning")
def chunk_text_for_tts(entry: Mapping[str, Any]) -> str:
"""Choose the best source text for synthesis.
We must prefer the raw chunk text (``text`` / ``original_text``) so
manual/pronunciation overrides can match against the original tokens
(e.g. censored words like ``Unfu*k``). ``normalized_text`` may have
already been run through ``normalize_for_pipeline``, which can remove
punctuation and prevent overrides from triggering.
"""
if not isinstance(entry, Mapping):
return ""
return str(
entry.get("text")
or entry.get("original_text")
or entry.get("normalized_text")
or ""
).strip()
-52
View File
@@ -1,52 +0,0 @@
"""Domain config types — shared contracts for domain functions.
These dataclasses group parameters that domain functions receive.
Domain defines them, app layer fills them.
Why here (domain) and not application:
- build_tts_context() is in domain → needs PronunciationConfig
- make_subtitle_writer() is in infrastructure → needs SubtitleConfig
- embed_m4b_metadata() is in infrastructure → needs CoverConfig
- Domain should not depend on application layer (DIP)
"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Dict, List, Optional
from abogen.domain.enums import SubtitleFormat, SubtitleMode
@dataclass(frozen=True)
class PronunciationConfig:
"""Pronunciation and normalization override settings.
Used by build_tts_context() to compile override rules.
"""
pronunciation_overrides: List[Dict[str, Any]] = field(default_factory=list)
manual_overrides: List[Dict[str, Any]] = field(default_factory=list)
heteronym_overrides: List[Dict[str, Any]] = field(default_factory=list)
normalization_overrides: Optional[Dict[str, Any]] = None
@dataclass(frozen=True)
class SubtitleConfig:
"""Subtitle output settings.
Used by make_subtitle_writer() and process_and_write_subtitles().
"""
mode: SubtitleMode = SubtitleMode.DISABLED
format: SubtitleFormat = SubtitleFormat.SRT
max_words: int = 50
@dataclass(frozen=True)
class CoverConfig:
"""Cover image settings.
Used by embed_m4b_metadata() and build_epub3_package().
"""
path: Optional[Path] = None
mime: Optional[str] = None
-241
View File
@@ -1,241 +0,0 @@
"""Shared TTS iteration loop used by both WebUI and PyQt conversion runners.
The core pattern is identical across both UIs:
for seg in tts_segments(text, backend, voice, speed, split_pattern, current_time):
check_cancel()
update_progress(seg)
write_audio(seg, sink)
accumulate_subtitles(seg)
After the loop, the caller processes accumulated subtitle tokens.
This module provides ``run_tts_segment_loop`` which encapsulates that
iteration, and ``synthesize_text`` which adds normalization on top —
the single entry point both UIs should call for text-to-speech.
"""
from __future__ import annotations
import time
from dataclasses import dataclass, field
from typing import Any, Callable, Optional, Protocol
from abogen.domain.audio_sink import AudioSink
from abogen.domain.conversion_pipeline import tts_segments
from abogen.domain.enums import Language, SubtitleMode
from abogen.domain.normalization import TTSContext
from abogen.domain.progress import calc_etr_str
from abogen.domain.subtitle_generation import process_subtitle_tokens
class CancelChecker(Protocol):
"""Returns True if conversion has been cancelled."""
def __call__(self) -> bool: ...
@dataclass
class SegmentStats:
"""Running statistics updated per TTS segment."""
processed_chars: int = 0
current_time: float = 0.0
etr_start_time: float = field(default_factory=time.time)
total_characters: int = 0
@dataclass
class SegmentInfo:
"""Read-only info about a TTS segment, passed to on_segment callback."""
graphemes: str
audio: Any
tokens: list
duration: float
chunk_start: float
def run_tts_segment_loop(
*,
text: str,
params: SynthParams,
backend: Any,
voice: Any,
speed: float,
split_pattern: str,
total_steps: Optional[int] = None,
chapter_sink: Optional[AudioSink] = None,
preview_callback: Optional[Callable[[str], None]] = None,
on_segment: Optional[Callable[[SegmentInfo], None]] = None,
) -> tuple[int, list]:
"""Run the core TTS segment iteration loop.
Args:
text: Normalized text to synthesize.
params: Common synthesis parameters (stats, callbacks, sinks, etc.).
backend: TTS pipeline instance (Kokoro or Supertonic).
voice: Voice name/id for the backend.
speed: Speech speed multiplier.
split_pattern: Regex pattern used by the TTS engine for sentence splitting.
total_steps: Inference quality steps (Supertonic only, ignored by Kokoro).
preview_callback: Called with a short preview string per segment.
on_segment: Called with a SegmentInfo for each segment *before*
audio is written. Useful for callers that need per-segment
subtitle processing (e.g. PyQt dual-writer pattern).
When provided, the default subtitle accumulation is skipped.
Returns:
Tuple of (segment_count, accumulated_subtitle_tokens).
The caller is responsible for processing subtitle tokens via
``process_subtitle_tokens`` and writing entries to subtitle writers.
"""
local_segments = 0
accumulated_tokens: list[dict] = []
for seg in tts_segments(
text,
backend=backend,
voice=voice,
speed=speed,
split_pattern=split_pattern,
current_time=params.stats.current_time,
total_steps=total_steps,
):
if params.check_cancel():
break
local_segments += 1
params.stats.processed_chars += len(seg.graphemes)
# Progress
if params.stats.total_characters:
percent = min(int(params.stats.processed_chars / params.stats.total_characters * 100), 99)
else:
percent = 0 if params.stats.processed_chars == 0 else 99
etr_str = calc_etr_str(
time.time() - params.stats.etr_start_time,
params.stats.processed_chars,
params.stats.total_characters,
)
params.on_progress(percent, etr_str)
# Preview / log
if preview_callback:
preview_callback(seg.graphemes or "[silence]")
# Per-segment callback (for callers needing segment-level access)
if on_segment:
info = SegmentInfo(
graphemes=seg.graphemes,
audio=seg.audio,
tokens=list(seg.tokens) if seg.tokens else [],
duration=seg.duration,
chunk_start=getattr(seg, "chunk_start", params.stats.current_time),
)
on_segment(info)
# Write audio
if chapter_sink:
chapter_sink.write(seg.audio)
if params.audio_sink:
params.audio_sink.write(seg.audio)
# Accumulate subtitle tokens (default path; skipped if on_segment handles it)
if not on_segment and params.subtitle_mode != SubtitleMode.DISABLED and seg.tokens:
accumulated_tokens.extend(seg.tokens)
# Update timing
if params.audio_sink:
params.stats.current_time += seg.duration
return local_segments, accumulated_tokens
def process_and_write_subtitles(
accumulated_tokens: list[dict],
subtitle_writer: Any,
*,
subtitle: "SubtitleConfig | str",
max_subtitle_words: int | None = None,
language: Language,
use_spacy_segmentation: bool,
fallback_end_time: float,
) -> None:
"""Process accumulated subtitle tokens and write entries to a subtitle writer.
Accepts a SubtitleConfig object or a subtitle mode string
for backward compatibility.
"""
from abogen.domain.config_types import SubtitleConfig
if isinstance(subtitle, SubtitleConfig):
mode_str = subtitle.mode.value
words = subtitle.max_words
else:
mode_str = subtitle
words = max_subtitle_words or 50
if not accumulated_tokens or not subtitle_writer:
return
new_entries: list[tuple] = []
process_subtitle_tokens(
accumulated_tokens,
new_entries,
words,
mode_str,
language,
use_spacy_segmentation=use_spacy_segmentation,
fallback_end_time=fallback_end_time,
)
for start, end, text in new_entries:
subtitle_writer.write_entry(start=start, end=end, text=text)
@dataclass(frozen=True)
class SynthParams:
"""Common parameters for synthesize_text calls.
Packed once by the executor to avoid repeating identical kwargs.
When adding new common params, change only this dataclass.
"""
tts_context: TTSContext
stats: SegmentStats
check_cancel: CancelChecker
on_progress: Callable[[int, str], None]
audio_sink: Optional[AudioSink] = None
subtitle_mode: str = "Disabled"
max_subtitle_words: int = 50
language: Language = Language.EN_US
use_spacy_segmentation: bool = False
def synthesize_text(
*,
text: str,
params: SynthParams,
backend: Any,
voice: Any,
speed: float,
total_steps: Optional[int] = None,
chapter_sink: Optional[AudioSink] = None,
preview_callback: Optional[Callable[[str], None]] = None,
on_segment: Optional[Callable[[SegmentInfo], None]] = None,
split_pattern_override: Optional[str] = None,
) -> tuple[int, list]:
"""Normalize text and run TTS — the single entry point for both UIs.
Combines TTSContext.normalize() + run_tts_segment_loop() into one call.
UI-specific concerns (provider resolution, progress display) stay in the UI.
"""
normalized = params.tts_context.normalize(text)
return run_tts_segment_loop(
text=normalized,
params=params,
backend=backend,
voice=voice,
speed=speed,
total_steps=total_steps,
split_pattern=split_pattern_override or params.tts_context.split_pattern,
chapter_sink=chapter_sink,
preview_callback=preview_callback,
on_segment=on_segment,
)
-372
View File
@@ -1,372 +0,0 @@
"""Shared TTS emission pipeline.
Provides the core TTS emission loop used by both WebUI and PyQt conversion runners.
The caller handles audio I/O, progress reporting, and subtitle writing.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass, field
from abogen.domain.enums import Language, SubtitleMode
from typing import Any, Callable, Dict, Iterator, List, Optional, Tuple
import numpy as np
from abogen.domain.audio_helpers import to_float32
from abogen.domain.normalization import prepare_text_for_tts
from abogen.domain.tokens import FakeToken
from abogen.domain.audio_buffer import SAMPLE_RATE
logger = logging.getLogger(__name__)
# Languages where spaCy is used for pre-TTS segmentation
# English ("a", "b") is excluded — spaCy only used for post-TTS subtitles
_SPACY_EXCLUDED_LANGS = {Language.EN_US, Language.EN_GB}
# CJK languages — different spacing pattern
_CJK_LANGS = {Language.ZH, Language.JA}
def spacy_pre_tts_segmentation(
text: str,
lang_code: Any,
subtitle_mode: Any,
*,
is_subtitle_input: bool = False,
use_spacy_segmentation: bool = True,
log_callback: Optional[Callable[[str], None]] = None,
) -> Tuple[List[str], str]:
"""Segment text using spaCy before TTS, with split_pattern override.
For non-English languages, spaCy sentence segmentation produces better
sentence boundaries than regex. This function:
1. Checks if spaCy should be used (toggle on, not disabled mode, not subtitle input)
2. For non-English: runs spaCy segmentation, computes split_pattern override
3. For English: returns single segment with default pattern (spaCy only for subtitles)
4. If spaCy fails: falls back to default pattern
Args:
text: Text to segment.
lang_code: Language code (Language enum or string like "a", "de", "fr").
subtitle_mode: SubtitleMode enum or string.
is_subtitle_input: True if source is .srt/.ass/.vtt file.
use_spacy_segmentation: User toggle for spaCy segmentation.
log_callback: Optional logging function.
Returns:
Tuple of (text_segments, active_split_pattern).
text_segments is a list of sentences (always at least one element).
active_split_pattern is the regex to use for TTS backend splitting.
"""
from abogen.domain.split_pattern import get_split_pattern
def _log(msg: str) -> None:
if log_callback:
log_callback(msg)
# Normalize language
lang_enum = _to_language_enum(lang_code)
# Default split pattern
default_split = get_split_pattern(lang_code, subtitle_mode)
# Check conditions
if not use_spacy_segmentation:
return [text], default_split
subtitle_mode_str = _to_subtitle_mode_str(subtitle_mode)
if subtitle_mode_str in ("Disabled", "Line"):
return [text], default_split
if is_subtitle_input:
return [text], default_split
# English: spaCy only for post-TTS subtitles, not pre-TTS
if lang_enum in _SPACY_EXCLUDED_LANGS:
return [text], default_split
# Non-English: run spaCy pre-TTS segmentation
from abogen.spacy_utils import segment_sentences
_log("Using spaCy for sentence segmentation (pre-TTS)...")
spacy_sentences = segment_sentences(text, lang_code, log_callback=log_callback)
if not spacy_sentences:
_log("spaCy: Fallback to default segmentation...")
return [text], default_split
_log(f"spaCy: Text segmented into {len(spacy_sentences)} sentences...")
# spaCy already split at sentence boundaries; the engine only needs to
# split on newlines. Commas are never used in the engine split pattern
# for non-English (Sentence + Comma splits at commas only at subtitle
# time, like English).
active_split = "\n"
return spacy_sentences, active_split
def _to_language_enum(lang_code: Any) -> Language:
"""Convert lang_code to Language enum (ISO code or Language enum)."""
if isinstance(lang_code, Language):
return lang_code
try:
return Language.from_str(str(lang_code))
except ValueError:
return Language.EN_US
def _to_subtitle_mode_str(subtitle_mode: Any) -> str:
"""Convert subtitle_mode to string."""
if isinstance(subtitle_mode, SubtitleMode):
return subtitle_mode.value
return str(subtitle_mode)
@dataclass
class SegmentResult:
"""One TTS segment emitted by the pipeline."""
graphemes: str
audio: np.ndarray
duration: float
chunk_start: float
tokens: List[Dict[str, Any]] = field(default_factory=list)
def tts_segments(
text: str,
*,
backend: Any,
voice: Any,
speed: float,
split_pattern: str,
current_time: float = 0.0,
total_steps: Optional[int] = None,
) -> Iterator[SegmentResult]:
"""Invoke TTS backend on (already normalized) text and yield SegmentResults.
Use this when you've already normalized the text yourself (e.g. after
spaCy sentence segmentation). For raw text, use emit_text_segments() instead.
Args:
text: Already-normalized text to synthesize.
backend: TTS pipeline callable.
voice: Resolved voice.
speed: TTS speed multiplier.
split_pattern: Regex pattern for sentence splitting.
current_time: Current position in the audio timeline (seconds).
total_steps: Inference quality steps (Supertonic only, ignored by Kokoro).
Yields:
SegmentResult for each non-empty TTS segment.
"""
kwargs: dict[str, Any] = dict(
voice=voice,
speed=speed,
split_pattern=split_pattern,
)
if total_steps is not None:
kwargs["total_steps"] = total_steps
segment_iter = backend(text, **kwargs)
chunk_start = current_time
prev_tokens: Optional[List[Dict[str, Any]]] = None
prev_was_fallback = True
for segment in segment_iter:
graphemes_raw = getattr(segment, "graphemes", "") or ""
graphemes = graphemes_raw.strip()
audio = to_float32(getattr(segment, "audio", None))
if audio.size == 0:
continue
duration = len(audio) / SAMPLE_RATE
tokens_list = getattr(segment, "tokens", [])
was_fallback = False
if not tokens_list and graphemes:
tokens_list = [FakeToken(graphemes, 0, duration)]
was_fallback = True
tokens = [
{
"start": chunk_start + (tok.start_ts or 0),
"end": chunk_start + (tok.end_ts or 0),
"text": tok.text,
"whitespace": tok.whitespace,
}
for tok in tokens_list
]
# When the engine splits text on a punctuation pattern, the
# whitespace between segments is consumed by the split. Restore a
# trailing space on the boundary token of the previous segment so
# subtitle processing sees the original spacing (only for real
# per-word tokens; FakeToken fallbacks split via their own logic).
if (
not prev_was_fallback
and prev_tokens
and not prev_tokens[-1].get("whitespace")
):
prev_tokens[-1]["whitespace"] = " "
yield SegmentResult(
graphemes=graphemes,
audio=audio,
duration=duration,
chunk_start=chunk_start,
tokens=tokens,
)
prev_tokens = tokens
prev_was_fallback = was_fallback
chunk_start += duration
def emit_text_segments(
text: str,
*,
backend: Any,
voice: Any,
speed: float,
split_pattern: str,
current_time: float = 0.0,
total_steps: Optional[int] = None,
# normalization
heteronym_rules: Any = None,
pronunciation_rules: Any = None,
normalization_overrides: Any = None,
usage_counter: Optional[Dict[str, int]] = None,
) -> Iterator[SegmentResult]:
"""Normalize text and yield SegmentResults from the TTS backend.
This is the innermost TTS emission loop shared by both UIs. It handles:
1. Text normalization (heteronym + pronunciation rules)
2. TTS backend invocation
3. Segment iteration with token extraction
The caller is responsible for:
- Writing audio to sinks
- Accumulating tokens for subtitle processing
- Progress tracking and cancellation
- Error handling
Args:
text: Raw text to synthesize.
backend: TTS pipeline callable (kokoro or supertonic).
voice: Resolved voice for TTS.
speed: TTS speed multiplier.
split_pattern: Regex pattern for sentence splitting.
current_time: Current position in the audio timeline (seconds).
heteronym_rules: Compiled heteronym rules.
pronunciation_rules: Compiled pronunciation rules.
normalization_overrides: User normalization overrides.
usage_counter: Counter for normalization statistics.
Yields:
SegmentResult for each non-empty TTS segment.
"""
source_text = str(text or "")
normalized = prepare_text_for_tts(
source_text,
heteronym_rules=heteronym_rules,
pronunciation_rules=pronunciation_rules,
normalization_overrides=normalization_overrides,
usage_counter=usage_counter,
)
yield from tts_segments(
normalized,
backend=backend,
voice=voice,
speed=speed,
split_pattern=split_pattern,
current_time=current_time,
total_steps=total_steps,
)
def emit_text_to_sinks(
text: str,
*,
backend: Any,
voice: Any,
speed: float,
split_pattern: str,
current_time: float = 0.0,
# sinks
audio_sink: Any = None,
chapter_sink: Any = None,
# subtitle
subtitle_writer: Any = None,
subtitle_mode: str = "Disabled",
subtitle_lang: Language = Language.EN_US,
max_subtitle_words: int = 50,
use_spacy_segmentation: bool = True,
# normalization
heteronym_rules: Any = None,
pronunciation_rules: Any = None,
normalization_overrides: Any = None,
usage_counter: Optional[Dict[str, int]] = None,
) -> tuple[int, float, List[Dict[str, Any]]]:
"""Emit TTS audio for text, writing to sinks and collecting subtitle tokens.
Convenience wrapper around emit_text_segments() that handles audio writing
and token accumulation. Returns stats for the caller to update progress.
Returns:
Tuple of (segments_emitted, new_current_time, accumulated_tokens).
"""
from abogen.domain.subtitle_generation import process_subtitle_tokens
segments_emitted = 0
accumulated_tokens: List[Dict[str, Any]] = []
for seg in emit_text_segments(
text,
backend=backend,
voice=voice,
speed=speed,
split_pattern=split_pattern,
current_time=current_time,
heteronym_rules=heteronym_rules,
pronunciation_rules=pronunciation_rules,
normalization_overrides=normalization_overrides,
usage_counter=usage_counter,
):
segments_emitted += 1
# Write audio
if chapter_sink:
chapter_sink.write(seg.audio)
if audio_sink:
audio_sink.write(seg.audio)
# Collect tokens
accumulated_tokens.extend(seg.tokens)
# Flush subtitle tokens
if subtitle_writer and accumulated_tokens:
_use_spacy = subtitle_mode not in (SubtitleMode.DISABLED, SubtitleMode.LINE)
new_entries: List[tuple] = []
process_subtitle_tokens(
accumulated_tokens,
new_entries,
max_subtitle_words,
subtitle_mode,
subtitle_lang,
use_spacy_segmentation=_use_spacy,
fallback_end_time=current_time + sum(t["end"] - t["start"] for t in accumulated_tokens if accumulated_tokens),
)
for start, end, text_entry in new_entries:
subtitle_writer.write_entry(start=start, end=end, text=text_entry)
new_time = current_time
if accumulated_tokens:
new_time = max(t["end"] for t in accumulated_tokens)
return segments_emitted, new_time, accumulated_tokens
-31
View File
@@ -1,31 +0,0 @@
from __future__ import annotations
import platform as _platform
def select_device() -> str:
"""Return the best available compute device (``"mps"``, ``"cuda"``, or ``"cpu"``).
Checks ``torch`` availability at runtime so this can be called from
any context without requiring torch at import time.
"""
try:
import torch # type: ignore[import-not-found]
except Exception:
return "cpu"
system = _platform.system()
if system == "Darwin" and _platform.processor() == "arm":
try:
if torch.backends.mps.is_available(): # type: ignore[union-attr]
return "mps"
except Exception:
pass
return "cpu"
try:
if torch.cuda.is_available(): # type: ignore[union-attr]
return "cuda"
except Exception:
pass
return "cpu"
-231
View File
@@ -1,231 +0,0 @@
"""Domain enums — typed constants for values tied to business logic.
Using Enum instead of bare strings ensures:
- Invalid values are caught at construction time
- IDE autocomplete and type checking work
- Adding new values is explicit (must update Enum)
"""
from __future__ import annotations
from enum import Enum
from pathlib import Path
class SubtitleMode(str, Enum):
"""Subtitle generation mode."""
DISABLED = "Disabled"
LINE = "Line"
SENTENCE = "Sentence"
SENTENCE_COMMA = "Sentence + Comma"
SENTENCE_HIGHLIGHT = "Sentence + Highlighting"
@classmethod
def from_str(cls, value: str) -> SubtitleMode:
"""Parse from user input: case-insensitive, strips whitespace."""
normalized = value.strip()
for member in cls:
if member.value.lower() == normalized.lower():
return member
raise ValueError(f"Invalid SubtitleMode: {value!r}. Valid: {[m.value for m in cls]}")
class OutputFormat(str, Enum):
"""Audio output format."""
WAV = "wav"
MP3 = "mp3"
FLAC = "flac"
OPUS = "opus"
M4B = "m4b"
@property
def dot_ext(self) -> str:
"""File extension with dot: '.wav', '.mp3', etc."""
return f".{self.value}"
@property
def is_lossless(self) -> bool:
"""True for lossless formats."""
return self in (self.WAV, self.FLAC)
@classmethod
def from_str(cls, value: str) -> OutputFormat:
"""Parse from user input: strips dot prefix, case-insensitive."""
normalized = value.strip().lstrip(".").lower()
for member in cls:
if member.value == normalized:
return member
raise ValueError(f"Invalid OutputFormat: {value!r}. Valid: {[m.value for m in cls]}")
class SaveMode(str, Enum):
"""Where to save the output file."""
SAVE_NEXT_TO_INPUT = "save_next_to_input"
SAVE_TO_DESKTOP = "save_to_desktop"
CHOOSE_OUTPUT_FOLDER = "choose_output_folder"
DEFAULT_OUTPUT = "default_output"
CUSTOM_FOLDER = "custom_folder"
class SubtitleFormat(str, Enum):
"""Subtitle file format."""
SRT = "srt"
ASS = "ass"
VTT = "vtt"
@property
def dot_ext(self) -> str:
"""File extension with dot: '.srt', '.ass'."""
return f".{self.value}"
@classmethod
def from_str(cls, value: str) -> SubtitleFormat:
"""Parse from user input: strips dot prefix, case-insensitive."""
normalized = value.strip().lstrip(".").lower()
for member in cls:
if member.value == normalized:
return member
raise ValueError(f"Invalid SubtitleFormat: {value!r}. Valid: {[m.value for m in cls]}")
class InputFormat(str, Enum):
"""Input file format."""
EPUB = "epub"
PDF = "pdf"
TXT = "txt"
MD = "md"
SRT = "srt"
ASS = "ass"
VTT = "vtt"
@property
def is_book(self) -> bool:
"""True for book/document formats (epub, pdf, txt, md)."""
return self in (self.EPUB, self.PDF, self.TXT, self.MD)
@property
def is_subtitle(self) -> bool:
"""True for subtitle formats (srt, ass, vtt)."""
return self in (self.SRT, self.ASS, self.VTT)
@property
def dot_ext(self) -> str:
"""File extension with dot: '.epub', '.srt', etc."""
return f".{self.value}"
@classmethod
def from_path(cls, path: Path) -> InputFormat:
"""Detect format from file path extension."""
suffix = path.suffix.lower().lstrip(".")
if suffix == "markdown":
return cls.MD
try:
return cls(suffix)
except ValueError:
raise ValueError(f"Unsupported input format: {path.suffix!r}. Supported: {[m.value for m in cls]}")
class Language(str, Enum):
"""TTS language code (ISO 639-1 with region where needed).
Each engine maps these to its own internal language identifiers.
Engines report which languages they support via ``supported_languages()``.
"""
EN_US = "en-US"
EN_GB = "en-GB"
ES = "es"
FR = "fr"
HI = "hi"
IT = "it"
JA = "ja"
PT_BR = "pt-BR"
ZH = "zh"
AR = "ar"
BG = "bg"
CS = "cs"
DA = "da"
DE = "de"
EL = "el"
ET = "et"
FI = "fi"
HR = "hr"
HU = "hu"
ID = "id"
KO = "ko"
LT = "lt"
LV = "lv"
NL = "nl"
PL = "pl"
RO = "ro"
RU = "ru"
SK = "sk"
SL = "sl"
SV = "sv"
TR = "tr"
UK = "uk"
VI = "vi"
@property
def display_name(self) -> str:
"""Human-readable language name."""
_names = {
"en-US": "American English",
"en-GB": "British English",
"es": "Spanish",
"fr": "French",
"hi": "Hindi",
"it": "Italian",
"ja": "Japanese",
"pt-BR": "Brazilian Portuguese",
"zh": "Mandarin Chinese",
"ar": "Arabic",
"bg": "Bulgarian",
"cs": "Czech",
"da": "Danish",
"de": "German",
"el": "Greek",
"et": "Estonian",
"fi": "Finnish",
"hr": "Croatian",
"hu": "Hungarian",
"id": "Indonesian",
"ko": "Korean",
"lt": "Lithuanian",
"lv": "Latvian",
"nl": "Dutch",
"pl": "Polish",
"ro": "Romanian",
"ru": "Russian",
"sk": "Slovak",
"sl": "Slovenian",
"sv": "Swedish",
"tr": "Turkish",
"uk": "Ukrainian",
"vi": "Vietnamese",
}
return _names[self.value]
@property
def is_cjk(self) -> bool:
"""True for CJK languages (Chinese, Japanese, Korean)."""
return self in (self.ZH, self.JA, self.KO)
@property
def supports_subtitle_tokens(self) -> bool:
"""True if this language supports subtitle generation.
All languages are supported: languages without per-word timestamped
tokens fall back to segment-level fake tokens in the pipeline.
"""
return True
@classmethod
def from_str(cls, value: str) -> Language:
"""Parse from user input: ISO code, case-insensitive."""
if isinstance(value, Language):
return value
normalized = value.strip()
for member in cls:
if member.value.lower() == normalized.lower():
return member
raise ValueError(f"Invalid Language: {value!r}. Valid: {[m.value for m in cls]}")
-136
View File
@@ -1,136 +0,0 @@
from __future__ import annotations
import re
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Tuple
from abogen.text_extractor import ExtractedChapter
_SIGNIFICANT_LENGTH_THRESHOLDS: Dict[str, int] = {"epub": 1000, "markdown": 500}
_MIN_SHORT_CONTENT: Dict[str, int] = {"epub": 240, "markdown": 160}
_STRUCTURAL_KEYWORDS = (
"preface",
"prologue",
"introduction",
"foreword",
"epilogue",
"afterword",
"appendix",
"acknowledgment",
"acknowledgement",
)
_STRUCTURAL_MIN_LENGTH = 120
_MAX_SHORT_CHAPTERS = 2
@dataclass
class ChapterFilterResult:
kept: List[ExtractedChapter]
skipped: List[Tuple[str, int]]
def infer_file_type(path: Path) -> str:
suffix = path.suffix.lower()
if suffix == ".epub":
return "epub"
if suffix in {".md", ".markdown"}:
return "markdown"
if suffix == ".pdf":
return "pdf"
if suffix == ".txt":
return "text"
return suffix.lstrip(".") or "text"
def looks_structural(title: str) -> bool:
lowered = title.strip().lower()
if not lowered:
return False
return any(keyword in lowered for keyword in _STRUCTURAL_KEYWORDS)
def chapter_label(file_type: str) -> str:
return "chapters" if file_type.lower() in {"epub", "markdown"} else "pages"
def auto_select_relevant_chapters(
chapters: List[ExtractedChapter],
file_type: str,
) -> ChapterFilterResult:
if not chapters:
return ChapterFilterResult(kept=[], skipped=[])
normalized = file_type.lower()
threshold = _SIGNIFICANT_LENGTH_THRESHOLDS.get(normalized, 0)
min_short = _MIN_SHORT_CONTENT.get(normalized, 0)
kept: List[ExtractedChapter] = []
skipped: List[Tuple[str, int]] = []
short_kept = 0
for chapter in chapters:
stripped = chapter.text.strip()
length = len(stripped)
if length == 0:
skipped.append((chapter.title, length))
continue
keep = False
if threshold == 0:
keep = True
elif length >= threshold:
keep = True
elif not kept:
keep = True
elif min_short and length >= min_short and short_kept < _MAX_SHORT_CHAPTERS:
keep = True
short_kept += 1
elif looks_structural(chapter.title) and length >= _STRUCTURAL_MIN_LENGTH:
keep = True
if keep:
kept.append(chapter)
else:
skipped.append((chapter.title, length))
if kept:
return ChapterFilterResult(kept=kept, skipped=skipped)
longest_idx = None
longest_length = 0
for idx, chapter in enumerate(chapters):
stripped = chapter.text.strip()
if stripped and len(stripped) > longest_length:
longest_length = len(stripped)
longest_idx = idx
if longest_idx is not None:
longest = chapters[longest_idx]
fallback_skipped = [
(chapter.title, len(chapter.text.strip()))
for idx, chapter in enumerate(chapters)
if idx != longest_idx and chapter.text.strip()
]
return ChapterFilterResult(kept=[longest], skipped=fallback_skipped)
return ChapterFilterResult(kept=[], skipped=skipped)
def update_metadata_for_chapter_count(
metadata: Dict[str, Any], count: int, file_type: str
) -> None:
if not metadata or count <= 0:
return
label = "Chapters" if file_type.lower() in {"epub", "markdown"} else "Pages"
metadata["chapter_count"] = str(count)
pattern = re.compile(r"\(\d+\s+(Chapters?|Pages?)\)")
replacement = f"({count} {label})"
for key in ("album", "ALBUM"):
value = metadata.get(key)
if not isinstance(value, str):
continue
metadata[key] = pattern.sub(replacement, value)
-83
View File
@@ -1,83 +0,0 @@
"""Intro/outro text building and voice resolution for audiobook conversion.
Both UIs (WebUI and Desktop) need to:
1. Build intro/outro text from book metadata
2. Resolve which voice to use for intro/outro synthesis
This module provides the shared domain logic. The actual TTS synthesis
and audio writing remain UI-specific.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any, Dict, Optional
from abogen.domain.title_builder import build_title_intro_text, build_outro_text
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
@dataclass
class IntroOutroSpec:
"""Resolved intro or outro specification ready for TTS synthesis."""
text: str
voice_spec: str
enabled: bool
def resolve_intro(
metadata: Optional[Dict[str, Any]],
original_filename: str,
read_title_intro: bool,
base_voice_spec: str,
job_voice: str,
voice_cache_keys: list[str],
) -> IntroOutroSpec:
"""Resolve the intro specification from job settings and metadata.
Returns an IntroOutroSpec with text and voice_spec populated,
or enabled=False if intro is disabled or text cannot be built.
"""
if not read_title_intro:
return IntroOutroSpec(text="", voice_spec="", enabled=False)
text = build_title_intro_text(metadata, original_filename)
if not text:
return IntroOutroSpec(text="", voice_spec="", enabled=False)
voice_spec = resolve_fallback_voice_spec(
base_voice_spec, job_voice, voice_cache_keys
)
if not voice_spec:
return IntroOutroSpec(text=text, voice_spec="", enabled=False)
return IntroOutroSpec(text=text, voice_spec=voice_spec, enabled=True)
def resolve_outro(
metadata: Optional[Dict[str, Any]],
original_filename: str,
read_closing_outro: bool,
base_voice_spec: str,
job_voice: str,
voice_cache_keys: list[str],
) -> IntroOutroSpec:
"""Resolve the outro specification from job settings and metadata.
Returns an IntroOutroSpec with text and voice_spec populated,
or enabled=False if outro is disabled or text cannot be built.
"""
if not read_closing_outro:
return IntroOutroSpec(text="", voice_spec="", enabled=False)
text = build_outro_text(metadata, original_filename)
if not text:
return IntroOutroSpec(text="", voice_spec="", enabled=False)
voice_spec = resolve_fallback_voice_spec(
base_voice_spec, job_voice, voice_cache_keys
)
if not voice_spec:
return IntroOutroSpec(text=text, voice_spec="", enabled=False)
return IntroOutroSpec(text=text, voice_spec=voice_spec, enabled=True)
-503
View File
@@ -1,503 +0,0 @@
"""Metadata extraction and processing utilities.
This module provides functions for extracting metadata from text content,
formatting metadata tags for TTS embedding, and generating ffmpeg metadata arguments.
"""
from __future__ import annotations
import datetime
import logging
import os
import re
import uuid
from typing import Any, Dict, List, Optional, Tuple
logger = logging.getLogger(__name__)
def extract_metadata_from_text(text: str) -> Dict[str, Optional[str]]:
"""Extract metadata tags from text content.
Looks for tags in format: <<METADATA_KEY:value>>
Supported tags:
- TITLE, ARTIST, ALBUM, YEAR
- ALBUM_ARTIST, COMPOSER, GENRE
- COVER_PATH
Args:
text: Text content to search for metadata tags.
Returns:
Dictionary with extracted metadata values (None if not found).
"""
metadata = {}
patterns = {
"title": r"<<METADATA_TITLE:([^>]*)>>",
"artist": r"<<METADATA_ARTIST:([^>]*)>>",
"album": r"<<METADATA_ALBUM:([^>]*)>>",
"year": r"<<METADATA_YEAR:([^>]*)>>",
"album_artist": r"<<METADATA_ALBUM_ARTIST:([^>]*)>>",
"composer": r"<<METADATA_COMPOSER:([^>]*)>>",
"genre": r"<<METADATA_GENRE:([^>]*)>>",
"cover_path": r"<<METADATA_COVER_PATH:([^>]*)>>",
}
for key, pattern in patterns.items():
match = re.search(pattern, text)
if match:
metadata[key] = match.group(1).strip()
else:
metadata[key] = None
return metadata
def get_filename_from_path(
file_path: str,
display_path: Optional[str] = None,
from_queue: bool = False,
) -> str:
"""Extract filename (without extension) from path.
Args:
file_path: The file path to extract from.
display_path: Optional display path (used if from_queue is False).
from_queue: Whether the file is from queue.
Returns:
Filename without extension.
"""
if from_queue:
base_path = file_path
else:
base_path = display_path if display_path else file_path
filename = os.path.splitext(os.path.basename(base_path))[0]
return filename
def build_ffmpeg_metadata_args(
metadata: Dict[str, Optional[str]],
filename: str,
) -> List[str]:
"""Build ffmpeg metadata arguments from metadata dictionary.
Args:
metadata: Dictionary with metadata keys and values.
filename: Fallback filename for title/album if not specified.
Returns:
List of ffmpeg metadata arguments.
"""
args = []
# Default values
defaults = {
"title": filename,
"artist": "Unknown",
"album": filename,
"date": str(datetime.datetime.now().year),
"album_artist": "Unknown",
"composer": "Narrator",
"genre": "Audiobook",
}
# Map of metadata keys to ffmpeg metadata keys
key_mapping = {
"title": "title",
"artist": "artist",
"album": "album",
"year": "date", # year -> date for ffmpeg
"album_artist": "album_artist",
"composer": "composer",
"genre": "genre",
}
for metadata_key, ffmpeg_key in key_mapping.items():
value = metadata.get(metadata_key)
if value is None:
value = defaults.get(metadata_key, "")
if value:
args.extend(["-metadata", f"{ffmpeg_key}={value}"])
return args
def extract_metadata_and_build_args(
text: str,
filename: str,
display_path: Optional[str] = None,
from_queue: bool = False,
) -> Tuple[List[str], Optional[str]]:
"""Extract metadata from text and build ffmpeg arguments.
Convenience function that combines extract_metadata_from_text and
build_ffmpeg_metadata_args.
Args:
text: Text content to search for metadata tags.
filename: Fallback filename for title/album.
display_path: Optional display path.
from_queue: Whether the file is from queue.
Returns:
Tuple of (ffmpeg_metadata_args, cover_path).
"""
metadata = extract_metadata_from_text(text)
cover_path = metadata.get("cover_path")
# Get actual filename from path
actual_filename = get_filename_from_path(
file_path=filename,
display_path=display_path,
from_queue=from_queue,
)
args = build_ffmpeg_metadata_args(metadata, actual_filename)
return args, cover_path
def read_text_for_metadata(
file_path: str,
is_direct_text: bool,
direct_text: Optional[str] = None,
encoding: Optional[str] = None,
) -> str:
"""Read text content for metadata extraction.
Args:
file_path: Path to file (or text if is_direct_text).
is_direct_text: Whether file_path contains direct text.
direct_text: Optional direct text (used if is_direct_text).
encoding: File encoding (detected if not provided).
Returns:
Text content for metadata extraction.
"""
if is_direct_text:
return direct_text or file_path
# Read from file
actual_path = direct_text if direct_text else file_path
try:
if encoding is None:
from abogen.utils import detect_encoding
encoding = detect_encoding(actual_path)
with open(actual_path, "r", encoding=encoding, errors="replace") as f:
return f.read()
except Exception:
return ""
def extract_metadata_for_file(
file_path: str,
is_direct_text: bool = False,
) -> Dict[str, Optional[str]]:
"""Extract metadata dict from a file or direct text.
Convenience function combining read_text_for_metadata + extract_metadata_from_text.
Returns empty dict on any error.
"""
try:
text = read_text_for_metadata(
file_path=file_path,
is_direct_text=is_direct_text,
direct_text=file_path if is_direct_text else None,
)
if text:
return extract_metadata_from_text(text) or {}
except Exception:
pass
return {}
def format_metadata_tags(
metadata: Dict[str, Any],
filename: str,
chapter_count: int,
file_type: str,
cover_bytes: Optional[bytes] = None,
cache_dir: Optional[str] = None,
) -> str:
"""Format metadata tags for insertion into TTS text.
Builds <<METADATA_KEY:value>> tags that are later parsed by
extract_metadata_from_text() and fed to ffmpeg.
Args:
metadata: Dict with keys like 'title', 'authors' (list),
'publication_year', 'description', 'cover_image' (bytes).
filename: Fallback filename (without extension) for title/album.
chapter_count: Number of chapters/pages.
file_type: 'epub', 'pdf', or 'markdown'.
cover_bytes: Optional cover image bytes to save to cache.
cache_dir: Directory for cover cache (uses default if None).
Returns:
Newline-joined string of <<METADATA_KEY:value>> tags.
"""
title = metadata.get("title") or filename
authors = metadata.get("authors") or ["Unknown"]
authors_text = ", ".join(authors) if isinstance(authors, list) else str(authors)
year = metadata.get("publication_year") or str(datetime.datetime.now().year)
chapter_label = "Chapters" if file_type in ("epub", "markdown") else "Pages"
chapter_text = f"{chapter_count} {chapter_label}"
tags = [
f"<<METADATA_TITLE:{title}>>",
f"<<METADATA_ARTIST:{authors_text}>>",
f"<<METADATA_ALBUM:{title} ({chapter_text})>>",
f"<<METADATA_YEAR:{year}>>",
f"<<METADATA_ALBUM_ARTIST:{authors_text}>>",
f"<<METADATA_COMPOSER:Narrator>>",
f"<<METADATA_GENRE:Audiobook>>",
]
cover_path = _save_cover_to_cache(cover_bytes, cache_dir)
if cover_path:
tags.append(f"<<METADATA_COVER_PATH:{cover_path}>>")
return "\n".join(tags)
def _save_cover_to_cache(
cover_bytes: Optional[bytes],
cache_dir: Optional[str] = None,
) -> Optional[str]:
"""Save cover image bytes to cache directory.
Args:
cover_bytes: Raw image bytes (e.g. JPEG/PNG).
cache_dir: Directory to save to. If None, returns None.
Returns:
Normalized path to saved cover file, or None on failure.
"""
if not cover_bytes:
return None
if cache_dir is None:
return None
try:
cover_path = os.path.join(cache_dir, f"cover_{uuid.uuid4()}.jpg")
cover_path = os.path.normpath(cover_path)
with open(cover_path, "wb") as f:
f.write(cover_bytes)
return cover_path
except Exception as e:
logger.warning("Failed to save cover image: %s", e)
return None
def extract_book_metadata_epub(book: Any) -> Dict[str, Any]:
"""Extract metadata from an opened ebooklib EPUB book.
Args:
book: An opened ebooklib EPUB book object.
Returns:
Dict with keys: title, authors, description, publisher,
publication_year, cover_image (bytes or None).
"""
import ebooklib
metadata: Dict[str, Any] = {
"title": None,
"authors": [],
"description": None,
"cover_image": None,
"publisher": None,
"publication_year": None,
}
try:
title_items = book.get_metadata("DC", "title")
if title_items and len(title_items) > 0:
metadata["title"] = title_items[0][0]
except Exception as e:
logger.warning("Error extracting title metadata: %s", e)
try:
author_items = book.get_metadata("DC", "creator")
if author_items:
metadata["authors"] = [
author[0] for author in author_items if len(author) > 0
]
except Exception as e:
logger.warning("Error extracting author metadata: %s", e)
try:
desc_items = book.get_metadata("DC", "description")
if desc_items and len(desc_items) > 0:
metadata["description"] = desc_items[0][0]
except Exception as e:
logger.warning("Error extracting description metadata: %s", e)
try:
publisher_items = book.get_metadata("DC", "publisher")
if publisher_items and len(publisher_items) > 0:
metadata["publisher"] = publisher_items[0][0]
except Exception as e:
logger.warning("Error extracting publisher metadata: %s", e)
try:
date_items = book.get_metadata("DC", "date")
if date_items and len(date_items) > 0:
date_str = date_items[0][0]
year_match = re.search(r"\b(19|20)\d{2}\b", date_str)
if year_match:
metadata["publication_year"] = year_match.group(0)
else:
metadata["publication_year"] = date_str
except Exception as e:
logger.warning("Error extracting publication date metadata: %s", e)
for item in book.get_items_of_type(ebooklib.ITEM_COVER):
metadata["cover_image"] = item.get_content()
break
if not metadata["cover_image"]:
for item in book.get_items_of_type(ebooklib.ITEM_IMAGE):
if "cover" in item.get_name().lower():
metadata["cover_image"] = item.get_content()
break
return metadata
def extract_book_metadata_pdf(pdf_doc: Any) -> Dict[str, Any]:
"""Extract metadata from an opened PyMuPDF document.
Args:
pdf_doc: An opened fitz.Document object.
Returns:
Dict with keys: title, authors, description, publisher,
publication_year, cover_image (bytes or None).
"""
metadata: Dict[str, Any] = {
"title": None,
"authors": [],
"description": None,
"cover_image": None,
"publisher": None,
"publication_year": None,
}
pdf_info = pdf_doc.metadata
if pdf_info:
metadata["title"] = pdf_info.get("title", None)
author = pdf_info.get("author", None)
if author:
metadata["authors"] = [author]
metadata["description"] = pdf_info.get("subject", None)
keywords = pdf_info.get("keywords", None)
if keywords:
if metadata["description"]:
metadata["description"] += f"\n\nKeywords: {keywords}"
else:
metadata["description"] = f"Keywords: {keywords}"
metadata["publisher"] = pdf_info.get("creator", None)
if "creationDate" in pdf_info:
date_str = pdf_info["creationDate"]
year_match = re.search(r"D:(\d{4})", date_str)
if year_match:
metadata["publication_year"] = year_match.group(1)
elif "modDate" in pdf_info:
date_str = pdf_info["modDate"]
year_match = re.search(r"D:(\d{4})", date_str)
if year_match:
metadata["publication_year"] = year_match.group(1)
if len(pdf_doc) > 0:
try:
import fitz
pix = pdf_doc[0].get_pixmap(matrix=fitz.Matrix(2, 2))
metadata["cover_image"] = pix.tobytes("png")
except Exception:
pass
return metadata
def extract_book_metadata_markdown(
markdown_text: str,
markdown_toc: Optional[List[Dict[str, Any]]] = None,
) -> Dict[str, Any]:
"""Extract metadata from markdown frontmatter and first heading.
Args:
markdown_text: Raw markdown text content.
markdown_toc: Optional table of contents list (each item has
'level' and 'name' keys).
Returns:
Dict with keys: title, authors, description, publication_year.
cover_image is always None for markdown.
"""
metadata: Dict[str, Any] = {
"title": None,
"authors": [],
"description": None,
"cover_image": None,
"publisher": None,
"publication_year": None,
}
if not markdown_text:
return metadata
frontmatter_match = re.match(
r"^---\s*\n(.*?)\n---\s*\n", markdown_text, re.DOTALL
)
if frontmatter_match:
try:
frontmatter = frontmatter_match.group(1)
title_match = re.search(
r"^title:\s*(.+)$", frontmatter, re.MULTILINE | re.IGNORECASE
)
if title_match:
metadata["title"] = title_match.group(1).strip().strip("\"'")
author_match = re.search(
r"^author:\s*(.+)$", frontmatter, re.MULTILINE | re.IGNORECASE
)
if author_match:
metadata["authors"] = [
author_match.group(1).strip().strip("\"'")
]
desc_match = re.search(
r"^description:\s*(.+)$", frontmatter, re.MULTILINE | re.IGNORECASE
)
if desc_match:
metadata["description"] = (
desc_match.group(1).strip().strip("\"'")
)
date_match = re.search(
r"^date:\s*(.+)$", frontmatter, re.MULTILINE | re.IGNORECASE
)
if date_match:
date_str = date_match.group(1).strip().strip("\"'")
year_match = re.search(r"\b(19|20)\d{2}\b", date_str)
if year_match:
metadata["publication_year"] = year_match.group(0)
except Exception as e:
logger.warning("Error parsing markdown frontmatter: %s", e)
if not metadata["title"] and markdown_toc:
first_h1 = next(
(h for h in markdown_toc if h.get("level") == 1), None
)
if first_h1:
metadata["title"] = first_h1.get("name")
return metadata
-496
View File
@@ -1,496 +0,0 @@
from __future__ import annotations
import json
import math
import re
from pathlib import Path
from typing import Any, Dict, List, Mapping, Optional, Tuple
_SERIES_NAME_KEYS = (
"series",
"series_name",
"series_title",
)
_SERIES_NUMBER_KEYS = (
"series_index",
"series_position",
"series_sequence",
"book_number",
"series_number",
)
_SERIES_NUMBER_RE = re.compile(r"\d+(?:\.\d+)?")
_SERIES_NAME_ALIASES = ("series", "series_name", "seriesname", "series_title", "seriestitle")
_SERIES_INDEX_ALIASES = ("series_index", "series_sequence", "series_position", "book_number")
_AUTHOR_ALIASES = ("author", "authors")
_DESCRIPTION_ALIASES = ("description", "summary")
_TAGS_ALIASES = ("tags", "keywords", "genre")
def expand_metadata_aliases(tags: Mapping[str, Any]) -> Dict[str, Any]:
"""Expand concept aliases so each concept has all canonical keys set.
One input concept fans out to multiple keys so that downstream consumers
can look up any variant and find the value.
Expanded concepts:
series -> series, series_name, seriesname, series_title, seriestitle
series_index -> series_index, series_sequence, series_position, book_number
author -> author, authors
description -> description, summary
tags -> tags, keywords, genre
"""
if not tags:
return {}
result: Dict[str, Any] = {}
for key, value in tags.items():
if value is None:
continue
text = str(value).strip() if not isinstance(value, (list, tuple, set)) else value
if not text:
continue
key_lower = str(key).strip().lower()
if not key_lower:
continue
if key_lower in _SERIES_NAME_ALIASES:
for alias in _SERIES_NAME_ALIASES:
result[alias] = text
elif key_lower in _SERIES_INDEX_ALIASES:
for alias in _SERIES_INDEX_ALIASES:
result[alias] = text
elif key_lower in _AUTHOR_ALIASES:
for alias in _AUTHOR_ALIASES:
result[alias] = text
elif key_lower in _DESCRIPTION_ALIASES:
for alias in _DESCRIPTION_ALIASES:
result[alias] = text
elif key_lower in _TAGS_ALIASES:
for alias in _TAGS_ALIASES:
result[alias] = text
else:
result[key_lower] = text
return result
def normalize_metadata_map(values: Optional[Mapping[str, Any]]) -> Dict[str, str]:
normalized: Dict[str, str] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
text = str(value).strip()
if not text:
continue
normalized[str(key).casefold()] = text
return normalized
def format_author_sentence(raw: Optional[str]) -> str:
if raw is None:
return ""
normalized = str(raw).strip()
if not normalized:
return ""
lowered = normalized.casefold()
if lowered in {"unknown", "various"}:
return ""
working = normalized.replace("&", " and ")
segments = [segment.strip() for segment in working.split(",") if segment.strip()]
tokens: List[str] = []
if segments:
for segment in segments:
parts = [part.strip() for part in re.split(r"\band\b", segment, flags=re.IGNORECASE) if part.strip()]
if parts:
tokens.extend(parts)
else:
tokens.append(segment)
else:
parts = [part.strip() for part in re.split(r"\band\b", working, flags=re.IGNORECASE) if part.strip()]
tokens.extend(parts or [normalized])
cleaned = [token for token in tokens if token and token.casefold() not in {"unknown", "various"}]
if not cleaned:
return ""
if len(cleaned) == 1:
return f"By {cleaned[0]}"
if len(cleaned) == 2:
return f"By {cleaned[0]} and {cleaned[1]}"
return f"By {', '.join(cleaned[:-1])}, and {cleaned[-1]}"
def ensure_sentence(text: str) -> str:
cleaned = text.strip()
if not cleaned:
return ""
if cleaned[-1] in ".!?":
return cleaned
return f"{cleaned}."
def normalize_series_number(value: Any) -> Optional[str]:
text = str(value or "").strip()
if not text:
return None
candidate = text.replace(",", ".")
if candidate.replace(".", "", 1).isdigit():
if "." in candidate:
normalized = candidate.rstrip("0").rstrip(".")
return normalized or "0"
try:
return str(int(candidate))
except ValueError:
pass
match = _SERIES_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
return normalized or "0"
try:
return str(int(normalized))
except ValueError:
return normalized
def extract_series_metadata(values: Mapping[str, str]) -> Tuple[Optional[str], Optional[str]]:
series_name: Optional[str] = None
for key in _SERIES_NAME_KEYS:
raw = values.get(key)
if raw:
cleaned = str(raw).strip()
if cleaned:
series_name = cleaned
break
series_number: Optional[str] = None
for key in _SERIES_NUMBER_KEYS:
raw = values.get(key)
if raw is None:
continue
normalized = normalize_series_number(raw)
if normalized:
series_number = normalized
break
return series_name, series_number
def format_series_sentence(series_name: Optional[str], series_number: Optional[str]) -> str:
if not series_name or not series_number:
return ""
name = series_name.strip()
number = series_number.strip()
if not name or not number:
return ""
article = "the " if not name.lower().startswith("the ") else ""
phrase = f"Book {number} of {article}{name}"
return re.sub(r"\s+", " ", phrase).strip()
_PEOPLE_SPLIT_RE = re.compile(r"[;,/&]|\band\b", re.IGNORECASE)
_LIST_SPLIT_RE = re.compile(r"[;,\n]")
_SERIES_SEQUENCE_TAG_KEYS: Tuple[str, ...] = (
"series_index",
"series_position",
"series_sequence",
"series_number",
"seriesnumber",
"book_number",
"booknumber",
)
def normalize_metadata_casefold(values: Optional[Mapping[str, Any]]) -> Dict[str, Any]:
normalized: Dict[str, Any] = {}
if not values:
return normalized
for key, value in values.items():
if value is None:
continue
key_text = str(key).strip().lower()
if not key_text:
continue
if isinstance(value, (list, tuple, set)):
normalized[key_text] = value
else:
text = str(value).strip()
if text:
normalized[key_text] = text
return normalized
def split_people_field(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(split_people_field(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _PEOPLE_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def split_simple_list(raw: Any) -> List[str]:
if raw is None:
return []
if isinstance(raw, (list, tuple, set)):
results: List[str] = []
for item in raw:
results.extend(split_simple_list(item))
return results
text = str(raw or "").strip()
if not text:
return []
tokens = [_token.strip() for _token in _LIST_SPLIT_RE.split(text) if _token.strip()]
seen: set[str] = set()
ordered: List[str] = []
for token in tokens:
key = token.casefold()
if key in seen:
continue
seen.add(key)
ordered.append(token)
return ordered
def first_nonempty(*values: Any) -> Optional[str]:
for value in values:
if value is None:
continue
if isinstance(value, (list, tuple, set)):
items = list(value)
if not items:
continue
value = items[0]
text = str(value).strip()
if text:
return text
return None
def extract_year(raw: Optional[str]) -> Optional[int]:
if not raw:
return None
text = str(raw).strip()
if not text:
return None
match = re.search(r"(19|20)\d{2}", text)
if match:
try:
return int(match.group(0))
except ValueError:
return None
try:
parsed = int(text)
except ValueError:
return None
if 0 < parsed < 3000:
return parsed
return None
def normalize_series_sequence(raw: Any) -> Optional[str]:
if raw is None:
return None
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return None
text = str(raw)
else:
text = str(raw).strip()
if not text:
return None
candidate = text.replace(",", ".")
match = _SERIES_NUMBER_RE.search(candidate)
if not match:
return None
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
def build_audiobookshelf_metadata(
tags: Mapping[str, Any],
*,
language: str = "",
filename: str = "",
) -> Dict[str, Any]:
normalized = normalize_metadata_casefold(tags)
title = first_nonempty(
normalized.get("title"),
normalized.get("book_title"),
normalized.get("name"),
normalized.get("album"),
filename,
)
authors = split_people_field(
normalized.get("authors")
or normalized.get("author")
or normalized.get("album_artist")
or normalized.get("artist")
)
narrators = split_people_field(normalized.get("narrators") or normalized.get("narrator"))
description = first_nonempty(
normalized.get("description"), normalized.get("summary"), normalized.get("comment")
)
genres = split_simple_list(normalized.get("genre"))
keywords = split_simple_list(normalized.get("tags") or normalized.get("keywords"))
lang = first_nonempty(normalized.get("language"), normalized.get("lang")) or language or ""
series_name = first_nonempty(
normalized.get("series"),
normalized.get("series_name"),
normalized.get("seriesname"),
normalized.get("series_title"),
normalized.get("seriestitle"),
)
series_sequence = None
for key in _SERIES_SEQUENCE_TAG_KEYS:
raw_value = normalized.get(key)
seq = normalize_series_sequence(raw_value)
if seq:
series_sequence = seq
break
if not series_name:
series_sequence = None
data: Dict[str, Any] = {
"title": title,
"subtitle": normalized.get("subtitle"),
"authors": authors,
"narrators": narrators,
"description": description,
"publisher": normalized.get("publisher"),
"genres": genres,
"tags": keywords,
"language": lang,
"publishedYear": extract_year(
normalized.get("published")
or normalized.get("publication_year")
or normalized.get("date")
or normalized.get("year")
),
"seriesName": series_name,
"seriesSequence": series_sequence,
"isbn": first_nonempty(normalized.get("isbn"), normalized.get("asin")),
}
published_date = first_nonempty(
normalized.get("published"), normalized.get("publication_date"), normalized.get("date")
)
if published_date:
data["publishedDate"] = published_date
rating_text = first_nonempty(normalized.get("rating"), normalized.get("my_rating"))
if rating_text:
try:
data["rating"] = float(str(rating_text).strip())
except ValueError:
pass
rating_max_text = first_nonempty(
normalized.get("rating_max"), normalized.get("rating_scale")
)
if rating_max_text:
try:
data["ratingMax"] = float(str(rating_max_text).strip())
except ValueError:
pass
cleaned: Dict[str, Any] = {}
for key, value in data.items():
if value is None:
continue
if isinstance(value, str) and not value.strip():
continue
if isinstance(value, (list, tuple)) and not value:
continue
cleaned[key] = value
return cleaned
def load_audiobookshelf_chapters(
metadata_path: Path,
) -> Optional[List[Dict[str, Any]]]:
if not metadata_path.exists():
return None
try:
payload = json.loads(metadata_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
chapters = payload.get("chapters")
if not isinstance(chapters, list):
return None
cleaned: List[Dict[str, Any]] = []
for entry in chapters:
if not isinstance(entry, Mapping):
continue
title = first_nonempty(entry.get("title"), entry.get("original_title"))
start = entry.get("start")
end = entry.get("end")
if title and start is not None and end is not None:
cleaned.append({"title": str(title), "start": start, "end": end})
return cleaned or None
def build_metadata_payload(
metadata: Optional[Dict[str, Any]] = None,
chapter_markers: Optional[List[Dict[str, Any]]] = None,
chunk_markers: Optional[List[Dict[str, Any]]] = None,
chunk_level: Optional[str] = None,
speaker_mode: Optional[str] = None,
speakers: Optional[Dict[str, Any]] = None,
generate_epub3: bool = False,
) -> Dict[str, Any]:
"""Build the canonical metadata payload dict for persistence and downstream use.
This is the single source of truth for metadata assembly. Both PyQt and WebUI
runners should call this instead of building the dict manually.
Args:
metadata: Normalized metadata tags dict.
chapter_markers: List of chapter marker dicts with title/start/end.
chunk_markers: List of chunk marker dicts.
chunk_level: Chunk granularity level (e.g. 'chapter', 'chunk').
speaker_mode: Speaker mode ('single', 'multi', etc.).
speakers: Speaker profile mapping.
generate_epub3: Whether EPUB3 generation is enabled.
Returns:
Complete metadata payload dict.
"""
return {
"metadata": dict(metadata or {}),
"chapters": chapter_markers or [],
"chunks": chunk_markers or [],
"chunk_level": chunk_level,
"speaker_mode": speaker_mode,
"speakers": dict(speakers or {}),
"generate_epub3": generate_epub3,
}
-23
View File
@@ -1,23 +0,0 @@
from __future__ import annotations
from typing import Any, Dict, Optional
def merge_metadata(
extracted: Optional[Dict[str, Any]],
overrides: Optional[Dict[str, Any]],
) -> Dict[str, str]:
merged: Dict[str, str] = {}
if extracted:
for key, value in extracted.items():
if value is None:
continue
merged[str(key)] = str(value)
if overrides:
for key, value in overrides.items():
key_str = str(key)
if value is None:
merged.pop(key_str, None)
else:
merged[key_str] = str(value)
return merged
-56
View File
@@ -1,56 +0,0 @@
"""OPDS metadata normalization.
Normalizes metadata keys from various OPDS/Calibre sources into
a canonical set of overrides for the audiobook conversion pipeline.
"""
from __future__ import annotations
from typing import Any, Dict, Mapping
from abogen.domain.metadata_helpers import expand_metadata_aliases
def normalize_opds_metadata(metadata_payload: Mapping[str, Any]) -> Dict[str, Any]:
"""Normalize OPDS/Calibre metadata into canonical override keys.
Takes a metadata payload with various key aliases (e.g. 'series'/'series_name',
'tags'/'keywords', 'authors'/'creator') and returns a dict with all
concept aliases expanded.
Args:
metadata_payload: Raw metadata dict from OPDS/Calibre import.
Returns:
Dict with all canonical metadata key aliases expanded.
"""
def _stringify(value: Any) -> str:
if value is None:
return ""
if isinstance(value, (list, tuple, set)):
parts = [str(item).strip() for item in value if item is not None]
return ", ".join(part for part in parts if part)
return str(value).strip()
# Map OPDS-specific keys to common concept keys before expansion
normalized_input: Dict[str, Any] = {}
for key, value in metadata_payload.items():
if value is None:
continue
key_lower = str(key).strip().lower()
if not key_lower:
continue
text = _stringify(value)
if not text:
continue
# Map OPDS-specific author aliases
if key_lower in ("creator", "dc_creator"):
normalized_input["author"] = text
# Map OPDS-specific subtitle aliases
elif key_lower in ("sub_title", "calibre_subtitle"):
normalized_input["subtitle"] = text
else:
normalized_input[key_lower] = text
return expand_metadata_aliases(normalized_input)
-245
View File
@@ -1,245 +0,0 @@
"""Text normalization convenience helpers.
Provides both the simple ``normalize_text_for_pipeline`` (apostrophe + LLM only)
and the comprehensive ``prepare_text_for_tts`` that chains all three normalization
stages used during conversion: heteronym rules → pronunciation rules → pipeline
normalization. The latter is the single entry point that both the Web UI and
PyQt Desktop GUI should use.
Also provides ``TTSContext`` — a dataclass bundling all pre-compiled normalization
resources so they can be created once and passed as a single object.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, List, Mapping, Optional
from abogen.domain.enums import Language
from abogen.kokoro_text_normalization import (
ApostropheConfig,
normalize_for_pipeline as _normalize_for_pipeline,
)
from abogen.normalization_settings import (
build_apostrophe_config,
get_runtime_settings,
apply_overrides as _apply_overrides,
)
_BASE_APOSTROPHE_CONFIG = ApostropheConfig()
@dataclass
class TTSContext:
"""Bundles pre-compiled normalization resources for TTS processing.
Created once per conversion job and passed to ``prepare_text_for_tts``
instead of threading 5 separate parameters.
"""
split_pattern: str = r"(?<=[.!?\-])\s+"
pronunciation_rules: Optional[List[Dict[str, Any]]] = None
heteronym_rules: Optional[List[Dict[str, Any]]] = None
normalization_overrides: Optional[Mapping[str, Any]] = None
usage_counter: Dict[str, int] = field(default_factory=dict)
def normalize(self, text: str) -> str:
"""Shorthand: normalize text using this context's compiled rules."""
return prepare_text_for_tts(
text,
heteronym_rules=self.heteronym_rules,
pronunciation_rules=self.pronunciation_rules,
normalization_overrides=self.normalization_overrides,
usage_counter=self.usage_counter,
)
def normalize_text_for_pipeline(
text: str,
*,
normalization_overrides: Optional[Mapping[str, Any]] = None,
) -> str:
"""Normalize text using runtime settings with optional overrides."""
runtime_settings = get_runtime_settings()
if normalization_overrides:
runtime_settings = _apply_overrides(runtime_settings, normalization_overrides)
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
return _normalize_for_pipeline(text, config=apostrophe_config, settings=runtime_settings)
def prepare_text_for_tts(
text: str,
*,
heteronym_rules: Optional[List[Dict[str, Any]]] = None,
pronunciation_rules: Optional[List[Dict[str, Any]]] = None,
normalization_overrides: Optional[Mapping[str, Any]] = None,
usage_counter: Optional[Dict[str, int]] = None,
) -> str:
"""Apply the full text normalization pipeline before TTS synthesis.
Chains three stages in order:
1. Heteronym sentence rules (context-dependent pronunciation)
2. Pronunciation rules (token-level replacements)
3. Pipeline normalization (apostrophe handling, LLM normalization)
This is the **single entry point** that both the Web UI conversion runner
and the PyQt conversion thread should call before passing text to the TTS
backend.
Parameters
----------
text:
Raw text to normalize.
heteronym_rules:
Compiled heteronym rules from ``compile_heteronym_sentence_rules``.
pronunciation_rules:
Compiled pronunciation rules from ``compile_pronunciation_rules``.
normalization_overrides:
User-level overrides for normalization settings (apostrophe mode, etc.).
usage_counter:
Mutable dict that tracks how many times each pronunciation override was
applied. Passed through to ``apply_pronunciation_rules``.
Returns
-------
str
Fully normalized text ready for TTS.
"""
from abogen.domain.pronunciation import (
apply_heteronym_sentence_rules,
apply_pronunciation_rules,
)
result = str(text or "")
if heteronym_rules:
result = apply_heteronym_sentence_rules(result, heteronym_rules)
if pronunciation_rules:
result = apply_pronunciation_rules(result, pronunciation_rules, usage_counter)
runtime_settings = get_runtime_settings()
if normalization_overrides:
runtime_settings = _apply_overrides(runtime_settings, normalization_overrides)
apostrophe_config = build_apostrophe_config(settings=runtime_settings, base=_BASE_APOSTROPHE_CONFIG)
return _normalize_for_pipeline(result, config=apostrophe_config, settings=runtime_settings)
def build_tts_context(
*,
language: Language,
subtitle: "SubtitleConfig | str" = "Disabled",
pronunciation: Optional["PronunciationConfig"] = None,
speakers: Optional[Dict[str, Any]] = None,
usage_counter: Optional[Dict[str, int]] = None,
log_callback: Optional[Callable[[str, str], None]] = None,
) -> TTSContext:
"""Build a TTSContext from raw data. Single entry point for both UIs.
Loads normalization settings, applies overrides, validates configuration,
merges pronunciation overrides, and compiles all rules.
Args:
language: Language enum value.
subtitle: SubtitleConfig object or subtitle mode string.
pronunciation: PronunciationConfig with override rules.
speakers: Speaker profile mapping.
usage_counter: Mutable dict for tracking override usage.
log_callback: Callable(level, message) for warnings.
Returns:
TTSContext ready for text normalization.
"""
from abogen.domain.config_types import PronunciationConfig, SubtitleConfig
from abogen.domain.enums import SubtitleMode
from abogen.domain.pronunciation import (
compile_heteronym_sentence_rules,
compile_pronunciation_rules,
merge_pronunciation_overrides,
)
from abogen.domain.split_pattern import get_split_pattern
def _log(msg: str, level: str = "warning") -> None:
if log_callback:
log_callback(level, msg)
# Resolve subtitle mode
if isinstance(subtitle, SubtitleConfig):
resolved_subtitle = subtitle.mode
else:
try:
resolved_subtitle = SubtitleMode.from_str(subtitle) if not isinstance(subtitle, SubtitleMode) else subtitle
except ValueError:
resolved_subtitle = SubtitleMode.DISABLED
# Resolve pronunciation config
if pronunciation is None:
pronunciation = PronunciationConfig()
# Get runtime normalization settings
runtime_settings = get_runtime_settings()
# Apply per-job normalization overrides
if pronunciation.normalization_overrides:
runtime_settings = _apply_overrides(runtime_settings, pronunciation.normalization_overrides)
# Build apostrophe config
apostrophe_config = build_apostrophe_config(settings=runtime_settings)
# Validate LLM apostrophe mode
apostrophe_mode = str(runtime_settings.get("normalization_apostrophe_mode", "spacy")).lower()
if apostrophe_mode == "llm":
from abogen.normalization_settings import build_llm_configuration
llm_config = build_llm_configuration(runtime_settings)
if not llm_config.is_configured():
raise RuntimeError(
"LLM-based apostrophe normalization is selected, but the LLM configuration is incomplete."
)
# Check for num2words availability
if apostrophe_config.convert_numbers:
try:
import num2words # noqa: F401
except ImportError:
_log(
"Number normalization is enabled but 'num2words' library is not available. "
"Numbers will NOT be converted to words."
)
# Compute split pattern
if not isinstance(language, Language):
raise TypeError(f"language must be Language enum, got {type(language).__name__}: {language!r}")
split_pattern = get_split_pattern(language, resolved_subtitle)
# Merge pronunciation overrides
source = {
"pronunciation_overrides": pronunciation.pronunciation_overrides,
"manual_overrides": pronunciation.manual_overrides,
"speakers": speakers or {},
"language": language,
}
merged_overrides = merge_pronunciation_overrides(source)
# Compile rules
pronunciation_rules = compile_pronunciation_rules(merged_overrides)
heteronym_rules = compile_heteronym_sentence_rules(pronunciation.heteronym_overrides)
if heteronym_rules:
_log(
f"Applying {len(heteronym_rules)} heteronym override(s) during conversion.",
level="debug",
)
if pronunciation_rules:
_log(
f"Applying {len(pronunciation_rules)} pronunciation override(s) during conversion.",
level="debug",
)
return TTSContext(
split_pattern=split_pattern,
pronunciation_rules=pronunciation_rules,
heteronym_rules=heteronym_rules,
normalization_overrides=pronunciation.normalization_overrides,
usage_counter=usage_counter if usage_counter is not None else {},
)
-226
View File
@@ -1,226 +0,0 @@
"""Output path resolution utilities.
Pure functions for resolving output directories, building file paths,
and computing project folder layouts.
"""
from __future__ import annotations
import os
import platform
import re
from datetime import datetime
from pathlib import Path
from typing import Callable, List, Optional, Tuple
from abogen.text_extractor import ExtractedChapter
_OUTPUT_SANITIZE_RE = re.compile(r"[^\w\-_.]+")
# OS-specific illegal characters for filenames
_WINDOWS_ILLEGAL_CHARS_RE = re.compile(r'[<>:"/\\|?*\x00-\x1f]')
_MACOS_ILLEGAL_CHARS_RE = re.compile(r"[:]")
_LINUX_ILLEGAL_CHARS_RE = re.compile(r"[/\x00]")
_CONTROL_CHARS_RE = re.compile(r"[\x00-\x1f]")
_UNIX_CONTROL_CHARS_RE = re.compile(r'[\x00-\x1f]')
_RESERVED_NAMES = frozenset(
{"CON", "PRN", "AUX", "NUL"}
| {f"COM{i}" for i in range(1, 10)}
| {f"LPT{i}" for i in range(1, 10)}
)
def sanitize_name_for_os(name: str, is_folder: bool = True) -> str:
"""Sanitize a filename or folder name based on the operating system.
Args:
name: The name to sanitize
is_folder: Whether this is a folder name (default: True)
Returns:
Sanitized name safe for the current OS
"""
if not name:
return "audiobook"
system = platform.system()
if system == "Windows":
sanitized = _WINDOWS_ILLEGAL_CHARS_RE.sub("_", name)
sanitized = _CONTROL_CHARS_RE.sub("_", sanitized)
sanitized = sanitized.rstrip(". ")
if sanitized.upper() in _RESERVED_NAMES or sanitized.upper().split(".")[0] in _RESERVED_NAMES:
sanitized = f"_{sanitized}"
elif system == "Darwin":
sanitized = _MACOS_ILLEGAL_CHARS_RE.sub("_", name)
sanitized = _CONTROL_CHARS_RE.sub("_", sanitized)
if is_folder and sanitized.startswith("."):
sanitized = "_" + sanitized[1:]
else:
sanitized = _LINUX_ILLEGAL_CHARS_RE.sub("_", name)
sanitized = _UNIX_CONTROL_CHARS_RE.sub("_", sanitized)
if is_folder and sanitized.startswith("."):
sanitized = "_" + sanitized[1:]
if not sanitized or sanitized.strip() == "":
sanitized = "audiobook"
if len(sanitized) > 255:
sanitized = sanitized[:255].rstrip(". ")
return sanitized
def slugify(title: str, index: int) -> str:
sanitized = re.sub(r"[^\w\-]+", "_", title.lower()).strip("_")
if not sanitized:
sanitized = f"chapter_{index:02d}"
return sanitized[:80]
def sanitize_filename_for_chapter(title: str, index: int, max_len: int = 80) -> str:
"""Sanitize a chapter name for use as a filename component.
Combines character sanitization, OS safety, and smart truncation
at word boundaries. Prepends zero-padded index prefix.
Args:
title: Raw chapter title.
index: 1-based chapter number for prefix.
max_len: Maximum length of the sanitized portion (excluding prefix).
Returns:
Sanitized string like "01_the_beginning".
"""
# Remove non-word/non-space/non-hyphen chars, then collapse spaces/hyphens
sanitized = re.sub(r"[^\w\s\-]", "", title)
sanitized = re.sub(r"[\s\-]+", "_", sanitized).strip("_")
if not sanitized:
sanitized = f"chapter_{index:02d}"
# OS-specific sanitization
system = platform.system()
if system == "Windows":
sanitized = _WINDOWS_ILLEGAL_CHARS_RE.sub("_", sanitized)
sanitized = sanitized.rstrip(". ")
base = sanitized.split(".")[0].upper()
if base in _RESERVED_NAMES:
sanitized = f"_{sanitized}"
# Linux: only NUL is truly illegal, but control chars are problematic
sanitized = _UNIX_CONTROL_CHARS_RE.sub("_", sanitized)
# Smart truncation at word boundary
if len(sanitized) > max_len:
pos = sanitized[:max_len].rfind("_")
sanitized = sanitized[: pos if pos > 0 else max_len].rstrip("_")
return f"{index:02d}_{sanitized}"
def sanitize_output_stem(name: str, index: int = 0) -> str:
base = Path(name or "").stem
sanitized = _OUTPUT_SANITIZE_RE.sub("_", base).strip("_")
return sanitized or "output"
def output_timestamp_token() -> str:
return datetime.now().strftime("%Y%m%d-%H%M%S")
def build_output_path(directory: Path, original_name: str, extension: str) -> Path:
sanitized = sanitize_output_stem(original_name)
return directory / f"{sanitized}.{extension}"
def apply_newline_policy(chapters: List[ExtractedChapter], replace_single_newlines: bool) -> None:
if not replace_single_newlines:
return
newline_regex = re.compile(r"(?<!\n)\n(?!\n)")
for chapter in chapters:
chapter.text = newline_regex.sub(" ", chapter.text)
from abogen.domain.enums import SaveMode
def resolve_output_directory(
*,
save_mode: str,
stored_path: Path,
output_folder: Optional[str],
desktop_dir: Optional[Path],
user_output_path: Optional[Path],
user_cache_outputs: Optional[Path],
) -> Path:
if save_mode in (SaveMode.SAVE_TO_DESKTOP, "Save to Desktop") and desktop_dir:
return desktop_dir
if save_mode in (SaveMode.SAVE_NEXT_TO_INPUT, "Save next to input file"):
return stored_path.parent
if save_mode in (SaveMode.CHOOSE_OUTPUT_FOLDER, "Choose output folder") and output_folder:
return Path(output_folder)
if save_mode in (SaveMode.DEFAULT_OUTPUT, "Use default save location") and user_output_path:
return user_output_path
return user_cache_outputs or Path(".")
def resolve_project_layout(
*,
original_filename: str,
save_as_project: bool,
base_dir: Path,
timestamp_fn: Callable[[], str] = output_timestamp_token,
sanitize_fn: Callable[[str, int], str] = sanitize_output_stem,
) -> Tuple[Path, Path, Path, Optional[Path]]:
sanitized = sanitize_fn(original_filename, 0)
folder_name = f"{timestamp_fn()}_{sanitized}"
project_root = base_dir / folder_name
project_root.mkdir(parents=True, exist_ok=True)
if save_as_project:
audio_dir = project_root / "audio"
subtitle_dir = project_root / "subtitles"
metadata_dir = project_root / "metadata"
for directory in (audio_dir, subtitle_dir, metadata_dir):
directory.mkdir(parents=True, exist_ok=True)
return project_root, audio_dir, subtitle_dir, metadata_dir
return project_root, project_root, project_root, None
def resolve_unique_path(
parent_dir: str,
base_name: str,
extension: str,
allowed_extensions: Optional[set] = None,
) -> str:
"""Find a unique file path by appending _2, _3, etc. on collision.
Args:
parent_dir: Directory to check for collisions.
base_name: Base filename (without extension).
extension: File extension (without dot).
allowed_extensions: Set of extensions to check against.
If None, checks any existing file/dir with same name.
Returns:
Full path without extension (e.g. "/path/to/name_2").
"""
sanitized = sanitize_name_for_os(base_name, is_folder=True)
counter = 1
while True:
suffix = f"_{counter}" if counter > 1 else ""
candidate = os.path.join(parent_dir, f"{sanitized}{suffix}")
if allowed_extensions is not None:
file_parts = (os.path.splitext(f) for f in os.listdir(parent_dir))
clash = any(
name == f"{sanitized}{suffix}"
and ext[1:].lower() in allowed_extensions
for name, ext in file_parts
)
else:
clash = os.path.exists(candidate)
if not clash:
return candidate
counter += 1
-117
View File
@@ -1,117 +0,0 @@
"""Pipeline creation, caching and lifecycle management.
Provides a unified interface for creating and managing TTS pipelines
across all UI layers (WebUI, PyQt, CLI).
Language handling: the engine owns the mapping between Language enum
and its internal format. Callers pass Language enum; the engine
converts internally. No engine-specific codes leak outside the engine.
"""
from __future__ import annotations
from typing import Any, Dict
from abogen.domain.device import select_device
from abogen.domain.enums import Language
from abogen.domain.voice_resolution import initialize_voice_cache
from abogen.tts_plugin.utils import create_pipeline, is_plugin_registered
def resolve_device(use_gpu: bool) -> str:
"""Determine compute device from job and global config flags."""
from abogen.utils import load_config
cfg = load_config()
if use_gpu and cfg.get("use_gpu", True):
return select_device()
return "cpu"
def create_pipeline_for_job(
provider: str,
language: Language,
use_gpu: bool,
) -> Any:
"""Create a TTS pipeline with proper device selection.
Args:
provider: TTS provider name ("kokoro" or "supertonic").
language: Language enum (app-layer type, not engine-specific).
use_gpu: Whether GPU acceleration is requested.
"""
provider = str(provider or "kokoro").strip().lower() or "kokoro"
if not is_plugin_registered(provider):
provider = "kokoro"
if provider == "supertonic":
return create_pipeline("supertonic", language=language)
device = resolve_device(use_gpu)
return create_pipeline("kokoro", language=language, device=device)
def dispose_pipelines(pipelines: Dict[str, Any]) -> None:
"""Dispose all pipelines in a dict and clear it."""
for p in pipelines.values():
try:
p.dispose()
except Exception:
pass
pipelines.clear()
class PipelinePool:
"""Cache and manage TTS pipelines by provider.
Usage::
pool = PipelinePool()
backend = pool.get("kokoro", Language.EN_US, use_gpu=True)
# ... use backend ...
pool.dispose_all()
"""
def __init__(self) -> None:
self._pipelines: Dict[str, Any] = {}
self._voice_cache_initialized = False
def get(
self,
provider: str,
language: Language,
use_gpu: bool,
*,
request: Any = None,
events: Any = None,
) -> Any:
"""Get or create a cached pipeline for the given provider.
Args:
provider: TTS provider name ("kokoro" or "supertonic").
language: Language enum (app-layer type).
use_gpu: Whether GPU acceleration is requested.
request: ConversionRequest for voice cache initialization.
events: ConversionEvents for logging during cache init.
"""
provider = str(provider or "kokoro").strip().lower() or "kokoro"
if not is_plugin_registered(provider):
provider = "kokoro"
existing = self._pipelines.get(provider)
if existing is not None:
return existing
pipeline = create_pipeline_for_job(provider, language, use_gpu)
self._pipelines[provider] = pipeline
if provider == "kokoro" and not self._voice_cache_initialized and request is not None:
initialize_voice_cache(request, events=events)
self._voice_cache_initialized = True
return pipeline
def dispose_all(self) -> None:
"""Dispose all cached pipelines."""
dispose_pipelines(self._pipelines)
self._voice_cache_initialized = False
-72
View File
@@ -1,72 +0,0 @@
from __future__ import annotations
"""Progress and ETR (estimated time remaining) calculation.
Shared by Web UI and PyQt desktop GUI. Pure math, no UI dependencies.
"""
import time
from dataclasses import dataclass, field
@dataclass
class ProgressTracker:
"""Tracks character-based progress with ETR calculation.
Usage:
tracker = ProgressTracker(total_chars=50000)
# ... as processing occurs:
tracker.update(chars_done=5000)
print(tracker.etr_str) # "00:04:30"
print(tracker.percent) # 10
"""
total_chars: int
_start_time: float = field(default_factory=time.time, repr=False)
_chars_done: int = field(default=0, repr=False)
def update(self, chars_done: int) -> None:
self._chars_done = chars_done
@property
def percent(self) -> int:
if self.total_chars <= 0:
return 0
return min(int(self._chars_done / self.total_chars * 100), 99)
@property
def etr_str(self) -> str:
elapsed = time.time() - self._start_time
if self._chars_done <= 0 or elapsed <= 0.5:
return "Processing..."
avg_time_per_char = elapsed / self._chars_done
remaining = self.total_chars - self._chars_done
if remaining <= 0:
return "00:00:00"
secs = avg_time_per_char * remaining
h = int(secs // 3600)
m = int((secs % 3600) // 60)
s = int(secs % 60)
return f"{h:02d}:{m:02d}:{s:02d}"
def calc_etr_str(elapsed: float, done: int, total: int) -> str:
"""Standalone ETR string calculation (matches PyQt original logic).
Args:
elapsed: seconds since processing started
done: items/characters processed so far
total: total items/characters to process
Returns:
ETR string like "01:23:45" or "Processing..."
"""
if done <= 0 or elapsed <= 0.5:
return "Processing..."
avg_time_per_item = elapsed / done
remaining = total - done
if remaining <= 0:
return "00:00:00"
secs = avg_time_per_item * remaining
h = int(secs // 3600)
m = int((secs % 3600) // 60)
s = int(secs % 60)
return f"{h:02d}:{m:02d}:{s:02d}"
-270
View File
@@ -1,270 +0,0 @@
"""Pronunciation rule compilation and application.
Pure functions for compiling token-level and sentence-level pronunciation
overrides into regex patterns, applying them to text, and merging multiple
override sources with precedence rules.
"""
from __future__ import annotations
import re
from typing import Any, Dict, Iterable, List, Mapping, Optional
from abogen.entity_analysis import normalize_token as normalize_entity_token
from abogen.entity_analysis import normalize_manual_override_token
def compile_pronunciation_rules(
overrides: Optional[Iterable[Mapping[str, Any]]],
) -> List[Dict[str, Any]]:
if not overrides:
return []
candidates: List[Dict[str, Any]] = []
seen: set[str] = set()
for entry in overrides:
if not isinstance(entry, Mapping):
continue
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not pronunciation_value:
continue
token_values: List[str] = []
token_raw = entry.get("token")
if token_raw:
token_value = str(token_raw).strip()
if token_value:
token_values.append(token_value)
normalized_raw = entry.get("normalized")
if normalized_raw:
normalized_value = str(normalized_raw).strip()
if normalized_value:
token_values.append(normalized_value)
if token_raw and not token_values:
fallback = normalize_entity_token(str(token_raw))
if fallback:
token_values.append(fallback)
if not token_values:
continue
usage_normalized = str(entry.get("normalized") or "").strip()
if not usage_normalized and token_values:
usage_normalized = normalize_entity_token(token_values[0]) or token_values[0]
usage_token = str(entry.get("token") or token_values[0])
for token_value in token_values:
key = token_value.casefold()
if key in seen:
continue
seen.add(key)
candidates.append(
{
"token": token_value,
"normalized": usage_normalized,
"replacement": pronunciation_value,
}
)
if not candidates:
return []
candidates.sort(key=lambda item: len(item["token"]), reverse=True)
compiled: List[Dict[str, Any]] = []
for candidate in candidates:
token_value = candidate["token"]
pronunciation_value = candidate["replacement"]
escaped = re.escape(token_value)
pattern = re.compile(rf"(?i)(?<!\w){escaped}(?P<possessive>'s|\u2019s|\u2019)?(?!\w)")
compiled.append(
{
"pattern": pattern,
"replacement": pronunciation_value,
"normalized": candidate.get("normalized") or token_value,
"token": candidate.get("token") or token_value,
}
)
return compiled
def compile_heteronym_sentence_rules(
overrides: Optional[Iterable[Mapping[str, Any]]],
) -> List[Dict[str, Any]]:
if not overrides:
return []
compiled: List[Dict[str, Any]] = []
seen: set[str] = set()
for entry in overrides:
if not isinstance(entry, Mapping):
continue
sentence = str(entry.get("sentence") or "").strip()
if not sentence:
continue
choice = str(entry.get("choice") or "").strip()
if not choice:
continue
replacement_sentence = ""
options = entry.get("options")
if isinstance(options, list):
for opt in options:
if not isinstance(opt, Mapping):
continue
if str(opt.get("key") or "").strip() == choice:
replacement_sentence = str(opt.get("replacement_sentence") or "").strip()
break
if not replacement_sentence:
continue
rule_key = f"{sentence}\n{choice}".casefold()
if rule_key in seen:
continue
seen.add(rule_key)
parts = [p for p in re.split(r"\s+", sentence) if p]
if not parts:
continue
pattern_text = r"\s+".join(re.escape(p) for p in parts)
pattern = re.compile(pattern_text)
compiled.append({"pattern": pattern, "replacement": replacement_sentence})
compiled.sort(key=lambda item: len(item["pattern"].pattern), reverse=True)
return compiled
def apply_heteronym_sentence_rules(text: str, rules: List[Dict[str, Any]]) -> str:
if not text or not rules:
return text
result = text
for rule in rules:
pattern = rule["pattern"]
replacement = rule["replacement"]
result = pattern.sub(replacement, result)
return result
def apply_pronunciation_rules(
text: str,
rules: List[Dict[str, Any]],
usage_counter: Optional[Dict[str, int]] = None,
) -> str:
if not text or not rules:
return text
result = text
for rule in rules:
pattern = rule["pattern"]
pronunciation_value = rule["replacement"]
usage_key = str(rule.get("normalized") or "").strip()
def _replacement(match: re.Match[str]) -> str:
suffix = match.group("possessive") or ""
if usage_counter is not None and usage_key:
usage_counter[usage_key] = usage_counter.get(usage_key, 0) + 1
return pronunciation_value + suffix
result = pattern.sub(_replacement, result)
return result
def merge_pronunciation_overrides(job: Any) -> List[Dict[str, Any]]:
"""Return pronunciation override entries, ensuring manual overrides are included.
Pending jobs keep both ``manual_overrides`` and ``pronunciation_overrides``, but the
latter can be stale if the UI didn't resync before enqueue. During conversion,
we must merge manual overrides so they always apply (before TTS).
Precedence: manual overrides win over existing entries for the same normalized key.
Args:
job: Either a job-like object with attributes, or a dict with keys:
``pronunciation_overrides``, ``manual_overrides``, ``speakers``, ``language``.
"""
collected: Dict[str, Dict[str, Any]] = {}
def _get(key: str, default: Any = None) -> Any:
if isinstance(job, Mapping):
return job.get(key, default)
return getattr(job, key, default)
existing = _get("pronunciation_overrides")
if isinstance(existing, list):
for entry in existing:
if not isinstance(entry, Mapping):
continue
token_value = str(entry.get("token") or "").strip()
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = str(entry.get("normalized") or "").strip() or normalize_entity_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(entry.get("voice") or "").strip() or None,
"notes": str(entry.get("notes") or "").strip() or None,
"context": str(entry.get("context") or "").strip() or None,
"source": str(entry.get("source") or "pronunciation"),
"language": _get("language"),
}
speakers = _get("speakers")
if isinstance(speakers, dict):
for payload in speakers.values():
if not isinstance(payload, Mapping):
continue
token_value = str(payload.get("token") or "").strip()
pronunciation_value = str(payload.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = normalize_entity_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(
payload.get("resolved_voice")
or payload.get("voice")
or _get("voice", "")
).strip()
or None,
"notes": None,
"context": None,
"source": "speaker",
"language": _get("language"),
}
manual = _get("manual_overrides")
if isinstance(manual, list):
for entry in manual:
if not isinstance(entry, Mapping):
continue
token_value = str(entry.get("token") or "").strip()
pronunciation_value = str(entry.get("pronunciation") or "").strip()
if not token_value or not pronunciation_value:
continue
normalized = str(entry.get("normalized") or "").strip() or normalize_manual_override_token(token_value)
if not normalized:
continue
collected[normalized] = {
"token": token_value,
"normalized": normalized,
"pronunciation": pronunciation_value,
"voice": str(entry.get("voice") or "").strip() or None,
"notes": str(entry.get("notes") or "").strip() or None,
"context": str(entry.get("context") or "").strip() or None,
"source": str(entry.get("source") or "manual"),
"language": _get("language"),
}
return list(collected.values())
-641
View File
@@ -1,641 +0,0 @@
"""Shared settings core.
Defines the SETTINGS_REGISTRY — the single source of truth for all settings.
Every setting has a key, type, default, validation rules, and UI scope.
Both Web UI and Desktop GUI must reference this registry.
"""
from __future__ import annotations
import os
import re
from dataclasses import dataclass
from typing import Any, Callable, Dict, Mapping, Optional
from abogen.constants import (
KOKORO_CODE_LABELS,
SUBTITLE_FORMATS,
SUPPORTED_SOUND_FORMATS,
)
from abogen.tts_plugin.utils import get_default_voice
from abogen.normalization_settings import (
DEFAULT_LLM_PROMPT,
environment_llm_defaults,
)
# ── Schema ───────────────────────────────────────────────────────────
@dataclass(frozen=True)
class Setting:
"""Contract for a single setting.
Attributes:
key: Config dict key (e.g. "output_format").
type_: Python type (bool, int, float, str, list).
default: Default value or callable returning one.
min_value: Minimum for numeric types.
max_value: Maximum for numeric types.
valid_values: Allowed values for str types (None = any).
gui_only: True if only used by PyQt Desktop GUI.
web_only: True if only used by Web UI.
normalizer: Optional callable(value, default) -> normalized_value.
description: Human-readable explanation.
"""
key: str
type_: type
default: Any
min_value: float | None = None
max_value: float | None = None
valid_values: tuple[Any, ...] | None = None
gui_only: bool = False
web_only: bool = False
normalizer: Callable | None = None
description: str = ""
def coerce(self, value: Any, fallback: Any | None = None) -> Any:
"""Coerce value to the declared type, returning fallback on failure."""
fb = fallback if fallback is not None else self.default
if self.type_ is bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() in {"true", "1", "yes", "on"}
if value is None:
return fb
return bool(value)
if self.type_ is int:
try:
v = int(value)
except (TypeError, ValueError):
return fb
if self.min_value is not None:
v = max(int(self.min_value), v)
if self.max_value is not None:
v = min(int(self.max_value), v)
return v
if self.type_ is float:
try:
v = float(value)
except (TypeError, ValueError):
return fb
if self.min_value is not None:
v = max(self.min_value, v)
if self.max_value is not None:
v = min(self.max_value, v)
return v
if self.type_ is str:
if isinstance(value, str):
v = value.strip()
if self.valid_values and v not in self.valid_values:
return fb
return v
return fb
if self.type_ is list:
if isinstance(value, (list, tuple, set)):
return list(value)
return fb
return value
# ── Normalizers (used by Setting.normalizer) ─────────────────────────
def _norm_save_mode(value: Any, default: str) -> str:
if isinstance(value, str):
if value in SAVE_MODE_LABELS:
return value
if value in LEGACY_SAVE_MODE_MAP:
return LEGACY_SAVE_MODE_MAP[value]
return default
def _norm_voice_spec(value: Any, default: str) -> str:
if isinstance(value, str):
text = value.strip()
if not text:
return default
spec, profile_name = split_profile_spec(text)
if profile_name:
return f"speaker:{profile_name}"
return spec
return default
def _norm_speaker_spec(value: Any, default: str) -> str:
if isinstance(value, str):
text = value.strip()
if not text:
return ""
spec, profile_name = split_profile_spec(text)
if profile_name:
return f"speaker:{profile_name}"
return spec
return ""
def _norm_language_list(value: Any, default: list) -> list:
if isinstance(value, (list, tuple, set)):
return [code for code in value if isinstance(code, str) and code in KOKORO_CODE_LABELS]
if isinstance(value, str):
parts = [item.strip().lower() for item in value.split(",") if item.strip()]
return [code for code in parts if code in KOKORO_CODE_LABELS]
return default
def _norm_stripped_str(value: Any, default: str) -> str:
return str(value or "").strip()
def _norm_prompt(value: Any, default: str) -> str:
candidate = str(value or "").strip()
return candidate if candidate else default
# ── Registry ─────────────────────────────────────────────────────────
def _default_output_format() -> str:
return "wav"
def _default_save_mode() -> str:
return "default_output" if has_output_override() else "save_next_to_input"
def _default_llm(key: str) -> str:
return environment_llm_defaults().get(key, "")
SETTINGS_REGISTRY: list[Setting] = [
# ── Core output ──────────────────────────────────────────────
Setting("output_format", str, "wav",
valid_values=tuple(SUPPORTED_SOUND_FORMATS),
description="Audio output format"),
Setting("subtitle_format", str, "srt",
valid_values=tuple(item[0] for item in SUBTITLE_FORMATS),
description="Subtitle file format"),
Setting("save_mode", str, _default_save_mode,
normalizer=_norm_save_mode,
description="Where to save output files"),
Setting("separate_chapters_format", str, "wav",
valid_values=("wav", "flac", "mp3", "opus"),
description="Format for separately saved chapters"),
Setting("chunk_level", str, "paragraph",
valid_values=("paragraph", "sentence"),
description="Text chunking granularity"),
# ── Voice ────────────────────────────────────────────────────
Setting("default_speaker", str, "",
normalizer=_norm_speaker_spec,
description="Default speaker name"),
Setting("default_voice", str, lambda: get_default_voice("kokoro"),
normalizer=_norm_voice_spec,
description="Default TTS voice"),
Setting("speed", float, 1.0, min_value=0.5, max_value=3.0,
gui_only=True,
description="TTS speed multiplier"),
Setting("supertonic_total_steps", int, 5, min_value=2, max_value=15,
description="SuperTonic processing steps"),
Setting("supertonic_speed", float, 1.0, min_value=0.7, max_value=2.0,
description="SuperTonic speed"),
# ── Chapter handling ─────────────────────────────────────────
Setting("silence_between_chapters", float, 2.0, min_value=0.0,
description="Silence gap between chapters (seconds)"),
Setting("chapter_intro_delay", float, 0.5, min_value=0.0,
description="Delay after chapter heading (seconds)"),
Setting("read_title_intro", bool, False,
description="Read chapter title as intro"),
Setting("read_closing_outro", bool, True,
description="Read closing/outro text"),
Setting("normalize_chapter_opening_caps", bool, True,
description="Normalize chapter opening caps"),
Setting("auto_prefix_chapter_titles", bool, True,
description="Auto-prefix chapter titles"),
Setting("save_chapters_separately", bool, False,
description="Save each chapter as separate file"),
Setting("merge_chapters_at_end", bool, True,
description="Merge chapters into single file"),
Setting("save_as_project", bool, False,
description="Save as editable project"),
Setting("generate_epub3", bool, False,
description="Generate EPUB3 output"),
# ── GPU / performance ────────────────────────────────────────
Setting("use_gpu", bool, True,
description="Use GPU acceleration"),
# ── Text processing ──────────────────────────────────────────
Setting("replace_single_newlines", bool, False,
description="Replace single newlines with spaces"),
Setting("max_subtitle_words", int, 50, min_value=1, max_value=500,
description="Max words per subtitle"),
Setting("enable_entity_recognition", bool, True,
description="Enable entity recognition"),
# ── Speaker analysis ─────────────────────────────────────────
Setting("speaker_analysis_threshold", int, 3, min_value=1, max_value=25,
description="Speaker analysis threshold"),
Setting("speaker_pronunciation_sentence", str, "This is {{name}} speaking.",
description="Template for pronunciation samples"),
Setting("speaker_random_languages", list, [],
normalizer=_norm_language_list,
description="Languages for random speaker assignment"),
# ── LLM ──────────────────────────────────────────────────────
Setting("llm_base_url", str, lambda: _default_llm("llm_base_url"),
normalizer=_norm_stripped_str,
description="LLM API base URL"),
Setting("llm_api_key", str, lambda: _default_llm("llm_api_key"),
normalizer=_norm_stripped_str,
description="LLM API key"),
Setting("llm_model", str, lambda: _default_llm("llm_model"),
normalizer=_norm_stripped_str,
description="LLM model name"),
Setting("llm_timeout", float, lambda: _default_llm("llm_timeout") or 30.0,
min_value=1.0,
description="LLM request timeout"),
Setting("llm_prompt", str, lambda: _default_llm("llm_prompt") or DEFAULT_LLM_PROMPT,
normalizer=_norm_prompt,
description="LLM normalization prompt"),
Setting("llm_context_mode", str, lambda: _default_llm("llm_context_mode") or "sentence",
valid_values=("sentence",),
description="LLM context mode"),
# ── Normalization (booleans) ─────────────────────────────────
Setting("normalization_numbers", bool, True,
description="Convert grouped numbers to words"),
Setting("normalization_currency", bool, True,
description="Convert currency symbols"),
Setting("normalization_footnotes", bool, True,
description="Remove footnote indicators"),
Setting("normalization_titles", bool, True,
description="Expand titles and suffixes"),
Setting("normalization_terminal", bool, True,
description="Ensure terminal punctuation"),
Setting("normalization_phoneme_hints", bool, True,
description="Add phoneme hints for possessives"),
Setting("normalization_caps_quotes", bool, True,
description="Convert ALL CAPS in quotes"),
Setting("normalization_internet_slang", bool, False,
description="Expand internet slang"),
Setting("normalization_apostrophes_contractions", bool, True,
description="Expand contractions"),
Setting("normalization_apostrophes_plural_possessives", bool, True,
description="Collapse plural possessives"),
Setting("normalization_apostrophes_sibilant_possessives", bool, True,
description="Mark sibilant possessives"),
Setting("normalization_apostrophes_decades", bool, True,
description="Expand decades"),
Setting("normalization_apostrophes_leading_elisions", bool, True,
description="Expand leading elisions"),
Setting("normalization_contraction_aux_be", bool, True,
description="Expand auxiliary 'be'"),
Setting("normalization_contraction_aux_have", bool, True,
description="Expand auxiliary 'have'"),
Setting("normalization_contraction_modal_will", bool, True,
description="Expand modal 'will'"),
Setting("normalization_contraction_modal_would", bool, True,
description="Expand modal 'would'"),
Setting("normalization_contraction_negation_not", bool, True,
description="Expand negation 'not'"),
Setting("normalization_contraction_let_us", bool, True,
description="Expand 'let's'"),
# ── Normalization (strings) ──────────────────────────────────
Setting("normalization_apostrophe_mode", str, "spacy",
valid_values=("off", "spacy", "llm"),
description="Apostrophe handling mode"),
Setting("normalization_numbers_year_style", str, "american",
valid_values=("american", "off"),
description="Year style for number normalization"),
# ── PyQt GUI-only ────────────────────────────────────────────
Setting("theme", str, "system",
gui_only=True,
description="UI theme"),
Setting("check_updates", bool, True,
gui_only=True,
description="Check for updates on startup"),
Setting("subtitle_mode", str, "Sentence",
gui_only=True,
description="Subtitle display mode"),
Setting("selected_format", str, "wav",
gui_only=True,
description="Last selected audio format"),
Setting("selected_voice", str, "af_heart",
gui_only=True,
description="Last selected voice"),
Setting("selected_profile_name", str, None,
gui_only=True,
description="Last selected profile name"),
Setting("log_window_max_lines", int, 2000, min_value=100,
gui_only=True,
description="Max lines in log window"),
Setting("use_silent_gaps", bool, True,
gui_only=True,
description="Use silent gaps between chunks"),
Setting("subtitle_speed_method", str, "tts",
gui_only=True,
valid_values=("tts", "ffmpeg"),
description="Speed adjustment method for subtitles"),
Setting("use_spacy_segmentation", bool, True,
gui_only=True,
description="Use spaCy for sentence segmentation"),
Setting("word_substitutions_enabled", bool, False,
gui_only=True,
description="Enable word substitutions"),
Setting("word_substitutions_list", str, "",
gui_only=True,
description="Word substitutions list"),
Setting("case_sensitive_substitutions", bool, False,
gui_only=True,
description="Case-sensitive substitutions"),
Setting("replace_all_caps", bool, False,
gui_only=True,
description="Replace ALL CAPS text"),
Setting("replace_numerals", bool, False,
gui_only=True,
description="Replace numerals with words"),
Setting("fix_nonstandard_punctuation", bool, False,
gui_only=True,
description="Fix nonstandard punctuation"),
Setting("queue_override_settings", bool, False,
gui_only=True,
description="Override settings per queue item"),
Setting("disable_kokoro_internet", bool, False,
description="Disable Kokoro internet access"),
]
# ── Registry helpers ─────────────────────────────────────────────────
_REGISTRY_BY_KEY: dict[str, Setting] = {s.key: s for s in SETTINGS_REGISTRY}
SETTING_KEYS: frozenset[str] = frozenset(_REGISTRY_BY_KEY.keys())
GUI_ONLY_KEYS: frozenset[str] = frozenset(s.key for s in SETTINGS_REGISTRY if s.gui_only)
WEB_ONLY_KEYS: frozenset[str] = frozenset(s.key for s in SETTINGS_REGISTRY if s.web_only)
SHARED_KEYS: frozenset[str] = SETTING_KEYS - GUI_ONLY_KEYS - WEB_ONLY_KEYS
BOOLEAN_SETTINGS: frozenset[str] = frozenset(s.key for s in SETTINGS_REGISTRY if s.type_ is bool)
FLOAT_SETTINGS: frozenset[str] = frozenset(s.key for s in SETTINGS_REGISTRY if s.type_ is float)
INT_SETTINGS: frozenset[str] = frozenset(s.key for s in SETTINGS_REGISTRY if s.type_ is int)
# Backward-compatible aliases (used by existing code)
_NORMALIZATION_BOOLEAN_KEYS: frozenset[str] = frozenset(
s.key for s in SETTINGS_REGISTRY
if s.type_ is bool and s.key.startswith("normalization_")
)
_NORMALIZATION_STRING_KEYS: frozenset[str] = frozenset(
s.key for s in SETTINGS_REGISTRY
if s.type_ is str and s.key.startswith("normalization_")
)
def get_setting(key: str) -> Setting | None:
"""Look up a setting by key."""
return _REGISTRY_BY_KEY.get(key)
def has_output_override() -> bool:
return bool(os.environ.get("ABOGEN_OUTPUT_DIR") or os.environ.get("ABOGEN_OUTPUT_ROOT"))
# ── Defaults ─────────────────────────────────────────────────────────
def settings_defaults() -> Dict[str, Any]:
"""Default values for all shared settings (excludes gui_only)."""
result: Dict[str, Any] = {}
for s in SETTINGS_REGISTRY:
if s.gui_only:
continue
result[s.key] = s.default() if callable(s.default) else s.default
return result
def all_settings_defaults() -> Dict[str, Any]:
"""Default values for ALL settings (including gui_only)."""
result: Dict[str, Any] = {}
for s in SETTINGS_REGISTRY:
result[s.key] = s.default() if callable(s.default) else s.default
return result
def load_settings() -> Dict[str, Any]:
"""Load and normalize settings from config file."""
from abogen.utils import load_config
defaults = settings_defaults()
cfg = load_config() or {}
settings: Dict[str, Any] = {}
for key, default in defaults.items():
raw_value = cfg.get(key, default)
settings[key] = normalize_setting_value(key, raw_value, defaults)
return settings
# ── Normalization (delegates to Setting.coerce) ──────────────────────
def normalize_setting_value(key: str, value: Any, defaults: Dict[str, Any]) -> Any:
"""Normalize a single setting value using the registry schema."""
setting = _REGISTRY_BY_KEY.get(key)
if setting is None:
return value if value is not None else defaults.get(key)
fallback = defaults.get(key, setting.default() if callable(setting.default) else setting.default)
if setting.normalizer is not None:
return setting.normalizer(value, fallback)
return setting.coerce(value, fallback)
def validate_setting(key: str, value: Any) -> tuple[bool, str]:
"""Validate a setting value against its schema. Returns (ok, error_message)."""
setting = _REGISTRY_BY_KEY.get(key)
if setting is None:
return False, f"Unknown setting: {key}"
if setting.type_ is str and setting.valid_values is not None:
v = str(value or "").strip()
if v and v not in setting.valid_values:
return False, f"Invalid value '{v}' for {key}. Allowed: {setting.valid_values}"
if setting.type_ is int:
try:
iv = int(value)
except (TypeError, ValueError):
return False, f"Invalid integer value for {key}: {value!r}"
if setting.min_value is not None and iv < setting.min_value:
return False, f"{key} must be >= {setting.min_value}, got {iv}"
if setting.max_value is not None and iv > setting.max_value:
return False, f"{key} must be <= {setting.max_value}, got {iv}"
if setting.type_ is float:
try:
fv = float(value)
except (TypeError, ValueError):
return False, f"Invalid float value for {key}: {value!r}"
if setting.min_value is not None and fv < setting.min_value:
return False, f"{key} must be >= {setting.min_value}, got {fv}"
if setting.max_value is not None and fv > setting.max_value:
return False, f"{key} must be <= {setting.max_value}, got {fv}"
return True, ""
# ── Constants (backward-compatible) ──────────────────────────────────
SAVE_MODE_LABELS = {
"save_next_to_input": "Save next to input file",
"save_to_desktop": "Save to Desktop",
"choose_output_folder": "Choose output folder",
"default_output": "Use default save location",
}
LEGACY_SAVE_MODE_MAP = {label: key for key, label in SAVE_MODE_LABELS.items()}
CHUNK_LEVEL_OPTIONS = [
{"value": "paragraph", "label": "Paragraphs"},
{"value": "sentence", "label": "Sentences"},
]
CHUNK_LEVEL_VALUES = frozenset(option["value"] for option in CHUNK_LEVEL_OPTIONS)
DEFAULT_ANALYSIS_THRESHOLD = 3
# ── Coercion helpers (backward-compatible, delegate to Setting.coerce) ──
def coerce_bool(value: Any, default: bool) -> bool:
return Setting("_", bool, default).coerce(value, default)
def coerce_float(value: Any, default: float) -> float:
return Setting("_", float, default).coerce(value, default)
def coerce_int(value: Any, default: int, *, minimum: int = 1, maximum: int = 200) -> int:
return Setting("_", int, default, min_value=minimum, max_value=maximum).coerce(value, default)
def split_profile_spec(value: Any) -> tuple[str, str | None]:
"""Split 'speaker:Name' or 'profile:Name' into (raw, name)."""
text = str(value or "").strip()
if not text:
return "", None
lowered = text.lower()
if lowered.startswith("profile:") or lowered.startswith("speaker:"):
_, _, remainder = text.partition(":")
name = remainder.strip()
return "", name or None
return text, None
def normalize_save_mode(value: Any, default: str) -> str:
return _norm_save_mode(value, default)
# ── LLM helpers ──────────────────────────────────────────────────────
_PROMPT_TOKEN_RE = re.compile(r"{{\s*([a-zA-Z0-9_]+)\s*}}")
def llm_ready(settings: Mapping[str, Any]) -> bool:
base_url = str(settings.get("llm_base_url") or "").strip()
return bool(base_url)
def render_prompt_template(template: str, context: Mapping[str, str]) -> str:
if not template:
return ""
def _replace(match: re.Match[str]) -> str:
key = match.group(1)
return context.get(key, "")
return _PROMPT_TOKEN_RE.sub(_replace, template)
# ── Integration defaults ─────────────────────────────────────────────
def integration_defaults() -> Dict[str, Dict[str, Any]]:
"""Default values for integration settings."""
return {
"calibre_opds": {
"enabled": False,
"base_url": "",
"username": "",
"password": "",
"verify_ssl": True,
},
"audiobookshelf": {
"enabled": False,
"base_url": "",
"api_token": "",
"library_id": "",
"collection_id": "",
"folder_id": "",
"verify_ssl": True,
"send_cover": True,
"send_chapters": True,
"send_subtitles": False,
"auto_send": False,
"timeout": 30.0,
},
}
def stored_integration_config(name: str) -> Dict[str, Any]:
"""Read raw integration config from config.json.
Reads ``config["integrations"][name]``.
"""
from abogen.utils import load_config
cfg = load_config() or {}
integrations = cfg.get("integrations")
if isinstance(integrations, Mapping):
entry = integrations.get(name)
if isinstance(entry, Mapping):
return dict(entry)
return {}
def load_audiobookshelf_config() -> Optional["AudiobookshelfConfig"]:
"""Read Audiobookshelf settings from config.json and build typed config.
Returns ``None`` when the integration is not configured or required
fields are missing.
"""
raw = stored_integration_config("audiobookshelf")
if not raw:
return None
return build_audiobookshelf_config(raw)
def build_audiobookshelf_config(
settings: Mapping[str, Any],
) -> Optional["AudiobookshelfConfig"]:
"""Build :class:`AudiobookshelfConfig` from a settings dict.
Returns ``None`` when required fields (base_url, api_token, library_id)
are missing.
"""
from abogen.integrations.audiobookshelf import AudiobookshelfConfig
base_url = str(settings.get("base_url") or "").strip()
api_token = str(settings.get("api_token") or "").strip()
library_id = str(settings.get("library_id") or "").strip()
if not (base_url and api_token and library_id):
return None
try:
timeout = float(settings.get("timeout", 3600.0))
except (TypeError, ValueError):
timeout = 3600.0
return AudiobookshelfConfig(
base_url=base_url,
api_token=api_token,
library_id=library_id,
collection_id=(str(settings.get("collection_id") or "").strip() or None),
folder_id=(str(settings.get("folder_id") or "").strip() or None),
verify_ssl=coerce_bool(settings.get("verify_ssl"), True),
send_cover=coerce_bool(settings.get("send_cover"), True),
send_chapters=coerce_bool(settings.get("send_chapters"), True),
send_subtitles=coerce_bool(settings.get("send_subtitles"), False),
timeout=timeout,
)
-381
View File
@@ -1,381 +0,0 @@
"""Speaker metadata functions for building and applying speaker rosters.
This module contains the core logic for:
- Building narrator and speaker rosters from analysis results
- Matching speakers to configured presets
- Applying speaker config presets to rosters
- Preparing full speaker metadata for conversion
Moved from webui/routes/utils/voice.py to be available across all UIs.
"""
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple, cast
from abogen.speaker_analysis import analyze_speakers
from abogen.speaker_configs import slugify_label
from abogen.domain.settings_core import load_settings
def build_narrator_roster(
voice: str,
voice_profile: Optional[str],
existing: Optional[Mapping[str, Any]] = None,
) -> Dict[str, Any]:
roster: Dict[str, Any] = {
"narrator": {
"id": "narrator",
"label": "Narrator",
"voice": voice,
}
}
if voice_profile:
roster["narrator"]["voice_profile"] = voice_profile
existing_entry: Optional[Mapping[str, Any]] = None
if existing is not None:
existing_entry = existing.get("narrator") if isinstance(existing, Mapping) else None
if isinstance(existing_entry, Mapping):
roster_entry = roster["narrator"]
for key in ("label", "voice", "voice_profile", "voice_formula", "pronunciation"):
value = existing_entry.get(key)
if value is not None and value != "":
roster_entry[key] = value
return roster
def build_speaker_roster(
analysis: Dict[str, Any],
base_voice: str,
voice_profile: Optional[str],
existing: Optional[Mapping[str, Any]] = None,
order: Optional[Iterable[str]] = None,
) -> Dict[str, Any]:
roster = build_narrator_roster(base_voice, voice_profile, existing)
existing_map: Dict[str, Any] = dict(existing) if isinstance(existing, Mapping) else {}
speakers = analysis.get("speakers", {}) if isinstance(analysis, dict) else {}
ordered_ids: Iterable[str]
if order is not None:
ordered_ids = [sid for sid in order if sid in speakers]
else:
ordered_ids = speakers.keys()
for speaker_id in ordered_ids:
payload = speakers.get(speaker_id, {})
if speaker_id == "narrator":
continue
if isinstance(payload, Mapping) and payload.get("suppressed"):
continue
previous = existing_map.get(speaker_id)
roster[speaker_id] = {
"id": speaker_id,
"label": payload.get("label") or speaker_id.replace("_", " ").title(),
"analysis_confidence": payload.get("confidence"),
"analysis_count": payload.get("count"),
"gender": payload.get("gender", "unknown"),
}
detected_gender = payload.get("detected_gender")
if detected_gender:
roster[speaker_id]["detected_gender"] = detected_gender
samples = payload.get("sample_quotes")
if isinstance(samples, list):
roster[speaker_id]["sample_quotes"] = samples
if isinstance(previous, Mapping):
for key in ("voice", "voice_profile", "voice_formula", "resolved_voice", "pronunciation"):
value = previous.get(key)
if value is not None and value != "":
roster[speaker_id][key] = value
if "sample_quotes" not in roster[speaker_id]:
prev_samples = previous.get("sample_quotes")
if isinstance(prev_samples, list):
roster[speaker_id]["sample_quotes"] = prev_samples
if "detected_gender" not in roster[speaker_id]:
prev_detected = previous.get("detected_gender")
if isinstance(prev_detected, str) and prev_detected:
roster[speaker_id]["detected_gender"] = prev_detected
return roster
def match_configured_speaker(
config_speakers: Mapping[str, Any],
roster_id: str,
roster_label: str,
) -> Optional[Mapping[str, Any]]:
if not config_speakers:
return None
entry = config_speakers.get(roster_id)
if entry:
return cast(Mapping[str, Any], entry)
slug = slugify_label(roster_label)
if slug != roster_id and slug in config_speakers:
return cast(Mapping[str, Any], config_speakers[slug])
lower_label = roster_label.strip().lower()
for record in config_speakers.values():
if not isinstance(record, Mapping):
continue
if str(record.get("label", "")).strip().lower() == lower_label:
return record
return None
def apply_speaker_config_to_roster(
roster: Mapping[str, Any],
config: Optional[Mapping[str, Any]],
*,
persist_changes: bool = False,
fallback_languages: Optional[Iterable[str]] = None,
) -> Tuple[Dict[str, Any], List[str], Optional[Dict[str, Any]]]:
if not isinstance(roster, Mapping):
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
return {}, effective_languages, None
updated_roster: Dict[str, Any] = {key: dict(value) for key, value in roster.items() if isinstance(value, Mapping)}
if not config:
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
return updated_roster, effective_languages, None
speakers_map = config.get("speakers")
if not isinstance(speakers_map, Mapping):
effective_languages = [code for code in (fallback_languages or []) if isinstance(code, str) and code]
return updated_roster, effective_languages, None
config_languages = config.get("languages")
if isinstance(config_languages, list):
allowed_languages = [code for code in config_languages if isinstance(code, str) and code]
else:
allowed_languages = []
if not allowed_languages and fallback_languages:
allowed_languages = [code for code in fallback_languages if isinstance(code, str) and code]
default_voice = config.get("default_voice") if isinstance(config.get("default_voice"), str) else ""
used_voices = {entry.get("resolved_voice") or entry.get("voice") for entry in updated_roster.values()} - {None}
narrator_voice = ""
narrator_entry = updated_roster.get("narrator") if isinstance(updated_roster, Mapping) else None
if isinstance(narrator_entry, Mapping):
narrator_voice = str(
narrator_entry.get("resolved_voice")
or narrator_entry.get("default_voice")
or ""
).strip()
if narrator_voice:
used_voices.add(narrator_voice)
config_changed = False
new_config_payload: Dict[str, Any] = {
"language": config.get("language", "a"),
"languages": allowed_languages,
"default_voice": default_voice,
"speakers": dict(speakers_map),
"version": config.get("version", 1),
"notes": config.get("notes", ""),
}
speakers_payload = new_config_payload["speakers"]
for speaker_id, roster_entry in updated_roster.items():
if speaker_id == "narrator":
continue
label = str(roster_entry.get("label") or speaker_id)
config_entry = match_configured_speaker(speakers_map, speaker_id, label)
if config_entry is None:
continue
voice_id = str(config_entry.get("voice") or "").strip()
voice_profile = str(config_entry.get("voice_profile") or "").strip()
voice_formula = str(config_entry.get("voice_formula") or "").strip()
resolved_voice = str(config_entry.get("resolved_voice") or "").strip()
languages = config_entry.get("languages") if isinstance(config_entry.get("languages"), list) else []
chosen_voice = resolved_voice or voice_formula or voice_id or roster_entry.get("voice")
usable_languages = languages or allowed_languages
if chosen_voice:
roster_entry["resolved_voice"] = chosen_voice
roster_entry["voice"] = chosen_voice if not voice_profile and not voice_formula else roster_entry.get("voice", chosen_voice)
if voice_profile:
roster_entry["voice_profile"] = voice_profile
if voice_formula:
roster_entry["voice_formula"] = voice_formula
roster_entry["resolved_voice"] = voice_formula
if not voice_formula and not voice_profile and resolved_voice:
roster_entry["resolved_voice"] = resolved_voice
roster_entry["config_languages"] = usable_languages or []
if chosen_voice:
used_voices.add(chosen_voice)
# persist updates back to config payload if required
if persist_changes:
slug = config_entry.get("id") or slugify_label(label)
speakers_payload[slug] = {
"id": slug,
"label": label,
"gender": config_entry.get("gender", "unknown"),
"voice": voice_id,
"voice_profile": voice_profile,
"voice_formula": voice_formula,
"resolved_voice": roster_entry.get("resolved_voice", resolved_voice or voice_id),
"languages": usable_languages,
}
new_config = new_config_payload if (persist_changes and config_changed) else None
return updated_roster, allowed_languages, new_config
def prepare_speaker_metadata(
*,
chapters: List[Dict[str, Any]],
chunks: List[Dict[str, Any]],
analysis_chunks: Optional[List[Dict[str, Any]]] = None,
voice: str,
voice_profile: Optional[str],
threshold: int,
existing_roster: Optional[Mapping[str, Any]] = None,
run_analysis: bool = True,
speaker_config: Optional[Mapping[str, Any]] = None,
apply_config: bool = False,
persist_config: bool = False,
inject_recommended: Optional[Any] = None,
) -> tuple[List[Dict[str, Any]], Dict[str, Any], Dict[str, Any], List[str], Optional[Dict[str, Any]]]:
chunk_list = [dict(chunk) for chunk in chunks]
analysis_source = [dict(chunk) for chunk in (analysis_chunks or chunks)]
threshold_value = max(1, int(threshold))
analysis_enabled = run_analysis
settings_state = load_settings()
global_random_languages = [
code
for code in settings_state.get("speaker_random_languages", [])
if isinstance(code, str) and code
]
if not analysis_enabled:
for chunk in chunk_list:
chunk["speaker_id"] = "narrator"
chunk["speaker_label"] = "Narrator"
analysis_payload = {
"version": "1.0",
"narrator": "narrator",
"assignments": {str(chunk.get("id")): "narrator" for chunk in chunk_list},
"speakers": {
"narrator": {
"id": "narrator",
"label": "Narrator",
"count": len(chunk_list),
"confidence": "low",
"sample_quotes": [],
"suppressed": False,
}
},
"suppressed": [],
"stats": {
"total_chunks": len(chunk_list),
"explicit_chunks": 0,
"active_speakers": 0,
"unique_speakers": 1,
"suppressed": 0,
},
}
roster = build_narrator_roster(voice, voice_profile, existing_roster)
narrator_pron = roster["narrator"].get("pronunciation")
if narrator_pron:
analysis_payload["speakers"]["narrator"]["pronunciation"] = narrator_pron
return chunk_list, roster, analysis_payload, [], None
analysis_result = analyze_speakers(
chapters,
analysis_source,
threshold=threshold_value,
max_speakers=0,
)
analysis_payload = analysis_result.to_dict()
speakers_payload = analysis_payload.get("speakers", {})
ordered_ids = [
sid
for sid, meta in sorted(
(
(sid, meta)
for sid, meta in speakers_payload.items()
if sid != "narrator" and isinstance(meta, Mapping) and not meta.get("suppressed")
),
key=lambda item: item[1].get("count", 0),
reverse=True,
)
]
analysis_payload["ordered_speakers"] = ordered_ids
assignments = analysis_payload.get("assignments", {})
suppressed_ids = analysis_payload.get("suppressed", [])
suppressed_details: List[Dict[str, Any]] = []
speakers_payload = analysis_payload.get("speakers", {})
if isinstance(suppressed_ids, Iterable):
for suppressed_id in suppressed_ids:
speaker_meta = speakers_payload.get(suppressed_id) if isinstance(speakers_payload, dict) else None
if isinstance(speaker_meta, dict):
suppressed_details.append(
{
"id": suppressed_id,
"label": speaker_meta.get("label")
or str(suppressed_id).replace("_", " ").title(),
"pronunciation": speaker_meta.get("pronunciation"),
}
)
else:
suppressed_details.append(
{
"id": suppressed_id,
"label": str(suppressed_id).replace("_", " ").title(),
"pronunciation": None,
}
)
analysis_payload["suppressed_details"] = suppressed_details
roster = build_speaker_roster(
analysis_payload,
voice,
voice_profile,
existing=existing_roster,
order=analysis_payload.get("ordered_speakers"),
)
applied_languages: List[str] = []
updated_config: Optional[Dict[str, Any]] = None
if apply_config and speaker_config:
roster, applied_languages, updated_config = apply_speaker_config_to_roster(
roster,
speaker_config,
persist_changes=persist_config,
fallback_languages=global_random_languages,
)
speakers_payload = analysis_payload.get("speakers")
if isinstance(speakers_payload, dict):
for roster_id, roster_payload in roster.items():
speaker_meta = speakers_payload.get(roster_id)
if isinstance(speaker_meta, dict):
for key in ("voice", "voice_profile", "voice_formula", "resolved_voice"):
value = roster_payload.get(key)
if value:
speaker_meta[key] = value
effective_languages: List[str] = []
if applied_languages:
effective_languages = applied_languages
elif isinstance(analysis_payload.get("config_languages"), list):
effective_languages = [
code for code in analysis_payload.get("config_languages", []) if isinstance(code, str) and code
]
elif global_random_languages:
effective_languages = list(global_random_languages)
if effective_languages:
analysis_payload["config_languages"] = effective_languages
speakers_payload = analysis_payload.get("speakers")
if isinstance(speakers_payload, dict):
for roster_id, roster_payload in roster.items():
if roster_id in speakers_payload and isinstance(roster_payload, dict):
pronunciation_value = roster_payload.get("pronunciation")
if pronunciation_value:
speakers_payload[roster_id]["pronunciation"] = pronunciation_value
fallback_languages = effective_languages or []
if callable(inject_recommended):
inject_recommended(roster, fallback_languages=fallback_languages)
for chunk in chunk_list:
chunk_id = str(chunk.get("id"))
speaker_id = assignments.get(chunk_id, "narrator")
chunk["speaker_id"] = speaker_id
speaker_meta = roster.get(speaker_id)
chunk["speaker_label"] = speaker_meta.get("label") if isinstance(speaker_meta, dict) else speaker_id
return chunk_list, roster, analysis_payload, applied_languages, updated_config
-58
View File
@@ -1,58 +0,0 @@
from __future__ import annotations
"""Unified split pattern logic extracted from 3 copies."""
from abogen.domain.enums import Language, SubtitleMode
# Canonical punctuation sets covering all supported scripts:
# ASCII (. ! ?), ellipsis (…), Arabic ؟, CJK (。!?), Devanagari ।
PUNCTUATION_SENTENCE = r".!?…؟。!?।"
# Commas: ASCII , CJK fullwidth CJK ideographic 、
PUNCTUATION_SENTENCE_COMMA = r".!?…,?。!?،,、।"
PUNCTUATION_COMMAS = ",,、"
def get_split_pattern(language: Language, subtitle_mode: str) -> str:
"""Get the appropriate split pattern based on language and subtitle mode.
Args:
language: Language enum value, ISO code, or kokoro letter code.
subtitle_mode: Subtitle mode ("Sentence", "Sentence + Comma", "Line", etc.)
Returns:
Split pattern string
"""
try:
mode = SubtitleMode.from_str(subtitle_mode) if not isinstance(subtitle_mode, SubtitleMode) else subtitle_mode
except ValueError:
mode = SubtitleMode.DISABLED
# English: spaCy is NOT used for pre-TTS segmentation (it is only used
# for post-TTS subtitle boundaries), so sentence boundaries for English
# are applied at subtitle time, not in the TTS engine. Disabled, Line,
# Sentence, and Sentence + Comma all keep newline-only engine splitting.
if language in (Language.EN_US, Language.EN_GB):
if mode in (
SubtitleMode.DISABLED,
SubtitleMode.LINE,
SubtitleMode.SENTENCE,
SubtitleMode.SENTENCE_COMMA,
):
return "\n"
# Determine spacing pattern based on language
spacing = r"\s*" if language.is_cjk else r"\s+"
# For CJK languages, when subtitle mode is Disabled or Line, prefer
# punctuation-based splitting instead of plain newline splitting.
if mode in (SubtitleMode.DISABLED, SubtitleMode.LINE) and language.is_cjk:
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
if mode == SubtitleMode.LINE:
return "\n"
elif mode == SubtitleMode.SENTENCE:
return rf"(?<=[{PUNCTUATION_SENTENCE}]){spacing}|\n+"
elif mode == SubtitleMode.SENTENCE_COMMA:
return rf"(?<=[{PUNCTUATION_SENTENCE_COMMA}]){spacing}|\n+"
else:
return r"\n+"
-493
View File
@@ -1,493 +0,0 @@
"""Subtitle generation utilities for audiobook generation.
This module provides functions for processing TTS tokens into subtitle entries
according to various subtitle modes (Line, Sentence, Sentence + Comma,
Sentence + Highlighting).
"""
from __future__ import annotations
import re
from typing import List, Optional, Tuple
from abogen.domain.enums import Language, SubtitleMode
from abogen.domain.split_pattern import PUNCTUATION_SENTENCE, PUNCTUATION_SENTENCE_COMMA
_CLOSING_DELIMS = "\"\"\"\"'\"”’»›)]}」』"
def _is_sentence_boundary(
token: dict,
current_sentence: List[dict],
separator: str,
) -> bool:
"""Check whether token ends a sentence, considering closing quotes and brackets."""
ws = token.get("whitespace", "") or ""
if not ws:
return False
# For Line mode, a newline in whitespace or text marks line boundary
if separator == r"\n":
return "\n" in ws or "\n" in str(token.get("text", ""))
text = str(token.get("text", ""))
if re.search(rf"{separator}[{re.escape(_CLOSING_DELIMS)}]*$", text):
return True
if len(current_sentence) >= 2 and text and all(c in _CLOSING_DELIMS for c in text):
prev_text = str(current_sentence[-2].get("text", ""))
if re.search(rf"{separator}$", prev_text):
return True
return False
def process_subtitle_tokens(
tokens_with_timestamps: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
language: Language,
use_spacy_segmentation: bool = False,
fallback_end_time: Optional[float] = None,
) -> None:
"""Process TTS tokens into subtitle entries according to the subtitle mode.
This function modifies subtitle_entries in-place by appending new entries.
Args:
tokens_with_timestamps: List of token dictionaries with 'start', 'end', 'text',
and 'whitespace' keys.
subtitle_entries: List to append subtitle entries to (modified in-place).
Each entry is a tuple of (start_time, end_time, text).
max_subtitle_words: Maximum number of words per subtitle entry.
subtitle_mode: One of "Disabled", "Line", "Sentence", "Sentence + Comma",
"Sentence + Highlighting", or a string like "5" for word-count mode.
language: Language enum value for spaCy processing.
use_spacy_segmentation: Whether to use spaCy for sentence boundary detection.
fallback_end_time: Fallback end time for the last entry if none is available.
"""
if not tokens_with_timestamps:
return
if not isinstance(language, Language):
try:
language = Language.from_str(str(language))
except ValueError:
language = Language.EN_US
if isinstance(subtitle_mode, SubtitleMode):
subtitle_mode_str = subtitle_mode.value
else:
subtitle_mode_str = str(subtitle_mode)
processed_tokens = tokens_with_timestamps
# For English with spaCy enabled and sentence-based modes, use spaCy for sentence boundaries
# spaCy is disabled when subtitle mode is "Disabled" or "Line"
use_spacy_for_english = (
use_spacy_segmentation
and subtitle_mode_str not in [SubtitleMode.DISABLED.value, SubtitleMode.LINE.value, "Disabled", "Line"]
and language in [Language.EN_US, Language.EN_GB]
and subtitle_mode_str in [SubtitleMode.SENTENCE.value, SubtitleMode.SENTENCE_COMMA.value, "Sentence", "Sentence + Comma"]
)
if subtitle_mode_str in (SubtitleMode.SENTENCE_HIGHLIGHT.value, "Sentence + Highlighting"):
_process_karaoke_highlighting(
processed_tokens, subtitle_entries, max_subtitle_words, fallback_end_time
)
elif subtitle_mode_str in [
SubtitleMode.SENTENCE.value,
SubtitleMode.SENTENCE_COMMA.value,
SubtitleMode.LINE.value,
"Sentence",
"Sentence + Comma",
"Line",
]:
if use_spacy_for_english and subtitle_mode_str not in (SubtitleMode.LINE.value, "Line"):
_process_spacy_sentences(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode_str, language, fallback_end_time
)
else:
_process_regex_sentences(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode_str, fallback_end_time
)
else:
# Word count-based grouping (e.g., "5" for 5-word groups)
_process_word_count(
processed_tokens, subtitle_entries, max_subtitle_words,
subtitle_mode_str, fallback_end_time
)
def _process_karaoke_highlighting(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens for Sentence + Highlighting mode (karaoke effect)."""
separator = rf"[{PUNCTUATION_SENTENCE}]"
current_sentence = []
word_count = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
is_boundary = _is_sentence_boundary(token, current_sentence, separator)
if is_boundary or word_count >= max_subtitle_words:
if current_sentence:
# Create karaoke subtitle entry for this sentence
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Generate karaoke text with timing
karaoke_text = ""
for t in current_sentence:
# Calculate duration in centiseconds
duration = (
t["end"] - t["start"]
if t.get("end") is not None and t.get("start") is not None
else 0.5
)
try:
duration_cs = int(duration * 100)
except (ValueError, OverflowError, TypeError):
duration_cs = 50
# Add karaoke effect
karaoke_text += f"{{\\kf{duration_cs}}}{t.get('text', '')}{t.get('whitespace', '') or ''}"
text_stripped = karaoke_text.strip()
if text_stripped:
subtitle_entries.append(
(start_time, end_time, text_stripped)
)
current_sentence = []
word_count = 0
# Add any remaining tokens as a sentence
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
# Generate karaoke text for remaining tokens
karaoke_text = ""
for t in current_sentence:
duration = t["end"] - t["start"] if t.get("end") and t.get("start") else 0.5
try:
duration_cs = int(duration * 100)
except (ValueError, OverflowError, TypeError):
duration_cs = 50
karaoke_text += f"{{\\kf{duration_cs}}}{t.get('text', '')}{t.get('whitespace', '') or ''}"
text_stripped = karaoke_text.strip()
if text_stripped:
subtitle_entries.append((start_time, end_time, text_stripped))
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_spacy_sentences(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
language: Language,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens using spaCy for sentence boundary detection."""
try:
from abogen.spacy_utils import get_spacy_model
except ImportError:
# Fall back to regex if spaCy is not available
_process_regex_sentences(
tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
return
nlp = get_spacy_model(language)
if not nlp:
_process_regex_sentences(
tokens, subtitle_entries, max_subtitle_words,
subtitle_mode, fallback_end_time
)
return
# Build full text and track character positions to token indices
full_text = ""
for token in tokens:
text_part = str(token.get("text", "")) + (token.get("whitespace") or "")
full_text += text_part
# Get sentence boundaries from spaCy
doc = nlp(full_text)
sentence_boundaries = [sent.end_char for sent in doc.sents]
# For "Sentence + Comma" mode, also split on commas
if subtitle_mode in (SubtitleMode.SENTENCE_COMMA.value, "Sentence + Comma"):
comma_positions = [
i + 1 for i, c in enumerate(full_text) if c == ","
]
sentence_boundaries = sorted(
set(sentence_boundaries + comma_positions)
)
# spaCy does not treat ellipsis ("...", "..", "…") as a sentence
# boundary ("Lorem ipsum... Lorem..." stays one sentence), so ellipsis
# runs followed by whitespace/end would merge into a single subtitle
# entry. Add explicit boundaries after them. Single dots ("Mr.") stay
# spaCy's responsibility so abbreviations don't regress.
for m in re.finditer(r"\.{2,}(?=[\s\"'”’»›)\]}]|$)|…(?=[\s\"'”’»›)\]}]|$)", full_text):
sentence_boundaries.append(m.end())
# Double newlines are paragraph breaks: always split, even when spaCy
# sees no sentence boundary.
for m in re.finditer(r"\n{2,}", full_text):
sentence_boundaries.append(m.end())
sentence_boundaries = sorted(set(sentence_boundaries))
# Multi-sentence single FakeToken handling
if len(tokens) == 1 and len(sentence_boundaries) > 1:
single = tokens[0]
start_time = single.get("start", 0.0) or 0.0
end_time = single.get("end")
duration = (end_time - start_time) if (end_time is not None and end_time > start_time) else 0.0
prev_pos = 0
cur_start = start_time
total_chars = max(len(full_text), 1)
for i, b_pos in enumerate(sentence_boundaries):
piece = full_text[prev_pos:b_pos].strip()
if not piece:
prev_pos = b_pos
continue
if i == len(sentence_boundaries) - 1:
cur_end = end_time if end_time is not None else (cur_start + 1.0)
else:
cur_end = cur_start + duration * len(piece) / total_chars
subtitle_entries.append((cur_start, cur_end, piece))
cur_start = cur_end
prev_pos = b_pos
if prev_pos < len(full_text):
remainder = full_text[prev_pos:].strip()
if remainder:
remainder_end = end_time
if remainder_end is None:
remainder_end = fallback_end_time
if remainder_end is None:
remainder_end = cur_start
subtitle_entries.append((cur_start, remainder_end, remainder))
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
return
# Group tokens by sentence boundaries
current_sentence = []
word_count = 0
current_char_pos = 0
boundary_idx = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
text_len = len(str(token.get("text", ""))) + len(token.get("whitespace") or "")
current_char_pos += text_len
# Check if we've hit a sentence boundary or max words
at_boundary = (
boundary_idx < len(sentence_boundaries)
and current_char_pos >= sentence_boundaries[boundary_idx]
)
if at_boundary or word_count >= max_subtitle_words:
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = "".join(
str(t.get("text", "")) + (t.get("whitespace") or "")
for t in current_sentence
).strip()
if sentence_text:
subtitle_entries.append(
(start_time, end_time, sentence_text)
)
current_sentence = []
word_count = 0
while (
boundary_idx < len(sentence_boundaries)
and current_char_pos >= sentence_boundaries[boundary_idx]
):
boundary_idx += 1
# Add remaining tokens
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = "".join(
str(t.get("text", "")) + (t.get("whitespace") or "")
for t in current_sentence
).strip()
if sentence_text:
subtitle_entries.append(
(start_time, end_time, sentence_text)
)
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_regex_sentences(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens using regex for sentence boundary detection."""
# Define separator pattern based on mode
if subtitle_mode in (SubtitleMode.LINE.value, "Line"):
separator = r"\n"
elif subtitle_mode in (SubtitleMode.SENTENCE.value, "Sentence"):
separator = rf"[{PUNCTUATION_SENTENCE}]"
else: # Sentence + Comma
separator = rf"[{PUNCTUATION_SENTENCE_COMMA}]"
current_sentence = []
word_count = 0
for token in tokens:
current_sentence.append(token)
word_count += 1
# Split sentences based on separator or word count
is_boundary = _is_sentence_boundary(token, current_sentence, separator)
if is_boundary or word_count >= max_subtitle_words:
if current_sentence:
# Create subtitle entry for this sentence
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = "".join(
str(t.get("text", "")) + (t.get("whitespace") or "")
for t in current_sentence
).strip()
if sentence_text:
subtitle_entries.append(
(start_time, end_time, sentence_text)
)
current_sentence = []
word_count = 0
# Add any remaining tokens as a sentence (split multi-sentence FakeToken)
if current_sentence:
start_time = current_sentence[0]["start"]
end_time = current_sentence[-1]["end"]
sentence_text = "".join(
str(t.get("text", "")) + (t.get("whitespace") or "")
for t in current_sentence
).strip()
if len(current_sentence) == 1:
split_pat = (
r"\n+"
if separator == r"\n"
else rf"(?<={separator})\s+|(?<={separator}[{re.escape(_CLOSING_DELIMS)}])\s+"
)
parts = [p.strip() for p in re.split(split_pat, sentence_text) if p.strip()]
if len(parts) > 1:
d = (end_time - start_time) if (end_time is not None and start_time is not None and end_time > start_time) else 0.0
total_len = max(len(sentence_text), 1)
cur_s = start_time if start_time is not None else 0.0
for i, p in enumerate(parts):
if i == len(parts) - 1 and end_time is not None:
e = end_time
else:
e = cur_s + d * len(p) / total_len
subtitle_entries.append((cur_s, e, p))
cur_s = e
current_sentence = []
if current_sentence and sentence_text:
safe_start = start_time if start_time is not None else 0.0
safe_end = end_time
if safe_end is None:
safe_end = fallback_end_time
if safe_end is None:
safe_end = safe_start
subtitle_entries.append((safe_start, safe_end, sentence_text))
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _process_word_count(
tokens: List[dict],
subtitle_entries: List[Tuple[float, float, str]],
max_subtitle_words: int,
subtitle_mode: str,
fallback_end_time: Optional[float],
) -> None:
"""Process tokens by counting spaces (word count mode)."""
try:
word_count = int(subtitle_mode.split()[0])
word_count = min(word_count, max_subtitle_words)
except (ValueError, IndexError):
word_count = 1
current_group = []
space_count = 0
for token in tokens:
current_group.append(token)
# Count spaces after tokens (in the whitespace field)
if token.get("whitespace", "") == " ":
space_count += 1
# Split after counting N spaces
if space_count >= word_count:
text = "".join(
str(t.get("text", "")) + (t.get("whitespace") or "")
for t in current_group
).strip()
if text:
subtitle_entries.append(
(
current_group[0]["start"],
current_group[-1]["end"],
text,
)
)
current_group = []
space_count = 0
# Add any remaining tokens
if current_group:
text = "".join(
str(t.get("text", "")) + (t.get("whitespace") or "") for t in current_group
).strip()
if text:
subtitle_entries.append(
(current_group[0]["start"], current_group[-1]["end"], text)
)
# Fallback for last entry
_apply_fallback_end_time(subtitle_entries, fallback_end_time)
def _apply_fallback_end_time(
subtitle_entries: List[Tuple[float, float, str]],
fallback_end_time: Optional[float],
) -> None:
"""Apply fallback end time to the last entry if needed."""
if subtitle_entries and fallback_end_time is not None:
last_entry = subtitle_entries[-1]
start, end, text = last_entry
if end is None or end <= start or end <= 0:
subtitle_entries[-1] = (start, fallback_end_time, text)
-278
View File
@@ -1,278 +0,0 @@
"""Subtitle-to-audio processing pipeline.
Converts subtitle files (SRT/ASS/VTT/timestamp text) into audio by
generating TTS for each entry and mixing into a buffer.
"""
from __future__ import annotations
import logging
import time
from dataclasses import dataclass
from typing import Any, Callable, List, Optional, Tuple
import numpy as np
from abogen.domain.audio_buffer import (
fit_audio_to_duration,
ffmpeg_time_stretch,
mix_audio,
normalize_audio,
SAMPLE_RATE,
)
from abogen.domain.audio_helpers import to_float32
from abogen.domain.progress import calc_etr_str
from abogen.subtitle_utils import (
parse_ass_file,
parse_srt_file,
parse_vtt_file,
parse_timestamp_text_file,
)
logger = logging.getLogger(__name__)
@dataclass
class SubtitleEntry:
"""A single subtitle entry with timing."""
start: float
end: Optional[float]
text: str
def parse_subtitle_file(
file_path: str,
is_timestamp_text: bool = False,
) -> List[Tuple[float, Optional[float], str]]:
"""Parse a subtitle file into (start, end, text) tuples.
Args:
file_path: Path to subtitle file.
is_timestamp_text: Whether to treat as timestamp text file.
Returns:
List of (start_time, end_time, text) tuples.
"""
if is_timestamp_text:
return parse_timestamp_text_file(file_path)
import os
ext = os.path.splitext(file_path)[1].lower()
if ext == ".srt":
return parse_srt_file(file_path)
elif ext == ".vtt":
return parse_vtt_file(file_path)
else:
return parse_ass_file(file_path)
def format_time_range(
start: float,
end: Optional[float],
is_auto_end: bool = False,
) -> str:
"""Format a time range for display in logs.
Args:
start: Start time in seconds.
end: End time in seconds, or None.
is_auto_end: Whether end time is auto-detected.
Returns:
Formatted string like "00:01:23,456 - 00:01:25,789" or "00:01:23 - AUTO".
"""
def _fmt(seconds: float) -> str:
h = int(seconds // 3600)
m = int(seconds % 3600 // 60)
s = int(seconds % 60)
ms = int((seconds - int(seconds)) * 1000)
result = f"{h:02d}:{m:02d}:{s:02d}"
if ms > 0:
result += f",{ms:03d}"
return result
if is_auto_end or end is None:
return f"{_fmt(start)} - AUTO"
return f"{_fmt(start)} - {_fmt(end)}"
def speed_up_audio(
audio: np.ndarray,
speed_factor: float,
method: str = "tts",
*,
backend: Any = None,
text: str = "",
voice: Any = None,
base_speed: float = 1.0,
sample_rate: int = SAMPLE_RATE,
) -> np.ndarray:
"""Speed up audio to fit a time window.
Args:
audio: Input audio buffer.
speed_factor: Required speed multiplier.
method: "ffmpeg" for time-stretch, "tts" for regeneration.
backend: TTS backend (required if method="tts").
text: Text to regenerate (required if method="tts").
voice: Voice to use for regeneration.
base_speed: Base speed for TTS.
sample_rate: Sample rate.
Returns:
Speed-adjusted audio buffer.
"""
if speed_factor <= 1.0:
return audio
if method == "ffmpeg":
logger.info("FFmpeg time-stretch: %.2fx", speed_factor)
return ffmpeg_time_stretch(audio, speed_factor, sample_rate)
# TTS regeneration
if backend is None:
return audio
new_speed = base_speed * speed_factor
logger.info("Regenerating at %.2fx speed", new_speed)
results = [
r for r in backend(text, voice=voice, speed=new_speed, split_pattern=None)
]
chunks = [r.audio for r in results]
if not chunks:
return audio
return np.concatenate([to_float32(c) for c in chunks])
def process_subtitle_entries(
subtitles: List[Tuple[float, Optional[float], str]],
*,
backend: Any,
voice: Any,
speed: float = 1.0,
cancel_check: Callable[[], bool] = lambda: False,
log_callback: Optional[Callable[[str], None]] = None,
progress_callback: Optional[Callable[[int, str], None]] = None,
replace_newlines: bool = True,
use_gaps: bool = False,
is_timestamp_text: bool = False,
subtitle_speed_method: str = "tts",
sample_rate: int = SAMPLE_RATE,
) -> np.ndarray:
"""Process subtitle entries: generate TTS for each and mix into buffer.
This is the core domain logic for subtitle-to-audio conversion.
UI-specific concerns (signals, widgets) are handled via callbacks.
Args:
subtitles: List of (start, end, text) tuples.
backend: TTS pipeline callable.
voice: Resolved voice for TTS.
speed: TTS speed.
cancel_check: Returns True if processing should stop.
log_callback: Called with log messages.
progress_callback: Called with (percent, etr_string).
replace_newlines: Replace \\n with spaces in text.
use_gaps: Whether to use silent gaps between subtitles.
is_timestamp_text: Whether input is timestamp text.
subtitle_speed_method: "ffmpeg" or "tts" for speed adjustment.
sample_rate: Audio sample rate.
Returns:
Mixed audio buffer (float32).
"""
if not subtitles:
return np.array([], dtype="float32")
max_end = max((end for _, end, _ in subtitles if end is not None), default=0)
buffer_samples = int(max_end * sample_rate) + sample_rate
audio_buffer = np.zeros(buffer_samples, dtype="float32")
etr_start = time.time()
total = len(subtitles)
for idx, (start_time, end_time, text) in enumerate(subtitles, 1):
if cancel_check():
break
processed_text = text.replace("\n", " ") if replace_newlines else text
next_start = (
subtitles[idx][0]
if (use_gaps and idx < total)
else float("inf")
)
subtitle_duration = None if end_time is None else end_time - start_time
is_auto_end = is_timestamp_text or (use_gaps and idx == total) or end_time is None
if log_callback:
log_callback(
f"\n[{idx}/{total}] {format_time_range(start_time, end_time, is_auto_end)}: {processed_text}"
)
# Generate TTS
results = [
r for r in backend(
processed_text, voice=voice, speed=speed, split_pattern=None
)
if not cancel_check()
]
if cancel_check():
break
audio_chunks = [r.audio for r in results]
full_audio = (
np.concatenate([to_float32(a) for a in audio_chunks])
if audio_chunks
else np.zeros(int((subtitle_duration or 0) * sample_rate), dtype="float32")
)
audio_duration = len(full_audio) / sample_rate
# Timing adjustment
if is_timestamp_text:
end_time = start_time + audio_duration
subtitle_duration = audio_duration
elif use_gaps:
end_time = min(start_time + audio_duration, next_start)
subtitle_duration = end_time - start_time
elif subtitle_duration is None:
subtitle_duration = audio_duration
end_time = start_time + audio_duration
# Speed up if needed
speedup_threshold = next_start - start_time if use_gaps else subtitle_duration
if audio_duration > speedup_threshold and speedup_threshold > 0:
speed_factor = audio_duration / speedup_threshold
full_audio = speed_up_audio(
full_audio, speed_factor,
method=subtitle_speed_method,
backend=backend, text=processed_text,
voice=voice, base_speed=speed,
sample_rate=sample_rate,
)
audio_duration = len(full_audio) / sample_rate
# Adjust duration after speed change
if use_gaps:
end_time = min(start_time + audio_duration, next_start)
subtitle_duration = end_time - start_time
elif subtitle_duration is None:
subtitle_duration = audio_duration
end_time = start_time + audio_duration
# Pad or trim to subtitle duration
full_audio = fit_audio_to_duration(full_audio, subtitle_duration, sample_rate)
# Mix into buffer
start_sample = int(start_time * sample_rate)
audio_buffer = mix_audio(audio_buffer, full_audio, start_sample)
# Progress
if progress_callback:
percent = min(int(idx / total * 100), 99)
etr = calc_etr_str(time.time() - etr_start, idx, total)
progress_callback(percent, etr)
# Normalize if needed
if np.abs(audio_buffer).max() > 1.0:
logger.info("Normalizing audio (peak: %.2f)", np.abs(audio_buffer).max())
audio_buffer = normalize_audio(audio_buffer)
return audio_buffer
-59
View File
@@ -1,59 +0,0 @@
"""Chapter parsing from raw text.
Provides a unified function for splitting text by chapter markers,
used by both WebUI and PyQt conversion runners.
"""
from __future__ import annotations
import re
from typing import List, Tuple
from abogen.subtitle_utils import clean_text
_CHAPTER_MARKER_RE = re.compile(r"<<CHAPTER_MARKER:(.*?)>>", re.IGNORECASE)
def parse_chapters_from_text(
text: str,
default_title: str = "text",
clean: bool = True,
) -> List[Tuple[str, str]]:
"""Split raw text into chapters using chapter marker patterns.
Preserves content before the first marker as "Introduction" if present.
Optionally applies clean_text() to each chapter segment.
Args:
text: Raw text possibly containing <<CHAPTER_MARKER:Title>> markers.
default_title: Fallback title when no markers are found.
clean: Whether to apply clean_text() to each segment.
Returns:
List of (title, text) tuples.
"""
matches = list(_CHAPTER_MARKER_RE.finditer(text))
if not matches:
cleaned = clean_text(text) if clean else text
return [(default_title, cleaned)]
chapters: List[Tuple[str, str]] = []
# Preserve content before first marker as "Introduction"
first_start = matches[0].start()
if first_start > 0:
intro_text = text[:first_start].strip()
if intro_text:
chapters.append(("Introduction", clean_text(intro_text) if clean else intro_text))
for idx, match in enumerate(matches):
start = match.end()
end = matches[idx + 1].start() if idx + 1 < len(matches) else len(text)
chapter_name = match.group(1).strip() or default_title
chapter_text = text[start:end].strip()
if clean:
chapter_text = clean_text(chapter_text)
chapters.append((chapter_name, chapter_text))
return chapters
-22
View File
@@ -1,22 +0,0 @@
"""Text utility functions for the domain layer."""
from __future__ import annotations
import re
# Pre-compiled patterns for calculate_text_length
_METADATA_TAG_PATTERN = re.compile(r"<<METADATA_[^:]+:[^>]*>>")
_CHAPTER_MARKER_PATTERN = re.compile(r"<<CHAPTER_MARKER:[^>]*>>")
_VOICE_MARKER_PATTERN = re.compile(r"<<VOICE:[^>]*>>")
def calculate_text_length(text: str) -> int:
"""Calculate character count, ignoring internal markers and newlines.
Strips chapter markers, voice markers, and metadata tags before counting.
"""
text = _CHAPTER_MARKER_PATTERN.sub("", text)
text = _VOICE_MARKER_PATTERN.sub("", text)
text = _METADATA_TAG_PATTERN.sub("", text)
text = text.replace("\n", "").strip()
return len(text)
-97
View File
@@ -1,97 +0,0 @@
from __future__ import annotations
from pathlib import Path
from typing import Any, List, Mapping, Optional
from .metadata_helpers import (
ensure_sentence,
extract_series_metadata,
format_author_sentence,
format_series_sentence,
normalize_metadata_map,
)
def build_title_intro_text(
metadata: Optional[Mapping[str, Any]],
fallback_basename: str,
) -> str:
"""Build the title introduction text from metadata."""
normalized = normalize_metadata_map(metadata)
fallback_title = Path(fallback_basename).stem if fallback_basename else ""
title = (
normalized.get("title")
or normalized.get("book_title")
or normalized.get("album")
or fallback_title
)
if not title:
title = fallback_title
subtitle = normalized.get("subtitle") or normalized.get("sub_title")
if subtitle and title and subtitle.casefold() == title.casefold():
subtitle = ""
author_value = ""
for candidate in ("artist", "album_artist", "author", "authors", "writer", "composer"):
value = normalized.get(candidate)
if value:
author_value = value
break
series_name, series_number = extract_series_metadata(normalized)
series_sentence = format_series_sentence(series_name, series_number)
sentences: List[str] = []
if series_sentence:
sentences.append(ensure_sentence(series_sentence))
if title:
sentences.append(ensure_sentence(title))
if subtitle:
sentences.append(ensure_sentence(subtitle))
author_sentence = format_author_sentence(author_value)
if author_sentence:
sentences.append(ensure_sentence(author_sentence))
return " ".join(sentences).strip()
def build_outro_text(
metadata: Optional[Mapping[str, Any]],
fallback_basename: str,
) -> str:
"""Build the outro/closing text from metadata."""
normalized = normalize_metadata_map(metadata)
fallback_title = Path(fallback_basename).stem if fallback_basename else ""
title = (
normalized.get("title")
or normalized.get("book_title")
or normalized.get("album")
or fallback_title
)
author_value = ""
for candidate in ("authors", "author", "album_artist", "artist", "writer", "composer"):
value = normalized.get(candidate)
if value:
author_value = value
break
author_sentence = format_author_sentence(author_value)
authors_fragment = (
author_sentence[3:].strip() if author_sentence.lower().startswith("by ") else author_sentence.strip()
)
if title and authors_fragment:
closing_line = f"The end of {title} from {authors_fragment}"
elif title:
closing_line = f"The end of {title}"
elif authors_fragment:
closing_line = f"The end from {authors_fragment}"
else:
closing_line = "The end"
series_name, series_number = extract_series_metadata(normalized)
series_sentence = format_series_sentence(series_name, series_number)
sentences: List[str] = [ensure_sentence(closing_line)]
if series_sentence:
sentences.append(ensure_sentence(series_sentence))
return " ".join(sentence for sentence in sentences if sentence).strip()
-13
View File
@@ -1,13 +0,0 @@
"""Shared token stubs for TTS processing."""
from __future__ import annotations
class FakeToken:
"""Minimal token stub for languages without per-word token support."""
def __init__(self, text: str, start: float, end: float):
self.text = text
self.start_ts = start
self.end_ts = end
self.whitespace = ""
-112
View File
@@ -1,112 +0,0 @@
"""Voice catalog — shared voice metadata for all UIs.
Builds a unified catalog of available voices with metadata (language,
gender, display name). Used by both WebUI and PyQt for voice selection UIs.
"""
from __future__ import annotations
from typing import Any, Dict, Iterable, List, Mapping, Optional
from abogen.constants import LANGUAGE_DESCRIPTIONS
from abogen.tts_plugin.utils import get_voices
def build_voice_catalog() -> List[Dict[str, str]]:
"""Build voice catalog with metadata for all available voices.
Returns a list of dicts, each containing:
- id: voice ID (e.g. "af_heart")
- language: language code (e.g. "a", "e")
- language_label: human-readable language name
- gender: "Female", "Male", or "Unknown"
- gender_code: "f", "m", or ""
- display_name: human-readable voice name
"""
from plugins.kokoro.engine import language_for_voice_id
catalog: List[Dict[str, str]] = []
gender_map = {"f": "Female", "m": "Male"}
for voice_id in get_voices("kokoro"):
prefix, _, rest = voice_id.partition("_")
gender_code = prefix[1] if len(prefix) > 1 else ""
lang = language_for_voice_id(voice_id)
catalog.append(
{
"id": voice_id,
"language": lang.value,
"language_label": LANGUAGE_DESCRIPTIONS.get(lang, lang.value.upper()),
"gender": gender_map.get(gender_code, "Unknown"),
"gender_code": gender_code,
"display_name": rest.replace("_", " ").title() if rest else voice_id,
}
)
return catalog
def filter_voice_catalog(
catalog: Iterable[Mapping[str, Any]],
*,
gender: str,
allowed_languages: Optional[Iterable[str]] = None,
) -> List[str]:
"""Filter voice catalog by gender and language.
Returns voice IDs that match the criteria. Falls back to broader
matches if no exact matches are found.
Args:
catalog: Voice catalog entries (from build_voice_catalog).
gender: Gender filter ("male", "female", or "unknown").
allowed_languages: Optional list of allowed language codes.
Returns:
List of matching voice IDs.
"""
allowed_set = {code.lower() for code in (allowed_languages or []) if isinstance(code, str) and code}
gender_normalized = (gender or "unknown").lower()
gender_code = ""
if gender_normalized == "male":
gender_code = "m"
elif gender_normalized == "female":
gender_code = "f"
matches: List[str] = []
seen: set[str] = set()
def _consider(entry: Mapping[str, Any]) -> None:
voice_id = entry.get("id")
if not isinstance(voice_id, str) or not voice_id:
return
if voice_id in seen:
return
seen.add(voice_id)
matches.append(voice_id)
primary: List[Mapping[str, Any]] = []
fallback: List[Mapping[str, Any]] = []
for entry in catalog:
if not isinstance(entry, Mapping):
continue
voice_lang = str(entry.get("language", "")).lower()
voice_gender_code = str(entry.get("gender_code", "")).lower()
if allowed_set and voice_lang not in allowed_set:
continue
if gender_code and voice_gender_code != gender_code:
fallback.append(entry)
continue
primary.append(entry)
for entry in primary:
_consider(entry)
if not matches:
for entry in fallback:
_consider(entry)
if not matches:
for entry in catalog:
if isinstance(entry, Mapping):
_consider(entry)
return matches
-128
View File
@@ -1,128 +0,0 @@
"""Voice loading and caching utilities.
This module provides unified voice loading with caching support for both
PyQt and WebUI interfaces.
"""
from __future__ import annotations
from typing import Any, Dict, Optional
from abogen.voice_formulas import get_new_voice
class VoiceCache:
"""Thread-safe voice cache for loaded voice tensors."""
def __init__(self):
self._cache: Dict[str, Any] = {}
def get(self, voice_spec: str) -> Optional[Any]:
"""Get cached voice by spec."""
return self._cache.get(voice_spec)
def set(self, voice_spec: str, voice: Any) -> None:
"""Cache a loaded voice."""
self._cache[voice_spec] = voice
def contains(self, voice_spec: str) -> bool:
"""Check if voice is in cache."""
return voice_spec in self._cache
def clear(self) -> None:
"""Clear all cached voices."""
self._cache.clear()
def keys(self):
"""Return cached voice specs."""
return self._cache.keys()
def __contains__(self, voice_spec: str) -> bool:
return self.contains(voice_spec)
def resolve_voice(
voice_spec: str,
pipeline: Any,
use_gpu: bool,
cache: Optional[VoiceCache] = None,
) -> Any:
"""Resolve voice spec to actual voice tensor or name.
If voice_spec contains '*' (formula), loads the voice using get_new_voice.
Otherwise, returns the voice_spec as-is (it's a voice name).
Uses optional cache to avoid reloading same voice multiple times.
Args:
voice_spec: Voice specification (name or formula string with '*').
pipeline: TTS pipeline instance for loading formula voices.
use_gpu: Whether to use GPU for voice loading.
cache: Optional VoiceCache instance for caching loaded voices.
Returns:
Loaded voice tensor (for formulas) or voice name string.
"""
# Check cache first
if cache and cache.contains(voice_spec):
return cache.get(voice_spec)
# Load voice
if "*" in voice_spec:
if pipeline is None or not hasattr(pipeline, "load_single_voice"):
return voice_spec
loaded_voice = get_new_voice(pipeline, voice_spec, use_gpu)
else:
loaded_voice = voice_spec
# Cache it
if cache:
cache.set(voice_spec, loaded_voice)
return loaded_voice
def load_voice_cached(
voice_name: str,
pipeline: Any,
use_gpu: bool,
cache: Any = None,
) -> Any:
"""Load voice with caching (compatibility wrapper for PyQt).
This function maintains backward compatibility with the PyQt interface
while using the unified voice loading logic.
Args:
voice_name: Voice name or formula string.
pipeline: TTS pipeline instance.
use_gpu: Whether to use GPU.
cache: Optional VoiceCache or dict to use as cache.
Returns:
Loaded voice tensor or voice name string.
"""
# Check cache (supports both VoiceCache and plain dict)
if cache is not None:
if isinstance(cache, VoiceCache):
if cache.contains(voice_name):
return cache.get(voice_name)
elif voice_name in cache:
return cache[voice_name]
# Load voice
if "*" in voice_name:
if pipeline is None or not hasattr(pipeline, "load_single_voice"):
return voice_name
loaded_voice = get_new_voice(pipeline, voice_name, use_gpu)
else:
loaded_voice = voice_name
# Cache it
if cache is not None:
if isinstance(cache, VoiceCache):
cache.set(voice_name, loaded_voice)
else:
cache[voice_name] = loaded_voice
return loaded_voice
-117
View File
@@ -1,117 +0,0 @@
"""Voice marker parsing and text splitting.
Handles <<VOICE:name>> markers in text, splitting text into voice-specific
segments. This is domain logic about text segmentation by voice, not subtitle
processing.
"""
from __future__ import annotations
import re
from typing import List, Tuple
_VOICE_MARKER_PATTERN = re.compile(r"<<VOICE:[^>]*>>")
_VOICE_MARKER_SEARCH_PATTERN = re.compile(r"<<VOICE:(.*?)>>")
def validate_voice_name(voice_name: str) -> Tuple[bool, str | None]:
"""Validate voice name against available voices (case-insensitive).
Handles both single voices and formulas like 'af_heart*0.5 + am_echo*0.5'.
Returns:
Tuple of (is_valid, invalid_voice_name):
- is_valid: True if all voices in the name/formula are valid
- invalid_voice_name: The first invalid voice found, or None if all valid
"""
from abogen.tts_plugin.utils import get_voices
voice_lookup_lower = {v.lower() for v in get_voices("kokoro")}
voice_name = voice_name.strip()
if "*" in voice_name:
voices = voice_name.split("+")
for term in voices:
if "*" in term:
base_voice = term.split("*")[0].strip()
if base_voice.lower() not in voice_lookup_lower:
return False, base_voice
return True, None
else:
if voice_name.lower() not in voice_lookup_lower:
return False, voice_name
return True, None
def split_text_by_voice_markers(
text: str, default_voice: str
) -> Tuple[List[Tuple[str, str]], str, int, int]:
"""Split text by voice markers, returning list of (voice, text) tuples.
Returns the last voice used so it can persist across chapters.
Voice names are normalized to lowercase to match canonical voice names.
Args:
text: Text potentially containing <<VOICE:name>> markers
default_voice: Voice to use if no markers found or before first marker
Returns:
Tuple of (segments_list, last_voice_used, valid_count, invalid_count):
- segments_list: List of (voice_name, segment_text) tuples
- last_voice_used: The voice that should continue into next chapter
- valid_count: Number of valid voice markers processed
- invalid_count: Number of invalid voice markers skipped
"""
from abogen.tts_plugin.utils import get_voices
voice_splits = list(_VOICE_MARKER_SEARCH_PATTERN.finditer(text))
if not voice_splits:
return [(default_voice, text)], default_voice, 0, 0
segments: List[Tuple[str, str]] = []
current_voice = default_voice
valid_markers = 0
invalid_markers = 0
first_start = voice_splits[0].start()
if first_start > 0:
intro_text = text[:first_start].strip()
if intro_text:
segments.append((current_voice, intro_text))
for idx, match in enumerate(voice_splits):
voice_name = match.group(1).strip()
start = match.end()
end = voice_splits[idx + 1].start() if idx + 1 < len(voice_splits) else len(text)
segment_text = text[start:end].strip()
is_valid, invalid_voice = validate_voice_name(voice_name)
if is_valid:
if "*" in voice_name:
normalized_parts = []
for part in voice_name.split("+"):
part = part.strip()
if "*" in part:
voice_part, weight = part.split("*", 1)
voice_part_lower = voice_part.strip().lower()
canonical_voice = next(
(v for v in get_voices("kokoro") if v.lower() == voice_part_lower),
voice_part.strip()
)
normalized_parts.append(f"{canonical_voice}*{weight.strip()}")
current_voice = " + ".join(normalized_parts)
else:
voice_name_lower = voice_name.lower()
current_voice = next(
(v for v in get_voices("kokoro") if v.lower() == voice_name_lower),
voice_name
)
valid_markers += 1
else:
invalid_markers += 1
if segment_text:
segments.append((current_voice, segment_text))
return segments, current_voice, valid_markers, invalid_markers
-355
View File
@@ -1,355 +0,0 @@
"""Voice resolution helpers.
Functions for resolving voice specifications, collecting required voice IDs,
and determining the voice to use for chapters and chunks.
All functions accept ConversionRequest (the app-layer contract) instead of
UI-specific objects. This keeps the domain layer UI-agnostic.
"""
from __future__ import annotations
from typing import Any, Dict, Mapping, Optional, Set, Tuple
from abogen.tts_plugin.utils import get_voices, get_default_voice
from abogen.voice_formulas import extract_voice_ids, pairs_to_formula
from abogen.voice_cache import ensure_voice_assets
def spec_to_voice_ids(spec: Any) -> Set[str]:
text = str(spec or "").strip()
if not text:
return set()
if text == "__custom_mix":
return set()
if "*" in text:
try:
return set(extract_voice_ids(text))
except ValueError:
return set()
if text in get_voices("kokoro"):
return {text}
return set()
def _get_chapter_overrides(request: Any) -> list:
"""Extract chapter overrides from ConversionRequest."""
cc = getattr(request, "chapter_chunk", None)
if cc is not None:
return getattr(cc, "chapter_overrides", []) or []
return []
def _get_chunks(request: Any) -> list:
"""Extract chunks from ConversionRequest."""
cc = getattr(request, "chapter_chunk", None)
if cc is not None:
return getattr(cc, "chunks", []) or []
return []
def job_voice_fallback(request: Any) -> str:
base = str(getattr(request, "voice", "") or "").strip()
if base and base != "__custom_mix":
return base
speakers = getattr(request, "speakers", None)
if isinstance(speakers, dict):
narrator = speakers.get("narrator")
if isinstance(narrator, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = narrator.get(key)
candidate = str(value or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
for payload in speakers.values() or []:
if not isinstance(payload, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
value = payload.get(key)
candidate = str(value or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
for chapter in _get_chapter_overrides(request):
if not isinstance(chapter, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
candidate = str(chapter.get(key) or "").strip()
if candidate and candidate != "__custom_mix":
return candidate
return ""
def collect_required_voice_ids(request: Any) -> Set[str]:
voices: Set[str] = set()
voices.update(spec_to_voice_ids(request.voice))
voices.update(spec_to_voice_ids(job_voice_fallback(request)))
for chapter in _get_chapter_overrides(request):
if not isinstance(chapter, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(chapter.get(key)))
for chunk in _get_chunks(request):
if not isinstance(chunk, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(chunk.get(key)))
speakers = getattr(request, "speakers", {})
if isinstance(speakers, dict):
for payload in speakers.values() or []:
if not isinstance(payload, dict):
continue
for key in ("resolved_voice", "voice_formula", "voice"):
voices.update(spec_to_voice_ids(payload.get(key)))
voices.update(get_voices("kokoro"))
return voices
def initialize_voice_cache(request: Any, events: Any = None) -> None:
"""Initialize voice cache by downloading required voice assets.
Args:
request: ConversionRequest with voice/chapter/chunk/speaker info.
events: ConversionEvents for logging (optional, for backward compat).
"""
log = (lambda msg, level="info": events.log(msg, level=level)) if events else (lambda msg, level="info": None)
try:
targets = collect_required_voice_ids(request)
downloaded, errors = ensure_voice_assets(
targets,
on_progress=lambda message: log(message, level="debug"),
)
except RuntimeError as exc:
log(f"Voice cache unavailable: {exc}", level="warning")
return
if downloaded:
log(
f"Cached {len(downloaded)} voice asset{'s' if len(downloaded) != 1 else ''} locally.",
level="info",
)
for voice_id, error in errors.items():
log(f"Failed to cache voice '{voice_id}': {error}", level="warning")
def chapter_voice_spec(request: Any, override: Optional[Dict[str, Any]]) -> str:
if not override:
return job_voice_fallback(request)
resolved = str(override.get("resolved_voice", "")).strip()
if resolved:
return resolved
formula = str(override.get("voice_formula", "")).strip()
if formula:
return formula
voice = str(override.get("voice", "")).strip()
if voice:
return voice
return job_voice_fallback(request)
def chunk_voice_spec(request: Any, chunk: Dict[str, Any], fallback: str) -> str:
for key in ("resolved_voice", "voice_formula", "voice"):
value = chunk.get(key)
if value:
return str(value)
speaker_id = chunk.get("speaker_id")
speakers = getattr(request, "speakers", None)
if isinstance(speakers, dict) and speaker_id in speakers:
speaker_entry = speakers.get(speaker_id) or {}
if isinstance(speaker_entry, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = speaker_entry.get(key)
if value:
return str(value)
profile_formula = speaker_entry.get("voice_formula")
if profile_formula:
return str(profile_formula)
profile_name = chunk.get("voice_profile")
if profile_name:
if isinstance(speakers, dict):
speaker_entry = speakers.get(profile_name)
if isinstance(speaker_entry, dict):
for key in ("resolved_voice", "voice_formula", "voice"):
value = speaker_entry.get(key)
if value:
return str(value)
if fallback:
return fallback
return job_voice_fallback(request)
def resolve_fallback_voice_spec(
base_spec: str,
job_voice: str,
voice_cache_keys: list[str],
provider: str = "kokoro",
) -> str:
"""Resolve the voice spec for intro/outro with a priority fallback chain.
Priority: base_spec job_voice first voice_cache key default voice.
``"__custom_mix"`` is treated as empty (it is not a usable voice spec).
"""
spec = base_spec or job_voice
if spec == "__custom_mix":
spec = job_voice or ""
if not spec:
for key in voice_cache_keys:
if key and key != "__custom_mix":
spec = key.split(":", 1)[-1]
break
if not spec:
spec = get_default_voice(provider)
return spec
# ---------------------------------------------------------------------------
# Voice choice resolution (shared by all UIs)
# ---------------------------------------------------------------------------
def formula_from_profile(entry: Dict[str, Any]) -> Optional[str]:
"""Convert a voice profile entry to a voice formula string.
Handles both Kokoro (voices list) and SuperTonic (single voice) profiles.
Returns None if the entry has no usable voice data.
"""
if not isinstance(entry, dict):
return None
voices = entry.get("voices") or []
if not voices:
return None
return pairs_to_formula(voices)
def resolve_profile_voice(
profile_name: Optional[str],
*,
profiles: Optional[Mapping[str, Any]] = None,
) -> Tuple[str, Optional[str]]:
"""Resolve a profile name to (formula, language).
Args:
profile_name: Name of the profile to resolve.
profiles: Pre-loaded profiles dict. If None, loads from disk.
Returns:
(formula_string, language_code) or ("", None) if not found.
"""
if not profile_name:
return "", None
source = profiles if isinstance(profiles, Mapping) else None
if source is None:
from abogen.voice_profiles import load_profiles
source = load_profiles()
entry = source.get(profile_name) if isinstance(source, Mapping) else None
if not isinstance(entry, Mapping):
return "", None
formula = formula_from_profile(dict(entry)) or ""
language = entry.get("language") if isinstance(entry.get("language"), str) else None
if isinstance(language, str):
language = language.strip().lower() or None
return formula, language
def resolve_voice_setting(
value: Any,
*,
profiles: Optional[Mapping[str, Any]] = None,
) -> Tuple[str, Optional[str], Optional[str]]:
"""Resolve a raw voice setting value into (spec, profile_name, language).
Parses 'profile:name' or 'speaker:name' prefixes and resolves
the profile to a formula string.
Args:
value: Raw voice value from user input (e.g. "af_heart", "profile:MyMix").
profiles: Pre-loaded profiles dict. If None, loads from disk.
Returns:
(resolved_spec, profile_name, language) profile_name and language
are None when the input is a plain voice spec.
"""
from abogen.domain.settings_core import split_profile_spec
base_spec, profile_name = split_profile_spec(value)
if profile_name:
formula, language = resolve_profile_voice(profile_name, profiles=profiles)
return formula or "", profile_name, language
return base_spec, None, None
def resolve_voice_choice(
language: str,
base_voice: str,
profile_name: str,
custom_formula: str,
profiles: Dict[str, Any],
) -> Tuple[str, str, Optional[str]]:
"""Resolve a user's voice selection into (resolved_voice, resolved_language, selected_profile).
Handles three input modes:
1. Profile selection resolves to formula (Kokoro) or speaker reference (SuperTonic)
2. Custom formula used directly
3. Plain voice spec passed through
Args:
language: Current language code (e.g. "a", "e").
base_voice: Base voice spec (voice ID or formula).
profile_name: Selected profile name (empty string if none).
custom_formula: Custom formula string (empty string if none).
profiles: Dict of all available profiles.
Returns:
(resolved_voice, resolved_language, selected_profile)
"""
from abogen.voice_profiles import normalize_profile_entry
resolved_voice = base_voice
resolved_language = language
selected_profile = None
if profile_name:
entry_raw = profiles.get(profile_name)
entry = normalize_profile_entry(entry_raw)
provider = str((entry or {}).get("provider") or "").strip().lower()
# Provider-aware behavior:
# - Kokoro profiles typically represent mixes (formula strings).
# - SuperTonic profiles represent a discrete voice id + settings.
# In that case, we return a speaker reference so downstream can
# resolve provider per-speaker and allow mixed-provider casting.
if provider == "supertonic":
resolved_voice = f"speaker:{profile_name}"
selected_profile = profile_name
profile_language = (entry or {}).get("language")
if profile_language:
resolved_language = str(profile_language)
else:
formula = formula_from_profile(entry or {}) if entry else None
if formula:
resolved_voice = formula
selected_profile = profile_name
profile_language = (entry or {}).get("language")
if profile_language:
resolved_language = profile_language
if custom_formula:
resolved_voice = custom_formula
selected_profile = None
return resolved_voice, resolved_language, selected_profile
-129
View File
@@ -1,129 +0,0 @@
from __future__ import annotations
from typing import Any, Dict, Mapping, Optional, Tuple
from abogen.tts_plugin.utils import get_voices
def infer_provider_from_spec(value: Any, fallback: str = "kokoro") -> str:
"""Infer TTS provider from voice specification."""
raw = str(value or "").strip()
if not raw:
return fallback
if raw.upper() == raw and raw.replace("_", "").isalnum():
return "supertonic"
if raw == "__custom_mix" or "*" in raw or "+" in raw:
return "kokoro"
if raw in get_voices("kokoro"):
return "kokoro"
return fallback
def supertonic_voice_from_spec(spec: Any, fallback: str) -> str:
"""Normalize a voice specification for Supertonic.
This function only performs Supertonic-specific normalization (uppercase conversion
and fallback handling). Backend resolution is handled by the registry.
"""
raw = str(spec or "").strip()
fallback_raw = str(fallback or "").strip()
# Normalize to uppercase for Supertonic voice IDs
upper = raw.upper() if raw else ""
# If empty or contains formula characters, use fallback
if not upper or "*" in upper or "+" in upper:
upper = fallback_raw.upper() if fallback_raw else ""
# If still empty, use default Supertonic voice
if not upper or "*" in upper or "+" in upper:
upper = "M1"
return upper
def split_speaker_reference(value: Any) -> Tuple[Optional[str], str]:
"""Parse speaker/profile reference from string.
Expected format: "speaker:name" or "profile:name"
Returns (name, original) or (None, original) if not a valid reference.
"""
raw = str(value or "").strip()
if not raw or ":" not in raw:
return None, raw
prefix, remainder = raw.split(":", 1)
prefix = prefix.strip().lower()
if prefix not in {"speaker", "profile"}:
return None, raw
name = remainder.strip()
return (name or None), raw
def formula_from_kokoro_entry(entry: Mapping[str, Any]) -> str:
"""Build voice formula string from kokoro entry."""
voices = entry.get("voices") or []
if not voices:
return ""
total = 0.0
parts: list[tuple[str, float]] = []
for item in voices:
if not isinstance(item, (list, tuple)) or len(item) < 2:
continue
name = str(item[0] or "").strip()
try:
weight = float(item[1])
except (TypeError, ValueError):
continue
if name and weight > 0:
parts.append((name, weight))
total += weight
if not parts:
return ""
normalized = [(name, weight / total) for name, weight in parts]
return " + ".join(f"{name}*{weight:.6f}" for name, weight in normalized)
def coerce_truthy(value: Any, default: bool = True) -> bool:
"""Coerce a value to boolean with default."""
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.lower() not in {"false", "0", "no", "off", ""}
if value is None:
return default
return bool(value)
def resolve_voice_target(
raw_spec: str,
normalized_profiles: Dict[str, Dict[str, Any]],
*,
job_voice: str = "M1",
job_tts_provider: str = "kokoro",
job_supertonic_total_steps: int = 5,
job_speed: float = 1.0,
) -> Tuple[str, str, Optional[float], Optional[int]]:
"""Resolve a raw voice spec into (provider, voice_spec, speed_override, steps_override).
Pure function all dependencies are passed as parameters.
"""
spec = str(raw_spec or "").strip()
speaker_name, _ = split_speaker_reference(spec)
if speaker_name and speaker_name in normalized_profiles:
entry = normalized_profiles[speaker_name]
provider = str(entry.get("provider") or "kokoro").strip().lower() or "kokoro"
if provider == "supertonic":
voice = str(entry.get("voice") or job_voice or "M1").strip() or "M1"
steps = int(entry.get("total_steps") or job_supertonic_total_steps or 5)
speed = float(entry.get("speed") or job_speed or 1.0)
return "supertonic", supertonic_voice_from_spec(voice, job_voice), speed, steps
formula = formula_from_kokoro_entry(entry)
return "kokoro", formula or spec, None, None
fallback_provider = str(job_tts_provider or "kokoro").strip().lower() or "kokoro"
inferred = infer_provider_from_spec(spec, fallback=fallback_provider)
if inferred == "supertonic":
return "supertonic", supertonic_voice_from_spec(spec, job_voice), None, None
return "kokoro", spec, None, None
+5 -17
View File
@@ -9,26 +9,15 @@ from collections import Counter
from dataclasses import dataclass, field
from typing import Any, Dict, Iterable, List, Mapping, Optional, Tuple
try: # pragma: no cover - fallback when spaCy not available during tests
import spacy # type: ignore[import-not-found]
except Exception: # pragma: no cover - spaCy optional during runtime bootstrap
spacy = None
_Language = Any # type: ignore[misc,assignment]
Doc = Any # type: ignore[misc,assignment]
Span = Any # type: ignore[misc,assignment]
_SPACY: Any = None
_SPACY_LOADED = False
def _get_spacy() -> Any:
"""Import spaCy lazily (it pulls in torch/thinc, ~2s at startup)."""
global _SPACY, _SPACY_LOADED
if not _SPACY_LOADED:
_SPACY_LOADED = True
try: # pragma: no cover - fallback when spaCy not available during tests
import spacy # type: ignore[import-not-found]
except Exception: # pragma: no cover - spaCy optional during runtime bootstrap
spacy = None
_SPACY = spacy
return _SPACY
_TITLE_PREFIXES = (
"mr",
@@ -178,7 +167,6 @@ def _resolve_model_name(language: str) -> str:
def _load_model(language: str) -> Any:
spacy = _get_spacy()
if spacy is None:
raise EntityModelError(
"spaCy is not available. Install spaCy to enable entity extraction."
+13 -15
View File
@@ -12,7 +12,6 @@ from typing import Any, Dict, Iterable, List, Optional, Pattern, Sequence, Tuple
import zipfile
from abogen.text_extractor import ExtractedChapter, ExtractionResult
from abogen.domain.metadata_helpers import normalize_metadata_map
@dataclass(slots=True)
@@ -23,7 +22,7 @@ class ChunkOverlay:
start: Optional[float]
end: Optional[float]
speaker_id: str
voice: Optional[Dict[str, str]]
voice: Optional[str]
level: Optional[str] = None
group_id: Optional[str] = None
@@ -60,7 +59,7 @@ class EPUB3PackageBuilder:
self.output_path = output_path
self.book_id = book_id or str(uuid.uuid4())
self.extraction = extraction
self.metadata_tags = normalize_metadata_map(metadata_tags)
self.metadata_tags = _normalize_metadata(metadata_tags)
self.chapter_markers = list(chapter_markers or [])
self.chunk_markers = list(chunk_markers or [])
self.chunks = list(chunks or [])
@@ -274,7 +273,7 @@ class EPUB3PackageBuilder:
start=_safe_float(marker.get("start")),
end=_safe_float(marker.get("end")),
speaker_id=speaker_id,
voice=voice if isinstance(voice, dict) else None,
voice=str(voice) if voice else None,
level=str(level) if level else None,
group_id=normalized_group_id,
)
@@ -517,14 +516,9 @@ def build_epub3_package(
chunks: Iterable[Dict[str, Any]],
audio_path: Path,
speaker_mode: str = "single",
cover: "CoverConfig | None" = None,
cover_image_path: Optional[Path] = None,
cover_image_mime: Optional[str] = None,
) -> Path:
from abogen.domain.config_types import CoverConfig
if isinstance(cover, CoverConfig):
cover_image_path = cover.path
cover_image_mime = cover.mime
builder = EPUB3PackageBuilder(
output_path=output_path,
book_id=book_id,
@@ -551,6 +545,15 @@ class ChunkLookup:
by_chapter: Dict[int, List[Dict[str, Any]]]
def _normalize_metadata(metadata: Optional[Dict[str, Any]]) -> Dict[str, str]:
normalized: Dict[str, str] = {}
for key, value in (metadata or {}).items():
if value is None:
continue
normalized[str(key).lower()] = str(value)
return normalized
def _combine_metadata(*sources: Dict[str, Any]) -> Dict[str, str]:
combined: Dict[str, str] = {}
for source in sources:
@@ -693,12 +696,7 @@ def _group_chunks_for_render(chunks: Sequence[ChunkOverlay]) -> List[Tuple[Optio
def _render_chunk_inline(chunk: ChunkOverlay) -> str:
escaped_id = html.escape(chunk.id)
speaker_attr = f" data-speaker=\"{html.escape(chunk.speaker_id)}\"" if chunk.speaker_id else ""
voice_str = None
if chunk.voice and isinstance(chunk.voice, dict):
name = chunk.voice.get("voice", "")
provider = chunk.voice.get("provider", "")
voice_str = f"{name}@{provider}" if name and provider else name or None
voice_attr = f" data-voice=\"{html.escape(voice_str)}\"" if voice_str else ""
voice_attr = f" data-voice=\"{html.escape(chunk.voice)}\"" if chunk.voice else ""
level_attr = f" data-level=\"{html.escape(chunk.level)}\"" if chunk.level else ""
raw_text = chunk.text or ""
escaped_text = html.escape(raw_text)
+8
View File
@@ -0,0 +1,8 @@
"""
Abogen Flet Frontend Package.
This package provides a unified, dual-target (desktop + web) user interface
for the Abogen audiobook generation application, built with the Flet framework.
"""
__all__ = ["main"]
+32
View File
@@ -0,0 +1,32 @@
"""Components sub-package."""
from .widgets import (
resolve_icon,
build_drop_zone,
build_log_terminal,
log_entry,
build_progress_row,
build_primary_button,
build_secondary_button,
build_card,
build_section_header,
build_status_badge,
labelled_row,
show_snack,
build_divider,
)
__all__ = [
"build_drop_zone",
"resolve_icon",
"build_log_terminal",
"log_entry",
"build_progress_row",
"build_primary_button",
"build_secondary_button",
"build_card",
"build_section_header",
"build_status_badge",
"labelled_row",
"show_snack",
"build_divider",
]
+630
View File
@@ -0,0 +1,630 @@
"""
Reusable UI components for the Abogen Flet frontend.
Each function in this module returns a standalone Flet control or small
widget tree. Components read the current palette from the page's theme
mode and should not hold any mutable state themselves state lives in the
session's ``AppState`` object.
"""
from __future__ import annotations
from typing import Any, Callable, List, Optional
import flet as ft
from ..utils.theme import get_palette, RADIUS_MD, RADIUS_SM, SPACE_SM, SPACE_MD, SPACE_LG
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def resolve_icon(icon: Any) -> Any:
"""Convert a snake_case icon name to Flet IconData when possible."""
if isinstance(icon, str):
return getattr(ft.Icons, icon.upper(), icon)
return icon
# ---------------------------------------------------------------------------
# Drop-zone (file input area)
# ---------------------------------------------------------------------------
def build_drop_zone(
*,
on_pick: Callable[[], None],
label: str = "Drag & drop your file here or click to browse",
sub_label: str = "Supports: .txt · .epub · .pdf · .md · .srt · .ass · .vtt",
accent: bool = False,
error: bool = False,
filename: Optional[str] = None,
file_size: Optional[str] = None,
char_count: Optional[str] = None,
page: Optional[ft.Page] = None,
) -> ft.GestureDetector:
"""
Build an interactive file drop-zone widget.
The zone shows a dashed border and centred instructions by default,
switching to an 'active' green style when a file is loaded and a red
style when an error has occurred.
Args:
on_pick: Callback invoked when the user clicks or activates the zone.
label: Primary instruction text.
sub_label: Secondary hint text shown beneath the label.
accent: When True, renders the 'active/success' green style.
error: When True, renders the 'error/red' style.
filename: When provided, replaces the instruction text with file info.
file_size: Human-readable file size to display alongside the filename.
char_count: Character count to display alongside file info.
page: The current Flet ``Page``; used to derive the active palette.
Returns:
A ``ft.GestureDetector`` wrapping the visual drop-zone container.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
p = get_palette(page) if page else None
# Colour scheme
if error:
border_color = "#e84e3c" if dark else "#c0392b"
bg_color = "#1a0a08" if dark else "#fff5f5"
text_color = "#e84e3c" if dark else "#c0392b"
icon_name = "error_outline"
elif accent:
border_color = "#42ad4a" if dark else "#2e9437"
bg_color = "#091810" if dark else "#f0fff1"
text_color = "#42ad4a" if dark else "#2e9437"
icon_name = "check_circle_outline"
else:
border_color = "#3a4466" if dark else "#a8b4d0"
bg_color = "#151928" if dark else "#f7f8fd"
text_color = "#9ba3b8" if dark else "#5a6172"
icon_name = "upload_file"
if filename:
# Compact file-info display
info_rows: List[ft.Control] = [
ft.Row(
[
ft.Icon(resolve_icon("insert_drive_file"), color=text_color, size=28),
ft.Column(
[
ft.Text(
filename,
weight=ft.FontWeight.W_600,
size=13,
color=text_color,
no_wrap=False,
max_lines=2,
overflow=ft.TextOverflow.ELLIPSIS,
),
],
tight=True,
expand=True,
),
],
alignment=ft.MainAxisAlignment.CENTER,
spacing=SPACE_SM,
)
]
if file_size or char_count:
chips: List[ft.Control] = []
if file_size:
chips.append(
ft.Text(f"📄 {file_size}", size=11, color=text_color, italic=True)
)
if char_count:
chips.append(
ft.Text(f"🔤 {char_count} chars", size=11, color=text_color, italic=True)
)
info_rows.append(
ft.Row(chips, alignment=ft.MainAxisAlignment.CENTER, spacing=SPACE_MD)
)
content = ft.Column(
info_rows,
alignment=ft.MainAxisAlignment.CENTER,
horizontal_alignment=ft.CrossAxisAlignment.CENTER,
spacing=SPACE_SM,
)
else:
content = ft.Column(
[
ft.Icon(resolve_icon(icon_name), size=48, color=border_color, opacity=0.8),
ft.Text(
label,
size=14,
weight=ft.FontWeight.W_500,
color=text_color,
text_align=ft.TextAlign.CENTER,
),
ft.Text(
sub_label,
size=11,
color=text_color,
opacity=0.6,
text_align=ft.TextAlign.CENTER,
),
],
alignment=ft.MainAxisAlignment.CENTER,
horizontal_alignment=ft.CrossAxisAlignment.CENTER,
spacing=SPACE_SM,
)
inner = ft.Container(
content=content,
border=ft.Border.all(2, border_color),
border_radius=RADIUS_MD,
bgcolor=bg_color,
padding=ft.Padding.all(SPACE_LG),
height=160,
alignment=ft.Alignment.CENTER,
expand=True,
)
return ft.GestureDetector(
content=ft.Row([inner], spacing=0),
on_tap=lambda _: on_pick(),
mouse_cursor=ft.MouseCursor.CLICK,
)
# ---------------------------------------------------------------------------
# Log terminal
# ---------------------------------------------------------------------------
def build_log_terminal(
*,
ref: Optional[ft.Ref] = None,
max_height: int = 260,
page: Optional[ft.Page] = None,
) -> ft.Container:
"""
Build a scrollable, read-only log terminal widget.
Args:
ref: Optional ``ft.Ref[ft.ListView]`` to bind the inner list-view so
callers can append entries programmatically.
max_height: Maximum pixel height before vertical scrolling activates.
page: Current Flet ``Page`` for palette derivation.
Returns:
A styled ``ft.Container`` wrapping a ``ft.ListView``.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
bg = "#0d1117" if dark else "#f8f9fc"
text_color = "#b0b8cc" if dark else "#3d4358"
border_color = "#252a38" if dark else "#dce0ea"
list_view = ft.ListView(
expand=True,
auto_scroll=True,
spacing=1,
padding=ft.Padding.all(SPACE_SM),
)
if ref is not None:
ref.current = list_view
return ft.Container(
content=list_view,
bgcolor=bg,
border=ft.Border.all(1, border_color),
border_radius=RADIUS_SM,
height=max_height,
clip_behavior=ft.ClipBehavior.HARD_EDGE,
)
def log_entry(message: str, level: str = "info", page: Optional[ft.Page] = None) -> ft.Text:
"""
Create a single log-line ``ft.Text`` widget with appropriate colour coding.
Args:
message: The log message string.
level: Severity string: ``'info'``, ``'success'``, ``'error'``,
``'warning'``, ``'debug'``, ``'critical'``.
page: Current Flet ``Page`` for dark/light mode detection.
Returns:
A styled ``ft.Text`` control.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
palette: dict[str, str] = {
"info": "#9ba3b8" if dark else "#5a6172",
"success": "#42ad4a" if dark else "#2e9437",
"error": "#e84e3c" if dark else "#c0392b",
"warning": "#f5a623" if dark else "#d4870a",
"debug": "#5a6172" if dark else "#9ba3b8",
"critical": "#ff5722",
"trace": "#4e5568" if dark else "#b0b8cc",
}
color = palette.get(level.lower(), palette["info"])
return ft.Text(message, size=12, color=color, selectable=True, no_wrap=False)
# ---------------------------------------------------------------------------
# Progress row
# ---------------------------------------------------------------------------
def build_progress_row(
*,
progress_value: float = 0.0,
etr_text: str = "",
page: Optional[ft.Page] = None,
) -> ft.Column:
"""
Build a progress-bar + ETR-label column.
Args:
progress_value: Float in [0.0, 1.0].
etr_text: Pre-formatted estimated-time-remaining string.
page: Current ``Page`` for palette derivation.
Returns:
A ``ft.Column`` containing the progress bar and label.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
fill = "#5b8af5" if dark else "#3a5fc4"
bg = "#1e2230" if dark else "#e4e8f0"
bar = ft.ProgressBar(
value=progress_value,
color=fill,
bgcolor=bg,
height=8,
border_radius=ft.BorderRadius.all(4),
expand=True,
)
label = ft.Text(
etr_text,
size=11,
color="#9ba3b8" if dark else "#5a6172",
text_align=ft.TextAlign.CENTER,
)
return ft.Column(
[bar, label],
spacing=SPACE_SM,
horizontal_alignment=ft.CrossAxisAlignment.CENTER,
)
# ---------------------------------------------------------------------------
# Primary action button
# ---------------------------------------------------------------------------
def build_primary_button(
text: str,
*,
icon: Optional[str] = None,
on_click: Optional[Callable] = None,
disabled: bool = False,
width: Optional[int] = None,
page: Optional[ft.Page] = None,
) -> ft.ElevatedButton:
"""
Build a prominent, styled primary action button.
Args:
text: Button label.
icon: Optional Flet icon name (e.g. ``'play_arrow'``).
on_click: Click callback.
disabled: Whether the button is non-interactive.
width: Optional fixed pixel width.
page: Current ``Page`` for accent colour derivation.
Returns:
A styled ``ft.ElevatedButton``.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
bg = "#5b8af5" if dark else "#3a5fc4"
on_bg = "#ffffff"
style = ft.ButtonStyle(
bgcolor={
ft.ControlState.DEFAULT: bg,
ft.ControlState.HOVERED: "#3a5fc4" if dark else "#2a4fae",
ft.ControlState.DISABLED: "#2a2f3f" if dark else "#c0c8d8",
},
color={
ft.ControlState.DEFAULT: on_bg,
ft.ControlState.DISABLED: "#4e5568" if dark else "#9ba3b8",
},
elevation={"default": 2, "hovered": 4},
padding=ft.Padding.symmetric(horizontal=SPACE_LG, vertical=SPACE_MD),
shape=ft.RoundedRectangleBorder(radius=RADIUS_SM),
animation_duration=150,
)
return ft.ElevatedButton(
content=text,
icon=resolve_icon(icon),
on_click=on_click,
disabled=disabled,
width=width,
style=style,
height=48,
)
# ---------------------------------------------------------------------------
# Secondary / ghost button
# ---------------------------------------------------------------------------
def build_secondary_button(
text: str,
*,
icon: Optional[str] = None,
on_click: Optional[Callable] = None,
disabled: bool = False,
page: Optional[ft.Page] = None,
) -> ft.OutlinedButton:
"""
Build a secondary outlined button.
Args:
text: Button label.
icon: Optional Flet icon name.
on_click: Click callback.
disabled: Whether the button is non-interactive.
page: Current ``Page`` for border colour derivation.
Returns:
A styled ``ft.OutlinedButton``.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
border_clr = "#3a4466" if dark else "#a8b4d0"
text_clr = "#e8eaf0" if dark else "#1a1d27"
style = ft.ButtonStyle(
side={
ft.ControlState.DEFAULT: ft.BorderSide(1.5, border_clr),
ft.ControlState.HOVERED: ft.BorderSide(1.5, "#5b8af5" if dark else "#3a5fc4"),
},
color={
ft.ControlState.DEFAULT: text_clr,
ft.ControlState.HOVERED: "#5b8af5" if dark else "#3a5fc4",
ft.ControlState.DISABLED: "#4e5568" if dark else "#9ba3b8",
},
padding=ft.Padding.symmetric(horizontal=SPACE_LG, vertical=SPACE_MD),
shape=ft.RoundedRectangleBorder(radius=RADIUS_SM),
animation_duration=150,
)
return ft.OutlinedButton(
content=text,
icon=resolve_icon(icon),
on_click=on_click,
disabled=disabled,
style=style,
height=44,
)
# ---------------------------------------------------------------------------
# Section card
# ---------------------------------------------------------------------------
def build_card(
content: ft.Control,
*,
padding: int = SPACE_LG,
page: Optional[ft.Page] = None,
) -> ft.Container:
"""
Wrap a control in a styled card container.
Args:
content: The child control to embed.
padding: Internal padding in pixels.
page: Current ``Page`` for palette derivation.
Returns:
A styled ``ft.Container``.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
bg = "#181b23" if dark else "#ffffff"
border_clr = "#2c3147" if dark else "#dce0ea"
return ft.Container(
content=content,
bgcolor=bg,
border=ft.Border.all(1, border_clr),
border_radius=RADIUS_MD,
padding=ft.Padding.all(padding),
shadow=ft.BoxShadow(
spread_radius=0,
blur_radius=12,
color=ft.Colors.with_opacity(0.12 if dark else 0.06, ft.Colors.BLACK),
offset=ft.Offset(0, 2),
),
)
# ---------------------------------------------------------------------------
# Section header
# ---------------------------------------------------------------------------
def build_section_header(
title: str,
*,
subtitle: Optional[str] = None,
icon: Optional[str] = None,
page: Optional[ft.Page] = None,
) -> ft.Row:
"""
Build a consistent section header row with an optional icon.
Args:
title: Section heading text.
subtitle: Optional explanatory sub-text.
icon: Optional Flet icon name.
page: Current ``Page`` for palette derivation.
Returns:
A ``ft.Row`` containing the icon and text column.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
title_color = "#e8eaf0" if dark else "#1a1d27"
sub_color = "#9ba3b8" if dark else "#5a6172"
accent = "#5b8af5" if dark else "#3a5fc4"
children: List[ft.Control] = []
if icon:
children.append(ft.Icon(resolve_icon(icon), size=20, color=accent))
text_parts: List[ft.Control] = [
ft.Text(title, size=15, weight=ft.FontWeight.W_600, color=title_color)
]
if subtitle:
text_parts.append(ft.Text(subtitle, size=11, color=sub_color))
children.append(
ft.Column(text_parts, spacing=1, tight=True, expand=True)
)
return ft.Row(children, spacing=SPACE_SM, vertical_alignment=ft.CrossAxisAlignment.START)
# ---------------------------------------------------------------------------
# Status badge
# ---------------------------------------------------------------------------
def build_status_badge(
label: str,
*,
variant: str = "info",
page: Optional[ft.Page] = None,
) -> ft.Container:
"""
Build a small status badge chip.
Args:
label: Badge text.
variant: Colour variant: ``'info'``, ``'success'``, ``'error'``,
``'warning'``, ``'neutral'``.
page: Current ``Page`` for theme derivation.
Returns:
A pill-shaped ``ft.Container``.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
palette = {
"info": ("#1a2a5e" if dark else "#dde8ff", "#5b8af5" if dark else "#3a5fc4"),
"success": ("#0d2010" if dark else "#d4f4d7", "#42ad4a" if dark else "#2e9437"),
"error": ("#2a0a08" if dark else "#ffe0dc", "#e84e3c" if dark else "#c0392b"),
"warning": ("#2a1a00" if dark else "#fff4d8", "#f5a623" if dark else "#d4870a"),
"neutral": ("#1e2230" if dark else "#edf0f5", "#9ba3b8" if dark else "#5a6172"),
}
bg, fg = palette.get(variant, palette["info"])
return ft.Container(
content=ft.Text(label, size=10, weight=ft.FontWeight.W_600, color=fg),
bgcolor=bg,
border_radius=999,
padding=ft.Padding.symmetric(horizontal=8, vertical=3),
)
# ---------------------------------------------------------------------------
# Labelled control row
# ---------------------------------------------------------------------------
def labelled_row(
label: str,
control: ft.Control,
*,
label_width: int = 200,
tooltip: Optional[str] = None,
page: Optional[ft.Page] = None,
) -> ft.Row:
"""
Lay a label and a control side-by-side in a consistent row.
Args:
label: Human-readable label text.
control: The UI control placed to the right of the label.
label_width: Fixed pixel width of the label column.
tooltip: Optional tooltip text on the label.
page: Current ``Page`` for palette derivation.
Returns:
A ``ft.Row`` with the label pinned to a fixed width.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
lbl_color = "#9ba3b8" if dark else "#5a6172"
lbl = ft.Text(label, size=13, color=lbl_color, weight=ft.FontWeight.W_500, width=label_width)
if tooltip:
lbl.tooltip = tooltip
return ft.Row(
[lbl, ft.Container(content=control, expand=True)],
alignment=ft.MainAxisAlignment.START,
vertical_alignment=ft.CrossAxisAlignment.CENTER,
spacing=SPACE_MD,
)
# ---------------------------------------------------------------------------
# Snack-bar helper
# ---------------------------------------------------------------------------
def show_snack(
page: ft.Page,
message: str,
*,
error: bool = False,
duration: int = 3000,
) -> None:
"""
Display a brief snack-bar notification.
Args:
page: The Flet ``Page`` instance.
message: Text to display.
error: When True, colours the bar red instead of the default accent.
duration: Visible duration in milliseconds.
"""
dark = page.theme_mode == ft.ThemeMode.DARK
bg = "#e84e3c" if error else ("#5b8af5" if dark else "#3a5fc4")
page.snack_bar = ft.SnackBar(
content=ft.Text(message, color="#ffffff", size=13),
bgcolor=bg,
duration=duration,
show_close_icon=True,
close_icon_color="#ffffff",
)
page.snack_bar.open = True
page.update()
# ---------------------------------------------------------------------------
# Divider helper
# ---------------------------------------------------------------------------
def build_divider(page: Optional[ft.Page] = None) -> ft.Divider:
"""
Build a styled horizontal rule divider.
Args:
page: Current ``Page`` for palette derivation.
Returns:
A ``ft.Divider``.
"""
dark = page is not None and page.theme_mode == ft.ThemeMode.DARK
return ft.Divider(color="#252a38" if dark else "#e8ebf2", height=1, thickness=1)
+365
View File
@@ -0,0 +1,365 @@
"""
Abogen Flet Frontend main entry point.
Run as desktop app:
python -m abogen.frontend.main
Run as web app (binds to port 8080 by default):
python -m abogen.frontend.main --web --port 8080
Architecture
------------
One ``ft.app()`` call launches the server. For every new browser tab (or the
desktop window) Flet invokes ``_app_entry(page)`` in its own coroutine, which
creates a fresh ``AppState`` and wires together the navigation rail and views.
This guarantees complete per-session isolation in multi-user web deployments.
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
from typing import Optional
import flet as ft
from .state import AppState
from .components import resolve_icon
from .views.dashboard import DashboardView
from .views.settings import SettingsView
from .views.queue_view import QueueView
from .utils.theme import make_theme, DARK, LIGHT, SPACE_SM, SPACE_MD, SPACE_LG, RADIUS_MD
from abogen.constants import PROGRAM_NAME as APP_NAME
# ---------------------------------------------------------------------------
# Navigation destinations
# ---------------------------------------------------------------------------
_NAV_ITEMS = [
("Convert", "swap_horiz", "swap_horiz"),
("Queue", "list_alt", "list_alt"),
("Settings", "settings", "settings"),
]
_ASSETS_DIR = Path(__file__).resolve().parents[1] / "assets"
def _build_sidebar_item(
*,
label: str,
icon: str,
selected: bool,
palette,
on_click,
) -> ft.Container:
accent = palette.accent if selected else palette.text_secondary
bg = palette.sidebar_selected_bg if selected else palette.sidebar_bg
return ft.Container(
content=ft.Row(
[
ft.Icon(resolve_icon(icon), size=20, color=accent),
ft.Text(
label,
size=13,
weight=ft.FontWeight.W_600 if selected else ft.FontWeight.W_500,
color=accent,
),
],
spacing=SPACE_MD,
vertical_alignment=ft.CrossAxisAlignment.CENTER,
),
bgcolor=bg,
border_radius=RADIUS_MD,
padding=ft.Padding.symmetric(horizontal=SPACE_MD, vertical=10),
ink=True,
on_click=on_click,
)
# ---------------------------------------------------------------------------
# Per-session entry point
# ---------------------------------------------------------------------------
def _app_entry(page: ft.Page) -> None:
try:
# ── State ────────────────────────────────────────────────────────────
state = AppState()
state.load_from_config()
# ── Page basics ──────────────────────────────────────────────────────
page.title = APP_NAME
page.padding = 0
page.spacing = 0
page.bgcolor = DARK.bg_base
page.theme_mode = ft.ThemeMode.DARK
page.theme = make_theme(dark=True)
page.dark_theme = make_theme(dark=True)
page.fonts = {}
page.window.min_width = 520
page.window.min_height = 600
page.update()
# ── Content area ref ─────────────────────────────────────────────────
content_area = ft.Column(expand=True, spacing=0)
sidebar_body = ft.Column(spacing=SPACE_SM)
theme_button_host = ft.Container()
brand_title = ft.Text(
APP_NAME,
size=18,
weight=ft.FontWeight.W_700,
color=DARK.text_primary,
)
brand_fallback_icon = ft.Icon(resolve_icon("speaker_notes"), size=32, color=DARK.accent)
divider = ft.VerticalDivider(width=1, color=DARK.border)
# ── Views ────────────────────────────────────────────────────────────
dashboard_view = DashboardView(page, state)
settings_view = SettingsView(page, state)
queue_view = QueueView(page, state)
views = [
dashboard_view.build,
queue_view.build,
settings_view.build,
]
_selected_index = [0]
def _refresh_sidebar() -> None:
dark = page.theme_mode == ft.ThemeMode.DARK
pal = DARK if dark else LIGHT
sidebar_body.controls = [
_build_sidebar_item(
label=label,
icon=icon,
selected=index == _selected_index[0],
palette=pal,
on_click=lambda _, i=index: _navigate(i),
)
for index, (label, icon, _) in enumerate(_NAV_ITEMS)
]
sidebar.bgcolor = pal.sidebar_bg
divider.color = pal.border
brand_title.color = pal.text_primary
brand_fallback_icon.color = pal.accent
theme_button_host.content = ft.Container(
content=ft.Icon(
resolve_icon("dark_mode" if dark else "light_mode"),
size=20,
color=pal.text_secondary,
),
tooltip="Toggle theme",
border_radius=RADIUS_MD,
padding=8,
ink=True,
on_click=lambda _: _toggle_theme(page, _refresh_sidebar),
)
def _navigate(index: int) -> None:
_selected_index[0] = index
content_area.controls.clear()
built = views[index]()
content_area.controls.append(
ft.Container(
content=built,
expand=True,
padding=ft.Padding.symmetric(horizontal=SPACE_LG, vertical=SPACE_LG),
)
)
_refresh_sidebar()
page.update()
# ── Sidebar ──────────────────────────────────────────────────────────
pal = DARK
sidebar = ft.Container(
width=220,
bgcolor=pal.sidebar_bg,
padding=ft.Padding.all(SPACE_MD),
content=ft.Column(
[
ft.Container(
content=ft.Row(
[
ft.Image(
src="icon.png",
width=36,
height=36,
fit=ft.BoxFit.CONTAIN,
error_content=brand_fallback_icon,
),
brand_title,
],
spacing=SPACE_MD,
vertical_alignment=ft.CrossAxisAlignment.CENTER,
),
padding=ft.Padding.only(top=SPACE_SM, bottom=SPACE_LG),
),
sidebar_body,
ft.Container(expand=True),
ft.Row([theme_button_host], alignment=ft.MainAxisAlignment.END),
],
expand=True,
spacing=SPACE_SM,
),
)
_refresh_sidebar()
# ── Page handle for pubsub (queue → dashboard) ───────────────────────
def _handle_pubsub(topic: str) -> None:
if topic == "start_queue":
_navigate(0)
page.pubsub.subscribe(_handle_pubsub)
# ── Layout ────────────────────────────────────────────────────────────
page.add(
ft.Row(
[
sidebar,
divider,
ft.Container(content=content_area, expand=True),
],
expand=True,
spacing=0,
vertical_alignment=ft.CrossAxisAlignment.START,
)
)
# Show dashboard by default
_navigate(0)
page.update()
except Exception as e:
import traceback
traceback.print_exc()
print(f"ERROR IN _app_entry: {e}")
raise
def _toggle_theme(page: ft.Page, refresh_sidebar) -> None:
"""Switch between dark and light theme modes."""
if page.theme_mode == ft.ThemeMode.DARK:
page.theme_mode = ft.ThemeMode.LIGHT
page.bgcolor = LIGHT.bg_base
else:
page.theme_mode = ft.ThemeMode.DARK
page.bgcolor = DARK.bg_base
page.theme = make_theme(page.theme_mode == ft.ThemeMode.DARK)
refresh_sidebar()
page.update()
# ---------------------------------------------------------------------------
# CLI helpers & entry point
# ---------------------------------------------------------------------------
def _is_port_free(host: str, port: int) -> bool:
import socket
try:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host, port))
return True
except OSError:
return False
def _find_free_port(host: str, start_port: int) -> int:
import socket
port = start_port
while port < 65535:
try:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host, port))
return port
except OSError:
port += 1
return start_port
def main() -> None:
"""
Start the Abogen Flet frontend.
Parses ``--web`` and ``--port`` CLI arguments to choose desktop vs. web
mode, then hands control to ``ft.app()``.
"""
import logging
logging.basicConfig(level=logging.INFO)
logging.getLogger("flet").setLevel(logging.INFO)
parser = argparse.ArgumentParser(description=f"{APP_NAME} Flet frontend")
parser.add_argument(
"--web", action="store_true",
help="Run as a web server instead of a desktop window.",
)
parser.add_argument(
"--port", type=int, default=8080,
help="Port for the web server (default: 8080). Ignored in desktop mode.",
)
parser.add_argument(
"--host", default="127.0.0.1",
help="Host for the web server (default: 127.0.0.1). Use 0.0.0.0 to expose publicly.",
)
args = parser.parse_args()
if args.web:
port_specified = "--port" in sys.argv
target_port = args.port
if not port_specified:
target_port = _find_free_port(args.host, 8080)
if target_port != 8080:
print(f"Port 8080 is in use. Automatically routed to free port: {target_port}")
else:
if not _is_port_free(args.host, target_port):
print(f"Error: Port {target_port} is already in use on {args.host}.", file=sys.stderr)
print("Please select a different port or omit the --port flag to find one automatically.", file=sys.stderr)
sys.exit(1)
print(f"Starting Abogen WebUI on http://{args.host}:{target_port} ...")
ft.app(
target=_app_entry,
view=ft.AppView.WEB_BROWSER,
port=target_port,
host=args.host,
assets_dir=str(_ASSETS_DIR) if _ASSETS_DIR.exists() else None,
no_cdn=True,
web_renderer="canvaskit",
)
else:
try:
ft.app(
target=_app_entry,
view=ft.AppView.FLET_APP,
assets_dir=str(_ASSETS_DIR) if _ASSETS_DIR.exists() else None,
)
except Exception as e:
print(f"Warning: Failed to launch native desktop window: {e}", file=sys.stderr)
print("Falling back to running as a web application in your default browser...", file=sys.stderr)
target_port = _find_free_port("127.0.0.1", 8080)
print(f"Starting Abogen WebUI on http://127.0.0.1:{target_port} ...")
ft.app(
target=_app_entry,
view=ft.AppView.WEB_BROWSER,
port=target_port,
host="127.0.0.1",
assets_dir=str(_ASSETS_DIR) if _ASSETS_DIR.exists() else None,
no_cdn=True,
web_renderer="canvaskit",
)
def main_web() -> None:
"""
Start the Abogen Flet frontend as a web server.
"""
import sys
if "--web" not in sys.argv:
sys.argv.insert(1, "--web")
main()
if __name__ == "__main__":
main()
+4
View File
@@ -0,0 +1,4 @@
"""State sub-package exports AppState and ConversionJob."""
from .app_state import AppState, ConversionJob
__all__ = ["AppState", "ConversionJob"]
+451
View File
@@ -0,0 +1,451 @@
"""
Centralized, per-session application state for the Abogen Flet frontend.
Each Flet page (session) gets its own instance of AppState, which guarantees
complete isolation between simultaneous web-browser clients and the desktop
window. The class carries every configuration variable, file buffer reference,
and generation progress field that the rest of the UI reads or writes.
This module intentionally has no Flet imports so it can be unit-tested without
a running Flet server.
"""
from __future__ import annotations
import threading
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional
from abogen.utils import load_config, save_config
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _default_config() -> Dict[str, Any]:
"""Load the persisted user config dict, returning an empty dict on failure."""
try:
return load_config() or {}
except Exception:
return {}
# ---------------------------------------------------------------------------
# Per-session state
# ---------------------------------------------------------------------------
@dataclass
class ConversionJob:
"""Lightweight descriptor of a single queued conversion job."""
file_path: str
"""Absolute path to the text/epub/pdf/txt input file."""
display_name: str
"""User-visible filename (may be the original epub/pdf path)."""
voice: str
"""Voice formula string (e.g. 'af_heart' or 'af_heart*0.5+am_adam*0.5')."""
lang_code: str
"""Single-char language prefix used by Kokoro (e.g. 'a', 'b', 'e')."""
speed: float = 1.0
"""Playback speed multiplier, range 0.1 2.0."""
output_format: str = "mp3"
"""Output audio container format."""
subtitle_mode: str = "Disabled"
"""Subtitle generation mode."""
save_option: str = "Save next to input file"
"""Save location strategy."""
output_folder: Optional[str] = None
"""Absolute path when save_option is 'Choose output folder'."""
char_count: int = 0
"""Pre-computed character count for ETR estimation."""
replace_single_newlines: bool = True
save_chapters_separately: Optional[bool] = None
merge_chapters_at_end: Optional[bool] = None
@dataclass
class AppState:
"""
Single source of truth for one Flet session.
Instantiated once per ``ft.app()`` call on desktop, and once per browser
tab on web. All UI components receive a reference to this object and
read/write it to keep themselves in sync.
Thread-safety: mutation from background threads should be done via the
provided ``_lock``. The UI update callbacks (``on_log``,
``on_progress``, etc.) are always invoked on the Flet event loop via
``page.run_task()`` and must be set by the view layer.
"""
# -----------------------------------------------------------------------
# Runtime identity
# -----------------------------------------------------------------------
_lock: threading.Lock = field(default_factory=threading.Lock, repr=False, compare=False)
# -----------------------------------------------------------------------
# Persisted user config (loaded once, written on every change)
# -----------------------------------------------------------------------
config: Dict[str, Any] = field(default_factory=_default_config)
# -----------------------------------------------------------------------
# File / input state
# -----------------------------------------------------------------------
selected_file: Optional[str] = None
"""Path to the processed text file (may be a temp cache copy for epub/pdf)."""
selected_file_type: Optional[str] = None
"""'txt' | 'epub' | 'pdf' | 'markdown' | None"""
selected_book_path: Optional[str] = None
"""Original epub/pdf path before being converted to txt."""
displayed_file_path: Optional[str] = None
"""Path shown in the UI drop-zone (original book or txt file)."""
selected_chapters: List[str] = field(default_factory=list)
"""Ordered list of selected chapter href tokens (or page numbers for PDFs)."""
save_chapters_separately: Optional[bool] = None
merge_chapters_at_end: Optional[bool] = None
save_as_project: bool = False
char_count: int = 0
# -----------------------------------------------------------------------
# Voice / language
# -----------------------------------------------------------------------
selected_voice: str = "af_heart"
selected_lang: str = "a"
selected_profile_name: Optional[str] = None
mixed_voice_state: Optional[List[Any]] = None
"""List of [voice_id, weight] pairs when the formula mixer is in use."""
# -----------------------------------------------------------------------
# Conversion parameters
# -----------------------------------------------------------------------
speed: float = 1.0
use_gpu: bool = True
selected_format: str = "wav"
subtitle_mode: str = "Sentence"
subtitle_format: str = "ass_centered_narrow"
replace_single_newlines: bool = True
save_option: str = "Save next to input file"
selected_output_folder: Optional[str] = None
silence_duration: float = 2.0
max_subtitle_words: int = 50
separate_chapters_format: str = "wav"
use_silent_gaps: bool = True
subtitle_speed_method: str = "tts"
use_spacy_segmentation: bool = True
chunk_level: str = "paragraph"
generate_epub3: bool = False
# TTS provider
tts_provider: str = "kokoro"
supertonic_total_steps: int = 5
# Chapter options
chapter_intro_delay: float = 0.5
read_title_intro: bool = False
read_closing_outro: bool = True
auto_prefix_chapter_titles: bool = True
normalize_chapter_opening_caps: bool = True
# Speaker analysis
speaker_analysis_threshold: int = 3
# Word substitutions
word_substitutions_enabled: bool = False
word_substitutions_list: str = ""
case_sensitive_substitutions: bool = False
replace_all_caps: bool = False
replace_numerals: bool = False
fix_nonstandard_punctuation: bool = False
# -----------------------------------------------------------------------
# Conversion runtime state
# -----------------------------------------------------------------------
is_converting: bool = False
is_cancelled: bool = False
progress: float = 0.0
"""Fractional progress 0.0 1.0."""
etr_seconds: Optional[float] = None
"""Estimated seconds remaining, or None if unknown."""
last_output_path: Optional[str] = None
log_lines: List[str] = field(default_factory=list)
"""Buffered log messages, capped at LOG_MAX_LINES."""
LOG_MAX_LINES: int = 2000
# -----------------------------------------------------------------------
# Queue
# -----------------------------------------------------------------------
queued_items: List[ConversionJob] = field(default_factory=list)
current_queue_index: int = 0
# -----------------------------------------------------------------------
# Callbacks (set by the view layer, not serialised)
# -----------------------------------------------------------------------
on_log: Optional[Callable[[str, str], None]] = field(default=None, repr=False, compare=False)
"""Called from any thread: ``on_log(message, level)``."""
on_progress: Optional[Callable[[float, Optional[float]], None]] = field(
default=None, repr=False, compare=False
)
"""Called from any thread: ``on_progress(fraction, etr_seconds)``."""
on_conversion_finished: Optional[Callable[[str, Optional[str]], None]] = field(
default=None, repr=False, compare=False
)
"""Called from any thread: ``on_conversion_finished(message, output_path)``."""
# -----------------------------------------------------------------------
# Integrations
# -----------------------------------------------------------------------
audiobookshelf_enabled: bool = False
audiobookshelf_base_url: str = ""
audiobookshelf_api_token: str = ""
audiobookshelf_library_id: str = ""
audiobookshelf_folder_id: str = ""
audiobookshelf_verify_ssl: bool = True
audiobookshelf_auto_send: bool = False
audiobookshelf_send_cover: bool = True
audiobookshelf_send_chapters: bool = True
audiobookshelf_send_subtitles: bool = False
audiobookshelf_timeout: float = 30.0
calibre_opds_enabled: bool = False
calibre_opds_base_url: str = ""
calibre_opds_username: str = ""
calibre_opds_password: str = ""
calibre_opds_verify_ssl: bool = True
# -----------------------------------------------------------------------
# Public helpers
# -----------------------------------------------------------------------
def load_from_config(self) -> None:
"""
Populate all fields from the persisted JSON config file.
Called once at startup and whenever the settings page is saved.
Thread-safe.
"""
with self._lock:
cfg = _default_config()
self.config = cfg
self.selected_voice = cfg.get("selected_voice", "af_heart")
self.selected_lang = self.selected_voice[0] if self.selected_voice else "a"
self.selected_profile_name = cfg.get("selected_profile_name")
self.speed = cfg.get("speed", 1.0)
self.use_gpu = cfg.get("use_gpu", True)
self.selected_format = cfg.get("selected_format", "wav")
self.subtitle_mode = cfg.get("subtitle_mode", "Sentence")
self.subtitle_format = cfg.get("subtitle_format", "ass_centered_narrow")
self.replace_single_newlines = cfg.get("replace_single_newlines", True)
self.save_option = cfg.get("save_option", "Save next to input file")
self.selected_output_folder = cfg.get("selected_output_folder")
self.silence_duration = cfg.get("silence_duration", 2.0)
self.max_subtitle_words = cfg.get("max_subtitle_words", 50)
self.separate_chapters_format = cfg.get("separate_chapters_format", "wav")
self.use_silent_gaps = cfg.get("use_silent_gaps", True)
self.subtitle_speed_method = cfg.get("subtitle_speed_method", "tts")
self.use_spacy_segmentation = cfg.get("use_spacy_segmentation", True)
self.chunk_level = cfg.get("chunk_level", "paragraph")
self.generate_epub3 = cfg.get("generate_epub3", False)
self.tts_provider = cfg.get("tts_provider", "kokoro")
self.supertonic_total_steps = cfg.get("supertonic_total_steps", 5)
self.chapter_intro_delay = cfg.get("chapter_intro_delay", 0.5)
self.read_title_intro = cfg.get("read_title_intro", False)
self.read_closing_outro = cfg.get("read_closing_outro", True)
self.auto_prefix_chapter_titles = cfg.get("auto_prefix_chapter_titles", True)
self.normalize_chapter_opening_caps = cfg.get("normalize_chapter_opening_caps", True)
self.speaker_analysis_threshold = cfg.get("speaker_analysis_threshold", 3)
self.word_substitutions_enabled = cfg.get("word_substitutions_enabled", False)
self.word_substitutions_list = cfg.get("word_substitutions_list", "")
self.case_sensitive_substitutions = cfg.get("case_sensitive_substitutions", False)
self.replace_all_caps = cfg.get("replace_all_caps", False)
self.replace_numerals = cfg.get("replace_numerals", False)
self.fix_nonstandard_punctuation = cfg.get("fix_nonstandard_punctuation", False)
# Integrations
integrations: Dict[str, Any] = cfg.get("integrations", {})
abs_cfg = integrations.get("audiobookshelf", {})
self.audiobookshelf_enabled = bool(abs_cfg.get("enabled", False))
self.audiobookshelf_base_url = str(abs_cfg.get("base_url", ""))
self.audiobookshelf_api_token = str(abs_cfg.get("api_token", ""))
self.audiobookshelf_library_id = str(abs_cfg.get("library_id", ""))
self.audiobookshelf_folder_id = str(abs_cfg.get("folder_id", ""))
self.audiobookshelf_verify_ssl = bool(abs_cfg.get("verify_ssl", True))
self.audiobookshelf_auto_send = bool(abs_cfg.get("auto_send", False))
self.audiobookshelf_send_cover = bool(abs_cfg.get("send_cover", True))
self.audiobookshelf_send_chapters = bool(abs_cfg.get("send_chapters", True))
self.audiobookshelf_send_subtitles = bool(abs_cfg.get("send_subtitles", False))
self.audiobookshelf_timeout = float(abs_cfg.get("timeout", 30.0))
cal_cfg = integrations.get("calibre_opds", {})
self.calibre_opds_enabled = bool(cal_cfg.get("enabled", False))
self.calibre_opds_base_url = str(cal_cfg.get("base_url", ""))
self.calibre_opds_username = str(cal_cfg.get("username", ""))
self.calibre_opds_password = str(cal_cfg.get("password", ""))
self.calibre_opds_verify_ssl = bool(cal_cfg.get("verify_ssl", True))
def persist_config(self) -> None:
"""
Write the current config snapshot back to disk.
Only the fields that map to the JSON config are written; runtime state
(progress, log_lines, callbacks) is not persisted.
Thread-safe.
"""
with self._lock:
cfg = self.config.copy()
cfg["selected_voice"] = self.selected_voice
cfg["selected_profile_name"] = self.selected_profile_name
cfg["speed"] = self.speed
cfg["use_gpu"] = self.use_gpu
cfg["selected_format"] = self.selected_format
cfg["subtitle_mode"] = self.subtitle_mode
cfg["subtitle_format"] = self.subtitle_format
cfg["replace_single_newlines"] = self.replace_single_newlines
cfg["save_option"] = self.save_option
cfg["selected_output_folder"] = self.selected_output_folder
cfg["silence_duration"] = self.silence_duration
cfg["max_subtitle_words"] = self.max_subtitle_words
cfg["separate_chapters_format"] = self.separate_chapters_format
cfg["use_silent_gaps"] = self.use_silent_gaps
cfg["subtitle_speed_method"] = self.subtitle_speed_method
cfg["use_spacy_segmentation"] = self.use_spacy_segmentation
cfg["chunk_level"] = self.chunk_level
cfg["generate_epub3"] = self.generate_epub3
cfg["tts_provider"] = self.tts_provider
cfg["supertonic_total_steps"] = self.supertonic_total_steps
cfg["chapter_intro_delay"] = self.chapter_intro_delay
cfg["read_title_intro"] = self.read_title_intro
cfg["read_closing_outro"] = self.read_closing_outro
cfg["auto_prefix_chapter_titles"] = self.auto_prefix_chapter_titles
cfg["normalize_chapter_opening_caps"] = self.normalize_chapter_opening_caps
cfg["speaker_analysis_threshold"] = self.speaker_analysis_threshold
cfg["word_substitutions_enabled"] = self.word_substitutions_enabled
cfg["word_substitutions_list"] = self.word_substitutions_list
cfg["case_sensitive_substitutions"] = self.case_sensitive_substitutions
cfg["replace_all_caps"] = self.replace_all_caps
cfg["replace_numerals"] = self.replace_numerals
cfg["fix_nonstandard_punctuation"] = self.fix_nonstandard_punctuation
# Integrations
cfg.setdefault("integrations", {})
cfg["integrations"]["audiobookshelf"] = {
"enabled": self.audiobookshelf_enabled,
"base_url": self.audiobookshelf_base_url,
"api_token": self.audiobookshelf_api_token,
"library_id": self.audiobookshelf_library_id,
"folder_id": self.audiobookshelf_folder_id,
"verify_ssl": self.audiobookshelf_verify_ssl,
"auto_send": self.audiobookshelf_auto_send,
"send_cover": self.audiobookshelf_send_cover,
"send_chapters": self.audiobookshelf_send_chapters,
"send_subtitles": self.audiobookshelf_send_subtitles,
"timeout": self.audiobookshelf_timeout,
}
cfg["integrations"]["calibre_opds"] = {
"enabled": self.calibre_opds_enabled,
"base_url": self.calibre_opds_base_url,
"username": self.calibre_opds_username,
"password": self.calibre_opds_password,
"verify_ssl": self.calibre_opds_verify_ssl,
}
self.config = cfg
try:
save_config(cfg)
except Exception:
pass
def append_log(self, message: str, level: str = "info") -> None:
"""
Thread-safely append a log line and trigger the UI callback.
Caps the internal buffer at ``LOG_MAX_LINES`` to prevent unbounded
memory growth during very long conversion tasks.
"""
with self._lock:
self.log_lines.append(f"[{level.upper()}] {message}")
if len(self.log_lines) > self.LOG_MAX_LINES:
# Trim oldest 10 % to amortise the cost of trimming
trim = self.LOG_MAX_LINES // 10
self.log_lines = self.log_lines[trim:]
cb = self.on_log
if cb is not None:
try:
cb(message, level)
except Exception:
pass
def update_progress(self, fraction: float, etr: Optional[float] = None) -> None:
"""
Update fractional progress and ETR, then notify the UI callback.
Args:
fraction: Value in [0.0, 1.0].
etr: Estimated seconds remaining, or None.
"""
with self._lock:
self.progress = max(0.0, min(1.0, fraction))
self.etr_seconds = etr
cb = self.on_progress
if cb is not None:
try:
cb(fraction, etr)
except Exception:
pass
def get_voice_formula(self) -> str:
"""
Return the effective voice formula string.
Uses the mixed_voice_state if the formula mixer is active, otherwise
returns the raw selected_voice.
"""
if self.mixed_voice_state:
parts = [f"{name}*{weight}" for name, weight in self.mixed_voice_state]
return " + ".join(filter(None, parts))
return self.selected_voice or "af_heart"
def reset_file_state(self) -> None:
"""Clear all file-related fields without touching voice/settings."""
with self._lock:
self.selected_file = None
self.selected_file_type = None
self.selected_book_path = None
self.displayed_file_path = None
self.selected_chapters = []
self.save_chapters_separately = None
self.merge_chapters_at_end = None
self.save_as_project = False
self.char_count = 0
def reset_conversion_state(self) -> None:
"""Clear all runtime conversion fields to start fresh."""
with self._lock:
self.is_converting = False
self.is_cancelled = False
self.progress = 0.0
self.etr_seconds = None
self.last_output_path = None
self.log_lines = []
+38
View File
@@ -0,0 +1,38 @@
"""Utils sub-package."""
from .helpers import (
human_readable_size,
format_duration,
format_etr,
detect_file_type,
is_supported_file,
is_book_type,
voice_lang_code,
language_label,
grouped_voices,
voice_display_name,
parse_voice_formula,
format_number,
safe_basename,
output_format_label,
subtitle_format_label,
SUPPORTED_EXTENSIONS,
)
__all__ = [
"human_readable_size",
"format_duration",
"format_etr",
"detect_file_type",
"is_supported_file",
"is_book_type",
"voice_lang_code",
"language_label",
"grouped_voices",
"voice_display_name",
"parse_voice_formula",
"format_number",
"safe_basename",
"output_format_label",
"subtitle_format_label",
"SUPPORTED_EXTENSIONS",
]
+462
View File
@@ -0,0 +1,462 @@
"""
Background conversion bridge for the Abogen Flet frontend.
This module wraps the existing ``abogen.webui.conversion_runner`` (and its
``ConversionService`` / ``Job`` machinery) in an async-friendly interface that
can push real-time progress and log updates back to the Flet event loop without
blocking the UI thread.
Key design decisions
--------------------
* All heavy work is offloaded to daemon threads. The Flet page event loop
is never blocked.
* Progress and log callbacks are scheduled back onto the Flet page via
``page.run_task()`` so Flet's session isolation remains intact.
* Cancellation is cooperative: the underlying job's ``cancel_requested``
flag is set, and the runner checks it at chunk boundaries.
* The module is a pure adapter it does NOT duplicate any processing logic
from the core pipeline.
"""
from __future__ import annotations
import asyncio
import os
import tempfile
import threading
import time
import traceback
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional
import flet as ft
from abogen.utils import (
get_gpu_acceleration,
get_user_cache_path,
get_user_output_path,
load_numpy_kpipeline,
prevent_sleep_end,
prevent_sleep_start,
)
from abogen.webui.service import (
ConversionService,
Job,
JobStatus,
PendingJob,
build_service,
)
from abogen.webui.conversion_runner import run_conversion_job
from ..state import AppState
# ---------------------------------------------------------------------------
# Module-level singleton ConversionService (shared across sessions, as in the
# web UI but each job carries its own output folder keyed by session).
# ---------------------------------------------------------------------------
_SERVICE_LOCK = threading.Lock()
_SERVICE: Optional[ConversionService] = None
def _get_service() -> ConversionService:
"""
Return (creating if necessary) the module-level ConversionService.
The service manages the background worker thread and persistent job state.
Thread-safe via a module-level lock.
"""
global _SERVICE
with _SERVICE_LOCK:
if _SERVICE is None:
output_root = Path(get_user_output_path("frontend"))
uploads_root = Path(get_user_cache_path("frontend/uploads"))
_SERVICE = build_service(
runner=run_conversion_job,
output_root=output_root,
uploads_root=uploads_root,
)
return _SERVICE
# ---------------------------------------------------------------------------
# Public conversion bridge
# ---------------------------------------------------------------------------
class ConversionBridge:
"""
Thin adapter between the Flet UI session and the core conversion pipeline.
One ``ConversionBridge`` instance is created per Flet page (session) and
is responsible for:
1. Accepting a conversion request from the UI.
2. Writing the input text to a temp file if needed.
3. Submitting the job to ``ConversionService``.
4. Polling the job from a daemon thread and forwarding progress/logs to
the Flet page via ``page.run_task()``.
5. Providing a ``cancel()`` method that sets the cooperative flag.
"""
def __init__(self, page: ft.Page, state: AppState) -> None:
"""
Initialise the bridge.
Args:
page: The Flet ``Page`` for this session. Used to schedule
UI callbacks on the correct event loop.
state: The session's ``AppState`` instance.
"""
self._page = page
self._state = state
self._current_job: Optional[Job] = None
self._poll_thread: Optional[threading.Thread] = None
self._stop_poll = threading.Event()
self._seen_log_count = 0
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def start(
self,
*,
input_file: str,
voice: str,
lang_code: str,
speed: float,
output_format: str,
subtitle_mode: str,
subtitle_format: str,
use_gpu: bool,
save_option: str,
output_folder: Optional[str],
replace_single_newlines: bool,
char_count: int,
chapters: Optional[List[Dict[str, Any]]] = None,
save_chapters_separately: bool = False,
merge_chapters_at_end: bool = True,
separate_chapters_format: str = "wav",
silence_between_chapters: float = 2.0,
max_subtitle_words: int = 50,
chapter_intro_delay: float = 0.5,
read_title_intro: bool = False,
read_closing_outro: bool = True,
auto_prefix_chapter_titles: bool = True,
normalize_chapter_opening_caps: bool = True,
tts_provider: str = "kokoro",
supertonic_total_steps: int = 5,
chunk_level: str = "paragraph",
generate_epub3: bool = False,
word_substitutions_enabled: bool = False,
word_substitutions_list: str = "",
case_sensitive_substitutions: bool = False,
replace_all_caps: bool = False,
replace_numerals: bool = False,
fix_nonstandard_punctuation: bool = False,
) -> None:
"""
Submit a conversion job and begin the progress-polling loop.
This method returns immediately; all heavy work runs on daemon threads.
UI callbacks (``state.on_log``, ``state.on_progress``,
``state.on_conversion_finished``) are scheduled on the Flet event loop.
Args:
input_file: Absolute path to the text/epub/pdf input file.
voice: Kokoro voice formula string.
lang_code: Single-char language code.
speed: Playback speed multiplier (0.1 2.0).
output_format: Audio container key (``'wav'``, ``'mp3'``, ).
subtitle_mode: Subtitle generation mode string.
subtitle_format: Subtitle container key (``'srt'``, ``'ass_wide'``, ).
use_gpu: Whether to request GPU acceleration.
save_option: Save-location strategy string.
output_folder: Explicit output folder or None.
replace_single_newlines: Pre-processing flag.
char_count: Pre-computed character count for ETR estimation.
chapters: Optional list of chapter dicts for epub/pdf.
save_chapters_separately: Split chapters into separate files.
merge_chapters_at_end: Merge chapter files into one after generation.
separate_chapters_format: Format for individual chapter files.
silence_between_chapters: Silence gap (seconds) between chapters.
max_subtitle_words: Maximum words per subtitle block.
chapter_intro_delay: Silence before chapter title announcement (s).
read_title_intro: Announce book title at the start.
read_closing_outro: Announce book title at the end.
auto_prefix_chapter_titles: Prepend "Chapter N." to titles.
normalize_chapter_opening_caps: Fix ALL-CAPS opening lines.
tts_provider: ``'kokoro'`` or ``'supertonic'``.
supertonic_total_steps: Quality steps for the Supertonic pipeline.
chunk_level: ``'paragraph'`` or ``'sentence'`` chunking granularity.
generate_epub3: Also produce an EPUB3 audiobook package.
word_substitutions_enabled: Toggle word-substitution pre-processing.
word_substitutions_list: Newline-delimited ``word|replacement`` rules.
case_sensitive_substitutions: Case-sensitive matching for substitutions.
replace_all_caps: Lowercase ALL-CAPS words.
replace_numerals: Convert digits to spoken words.
fix_nonstandard_punctuation: Normalise curly quotes etc.
"""
if self._state.is_converting:
return
# Resolve the effective output folder
resolved_output: Optional[Path] = self._resolve_output_folder(
save_option=save_option,
output_folder=output_folder,
input_file=input_file,
)
# Store the input file as a Path
stored_path = Path(input_file)
original_filename = stored_path.name
# Block signals until the job is submitted
prevent_sleep_start()
self._state.is_converting = True
self._state.is_cancelled = False
self._state.progress = 0.0
self._state.etr_seconds = None
self._state.log_lines = []
self._seen_log_count = 0
# Enqueue the job on the service
service = _get_service()
job = service.enqueue(
original_filename=original_filename,
stored_path=stored_path,
language=lang_code,
voice=voice,
speed=speed,
tts_provider=tts_provider,
supertonic_total_steps=supertonic_total_steps,
use_gpu=use_gpu,
subtitle_mode=subtitle_mode,
output_format=output_format,
save_mode=self._save_mode_key(save_option),
output_folder=resolved_output,
replace_single_newlines=replace_single_newlines,
subtitle_format=subtitle_format,
total_characters=char_count,
chapters=chapters or [],
save_chapters_separately=save_chapters_separately,
merge_chapters_at_end=merge_chapters_at_end,
separate_chapters_format=separate_chapters_format,
silence_between_chapters=silence_between_chapters,
max_subtitle_words=max_subtitle_words,
chapter_intro_delay=chapter_intro_delay,
read_title_intro=read_title_intro,
read_closing_outro=read_closing_outro,
auto_prefix_chapter_titles=auto_prefix_chapter_titles,
normalize_chapter_opening_caps=normalize_chapter_opening_caps,
chunk_level=chunk_level,
generate_epub3=generate_epub3,
)
self._current_job = job
# Persist word-substitution settings to config so the runner picks them up
self._state.word_substitutions_enabled = word_substitutions_enabled
self._state.word_substitutions_list = word_substitutions_list
self._state.case_sensitive_substitutions = case_sensitive_substitutions
self._state.replace_all_caps = replace_all_caps
self._state.replace_numerals = replace_numerals
self._state.fix_nonstandard_punctuation = fix_nonstandard_punctuation
self._state.persist_config()
# Start the poll thread
self._stop_poll.clear()
self._poll_thread = threading.Thread(
target=self._poll_job_loop, daemon=True, name="abogen-poll"
)
self._poll_thread.start()
def cancel(self) -> None:
"""
Request cancellation of the currently running job.
Sets the cooperative flag on the underlying ``Job`` object; the runner
will stop after completing the current text chunk.
"""
if self._current_job is not None:
self._state.is_cancelled = True
try:
_get_service().cancel(self._current_job.id)
except Exception:
pass
# ------------------------------------------------------------------
# Internal helpers
# ------------------------------------------------------------------
@staticmethod
def _save_mode_key(option: str) -> str:
"""
Convert the human-readable save option to the service's internal key.
Args:
option: UI-facing string (``'Save next to input file'``, ).
Returns:
Service key string.
"""
mapping = {
"Save next to input file": "save_next_to_input",
"Save to Desktop": "save_to_desktop",
"Choose output folder": "custom",
}
return mapping.get(option, "save_next_to_input")
@staticmethod
def _resolve_output_folder(
save_option: str,
output_folder: Optional[str],
input_file: str,
) -> Optional[Path]:
"""
Return the output ``Path`` based on the save option, or None for
the "next to input" strategy (the runner handles that internally).
Args:
save_option: UI-facing save strategy string.
output_folder: Explicit path when ``save_option`` is ``'Choose output folder'``.
input_file: Path to the source file for the ``'Save to Desktop'`` strategy.
Returns:
Resolved ``Path`` or ``None``.
"""
if save_option == "Choose output folder" and output_folder:
p = Path(output_folder)
p.mkdir(parents=True, exist_ok=True)
return p
if save_option == "Save to Desktop":
desktop = Path.home() / "Desktop"
desktop.mkdir(exist_ok=True)
return desktop
# "Save next to input file" let the runner decide
return None
def _poll_job_loop(self) -> None:
"""
Background daemon loop that polls the current Job for updates.
Runs until the job enters a terminal state or until ``_stop_poll``
is set. Uses ``page.run_task()`` to schedule UI updates on the Flet
event loop without triggering thread-safety violations.
"""
job = self._current_job
if job is None:
return
service = _get_service()
POLL_INTERVAL = 0.25 # seconds
while not self._stop_poll.is_set():
# Re-fetch the current job state (it's mutated in-place by the runner)
current = service.get_job(job.id)
if current is None:
break
# Forward new log lines
new_logs = current.logs[self._seen_log_count:]
self._seen_log_count += len(new_logs)
for log_entry in new_logs:
level = getattr(log_entry, "level", "info")
message = getattr(log_entry, "message", str(log_entry))
self._schedule_log(message, level)
# Forward progress
if current.progress is not None:
etr = getattr(current, "estimated_time_remaining", None)
self._schedule_progress(float(current.progress), etr)
# Check for terminal states
status = current.status
if status in (
JobStatus.COMPLETED,
JobStatus.FAILED,
JobStatus.CANCELLED,
):
output_path: Optional[str] = None
if current.result and current.result.audio_path:
output_path = str(current.result.audio_path)
if status == JobStatus.COMPLETED:
finish_msg = "Conversion completed successfully."
elif status == JobStatus.CANCELLED:
finish_msg = "Cancelled"
else:
finish_msg = f"Conversion failed: {current.error or 'Unknown error'}"
self._schedule_finished(finish_msg, output_path)
break
time.sleep(POLL_INTERVAL)
prevent_sleep_end()
self._state.is_converting = False
def _schedule_log(self, message: str, level: str) -> None:
"""Schedule a log update on the Flet event loop."""
state = self._state
page = self._page
state.append_log(message, level)
async def _update() -> None:
cb = state.on_log
if cb:
cb(message, level)
try:
page.update()
except Exception:
pass
try:
page.run_task(_update)
except Exception:
pass
def _schedule_progress(self, fraction: float, etr: Optional[float]) -> None:
"""Schedule a progress update on the Flet event loop."""
state = self._state
page = self._page
state.progress = max(0.0, min(1.0, fraction))
state.etr_seconds = etr
async def _update() -> None:
cb = state.on_progress
if cb:
cb(fraction, etr)
try:
page.update()
except Exception:
pass
try:
page.run_task(_update)
except Exception:
pass
def _schedule_finished(
self, message: str, output_path: Optional[str]
) -> None:
"""Schedule a completion notification on the Flet event loop."""
state = self._state
page = self._page
state.last_output_path = output_path
self._stop_poll.set()
async def _update() -> None:
state.is_converting = False
state.progress = 1.0
state.last_output_path = output_path
cb = state.on_conversion_finished
if cb:
cb(message, output_path)
try:
page.update()
except Exception:
pass
try:
page.run_task(_update)
except Exception:
pass
+313
View File
@@ -0,0 +1,313 @@
"""
Frontend-specific utilities for the Abogen Flet application.
Contains helpers for:
- Human-readable size / duration formatting
- Voice formula parsing and display
- File-type detection
- ETR (Estimated Time Remaining) formatting
- Path resolution that adapts to desktop vs. web context
"""
from __future__ import annotations
import os
import re
from pathlib import Path
from typing import List, Optional, Tuple
from abogen.constants import (
LANGUAGE_DESCRIPTIONS,
SUPPORTED_INPUT_FORMATS,
SUPPORTED_SOUND_FORMATS,
SUBTITLE_FORMATS,
VOICES_INTERNAL,
)
# ---------------------------------------------------------------------------
# Size / duration helpers
# ---------------------------------------------------------------------------
def human_readable_size(size_bytes: int, decimal_places: int = 2) -> str:
"""
Convert a byte count into a human-readable string.
Args:
size_bytes: Number of bytes.
decimal_places: Significant decimal digits in the output.
Returns:
A string like ``"3.14 MB"`` or ``"1.00 KB"``.
"""
for unit in ("B", "KB", "MB", "GB", "TB"):
if size_bytes < 1024.0:
return f"{size_bytes:.{decimal_places}f} {unit}"
size_bytes /= 1024.0 # type: ignore[assignment]
return f"{size_bytes:.{decimal_places}f} PB"
def format_duration(seconds: float) -> str:
"""
Format a duration in seconds as ``HH:MM:SS``.
Args:
seconds: Non-negative floating-point duration.
Returns:
A colon-delimited time string, e.g. ``"00:03:42"``.
"""
total = max(0, int(seconds))
h, remainder = divmod(total, 3600)
m, s = divmod(remainder, 60)
return f"{h:02d}:{m:02d}:{s:02d}"
def format_etr(etr_seconds: Optional[float]) -> str:
"""
Format an estimated time remaining value for the UI.
Args:
etr_seconds: Seconds remaining, or None when unknown.
Returns:
Human-readable string such as ``"~3 min 42 sec"`` or ``"Calculating…"``.
"""
if etr_seconds is None:
return "Calculating…"
total = max(0, int(etr_seconds))
if total < 60:
return f"~{total} sec"
m, s = divmod(total, 60)
if m < 60:
return f"~{m} min {s} sec"
h, m = divmod(m, 60)
return f"~{h} h {m} min"
# ---------------------------------------------------------------------------
# File helpers
# ---------------------------------------------------------------------------
SUPPORTED_EXTENSIONS: Tuple[str, ...] = (
".txt",
".epub",
".pdf",
".md",
".markdown",
".srt",
".ass",
".vtt",
)
"""All file extensions that the drop-zone accepts."""
def detect_file_type(file_path: str) -> str:
"""
Return a normalised file-type token for the given path.
Args:
file_path: Absolute or relative path to the input file.
Returns:
One of ``'txt'``, ``'epub'``, ``'pdf'``, ``'markdown'``,
``'subtitle'``, or ``'unknown'``.
"""
ext = Path(file_path).suffix.lower()
if ext == ".epub":
return "epub"
if ext == ".pdf":
return "pdf"
if ext in (".md", ".markdown"):
return "markdown"
if ext in (".srt", ".ass", ".vtt"):
return "subtitle"
if ext == ".txt":
return "txt"
return "unknown"
def is_supported_file(file_path: str) -> bool:
"""
Return True when the file extension is in the supported set.
Args:
file_path: Path whose extension is inspected.
"""
return Path(file_path).suffix.lower() in SUPPORTED_EXTENSIONS
def is_book_type(file_type: str) -> bool:
"""
Return True for file types that contain chapters / pages.
Args:
file_type: Token from ``detect_file_type()``.
"""
return file_type in ("epub", "pdf", "markdown")
# ---------------------------------------------------------------------------
# Voice helpers
# ---------------------------------------------------------------------------
def voice_lang_code(voice: str) -> str:
"""
Extract the language code character from a Kokoro voice name.
The first character of every internal voice name encodes the language
(e.g. ``'a'`` for American English, ``'b'`` for British English).
Args:
voice: Raw voice string like ``'af_heart'`` or a formula.
Returns:
Single lowercase character, defaulting to ``'a'`` on failure.
"""
if not voice:
return "a"
# For plain voice IDs the first char is the language
if voice[0].isalpha() and "_" in voice[:4]:
return voice[0].lower()
# Formula: extract first alpha char
match = re.search(r"\b([a-z])", voice)
return match.group(1) if match else "a"
def language_label(lang_code: str) -> str:
"""
Return the human-readable label for a language code.
Args:
lang_code: Single-character code (``'a'``, ``'b'``, ).
Returns:
Display string, e.g. ``"American English"``.
"""
return LANGUAGE_DESCRIPTIONS.get(lang_code, lang_code.upper())
def grouped_voices() -> List[Tuple[str, List[str]]]:
"""
Return the internal voice list grouped by language for display.
Returns:
List of ``(language_label, [voice_id, ])`` tuples.
"""
groups: dict[str, List[str]] = {}
for v in VOICES_INTERNAL:
lang = language_label(v[0])
groups.setdefault(lang, []).append(v)
return sorted(groups.items())
def voice_display_name(voice_id: str) -> str:
"""
Convert a raw voice ID like ``'af_heart'`` to a prettier display name.
Args:
voice_id: Raw internal voice identifier.
Returns:
Formatted string, e.g. ``"af_heart"`` (unchanged; may be enhanced later).
"""
return voice_id
def parse_voice_formula(formula: str) -> List[Tuple[str, float]]:
"""
Parse a Kokoro voice mix formula into a list of ``(voice_id, weight)`` tuples.
Example:
``"af_heart*0.7+am_adam*0.3"`` ``[('af_heart', 0.7), ('am_adam', 0.3)]``
Args:
formula: Space- or ``+``-joined mix formula string.
Returns:
Parsed list; empty if parsing fails.
"""
parts: List[Tuple[str, float]] = []
for token in re.split(r"[+\s]+", formula.strip()):
token = token.strip()
if not token:
continue
if "*" in token:
name, _, weight_str = token.partition("*")
try:
parts.append((name.strip(), float(weight_str.strip())))
except ValueError:
pass
else:
# Bare voice id — assume full weight
if token in VOICES_INTERNAL:
parts.append((token, 1.0))
return parts
# ---------------------------------------------------------------------------
# Number formatting
# ---------------------------------------------------------------------------
def format_number(n: int) -> str:
"""
Format an integer with thousands separators.
Args:
n: Integer value.
Returns:
Formatted string, e.g. ``"1,234,567"``.
"""
return f"{n:,}"
# ---------------------------------------------------------------------------
# Path helpers
# ---------------------------------------------------------------------------
def safe_basename(path: Optional[str]) -> str:
"""
Return the basename of a path, or an empty string when path is None/empty.
Args:
path: Optional file-system path.
"""
if not path:
return ""
return os.path.basename(path)
def output_format_label(fmt: str) -> str:
"""
Return a display label for an audio output format key.
Args:
fmt: Lowercase format key (``'wav'``, ``'mp3'``, ).
"""
labels = {
"wav": "WAV (lossless)",
"flac": "FLAC (lossless compressed)",
"mp3": "MP3",
"opus": "Opus (best compression)",
"m4b": "M4B (with chapters)",
}
return labels.get(fmt, fmt.upper())
def subtitle_format_label(key: str) -> str:
"""
Return the display label for a subtitle format key.
Args:
key: Internal subtitle format key (e.g. ``'ass_centered_narrow'``).
"""
for k, label in SUBTITLE_FORMATS:
if k == key:
return label
return key
+264
View File
@@ -0,0 +1,264 @@
"""
Design tokens and theme configuration for the Abogen Flet frontend.
This module defines the application's complete colour palette, typography
scale, spacing constants, and border radii in one canonical place.
All component modules import from here; changing a value here propagates
instantly across the entire UI.
Flet's ``ft.Theme`` uses ``ColorScheme``, but for custom widgets we paint
directly with hex colours drawn from ``LIGHT`` and ``DARK`` palettes.
"""
from __future__ import annotations
import flet as ft
from dataclasses import dataclass
# ---------------------------------------------------------------------------
# Colour palettes
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class _Palette:
"""A complete colour palette for one theme mode."""
# Backgrounds
bg_base: str # Deepest background (window / page)
bg_surface: str # Cards, panels, dialogs
bg_elevated: str # Slightly raised elements (toolbar, sidebar)
bg_input: str # Text-field / dropdown backgrounds
# Brand accent
accent: str # Primary interactive colour (buttons, links)
accent_muted: str # Hover tint over accents
accent_on: str # Text drawn on top of accent fills
# Semantic
success: str
error: str
warning: str
info: str
# Text hierarchy
text_primary: str
text_secondary: str
text_disabled: str
text_on_accent: str
# Borders / dividers
border: str
border_focused: str
divider: str
# Specific UI atoms
drop_zone_border: str
drop_zone_bg: str
drop_zone_active_border: str
drop_zone_active_bg: str
log_bg: str
log_text: str
progress_bar_bg: str
progress_bar_fill: str
sidebar_bg: str
sidebar_selected_bg: str
sidebar_selected_text: str
nav_indicator: str
DARK = _Palette(
bg_base="#0f1117",
bg_surface="#181b23",
bg_elevated="#1e2230",
bg_input="#252a38",
accent="#5b8af5",
accent_muted="#3a5fc4",
accent_on="#ffffff",
success="#42ad4a",
error="#e84e3c",
warning="#f5a623",
info="#5b8af5",
text_primary="#e8eaf0",
text_secondary="#9ba3b8",
text_disabled="#4e5568",
text_on_accent="#ffffff",
border="#2c3147",
border_focused="#5b8af5",
divider="#252a38",
drop_zone_border="#3a4466",
drop_zone_bg="#151928",
drop_zone_active_border="#42ad4a",
drop_zone_active_bg="#0d1f10",
log_bg="#0d1117",
log_text="#b0b8cc",
progress_bar_bg="#1e2230",
progress_bar_fill="#5b8af5",
sidebar_bg="#13161f",
sidebar_selected_bg="#252a38",
sidebar_selected_text="#5b8af5",
nav_indicator="#5b8af5",
)
LIGHT = _Palette(
bg_base="#f4f5f8",
bg_surface="#ffffff",
bg_elevated="#edf0f5",
bg_input="#f0f2f7",
accent="#3a5fc4",
accent_muted="#2a4fae",
accent_on="#ffffff",
success="#2e9437",
error="#c0392b",
warning="#d4870a",
info="#3a5fc4",
text_primary="#1a1d27",
text_secondary="#5a6172",
text_disabled="#9ba3b8",
text_on_accent="#ffffff",
border="#dce0ea",
border_focused="#3a5fc4",
divider="#e8ebf2",
drop_zone_border="#a8b4d0",
drop_zone_bg="#f7f8fd",
drop_zone_active_border="#2e9437",
drop_zone_active_bg="#f0fff1",
log_bg="#f8f9fc",
log_text="#3d4358",
progress_bar_bg="#e4e8f0",
progress_bar_fill="#3a5fc4",
sidebar_bg="#eff1f5",
sidebar_selected_bg="#dde3f2",
sidebar_selected_text="#3a5fc4",
nav_indicator="#3a5fc4",
)
# ---------------------------------------------------------------------------
# Typography
# ---------------------------------------------------------------------------
FONT_FAMILY = "Inter, Segoe UI, Roboto, system-ui, sans-serif"
FONT_SIZE_XS = 11
FONT_SIZE_SM = 12
FONT_SIZE_BASE = 14
FONT_SIZE_MD = 16
FONT_SIZE_LG = 20
FONT_SIZE_XL = 26
FONT_SIZE_DISPLAY = 34
# ---------------------------------------------------------------------------
# Spacing scale (pixels)
# ---------------------------------------------------------------------------
SPACE_XS = 4
SPACE_SM = 8
SPACE_MD = 12
SPACE_LG = 16
SPACE_XL = 24
SPACE_2XL = 32
SPACE_3XL = 48
# ---------------------------------------------------------------------------
# Border radii
# ---------------------------------------------------------------------------
RADIUS_SM = 6
RADIUS_MD = 10
RADIUS_LG = 16
RADIUS_FULL = 999 # Pill-shaped
# ---------------------------------------------------------------------------
# Flet ColorScheme builders
# ---------------------------------------------------------------------------
def build_color_scheme(palette: _Palette) -> ft.ColorScheme:
"""
Construct a ``ft.ColorScheme`` from a ``_Palette`` object.
Args:
palette: The ``DARK`` or ``LIGHT`` palette.
Returns:
A fully-populated Flet ``ColorScheme``.
"""
return ft.ColorScheme(
primary=palette.accent,
on_primary=palette.accent_on,
primary_container=palette.accent_muted,
secondary=palette.accent,
on_secondary=palette.text_on_accent,
surface=palette.bg_surface,
on_surface=palette.text_primary,
on_surface_variant=palette.text_secondary,
error=palette.error,
on_error=palette.text_on_accent,
outline=palette.border,
)
def build_text_theme() -> ft.TextTheme:
"""
Construct a ``ft.TextTheme`` using the application's type scale.
Returns:
A Flet ``TextTheme`` with consistent font-size assignments.
"""
return ft.TextTheme(
display_large=ft.TextStyle(size=FONT_SIZE_DISPLAY, weight=ft.FontWeight.W_700),
headline_large=ft.TextStyle(size=FONT_SIZE_XL, weight=ft.FontWeight.W_700),
headline_medium=ft.TextStyle(size=FONT_SIZE_LG, weight=ft.FontWeight.W_600),
title_large=ft.TextStyle(size=FONT_SIZE_MD, weight=ft.FontWeight.W_600),
title_medium=ft.TextStyle(size=FONT_SIZE_BASE, weight=ft.FontWeight.W_500),
body_large=ft.TextStyle(size=FONT_SIZE_BASE),
body_medium=ft.TextStyle(size=FONT_SIZE_SM),
label_large=ft.TextStyle(size=FONT_SIZE_SM, weight=ft.FontWeight.W_500),
label_medium=ft.TextStyle(size=FONT_SIZE_XS),
)
def make_theme(dark: bool) -> ft.Theme:
"""
Build a complete Flet ``Theme`` for the requested mode.
Args:
dark: True for dark-mode theme, False for light-mode theme.
Returns:
A configured ``ft.Theme`` instance.
"""
palette = DARK if dark else LIGHT
return ft.Theme(
color_scheme=build_color_scheme(palette),
text_theme=build_text_theme(),
color_scheme_seed=palette.accent,
use_material3=True,
)
def get_palette(page: ft.Page) -> _Palette:
"""
Return the active colour palette for the given page.
Args:
page: The Flet ``Page`` instance.
Returns:
``DARK`` or ``LIGHT`` depending on the page's theme mode.
"""
return DARK if page.theme_mode == ft.ThemeMode.DARK else LIGHT
+6
View File
@@ -0,0 +1,6 @@
"""Views sub-package for the Abogen Flet frontend."""
from .dashboard import DashboardView
from .settings import SettingsView
from .queue_view import QueueView
__all__ = ["DashboardView", "SettingsView", "QueueView"]
+587
View File
@@ -0,0 +1,587 @@
"""
Dashboard view the primary conversion screen.
Hosts the file drop-zone, voice/speed/format controls, real-time log
terminal, progress bar, and the Start/Cancel/Finish action row.
All heavy work is delegated to ConversionBridge which runs on daemon
threads and schedules UI updates back onto the Flet event loop.
"""
from __future__ import annotations
import os
import tempfile
from pathlib import Path
from typing import Optional
import flet as ft
from ..state import AppState
from ..utils.helpers import (
detect_file_type, human_readable_size, format_number,
format_etr, grouped_voices, output_format_label,
subtitle_format_label, is_book_type, voice_lang_code, SUPPORTED_EXTENSIONS
)
from ..utils.theme import get_palette, RADIUS_MD, RADIUS_SM, SPACE_SM, SPACE_MD, SPACE_LG, SPACE_XL
from ..utils.conversion_bridge import ConversionBridge
from ..components import (
build_drop_zone, build_log_terminal, log_entry,
build_primary_button, build_secondary_button,
build_card, build_section_header, labelled_row, show_snack,
)
from abogen.constants import (
SUBTITLE_FORMATS, SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
LANGUAGE_DESCRIPTIONS, VOICES_INTERNAL,
)
from abogen.utils import get_gpu_acceleration, get_user_cache_path, calculate_text_length, clean_text
class DashboardView:
"""
The main conversion dashboard.
Instantiated once per Flet session and mounted as a ``ft.Column``
inside the page's content area.
"""
def __init__(self, page: ft.Page, state: AppState) -> None:
self._page = page
self._state = state
self._bridge = ConversionBridge(page, state)
# Internal refs
self._log_list: Optional[ft.ListView] = None
self._progress_bar: Optional[ft.ProgressBar] = None
self._etr_label: Optional[ft.Text] = None
self._drop_zone_ref: Optional[ft.GestureDetector] = None
self._drop_zone_container: Optional[ft.Container] = None
self._file_picker: Optional[ft.FilePicker] = None
# Wire state callbacks
state.on_log = self._on_log
state.on_progress = self._on_progress
state.on_conversion_finished = self._on_finished
# Build UI refs
self._voice_dd: Optional[ft.Dropdown] = None
self._speed_slider: Optional[ft.Slider] = None
self._speed_label: Optional[ft.Text] = None
self._format_dd: Optional[ft.Dropdown] = None
self._subtitle_dd: Optional[ft.Dropdown] = None
self._subtitle_fmt_dd: Optional[ft.Dropdown] = None
self._gpu_switch: Optional[ft.Switch] = None
self._start_btn: Optional[ft.ElevatedButton] = None
self._cancel_btn: Optional[ft.OutlinedButton] = None
self._finish_col: Optional[ft.Column] = None
self._controls_col: Optional[ft.Column] = None
self._log_section: Optional[ft.Container] = None
self._progress_col: Optional[ft.Column] = None
# ------------------------------------------------------------------
# Build
# ------------------------------------------------------------------
def build(self) -> ft.Column:
"""Return the complete dashboard column."""
p = self._page
dark = p.theme_mode == ft.ThemeMode.DARK
pal = get_palette(p)
if self._file_picker is None:
self._file_picker = ft.FilePicker()
# --- Drop zone ---
self._drop_zone_container = ft.Container()
self._refresh_drop_zone()
# --- Voice selector ---
voice_items = []
for lang_label, voices in grouped_voices():
voice_items.append(ft.dropdown.Option(key=f"__hdr_{lang_label}", text=f"── {lang_label} ──", disabled=True))
for v in voices:
voice_items.append(ft.dropdown.Option(key=v, text=v))
self._voice_dd = ft.Dropdown(
options=voice_items,
value=self._state.selected_voice,
on_select=self._on_voice_changed,
dense=True,
expand=True,
border_radius=RADIUS_SM,
)
# --- Speed slider ---
self._speed_label = ft.Text(f"{self._state.speed:.2f}", size=13, width=40)
self._speed_slider = ft.Slider(
min=0.1, max=2.0, value=self._state.speed,
divisions=190, label="{value}",
on_change=self._on_speed_changed,
expand=True,
)
# --- Format ---
self._format_dd = ft.Dropdown(
options=[ft.dropdown.Option(key=k, text=output_format_label(k))
for k in ("wav", "flac", "mp3", "opus", "m4b")],
value=self._state.selected_format,
on_select=lambda e: self._set_field("selected_format", e.control.value),
dense=True, expand=True, border_radius=RADIUS_SM,
)
# --- Subtitle mode ---
sub_modes = ["Disabled", "Line", "Sentence", "Sentence + Comma",
"Sentence + Highlighting"] + [f"{i} word{'s' if i > 1 else ''}" for i in range(1, 11)]
self._subtitle_dd = ft.Dropdown(
options=[ft.dropdown.Option(m) for m in sub_modes],
value=self._state.subtitle_mode,
on_select=lambda e: self._set_field("subtitle_mode", e.control.value),
dense=True, expand=True, border_radius=RADIUS_SM,
)
# --- Subtitle format ---
self._subtitle_fmt_dd = ft.Dropdown(
options=[ft.dropdown.Option(key=k, text=lbl) for k, lbl in SUBTITLE_FORMATS],
value=self._state.subtitle_format,
on_select=lambda e: self._set_field("subtitle_format", e.control.value),
dense=True, expand=True, border_radius=RADIUS_SM,
)
# --- GPU ---
self._gpu_switch = ft.Switch(
value=self._state.use_gpu, label="",
on_change=lambda e: self._set_field("use_gpu", e.control.value),
active_color="#5b8af5" if dark else "#3a5fc4",
)
# --- Log ---
log_lv = ft.ListView(expand=True, auto_scroll=True, spacing=1, padding=ft.Padding.all(8))
self._log_list = log_lv
bg_log = "#0d1117" if dark else "#f8f9fc"
bd_log = "#252a38" if dark else "#dce0ea"
self._log_section = ft.Container(
content=log_lv, bgcolor=bg_log,
border=ft.Border.all(1, bd_log),
border_radius=RADIUS_SM, height=220,
clip_behavior=ft.ClipBehavior.HARD_EDGE,
visible=False,
)
# --- Progress ---
fill = "#5b8af5" if dark else "#3a5fc4"
bg_p = "#1e2230" if dark else "#e4e8f0"
self._progress_bar = ft.ProgressBar(
value=0, color=fill, bgcolor=bg_p, height=8,
border_radius=ft.BorderRadius.all(4), expand=True,
)
self._etr_label = ft.Text("", size=11, color=pal.text_secondary, text_align=ft.TextAlign.CENTER)
self._progress_col = ft.Column([
ft.Row([self._progress_bar], spacing=0),
self._etr_label,
], spacing=SPACE_SM, horizontal_alignment=ft.CrossAxisAlignment.CENTER, visible=False)
# --- Buttons ---
self._start_btn = build_primary_button(
"Start Conversion",
icon="play_arrow",
on_click=self._on_start,
page=p,
)
self._cancel_btn = build_secondary_button(
"Cancel", icon="stop",
on_click=self._on_cancel, page=p,
)
self._cancel_btn.visible = False
# --- Finish row ---
self._finish_col = ft.Column([
ft.Row([
build_secondary_button("Open File", icon="open_in_new",
on_click=self._on_open_file, page=p),
build_secondary_button("Go to Folder", icon="folder_open",
on_click=self._on_go_folder, page=p),
build_secondary_button("New Conversion", icon="refresh",
on_click=self._on_reset, page=p),
], wrap=True, spacing=SPACE_SM, run_spacing=SPACE_SM),
], visible=False)
# --- Controls column ---
self._controls_col = ft.Column([
build_section_header("Voice & Speed", icon="record_voice_over", page=p),
labelled_row("Voice", self._voice_dd, page=p),
labelled_row("Speed", ft.Row([self._speed_slider, self._speed_label], expand=True, spacing=SPACE_SM), page=p),
ft.Divider(height=1, color=pal.divider),
build_section_header("Output", icon="audio_file", page=p),
labelled_row("Format", self._format_dd, page=p),
labelled_row("Subtitles", self._subtitle_dd, page=p),
labelled_row("Subtitle Format", self._subtitle_fmt_dd, page=p),
ft.Divider(height=1, color=pal.divider),
build_section_header("Processing", icon="memory", page=p),
labelled_row("GPU Acceleration", self._gpu_switch, page=p),
], spacing=SPACE_MD)
outer = ft.Column([
self._drop_zone_container,
ft.Container(height=SPACE_MD),
build_card(self._controls_col, page=p),
ft.Container(height=SPACE_SM),
self._log_section,
self._progress_col,
ft.Row([self._start_btn, self._cancel_btn], spacing=SPACE_SM, wrap=True),
self._finish_col,
], spacing=SPACE_MD, expand=True, scroll=ft.ScrollMode.AUTO)
return outer
# ------------------------------------------------------------------
# Drop-zone management
# ------------------------------------------------------------------
def _refresh_drop_zone(self, *, accent: bool = False, error: bool = False, err_msg: str = "") -> None:
"""Rebuild the drop-zone widget and update its container."""
p = self._page
s = self._state
fname = None; fsize = None; fchars = None
if s.selected_file and os.path.exists(s.selected_file):
disp = s.displayed_file_path or s.selected_file
fname = os.path.basename(disp)
try:
fsize = human_readable_size(os.path.getsize(s.selected_file))
except Exception:
fsize = ""
if s.char_count:
fchars = format_number(s.char_count)
label = err_msg if error else "Drag & drop your file here or click to browse"
sub = "Supports .txt · .epub · .pdf · .md · .srt · .ass · .vtt"
dz = build_drop_zone(
on_pick=self._open_file_picker,
label=label, sub_label=sub,
accent=accent, error=error,
filename=fname, file_size=fsize, char_count=fchars,
page=p,
)
if self._drop_zone_container is not None:
self._drop_zone_container.content = dz
self._drop_zone_ref = dz
# ------------------------------------------------------------------
# File picking
# ------------------------------------------------------------------
def _open_file_picker(self) -> None:
"""Open the native file picker dialog."""
self._page.run_task(self._pick_files_async)
async def _pick_files_async(self) -> None:
"""Run the file picker using Flet's async service API."""
picker = self._file_picker
if picker is None:
picker = ft.FilePicker()
self._file_picker = picker
try:
files = await picker.pick_files(
dialog_title="Select Input File",
file_type=ft.FilePickerFileType.CUSTOM,
allowed_extensions=["txt", "epub", "pdf", "md", "markdown", "srt", "ass", "vtt"],
allow_multiple=False,
)
except Exception as ex:
self._refresh_drop_zone(error=True, err_msg="Could not open file picker.")
show_snack(self._page, f"File picker error: {ex}", error=True)
self._page.update()
return
if not files:
return
file_path = files[0].path
if not file_path or not os.path.exists(file_path):
return
self._load_file(file_path)
def _load_file(self, file_path: str) -> None:
"""Validate and load a file into the session state."""
from pathlib import Path as _Path
ext = _Path(file_path).suffix.lower()
if ext not in SUPPORTED_EXTENSIONS:
self._state.reset_file_state()
self._refresh_drop_zone(error=True, err_msg=f"Unsupported file type: {ext}")
self._page.update()
return
ftype = detect_file_type(file_path)
s = self._state
if ftype in ("epub", "pdf", "markdown"):
# For book types: extract text to temp cache
self._handle_book_file(file_path, ftype)
else:
# Plain text / subtitle files
s.selected_file = file_path
s.selected_file_type = ftype
s.displayed_file_path = file_path
try:
with open(file_path, "r", encoding="utf-8", errors="ignore") as f:
text = f.read()
s.char_count = calculate_text_length(clean_text(text))
except Exception:
s.char_count = 0
self._refresh_drop_zone(accent=True)
self._update_subtitle_availability()
self._page.update()
def _handle_book_file(self, book_path: str, ftype: str) -> None:
"""Extract text from epub/pdf/markdown and store as temp txt."""
import threading as _t
s = self._state
def _extract():
try:
from abogen.text_extractor import extract_from_path
chapters = extract_from_path(book_path, file_type=ftype)
combined = "\n\n".join(ch.text for ch in chapters if ch.text.strip())
cache_dir = get_user_cache_path()
base = os.path.splitext(os.path.basename(book_path))[0]
fd, tmp = tempfile.mkstemp(prefix=f"{base}_", suffix=".txt", dir=cache_dir)
os.close(fd)
with open(tmp, "w", encoding="utf-8") as f:
f.write(combined)
s.selected_file = tmp
s.selected_file_type = ftype
s.selected_book_path = book_path
s.displayed_file_path = book_path
s.char_count = calculate_text_length(clean_text(combined))
s.selected_chapters = [f"ch_{i}" for i in range(len(chapters))]
self._refresh_drop_zone(accent=True)
self._update_subtitle_availability()
self._page.update()
except Exception as ex:
s.reset_file_state()
self._refresh_drop_zone(error=True, err_msg=f"Could not parse file: {ex}")
self._page.update()
_t.Thread(target=_extract, daemon=True).start()
# ------------------------------------------------------------------
# Control event handlers
# ------------------------------------------------------------------
def _set_field(self, attr: str, value) -> None:
setattr(self._state, attr, value)
self._state.persist_config()
def _on_voice_changed(self, e: ft.ControlEvent) -> None:
v = e.control.value or "af_heart"
self._state.selected_voice = v
self._state.selected_lang = voice_lang_code(v)
self._state.persist_config()
self._update_subtitle_availability()
self._page.update()
def _on_speed_changed(self, e: ft.ControlEvent) -> None:
val = round(float(e.control.value), 2)
self._state.speed = val
if self._speed_label:
self._speed_label.value = f"{val:.2f}"
self._state.persist_config()
self._page.update()
def _update_subtitle_availability(self) -> None:
"""Enable or disable subtitle controls based on selected language."""
lang = self._state.selected_lang
enabled = lang in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION
if self._subtitle_dd:
self._subtitle_dd.disabled = not enabled
if self._subtitle_fmt_dd:
self._subtitle_fmt_dd.disabled = not enabled
# ------------------------------------------------------------------
# Conversion control
# ------------------------------------------------------------------
def _on_start(self, _: ft.ControlEvent) -> None:
"""Validate inputs and kick off conversion."""
s = self._state
if not s.selected_file or not os.path.exists(s.selected_file):
self._refresh_drop_zone(error=True, err_msg="Please select an input file first.")
self._page.update()
return
# Transition UI to converting state
self._set_converting_ui(True)
self._bridge.start(
input_file=s.selected_file,
voice=s.get_voice_formula(),
lang_code=s.selected_lang,
speed=s.speed,
output_format=s.selected_format,
subtitle_mode=s.subtitle_mode,
subtitle_format=s.subtitle_format,
use_gpu=s.use_gpu,
save_option=s.save_option,
output_folder=s.selected_output_folder,
replace_single_newlines=s.replace_single_newlines,
char_count=s.char_count,
save_chapters_separately=s.save_chapters_separately or False,
merge_chapters_at_end=True if s.merge_chapters_at_end is None else s.merge_chapters_at_end,
separate_chapters_format=s.separate_chapters_format,
silence_between_chapters=s.silence_duration,
max_subtitle_words=s.max_subtitle_words,
chapter_intro_delay=s.chapter_intro_delay,
read_title_intro=s.read_title_intro,
read_closing_outro=s.read_closing_outro,
auto_prefix_chapter_titles=s.auto_prefix_chapter_titles,
normalize_chapter_opening_caps=s.normalize_chapter_opening_caps,
tts_provider=s.tts_provider,
supertonic_total_steps=s.supertonic_total_steps,
chunk_level=s.chunk_level,
generate_epub3=s.generate_epub3,
word_substitutions_enabled=s.word_substitutions_enabled,
word_substitutions_list=s.word_substitutions_list,
case_sensitive_substitutions=s.case_sensitive_substitutions,
replace_all_caps=s.replace_all_caps,
replace_numerals=s.replace_numerals,
fix_nonstandard_punctuation=s.fix_nonstandard_punctuation,
)
def _on_cancel(self, _: ft.ControlEvent) -> None:
self._bridge.cancel()
def _set_converting_ui(self, converting: bool) -> None:
"""Toggle UI between idle and converting states."""
if self._start_btn:
self._start_btn.visible = not converting
if self._cancel_btn:
self._cancel_btn.visible = converting
if self._controls_col:
self._controls_col.visible = not converting
if self._log_section:
self._log_section.visible = converting
if self._log_list:
self._log_list.controls.clear()
if self._progress_col:
self._progress_col.visible = converting
if self._progress_bar:
self._progress_bar.value = 0
if self._etr_label:
self._etr_label.value = "Estimating…"
if self._finish_col:
self._finish_col.visible = False
self._page.update()
# ------------------------------------------------------------------
# State callbacks (called from background thread via page.run_task)
# ------------------------------------------------------------------
def _on_log(self, message: str, level: str) -> None:
if self._log_list is None:
return
entry = log_entry(message, level, self._page)
self._log_list.controls.append(entry)
# Cap log lines
if len(self._log_list.controls) > 2000:
self._log_list.controls = self._log_list.controls[-1800:]
try:
self._page.update()
except Exception:
pass
def _on_progress(self, fraction: float, etr: Optional[float]) -> None:
if self._progress_bar:
self._progress_bar.value = min(fraction, 0.99)
if self._etr_label:
self._etr_label.value = format_etr(etr)
try:
self._page.update()
except Exception:
pass
def _on_finished(self, message: str, output_path: Optional[str]) -> None:
if self._progress_bar:
self._progress_bar.value = 1.0
if self._cancel_btn:
self._cancel_btn.visible = False
if message == "Cancelled":
# Restore idle state
self._set_converting_ui(False)
show_snack(self._page, "Conversion cancelled.", error=True)
return
if "failed" in message.lower() or "error" in message.lower():
self._log_on_log(message, "error")
self._set_converting_ui(False)
show_snack(self._page, f"Error: {message}", error=True)
return
# Success
if self._log_section:
self._log_section.visible = True
if self._progress_col:
self._progress_col.visible = False
if self._controls_col:
self._controls_col.visible = False
if self._finish_col:
self._finish_col.visible = True
if self._start_btn:
self._start_btn.visible = False
show_snack(self._page, "Conversion completed!")
try:
self._page.update()
except Exception:
pass
def _log_on_log(self, message: str, level: str) -> None:
self._on_log(message, level)
# ------------------------------------------------------------------
# Finish actions
# ------------------------------------------------------------------
def _on_open_file(self, _: ft.ControlEvent) -> None:
path = self._state.last_output_path
if path and os.path.exists(path):
import subprocess, platform
try:
if platform.system() == "Darwin":
subprocess.Popen(["open", path])
elif platform.system() == "Windows":
os.startfile(path)
else:
subprocess.Popen(["xdg-open", path])
except Exception as ex:
show_snack(self._page, f"Cannot open file: {ex}", error=True)
else:
show_snack(self._page, "Output file not found.", error=True)
def _on_go_folder(self, _: ft.ControlEvent) -> None:
path = self._state.last_output_path
folder = os.path.dirname(path) if path and os.path.isfile(path) else path
if folder and os.path.isdir(folder):
import subprocess, platform
try:
if platform.system() == "Darwin":
subprocess.Popen(["open", folder])
elif platform.system() == "Windows":
subprocess.Popen(["explorer", folder])
else:
subprocess.Popen(["xdg-open", folder])
except Exception as ex:
show_snack(self._page, f"Cannot open folder: {ex}", error=True)
else:
show_snack(self._page, "Output folder not found.", error=True)
def _on_reset(self, _: ft.ControlEvent) -> None:
self._state.reset_file_state()
self._state.reset_conversion_state()
self._refresh_drop_zone()
self._set_converting_ui(False)
if self._finish_col:
self._finish_col.visible = False
if self._controls_col:
self._controls_col.visible = True
if self._start_btn:
self._start_btn.visible = True
self._page.update()
+154
View File
@@ -0,0 +1,154 @@
"""
Queue management view.
Displays the current conversion queue, allowing the user to reorder,
remove, and inspect queued items before starting batch processing.
"""
from __future__ import annotations
from typing import Optional
import flet as ft
from ..state import AppState, ConversionJob
from ..utils.theme import get_palette, RADIUS_SM, SPACE_SM, SPACE_MD, SPACE_LG
from ..utils.helpers import safe_basename, output_format_label, format_number
from ..components import (
build_card, build_section_header, build_primary_button,
build_secondary_button, show_snack, build_divider,
resolve_icon,
)
class QueueView:
"""Queue manager view."""
def __init__(self, page: ft.Page, state: AppState) -> None:
self._page = page
self._state = state
self._list_col: Optional[ft.Column] = None
def build(self) -> ft.Column:
p = self._page
s = self._state
pal = get_palette(p)
dark = p.theme_mode == ft.ThemeMode.DARK
self._list_col = ft.Column(spacing=SPACE_SM)
self._refresh_list()
header = build_section_header("Conversion Queue",
icon="list_alt", page=p)
action_row = ft.Row([
build_primary_button(
"Start Queue",
icon="play_arrow",
on_click=self._on_start_queue,
page=p,
disabled=not s.queued_items,
),
build_secondary_button(
"Clear All",
icon="delete_sweep",
on_click=self._on_clear_queue,
page=p,
),
], spacing=SPACE_SM, wrap=True)
queue_card = build_card(ft.Column([
header,
ft.Divider(height=1, color=pal.divider),
self._list_col,
ft.Container(height=SPACE_SM),
action_row,
], spacing=SPACE_MD), page=p)
return ft.Column([queue_card], scroll=ft.ScrollMode.AUTO, expand=True)
# ------------------------------------------------------------------
def _refresh_list(self) -> None:
if self._list_col is None:
return
self._list_col.controls.clear()
s = self._state
pal = get_palette(self._page)
dark = self._page.theme_mode == ft.ThemeMode.DARK
if not s.queued_items:
self._list_col.controls.append(
ft.Text("No items in the queue.", size=13,
color=pal.text_secondary,
text_align=ft.TextAlign.CENTER)
)
return
for idx, job in enumerate(s.queued_items):
tile = self._build_job_tile(idx, job, dark, pal)
self._list_col.controls.append(tile)
try:
self._page.update()
except Exception:
pass
def _build_job_tile(self, idx: int, job: ConversionJob, dark: bool, pal) -> ft.Container:
"""Build a single queue-item tile."""
bg = pal.bg_elevated
border_clr = pal.border
accent = "#5b8af5" if dark else "#3a5fc4"
text_primary = pal.text_primary
text_secondary = pal.text_secondary
def _remove(_):
self._state.queued_items.pop(idx)
self._refresh_list()
name = safe_basename(job.display_name or job.file_path)
details = (
f"Voice: {job.voice} · Format: {output_format_label(job.output_format)}"
f" · Speed: {job.speed:.2f}x · Chars: {format_number(job.char_count)}"
)
return ft.Container(
content=ft.Row([
ft.Container(
content=ft.Text(str(idx + 1), size=12, weight=ft.FontWeight.W_700,
color=accent),
width=32,
),
ft.Column([
ft.Text(name, size=13, weight=ft.FontWeight.W_600, color=text_primary,
no_wrap=True, overflow=ft.TextOverflow.ELLIPSIS),
ft.Text(details, size=11, color=text_secondary),
], expand=True, tight=True, spacing=2),
ft.IconButton(
icon=resolve_icon("delete_outline"),
icon_color=pal.error if hasattr(pal, "error") else "#e84e3c",
icon_size=18,
tooltip="Remove",
on_click=_remove,
),
], vertical_alignment=ft.CrossAxisAlignment.CENTER, spacing=SPACE_SM),
bgcolor=bg,
border=ft.Border.all(1, border_clr),
border_radius=RADIUS_SM,
padding=ft.Padding.symmetric(horizontal=SPACE_MD, vertical=SPACE_SM),
)
# ------------------------------------------------------------------
def _on_start_queue(self, _: ft.ControlEvent) -> None:
if not self._state.queued_items:
show_snack(self._page, "Queue is empty.", error=True)
return
# Navigate to dashboard and trigger queue start
# This is wired in main.py via the nav controller
self._page.pubsub.send_all("start_queue")
def _on_clear_queue(self, _: ft.ControlEvent) -> None:
if not self._state.queued_items:
return
self._state.queued_items.clear()
self._refresh_list()
show_snack(self._page, "Queue cleared.")
+305
View File
@@ -0,0 +1,305 @@
"""
Settings view a categorised, scrollable settings page.
Groups settings into collapsible cards:
- Output (format, save location, chapters)
- Text processing (newlines, caps, substitutions, numerals)
- Subtitle options
- TTS pipeline (provider, GPU, chunking)
- Integrations (Audiobookshelf, Calibre OPDS)
"""
from __future__ import annotations
from typing import Optional
import flet as ft
from ..state import AppState
from ..utils.theme import get_palette, RADIUS_MD, RADIUS_SM, SPACE_SM, SPACE_MD, SPACE_LG
from ..utils.helpers import output_format_label, subtitle_format_label, SUPPORTED_EXTENSIONS
from ..components import (
build_card, build_section_header, labelled_row, show_snack, build_divider,
build_primary_button,
)
from abogen.constants import SUBTITLE_FORMATS
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _dd(options, value, on_change, **kw):
"""Compact dropdown factory."""
return ft.Dropdown(
options=[ft.dropdown.Option(key=k, text=v) for k, v in options],
value=value, on_select=on_change, dense=True,
border_radius=RADIUS_SM, expand=True, **kw
)
def _sw(value, on_change, label=""):
return ft.Switch(value=value, on_change=on_change, label=label)
class SettingsView:
"""The full settings panel."""
def __init__(self, page: ft.Page, state: AppState) -> None:
self._page = page
self._state = state
def build(self) -> ft.Column:
p = self._page
s = self._state
pal = get_palette(p)
# ── Output card ──────────────────────────────────────────────
format_dd = _dd(
[(k, output_format_label(k)) for k in ("wav", "flac", "mp3", "opus", "m4b")],
s.selected_format,
lambda e: self._save("selected_format", e.control.value),
)
save_dd = _dd(
[
("Save next to input file", "Save next to input file"),
("Save to Desktop", "Save to Desktop"),
("Choose output folder", "Choose output folder"),
],
s.save_option,
lambda e: self._save("save_option", e.control.value),
)
chapters_sw = _sw(s.save_chapters_separately or False,
lambda e: self._save("save_chapters_separately", e.control.value))
merge_sw = _sw(True if s.merge_chapters_at_end is None else s.merge_chapters_at_end,
lambda e: self._save("merge_chapters_at_end", e.control.value))
sep_fmt_dd = _dd(
[(k, output_format_label(k)) for k in ("wav", "flac", "mp3", "opus")],
s.separate_chapters_format,
lambda e: self._save("separate_chapters_format", e.control.value),
)
epub3_sw = _sw(s.generate_epub3, lambda e: self._save("generate_epub3", e.control.value))
output_card = build_card(ft.Column([
build_section_header("Output", icon="audio_file", page=p),
labelled_row("Audio Format", format_dd, page=p),
labelled_row("Save Location", save_dd, page=p),
build_divider(p),
labelled_row("Save Chapters Separately", chapters_sw, page=p),
labelled_row("Merge at End", merge_sw, page=p),
labelled_row("Chapter Format", sep_fmt_dd, page=p),
labelled_row("Generate EPUB3", epub3_sw, page=p),
], spacing=SPACE_MD), page=p)
# ── Text processing card ─────────────────────────────────────
newlines_sw = _sw(s.replace_single_newlines,
lambda e: self._save("replace_single_newlines", e.control.value))
caps_sw = _sw(s.replace_all_caps, lambda e: self._save("replace_all_caps", e.control.value))
norm_sw = _sw(s.normalize_chapter_opening_caps,
lambda e: self._save("normalize_chapter_opening_caps", e.control.value))
numerals_sw = _sw(s.replace_numerals, lambda e: self._save("replace_numerals", e.control.value))
punct_sw = _sw(s.fix_nonstandard_punctuation,
lambda e: self._save("fix_nonstandard_punctuation", e.control.value))
wordsub_sw = _sw(s.word_substitutions_enabled,
lambda e: self._save("word_substitutions_enabled", e.control.value))
wordsub_tf = ft.TextField(
value=s.word_substitutions_list,
multiline=True, min_lines=3, max_lines=6,
hint_text="word|replacement (one per line)",
on_change=lambda e: self._save("word_substitutions_list", e.control.value),
expand=True, border_radius=RADIUS_SM, text_size=12,
)
case_sw = _sw(s.case_sensitive_substitutions,
lambda e: self._save("case_sensitive_substitutions", e.control.value))
spacy_sw = _sw(s.use_spacy_segmentation,
lambda e: self._save("use_spacy_segmentation", e.control.value))
chunk_dd = _dd(
[("paragraph", "Paragraph"), ("sentence", "Sentence")],
s.chunk_level,
lambda e: self._save("chunk_level", e.control.value),
)
title_intro_sw = _sw(s.read_title_intro, lambda e: self._save("read_title_intro", e.control.value))
outro_sw = _sw(s.read_closing_outro, lambda e: self._save("read_closing_outro", e.control.value))
prefix_sw = _sw(s.auto_prefix_chapter_titles,
lambda e: self._save("auto_prefix_chapter_titles", e.control.value))
text_card = build_card(ft.Column([
build_section_header("Text Processing", icon="text_fields", page=p),
labelled_row("Replace Single Newlines", newlines_sw,
tooltip="Replace single newlines with spaces before processing.", page=p),
labelled_row("Replace ALL CAPS Words", caps_sw, page=p),
labelled_row("Normalize Opening CAPS", norm_sw, page=p),
labelled_row("Replace Numerals (spoken)", numerals_sw, page=p),
labelled_row("Fix Non-standard Punctuation", punct_sw, page=p),
build_divider(p),
labelled_row("Word Substitutions", wordsub_sw, page=p),
labelled_row("Case Sensitive", case_sw, page=p),
ft.Text("Substitution rules (word|replacement, one per line):",
size=12, color=pal.text_secondary),
wordsub_tf,
build_divider(p),
build_section_header("Chapter Options", icon="library_books", page=p),
labelled_row("Announce Book Title (intro)", title_intro_sw, page=p),
labelled_row("Announce Book Title (outro)", outro_sw, page=p),
labelled_row("Auto-prefix Chapter Titles", prefix_sw, page=p),
labelled_row("Chunk Level", chunk_dd, page=p),
labelled_row("Use spaCy Segmentation", spacy_sw, page=p),
], spacing=SPACE_MD), page=p)
# ── Subtitle card ─────────────────────────────────────────────
sub_modes = ["Disabled", "Line", "Sentence", "Sentence + Comma",
"Sentence + Highlighting"] + [f"{i} word{'s' if i > 1 else ''}" for i in range(1, 11)]
sub_mode_dd = _dd(
[(m, m) for m in sub_modes],
s.subtitle_mode,
lambda e: self._save("subtitle_mode", e.control.value),
)
sub_fmt_dd = _dd(
[(k, lbl) for k, lbl in SUBTITLE_FORMATS],
s.subtitle_format,
lambda e: self._save("subtitle_format", e.control.value),
)
def _mk_mw_slider():
lbl = ft.Text(str(s.max_subtitle_words), size=12, width=36)
sl = ft.Slider(
min=1, max=200, value=s.max_subtitle_words, divisions=199, label="{value}",
expand=True,
on_change=lambda e: (self._save("max_subtitle_words", int(e.control.value)),
setattr(lbl, "value", str(int(e.control.value))),
self._page.update()),
)
return ft.Row([sl, lbl], expand=True, spacing=SPACE_SM)
sub_speed_dd = _dd(
[("tts", "TTS duration"), ("silence", "Silence detection")],
s.subtitle_speed_method,
lambda e: self._save("subtitle_speed_method", e.control.value),
)
silent_gaps_sw = _sw(s.use_silent_gaps,
lambda e: self._save("use_silent_gaps", e.control.value))
subtitle_card = build_card(ft.Column([
build_section_header("Subtitles", icon="subtitles", page=p),
labelled_row("Mode", sub_mode_dd, page=p),
labelled_row("Format", sub_fmt_dd, page=p),
labelled_row("Max Words / Block", _mk_mw_slider(), page=p),
labelled_row("Speed Method", sub_speed_dd, page=p),
labelled_row("Silent Gaps", silent_gaps_sw, page=p),
], spacing=SPACE_MD), page=p)
# ── Pipeline card ─────────────────────────────────────────────
provider_dd = _dd(
[("kokoro", "Kokoro (default)"), ("supertonic", "Supertonic")],
s.tts_provider,
lambda e: self._save("tts_provider", e.control.value),
)
gpu_sw = _sw(s.use_gpu, lambda e: self._save("use_gpu", e.control.value),
label="GPU acceleration (if available)")
def _mk_steps_slider():
lbl = ft.Text(str(s.supertonic_total_steps), size=12, width=28)
sl = ft.Slider(
min=2, max=15, value=s.supertonic_total_steps, divisions=13,
label="{value}", expand=True,
on_change=lambda e: (self._save("supertonic_total_steps", int(e.control.value)),
setattr(lbl, "value", str(int(e.control.value))),
self._page.update()),
)
return ft.Row([sl, lbl], expand=True, spacing=SPACE_SM)
thresh_tf = ft.TextField(
value=str(s.speaker_analysis_threshold), width=80,
keyboard_type=ft.KeyboardType.NUMBER, border_radius=RADIUS_SM,
on_change=lambda e: self._save_int("speaker_analysis_threshold", e.control.value, 1, 25),
)
silence_tf = ft.TextField(
value=str(s.silence_duration), width=80,
keyboard_type=ft.KeyboardType.NUMBER, border_radius=RADIUS_SM,
on_change=lambda e: self._save_float("silence_duration", e.control.value, 0.0),
)
intro_tf = ft.TextField(
value=str(s.chapter_intro_delay), width=80,
keyboard_type=ft.KeyboardType.NUMBER, border_radius=RADIUS_SM,
on_change=lambda e: self._save_float("chapter_intro_delay", e.control.value, 0.0),
)
pipeline_card = build_card(ft.Column([
build_section_header("TTS Pipeline", icon="settings", page=p),
labelled_row("Provider", provider_dd, page=p),
labelled_row("GPU Acceleration", gpu_sw, page=p),
labelled_row("Supertonic Steps", _mk_steps_slider(), page=p),
build_divider(p),
labelled_row("Speaker Analysis Threshold", thresh_tf, page=p),
labelled_row("Silence Between Chapters (s)", silence_tf, page=p),
labelled_row("Chapter Intro Delay (s)", intro_tf, page=p),
], spacing=SPACE_MD), page=p)
# ── Integration card (Audiobookshelf) ─────────────────────────
abs_enabled_sw = _sw(s.audiobookshelf_enabled,
lambda e: self._save("audiobookshelf_enabled", e.control.value))
abs_url_tf = ft.TextField(value=s.audiobookshelf_base_url, hint_text="http://abs-server:13378",
expand=True, border_radius=RADIUS_SM, text_size=12,
on_change=lambda e: self._save("audiobookshelf_base_url", e.control.value))
abs_token_tf = ft.TextField(value=s.audiobookshelf_api_token, password=True,
can_reveal_password=True, expand=True,
border_radius=RADIUS_SM, text_size=12,
on_change=lambda e: self._save("audiobookshelf_api_token", e.control.value))
abs_lib_tf = ft.TextField(value=s.audiobookshelf_library_id, hint_text="Library ID",
expand=True, border_radius=RADIUS_SM, text_size=12,
on_change=lambda e: self._save("audiobookshelf_library_id", e.control.value))
abs_auto_sw = _sw(s.audiobookshelf_auto_send,
lambda e: self._save("audiobookshelf_auto_send", e.control.value))
integ_card = build_card(ft.Column([
build_section_header("Audiobookshelf Integration",
icon="cloud_upload", page=p),
labelled_row("Enabled", abs_enabled_sw, page=p),
labelled_row("Server URL", abs_url_tf, page=p),
labelled_row("API Token", abs_token_tf, page=p),
labelled_row("Library ID", abs_lib_tf, page=p),
labelled_row("Auto-upload on finish", abs_auto_sw, page=p),
], spacing=SPACE_MD), page=p)
save_btn = build_primary_button(
"Save Settings", icon="save",
on_click=self._on_save, page=p,
)
return ft.Column([
output_card,
ft.Container(height=SPACE_MD),
text_card,
ft.Container(height=SPACE_MD),
subtitle_card,
ft.Container(height=SPACE_MD),
pipeline_card,
ft.Container(height=SPACE_MD),
integ_card,
ft.Container(height=SPACE_LG),
save_btn,
ft.Container(height=SPACE_LG),
], spacing=0, scroll=ft.ScrollMode.AUTO, expand=True)
# ------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------
def _save(self, attr: str, value) -> None:
setattr(self._state, attr, value)
def _save_int(self, attr: str, raw: str, lo: int, hi: int) -> None:
try:
v = max(lo, min(hi, int(raw)))
setattr(self._state, attr, v)
except ValueError:
pass
def _save_float(self, attr: str, raw: str, lo: float) -> None:
try:
v = max(lo, float(raw))
setattr(self._state, attr, v)
except ValueError:
pass
def _on_save(self, _: ft.ControlEvent) -> None:
self._state.persist_config()
show_snack(self._page, "Settings saved.")
+5 -17
View File
@@ -5,21 +5,10 @@ import re
from dataclasses import dataclass
from typing import Any, Dict, Iterable, List, Mapping, Optional, Sequence, Tuple
_SPACY: Any = None
_SPACY_LOADED = False
def _get_spacy() -> Any:
"""Import spaCy lazily (it pulls in torch/thinc, ~2s at startup)."""
global _SPACY, _SPACY_LOADED
if not _SPACY_LOADED:
_SPACY_LOADED = True
try: # pragma: no cover - optional dependency
import spacy # type: ignore
except Exception: # pragma: no cover - spaCy may be unavailable in minimal environments
spacy = None
_SPACY = spacy
return _SPACY
try: # pragma: no cover - optional dependency
import spacy # type: ignore
except Exception: # pragma: no cover - spaCy may be unavailable in minimal environments
spacy = None
@dataclass(frozen=True)
@@ -195,7 +184,6 @@ def _build_replacement_sentence(
def _load_spacy(language: str) -> Any:
spacy = _get_spacy()
if spacy is None:
return None
@@ -233,7 +221,7 @@ def extract_heteronym_overrides(
if not lang.startswith("en"):
return []
if _get_spacy() is None:
if spacy is None:
return []
nlp = _load_spacy(lang)
+1 -1
View File
@@ -19,7 +19,7 @@ def tracked_hf_hub_download(*args, **kwargs):
try:
local_kwargs = dict(kwargs)
local_kwargs["local_files_only"] = True
return hf_hub_download(*args, **local_kwargs)
hf_hub_download(*args, **local_kwargs)
except Exception:
repo_id = kwargs.get("repo_id", "<unknown repo>")
filename = kwargs.get("filename", "<unknown file>")
-324
View File
@@ -1,324 +0,0 @@
from __future__ import annotations
import json
import logging
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Optional, Mapping, Sequence
import static_ffmpeg
from abogen.domain.metadata_helpers import (
split_people_field,
split_simple_list,
first_nonempty,
extract_year,
normalize_series_sequence,
_SERIES_SEQUENCE_TAG_KEYS,
)
from abogen.epub3.exporter import build_epub3_package
from abogen.utils import create_process
logger = logging.getLogger(__name__)
@dataclass
class ExportConfig:
"""Configuration for export operations."""
ffmpeg_path: str = "ffmpeg"
verify_ssl: bool = True
class ExportService:
"""Unified service for audiobook exports (M4B, FFMETADATA, EPUB3, Audiobookshelf)."""
def __init__(self, config: Optional[ExportConfig] = None):
self.config = config or ExportConfig()
static_ffmpeg.add_paths()
# ----------------------------------------------------------------------
# FFMETADATA
# ----------------------------------------------------------------------
def render_ffmetadata(
self,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
) -> str:
"""Render FFMETADATA content."""
lines = [";FFMETADATA1"]
for key, value in (metadata or {}).items():
if value is None:
continue
key_str = str(key).strip()
if not key_str:
continue
lines.append(f"{key_str}={self._escape_ffmetadata_value(value)}")
for chapter in chapters or []:
start = chapter.get("start")
end = chapter.get("end")
if start is None or end is None:
continue
try:
start_ms = max(0, int(round(float(start) * 1000)))
end_ms = int(round(float(end) * 1000))
except (TypeError, ValueError):
continue
if end_ms <= start_ms:
end_ms = start_ms + 1
lines.append("[CHAPTER]")
lines.append("TIMEBASE=1/1000")
lines.append(f"START={start_ms}")
lines.append(f"END={end_ms}")
title = chapter.get("title")
if title:
lines.append(f"title={self._escape_ffmetadata_value(title)}")
voices = chapter.get("voices")
if voices and isinstance(voices, list):
voice_str = ", ".join(
f"{v.get('voice', '')}@{v.get('provider', '')}"
for v in voices if v.get("voice")
)
if voice_str:
lines.append(f"voice={self._escape_ffmetadata_value(voice_str)}")
return "\n".join(lines) + "\n"
@staticmethod
def _escape_ffmetadata_value(value: Any) -> str:
escaped = str(value).replace("\\", "\\\\").replace("\n", "\\n")
escaped = escaped.replace("=", "\\=").replace(";", "\\;").replace("#", "\\#")
return escaped
def write_ffmetadata_file(
self,
audio_path: Path,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
) -> Optional[Path]:
"""Write FFMETADATA file to temp location."""
content = self.render_ffmetadata(metadata, chapters)
if content.strip() == ";FFMETADATA1":
return None
directory = audio_path.parent if audio_path.parent.exists() else Path(tempfile.gettempdir())
with tempfile.NamedTemporaryFile(
mode="w",
encoding="utf-8",
suffix=".ffmeta",
delete=False,
dir=str(directory),
) as handle:
handle.write(content)
return Path(handle.name)
# ----------------------------------------------------------------------
# M4B Export
# ----------------------------------------------------------------------
def embed_m4b_metadata(
self,
audio_path: Path,
metadata: Dict[str, Any],
chapters: List[Dict[str, Any]],
cover: "CoverConfig | None" = None,
cover_path: Optional[Path] = None,
cover_mime: Optional[str] = None,
log_callback: Optional[callable] = None,
) -> None:
"""Embed metadata and chapters into M4B file using FFmpeg + Mutagen."""
from abogen.domain.config_types import CoverConfig
if isinstance(cover, CoverConfig):
cover_path = cover.path
cover_mime = cover.mime
ffmetadata_path = self.write_ffmetadata_file(audio_path, metadata, chapters)
metadata_args = self._metadata_to_ffmpeg_args(metadata)
cmd = ["ffmpeg", "-y", "-i", str(audio_path)]
if ffmetadata_path:
cmd.extend(["-f", "ffmetadata", "-i", str(ffmetadata_path)])
if cover_path and cover_path.exists():
cmd.extend(["-i", str(cover_path)])
cmd.extend(["-map", "0:a"])
cmd.extend(["-map", "1:v:0", "-c:v:0", "mjpeg", "-disposition:v:0", "attached_pic"])
if cover_mime:
cmd.extend(["-metadata:s:v:0", f"mimetype={cover_mime}"])
cmd.extend(["-metadata:s:v:0", "title=Cover Art"])
else:
cmd.extend(["-map", "0:a"])
cmd.extend(["-c:a", "copy"])
if ffmetadata_path:
cmd.extend(["-map_metadata", "1", "-map_chapters", "1"])
else:
cmd.extend(["-map_metadata", "0"])
if metadata_args:
cmd.extend(metadata_args)
cmd.extend(["-movflags", "+faststart+use_metadata_tags"])
temp_output = audio_path.with_suffix(audio_path.suffix + ".tmp")
if audio_path.suffix.lower() in {".m4b", ".mp4", ".m4a"}:
cmd.extend(["-f", "mp4"])
cmd.append(str(temp_output))
if log_callback:
log_callback("Embedding metadata into M4B output")
process = create_process(cmd, text=True)
return_code = process.wait()
if ffmetadata_path and ffmetadata_path.exists():
try:
ffmetadata_path.unlink()
except OSError:
pass
if return_code != 0:
if temp_output.exists():
temp_output.unlink(missing_ok=True)
raise RuntimeError(f"ffmpeg failed to embed metadata (exit code {return_code})")
temp_output.replace(audio_path)
if log_callback:
log_callback("Embedded metadata and chapters into M4B output", "info")
# Apply chapters via Mutagen for better compatibility
self._apply_m4b_chapters_mutagen(audio_path, chapters, log_callback)
@staticmethod
def _metadata_to_ffmpeg_args(metadata: Dict[str, Any]) -> List[str]:
args = []
for key, value in (metadata or {}).items():
if value in (None, ""):
continue
key_str = str(key).strip()
if not key_str:
continue
normalized_key = key_str.lower()
if normalized_key == "year":
ffmpeg_key = "date"
else:
ffmpeg_key = key_str
args.extend(["-metadata", f"{ffmpeg_key}={value}"])
return args
def _apply_m4b_chapters_mutagen(
self,
audio_path: Path,
chapters: List[Dict[str, Any]],
log_callback: Optional[callable] = None,
) -> bool:
"""Apply chapter atoms using Mutagen."""
if not chapters:
return False
try:
from fractions import Fraction
from mutagen.mp4 import MP4, MP4Chapter
except ImportError:
if log_callback:
log_callback("Unable to write MP4 chapter atoms because mutagen is not installed.", "warning")
return False
try:
mp4 = MP4(str(audio_path))
except Exception as exc:
if log_callback:
log_callback(f"Failed to open m4b for chapter embedding: {exc}", "warning")
return False
chapter_objects = []
for index, entry in enumerate(sorted(chapters, key=lambda item: float(item.get("start") or 0.0))):
start_raw = entry.get("start")
if start_raw is None:
continue
try:
start_seconds = max(0.0, float(start_raw))
except (TypeError, ValueError):
continue
title_value = entry.get("title")
title_text = str(title_value) if title_value else f"Chapter {index + 1}"
start_fraction = Fraction(int(round(start_seconds * 1000)), 1000)
chapter_atom = MP4Chapter(start_fraction, title_text)
end_raw = entry.get("end")
if end_raw is not None:
try:
end_seconds = float(end_raw)
except (TypeError, ValueError):
end_seconds = None
if end_seconds is not None and end_seconds > start_seconds:
chapter_atom.end = Fraction(int(round(end_seconds * 1000)), 1000)
chapter_objects.append(chapter_atom)
if not chapter_objects:
return False
try:
mp4.chapters = chapter_objects
mp4.save()
except Exception as exc:
if log_callback:
log_callback(f"Failed to persist MP4 chapter atoms: {exc}", "warning")
return False
if log_callback:
log_callback(f"Applied {len(chapter_objects)} chapter markers via mutagen", "info")
return True
# ----------------------------------------------------------------------
# EPUB3 Export
# ----------------------------------------------------------------------
def export_epub3(
self,
output_path: Path,
book_id: str,
extraction: Any, # ExtractionResult
metadata_tags: Dict[str, Any],
chapter_markers: Sequence[Dict[str, Any]],
chunk_markers: Sequence[Dict[str, Any]],
chunks: Iterable[Dict[str, Any]],
audio_path: Path,
speaker_mode: str = "single",
cover_path: Optional[Path] = None,
cover_mime: Optional[str] = None,
) -> Path:
"""Export EPUB3 with media overlays."""
return build_epub3_package(
output_path=output_path,
book_id=book_id,
extraction=extraction,
metadata_tags=metadata_tags,
chapter_markers=chapter_markers,
chunk_markers=chunk_markers,
chunks=chunks,
audio_path=audio_path,
speaker_mode=speaker_mode,
cover_image_path=cover_path,
cover_image_mime=cover_mime,
)
if value is None:
return default
return bool(value)
__all__ = [
"ExportConfig",
"ExportService",
]
-389
View File
@@ -1,389 +0,0 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass
from enum import Enum
from pathlib import Path
from typing import List, Optional, TextIO
from abogen.domain.enums import SubtitleFormat, SubtitleMode
from abogen.subtitle_utils import clean_subtitle_text
class SubtitleAlignment(Enum):
LEFT = "left"
CENTER = "center"
NARROW = "narrow"
CENTER_NARROW = "center_narrow"
@dataclass
class SubtitleConfig:
"""Configuration for subtitle writer."""
format: SubtitleFormat
mode: SubtitleMode
alignment: SubtitleAlignment = SubtitleAlignment.LEFT
max_words: int = 50
highlight_color: str = "&H00FFFF00" # ASS highlight color
class SubtitleWriter(ABC):
"""Abstract base class for subtitle writers."""
def __init__(self, path: Path, config: SubtitleConfig):
self.path = path
self.config = config
self._file: Optional[TextIO] = None
self._index = 0
self._opened = False
def open(self) -> None:
"""Open the subtitle file and write header."""
if self._opened:
return
self._file = open(self.path, "w", encoding="utf-8", errors="replace")
self._write_header()
self._opened = True
@abstractmethod
def _write_header(self) -> None:
pass
def write_entry(
self,
start: float,
end: float,
text: str,
voice: Optional[str] = None,
) -> None:
"""Write a subtitle entry."""
if not self._opened:
self.open()
text = clean_subtitle_text(text)
if not text:
return
self._index += 1
self._write_entry(self._index, start, end, text, voice)
@abstractmethod
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
pass
def close(self) -> None:
"""Close the subtitle file."""
if self._file:
self._file.close()
self._file = None
self._opened = False
def __enter__(self) -> "SubtitleWriter":
self.open()
return self
def __exit__(self, exc_type, exc_val, exc_tb) -> None:
self.close()
class SrtWriter(SubtitleWriter):
"""SRT subtitle writer."""
def _write_header(self) -> None:
pass # SRT has no header
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
self._file.write(f"{index}\n")
self._file.write(f"{start_str} --> {end_str}\n")
self._file.write(f"{text}\n\n")
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = int(seconds % 60)
millis = int((seconds - int(seconds)) * 1000)
return f"{hours:02d}:{minutes:02d}:{secs:02d},{millis:03d}"
class VttWriter(SubtitleWriter):
"""WebVTT subtitle writer."""
def _write_header(self) -> None:
self._file.write("WEBVTT\n\n")
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
self._file.write(f"{index}\n")
self._file.write(f"{start_str} --> {end_str}\n")
self._file.write(f"{text}\n\n")
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = seconds % 60
return f"{hours:02d}:{minutes:02d}:{secs:06.3f}".replace(".", ".")
class AssWriter(SubtitleWriter):
"""ASS subtitle writer with karaoke highlighting support."""
def __init__(self, path: Path, config: SubtitleConfig):
super().__init__(path, config)
self._is_centered = config.alignment in (SubtitleAlignment.CENTER, SubtitleAlignment.CENTER_NARROW)
self._is_narrow = config.alignment in (SubtitleAlignment.NARROW, SubtitleAlignment.CENTER_NARROW)
def _write_header(self) -> None:
margin = "90" if self._is_narrow else "10"
alignment = "5" if self._is_centered else "2"
self._file.write("[Script Info]\n")
self._file.write("Title: Generated by Abogen\n")
self._file.write("ScriptType: v4.00+\n\n")
# Styles
self._file.write("[V4+ Styles]\n")
self._file.write(
"Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, "
"OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, "
"ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, "
"Alignment, MarginL, MarginR, MarginV, Encoding\n"
)
if self.config.mode == SubtitleMode.SENTENCE_HIGHLIGHT:
# Karaoke style with highlighting
self._file.write(
f"Style: Default,Arial,24,&H00FFFFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n"
)
self._file.write(
f"Style: Highlight,Arial,24,&H0000FFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n\n"
)
else:
self._file.write(
f"Style: Default,Arial,24,&H00FFFFFF,&H00808080,&H00000000,&H00404040,"
f"0,0,0,0,100,100,0,0,3,2,0,{alignment},{margin},{margin},10,1\n\n"
)
self._file.write("[Events]\n")
self._file.write(
"Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text\n"
)
def _write_entry(
self,
index: int,
start: float,
end: float,
text: str,
voice: Optional[str],
) -> None:
start_str = self._format_time(start)
end_str = self._format_time(end)
if voice:
text = f"[{voice}] {text}"
style = "Default"
if self.config.mode == SubtitleMode.SENTENCE_HIGHLIGHT:
# Entries from process_subtitle_tokens already carry per-word
# {\kf...} timing; only synthesize simplified tags when absent.
if "{\\k" not in text:
text = self._add_karaoke_tags(text)
style = "Highlight"
alignment_tag = r"{\an5}" if self._is_centered else ""
self._file.write(
f"Dialogue: 0,{start_str},{end_str},{style},,0,0,0,,{alignment_tag}{text}\n"
)
def _add_karaoke_tags(self, text: str) -> str:
"""Add karaoke highlighting tags to text."""
# Simple word-level karaoke timing
words = text.split()
if not words:
return text
# This is a simplified version - real karaoke needs per-word timing
# For now, just return the text with the highlight color
return r"{\k100}" + r"{\k100}".join(words) + r"{\k0}"
@staticmethod
def _format_time(seconds: float) -> str:
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = seconds % 60
return f"{hours}:{minutes:02d}:{secs:05.2f}"
def _coerce_mode(mode: str) -> SubtitleMode:
"""Parse a subtitle mode, tolerating word-count strings like "5 words".
Word-count modes are grouped upstream (subtitle_generation) and the writer
only branches on SubtitleMode.SENTENCE_HIGHLIGHT, so any non-highlight
fallback is behaviorally equivalent for the writers.
"""
try:
return SubtitleMode(mode)
except ValueError:
return SubtitleMode.SENTENCE
def create_subtitle_writer(
path: Path,
format: str,
mode: str,
alignment: str = "left",
max_words: int = 50,
) -> SubtitleWriter:
"""Factory function to create subtitle writer."""
fmt = SubtitleFormat(format.lower())
mode = _coerce_mode(mode)
align = SubtitleAlignment(alignment.lower())
config = SubtitleConfig(
format=fmt,
mode=mode,
alignment=align,
max_words=max_words,
)
if fmt == SubtitleFormat.SRT:
return SrtWriter(path, config)
elif fmt == SubtitleFormat.VTT:
return VttWriter(path, config)
elif fmt == SubtitleFormat.ASS:
return AssWriter(path, config)
else:
raise ValueError(f"Unsupported subtitle format: {format}")
def resolve_subtitle_format(
subtitle: "SubtitleConfig | str | None",
subtitle_mode: str | None = None,
) -> tuple[str, str]:
"""Resolve a subtitle config to (file_extension, alignment).
Accepts a SubtitleConfig object or individual format/mode strings
for backward compatibility.
Returns:
Tuple of (file_extension, alignment) suitable for
:func:`create_subtitle_writer`.
"""
from abogen.domain.config_types import SubtitleConfig
if isinstance(subtitle, SubtitleConfig):
fmt = subtitle.format.value.lower()
mode_str = subtitle.mode.value
else:
fmt = (subtitle or "srt").lower()
mode_str = subtitle_mode or "Disabled"
if mode_str == "Sentence + Highlighting" and fmt == "srt":
fmt = "ass"
if "ass" in fmt:
extension = "ass"
if "centered_narrow" in fmt:
alignment = "center_narrow"
elif "centered" in fmt:
alignment = "center"
elif "narrow" in fmt:
alignment = "narrow"
else:
alignment = "left"
else:
extension = fmt if fmt in ("srt", "vtt") else "srt"
alignment = "left"
return extension, alignment
def make_subtitle_writer(
audio_path: Path,
subtitle: "SubtitleConfig | str | None",
subtitle_mode: str | None = None,
max_words: int | None = None,
) -> SubtitleWriter | None:
"""Convenience: resolve format and create a writer, or return None if disabled.
Accepts a SubtitleConfig object or individual format/mode strings
for backward compatibility.
Returns ``None`` when subtitle mode is ``"Disabled"`` or the
format is unsupported.
"""
from abogen.domain.config_types import SubtitleConfig
if isinstance(subtitle, SubtitleConfig):
mode_str = subtitle.mode.value
if mode_str == "Disabled":
return None
words = subtitle.max_words
else:
mode_str = subtitle_mode or subtitle or "Disabled"
if mode_str == "Disabled":
return None
words = max_words or 50
extension, alignment = resolve_subtitle_format(subtitle, subtitle_mode)
try:
return create_subtitle_writer(
audio_path.with_suffix(f".{extension}"),
extension,
mode_str,
alignment=alignment,
max_words=words,
)
except (ValueError, KeyError):
return None
__all__ = [
"SubtitleFormat",
"SubtitleMode",
"SubtitleAlignment",
"SubtitleConfig",
"SubtitleWriter",
"SrtWriter",
"VttWriter",
"AssWriter",
"create_subtitle_writer",
"resolve_subtitle_format",
"make_subtitle_writer",
]
+36 -3
View File
@@ -2,7 +2,9 @@ from __future__ import annotations
import json
import logging
import math
import mimetypes
import re
from contextlib import ExitStack
from dataclasses import dataclass
from pathlib import Path
@@ -10,8 +12,6 @@ from typing import Any, Dict, Iterable, List, Mapping, Optional, Sequence, Tuple
import httpx
from abogen.domain.metadata_helpers import normalize_series_sequence
logger = logging.getLogger(__name__)
@@ -641,7 +641,40 @@ class AudiobookshelfClient:
for key in preferred_keys:
if key not in metadata:
continue
normalized = normalize_series_sequence(metadata.get(key))
normalized = AudiobookshelfClient._normalize_series_sequence(metadata.get(key))
if normalized:
return normalized
return ""
@staticmethod
def _normalize_series_sequence(raw: Any) -> str:
if raw is None:
return ""
if isinstance(raw, (int, float)):
if isinstance(raw, float) and (math.isnan(raw) or math.isinf(raw)):
return ""
text = str(raw)
else:
text = str(raw).strip()
if not text:
return ""
candidate = text.replace(",", ".")
match = re.search(r"\d+(?:\.\d+)?", candidate)
if not match:
return ""
normalized = match.group(0)
if "." in normalized:
normalized = normalized.rstrip("0").rstrip(".")
if not normalized:
normalized = "0"
return normalized
try:
return str(int(normalized))
except ValueError:
cleaned = normalized.lstrip("0")
return cleaned or "0"
+15 -63
View File
@@ -672,15 +672,6 @@ def tokenize_with_spans(text: str) -> List[Tuple[str, int, int]]:
]
_OPENING_PUNCTUATION_CHARS = "«‹“‘([{¡¿「『"
_CLOSING_PUNCTUATION_CHARS = "»›”’)]}」』"
_STANDARD_PUNCTUATION_CHARS = ",.;:!?%"
_OPENING_PUNCT_CLASS = re.escape(_OPENING_PUNCTUATION_CHARS)
_CLOSING_PUNCT_CLASS = re.escape(_CLOSING_PUNCTUATION_CHARS)
_STANDARD_PUNCT_CLASS = re.escape(_STANDARD_PUNCTUATION_CHARS)
def _cleanup_spacing(text: str) -> str:
if not text:
return text
@@ -688,39 +679,22 @@ def _cleanup_spacing(text: str) -> str:
for marker in ("\ufeff", "\u200b", "\u200c", "\u200d", "\u2060"):
text = text.replace(marker, "")
# Collapse spaces before standard punctuation and unambiguous closing quotes/brackets.
text = re.sub(rf"\s+([{_STANDARD_PUNCT_CLASS}])", r"\1", text)
text = re.sub(rf"\s+([{_CLOSING_PUNCT_CLASS}])", r"\1", text)
# Collapse spaces before closing punctuation.
text = re.sub(r"\s+([,.;:!?%])", r"\1", text)
text = re.sub(r"\s+([\"”»›)\]\}])", r"\1", text)
# Remove spaces directly after unambiguous opening punctuation/quotes.
text = re.sub(rf"([{_OPENING_PUNCT_CLASS}])\s+", r"\1", text)
# Handle ambiguous straight quotes (\", ')
# 1. Remove spaces directly after opening straight quotes:
# e.g. ' \" word' -> ' \"word', '^\" word' -> '\"word', '(\" word' -> '(\"word'
text = re.sub(rf"(^|[\s{_OPENING_PUNCT_CLASS}])([\"\'])\s+", r"\1\2", text)
# 2. Collapse spaces directly before closing straight quotes:
# e.g. 'word \" ' -> 'word\" ', 'word \".' -> 'word\".'
text = re.sub(rf"\s+([\"\'])([\s{_STANDARD_PUNCT_CLASS}{_CLOSING_PUNCT_CLASS}]|$)", r"\1\2", text)
# Remove spaces directly after opening punctuation/quotes.
text = re.sub(r"([«‹“‘\"'(\[\{])\s+", r"\1", text)
# Ensure spaces exist after sentence punctuation when followed by a word/quote.
# Runs of punctuation ("...", "?!?", "!!") must stay together: no space
# inside the run, only after it ("a...b" -> "a... b").
text = re.sub(rf"([{_STANDARD_PUNCT_CLASS}])(?![\s{_STANDARD_PUNCT_CLASS}{_CLOSING_PUNCT_CLASS}\"\'”’»›)])", r"\1 ", text)
# Ensure space after unambiguous closing quote when followed by a word (e.g. '”Next' -> '” Next')
text = re.sub(rf"([{_CLOSING_PUNCT_CLASS}])(?![\s{_STANDARD_PUNCT_CLASS}{_CLOSING_PUNCT_CLASS}\"\'”’»›)])", r"\1 ", text)
# Straight double quote closing (preceded by non-whitespace) followed directly by a word/number/opening
text = re.sub(rf"(\S\")([A-Za-z0-9{_OPENING_PUNCT_CLASS}])", r"\1 \2", text)
# Straight single quote closing (preceded by punctuation, not internal word apostrophe) followed by a word
text = re.sub(rf"([{_STANDARD_PUNCT_CLASS}{_CLOSING_PUNCT_CLASS}]\')([A-Za-z0-9{_OPENING_PUNCT_CLASS}])", r"\1 \2", text)
text = re.sub(r"([,.;:!?%])(?![\s”'\"’»›)])", r"\1 ", text)
text = re.sub(r"([”\"])(?![\s.,;:!?\"”’»›)])", r"\1 ", text)
# Tighten hyphen/em dash spacing between word characters.
text = re.sub(r"(?<=\w)\s*([-–—])\s*(?=\w)", r"\1", text)
# Normalize multiple spaces, preserving paragraph breaks (double
# newlines must survive so the TTS engine can split on them).
text = re.sub(r"[^\S\n]{2,}", " ", text)
text = re.sub(r"\n{3,}", "\n\n", text)
# Normalize multiple spaces.
text = re.sub(r"\s{2,}", " ", text)
return text.strip()
@@ -1648,18 +1622,8 @@ def normalize_apostrophes(
results.append((tok, category, norm))
normalized_tokens.append(norm)
out_pieces: List[str] = []
last_end = 0
for (tok, start, end), norm in zip(token_entries, normalized_tokens):
if start > last_end:
out_pieces.append(text[last_end:start])
out_pieces.append(norm)
last_end = end
if last_end < len(text):
out_pieces.append(text[last_end:])
reconstructed = "".join(out_pieces)
normalized_text = _cleanup_spacing(reconstructed)
filtered = [token for token in normalized_tokens if token]
normalized_text = _cleanup_spacing(" ".join(filtered))
return normalized_text, results
@@ -1860,10 +1824,7 @@ def _normalize_grouped_numbers(text: str, cfg: ApostropheConfig) -> str:
for digit in trimmed_fraction:
if not digit.isdigit():
return token
try:
digit_words.append(_DIGIT_WORDS[int(digit)])
except (ValueError, IndexError):
return token
digit_words.append(_DIGIT_WORDS[int(digit)])
spoken = f"{integer_words} point {' '.join(digit_words)}"
return f"minus {spoken}" if is_negative else spoken
@@ -1885,27 +1846,18 @@ def _normalize_grouped_numbers(text: str, cfg: ApostropheConfig) -> str:
# Magnitude case: $2.5 million -> two point five million dollars
if "." in amount_str:
integer_part, fraction_part = amount_str.split(".", 1)
try:
integer_val = int(integer_part)
except ValueError:
return match.group(0)
integer_val = int(integer_part)
integer_words = _int_to_words(integer_val, language)
# Spell out fraction digits
digit_words = []
for digit in fraction_part:
if digit.isdigit():
try:
digit_words.append(_DIGIT_WORDS[int(digit)])
except (ValueError, IndexError):
return match.group(0)
digit_words.append(_DIGIT_WORDS[int(digit)])
amount_spoken = f"{integer_words} point {' '.join(digit_words)}"
else:
try:
amount_spoken = _int_to_words(int(amount), language)
except (ValueError, OverflowError):
return match.group(0)
amount_spoken = _int_to_words(int(amount), language)
currency_names = {
"$": "dollars",
+15 -5
View File
@@ -2,14 +2,13 @@
from __future__ import annotations
import atexit
import os
import platform
import signal
import sys
# Initialise global shutdown handling (atexit, signals, Qt) as early as possible.
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
from abogen.utils import load_config
from abogen.utils import load_config, prevent_sleep_end
from abogen.webui.app import main as _run_web_ui
# Configure Hugging Face Hub behaviour (mirrors legacy GUI defaults).
@@ -28,6 +27,17 @@ os.environ.setdefault("MIOPEN_CONV_PRECISE_ROCM_TUNING", "0")
if platform.system() == "Darwin" and platform.processor() == "arm":
os.environ.setdefault("PYTORCH_ENABLE_MPS_FALLBACK", "1")
atexit.register(prevent_sleep_end)
def _cleanup_sleep(signum, _frame):
prevent_sleep_end()
sys.exit(0)
signal.signal(signal.SIGINT, _cleanup_sleep)
signal.signal(signal.SIGTERM, _cleanup_sleep)
def main() -> None:
"""Launch the Flask-based web UI."""
+4 -5
View File
@@ -21,8 +21,7 @@ from PyQt6.QtWidgets import (
)
from PyQt6.QtCore import QThread, pyqtSignal
from abogen.constants import COLORS
from abogen.tts_plugin.utils import get_voices
from abogen.constants import COLORS, VOICES_INTERNAL
from abogen.spacy_utils import SPACY_MODELS
import abogen.hf_tracker
@@ -115,7 +114,7 @@ class PreDownloadWorker(QThread):
self._voices_success = False
return
voice_list = get_voices("kokoro")
voice_list = VOICES_INTERNAL
for idx, voice in enumerate(voice_list, start=1):
if self._cancelled:
self._voices_success = False
@@ -463,14 +462,14 @@ class PreDownloadDialog(QDialog):
try:
from huggingface_hub import try_to_load_from_cache
for voice in get_voices("kokoro"):
for voice in VOICES_INTERNAL:
if not try_to_load_from_cache(
repo_id="hexgrad/Kokoro-82M", filename=f"voices/{voice}.pt"
):
missing.append(voice)
except Exception:
# If HF missing, report all as missing
return False, list(get_voices("kokoro"))
return False, list(VOICES_INTERNAL)
return (len(missing) == 0), missing
def _check_kokoro_model(self) -> bool:
+214 -25
View File
@@ -29,15 +29,11 @@ from abogen.utils import (
get_resource_path,
)
from abogen.book_parser import get_book_parser
from abogen.domain.metadata_extraction import (
extract_book_metadata_epub,
extract_book_metadata_pdf,
extract_book_metadata_markdown,
format_metadata_tags,
)
from abogen.subtitle_utils import clean_text
from abogen.domain.text_utils import calculate_text_length
from abogen.subtitle_utils import (
clean_text,
calculate_text_length,
)
import os
import logging
@@ -45,9 +41,9 @@ import urllib.parse
import textwrap
# Setup logging
from abogen.utils import setup_console_logging
setup_console_logging()
logging.basicConfig(
level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s"
)
_HTML_TAG_PATTERN = re.compile(r"<[^>]+>")
_LEADING_DASH_PATTERN = re.compile(r"^\s*[-–—]\s*")
@@ -952,14 +948,169 @@ class HandlerDialog(QDialog):
self.previewEdit.setHtml(html_content)
def _extract_book_metadata(self):
metadata = {
"title": None,
"authors": [],
"description": None,
"cover_image": None,
"publisher": None,
"publication_year": None,
}
if self.parser.file_type == "epub":
return extract_book_metadata_epub(self.book)
try:
title_items = self.book.get_metadata("DC", "title")
if title_items and len(title_items) > 0:
metadata["title"] = title_items[0][0]
except Exception as e:
logging.warning(f"Error extracting title metadata: {e}")
try:
author_items = self.book.get_metadata("DC", "creator")
if author_items:
metadata["authors"] = [
author[0] for author in author_items if len(author) > 0
]
except Exception as e:
logging.warning(f"Error extracting author metadata: {e}")
try:
desc_items = self.book.get_metadata("DC", "description")
if desc_items and len(desc_items) > 0:
metadata["description"] = desc_items[0][0]
except Exception as e:
logging.warning(f"Error extracting description metadata: {e}")
try:
publisher_items = self.book.get_metadata("DC", "publisher")
if publisher_items and len(publisher_items) > 0:
metadata["publisher"] = publisher_items[0][0]
except Exception as e:
logging.warning(f"Error extracting publisher metadata: {e}")
# Try to extract publication year
try:
date_items = self.book.get_metadata("DC", "date")
if date_items and len(date_items) > 0:
date_str = date_items[0][0]
# Try to extract just the year from the date string
year_match = re.search(r"\b(19|20)\d{2}\b", date_str)
if year_match:
metadata["publication_year"] = year_match.group(0)
else:
metadata["publication_year"] = date_str
except Exception as e:
logging.warning(f"Error extracting publication date metadata: {e}")
for item in self.book.get_items_of_type(ebooklib.ITEM_COVER):
metadata["cover_image"] = item.get_content()
break
if not metadata["cover_image"]:
for item in self.book.get_items_of_type(ebooklib.ITEM_IMAGE):
if "cover" in item.get_name().lower():
metadata["cover_image"] = item.get_content()
break
elif self.parser.file_type == "markdown":
return extract_book_metadata_markdown(
self.markdown_text, self.markdown_toc
)
# Extract metadata from markdown frontmatter or first heading
if self.markdown_text:
# Try to extract YAML frontmatter
frontmatter_match = re.match(
r"^---\s*\n(.*?)\n---\s*\n", self.markdown_text, re.DOTALL
)
if frontmatter_match:
try:
frontmatter = frontmatter_match.group(1)
# Simple YAML-like parsing for common fields
title_match = re.search(
r"^title:\s*(.+)$",
frontmatter,
re.MULTILINE | re.IGNORECASE,
)
if title_match:
metadata["title"] = (
title_match.group(1).strip().strip("\"'")
)
author_match = re.search(
r"^author:\s*(.+)$",
frontmatter,
re.MULTILINE | re.IGNORECASE,
)
if author_match:
metadata["authors"] = [
author_match.group(1).strip().strip("\"'")
]
desc_match = re.search(
r"^description:\s*(.+)$",
frontmatter,
re.MULTILINE | re.IGNORECASE,
)
if desc_match:
metadata["description"] = (
desc_match.group(1).strip().strip("\"'")
)
date_match = re.search(
r"^date:\s*(.+)$", frontmatter, re.MULTILINE | re.IGNORECASE
)
if date_match:
date_str = date_match.group(1).strip().strip("\"'")
year_match = re.search(r"\b(19|20)\d{2}\b", date_str)
if year_match:
metadata["publication_year"] = year_match.group(0)
except Exception as e:
logging.warning(f"Error parsing markdown frontmatter: {e}")
# Fallback: use first H1 header as title if no frontmatter title
if not metadata["title"] and self.markdown_toc:
# Find the first level 1 header
first_h1 = next(
(h for h in self.markdown_toc if h["level"] == 1), None
)
if first_h1:
metadata["title"] = first_h1["name"]
else:
return extract_book_metadata_pdf(self.pdf_doc)
pdf_info = self.pdf_doc.metadata
if pdf_info:
metadata["title"] = pdf_info.get("title", None)
author = pdf_info.get("author", None)
if author:
metadata["authors"] = [author]
metadata["description"] = pdf_info.get("subject", None)
keywords = pdf_info.get("keywords", None)
if keywords:
if metadata["description"]:
metadata["description"] += f"\n\nKeywords: {keywords}"
else:
metadata["description"] = f"Keywords: {keywords}"
metadata["publisher"] = pdf_info.get("creator", None)
# Try to extract publication date from PDF metadata
if "creationDate" in pdf_info:
date_str = pdf_info["creationDate"]
year_match = re.search(r"D:(\d{4})", date_str)
if year_match:
metadata["publication_year"] = year_match.group(1)
elif "modDate" in pdf_info:
date_str = pdf_info["modDate"]
year_match = re.search(r"D:(\d{4})", date_str)
if year_match:
metadata["publication_year"] = year_match.group(1)
if len(self.pdf_doc) > 0:
try:
pix = self.pdf_doc[0].get_pixmap(matrix=fitz.Matrix(2, 2))
metadata["cover_image"] = pix.tobytes("png")
except Exception:
pass
return metadata
def get_selected_text(self):
# If a background loader thread is running, wait for it to finish to
@@ -985,21 +1136,59 @@ class HandlerDialog(QDialog):
def _format_metadata_tags(self):
"""Format metadata tags for insertion at the beginning of the text"""
import datetime
from abogen.utils import get_user_cache_path
metadata = self.book_metadata
filename = os.path.splitext(os.path.basename(self.book_path))[0]
chapter_count = len(self.checked_chapters)
cache_dir = get_user_cache_path()
current_year = str(datetime.datetime.now().year)
return format_metadata_tags(
self.book_metadata,
filename,
chapter_count,
self.parser.file_type,
cover_bytes=self.book_metadata.get("cover_image"),
cache_dir=cache_dir,
# Get values with fallbacks
title = metadata.get("title") or filename
authors = metadata.get("authors") or ["Unknown"]
authors_text = ", ".join(authors)
album_artist = authors_text or "Unknown"
year = (
metadata.get("publication_year") or current_year
) # Use publication year if available
# Count chapters/pages
total_chapters = len(self.checked_chapters)
chapter_text = (
f"{total_chapters} {'Chapters' if self.parser.file_type == 'epub' else 'Pages'}"
)
# Handle cover image
cover_tag = ""
if metadata.get("cover_image"):
try:
import uuid
cache_dir = get_user_cache_path()
cover_path = os.path.join(cache_dir, f"cover_{uuid.uuid4()}.jpg")
cover_path = os.path.normpath(cover_path)
with open(cover_path, "wb") as f:
f.write(metadata["cover_image"])
cover_tag = f"<<METADATA_COVER_PATH:{cover_path}>>"
except Exception as e:
logging.warning(f"Failed to save cover image: {e}")
# Format metadata tags
metadata_tags = [
f"<<METADATA_TITLE:{title}>>",
f"<<METADATA_ARTIST:{authors_text}>>",
f"<<METADATA_ALBUM:{title} ({chapter_text})>>",
f"<<METADATA_YEAR:{year}>>",
f"<<METADATA_ALBUM_ARTIST:{album_artist}>>",
f"<<METADATA_COMPOSER:Narrator>>",
f"<<METADATA_GENRE:Audiobook>>",
]
if cover_tag:
metadata_tags.append(cover_tag)
return "\n".join(metadata_tags)
def _get_markdown_selected_text(self):
"""Get selected text from markdown chapters"""
all_checked_identifiers = set()
+1513 -463
View File
File diff suppressed because it is too large Load Diff
-209
View File
@@ -1,209 +0,0 @@
"""PyQt adapter: ConversionThread -> ConversionRequest.
Converts a PyQt ConversionThread into a ConversionRequest that the application layer can process.
This adapter is the bridge between the PyQt layer and the application/domain layer.
The adapter is responsible for:
- Mapping ConversionThread fields to ConversionRequest fields
- Handling UI-specific state (signals, dialogs, cancellation)
- Providing PipelineProvider and VoiceResolver implementations
Subtitle file/timestamp special paths remain in ConversionThread.run() early return.
"""
from __future__ import annotations
import os
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional
from abogen.application.conversion_config import (
ChapterChunkConfig,
Epub3ExportConfig,
PronunciationConfig,
WordSubstitutionConfig,
)
from abogen.application.conversion_request import ConversionRequest
from abogen.application.conversion_ports import ConversionCancelled, ResolvedVoice
def build_conversion_request_from_thread(thread: Any) -> ConversionRequest:
"""Convert a PyQt ConversionThread into a ConversionRequest.
This is the primary function that maps thread fields to ConversionRequest.
All fields are copied the request is independent of the thread.
Args:
thread: PyQt ConversionThread instance
Returns:
ConversionRequest with all thread data mapped
"""
# Determine source path
source_path = None
is_direct_text = getattr(thread, "is_direct_text", False)
if not is_direct_text and thread.file_name:
source_path = Path(thread.file_name)
# Determine original filename
original_filename = ""
if getattr(thread, "from_queue", False):
base_path = getattr(thread, "save_base_path", None) or thread.file_name
else:
base_path = getattr(thread, "display_path", None) or thread.file_name
if base_path:
original_filename = os.path.basename(base_path)
# Determine output folder
output_folder = None
if thread.output_folder:
output_folder = Path(thread.output_folder)
# Build pronunciation config
pronunciation = None
pron_overrides = getattr(thread, "pronunciation_overrides", []) or []
manual_overrides = getattr(thread, "manual_overrides", []) or []
heteronym_overrides = getattr(thread, "heteronym_overrides", []) or []
norm_overrides = getattr(thread, "normalization_overrides", None)
if pron_overrides or manual_overrides or heteronym_overrides or norm_overrides:
pronunciation = PronunciationConfig(
pronunciation_overrides=pron_overrides,
manual_overrides=manual_overrides,
heteronym_overrides=heteronym_overrides,
normalization_overrides=norm_overrides,
)
# Build epub3 config
epub3_export = None
if getattr(thread, "generate_epub3", False):
epub3_export = Epub3ExportConfig()
return ConversionRequest(
# Source
source_path=source_path,
direct_text=thread.file_name if is_direct_text else None,
original_filename=original_filename,
# TTS Settings
language=thread.lang_code,
tts_provider="kokoro", # PyQt uses Kokoro by default
voice=thread.voice,
voice_profile=getattr(thread, "voice_profile", None),
speed=thread.speed,
use_gpu=thread.use_gpu,
supertonic_total_steps=getattr(thread, "supertonic_total_steps", 5),
# Output Format
output_format=thread.output_format,
subtitle_mode=thread.subtitle_mode,
subtitle_format=getattr(thread, "subtitle_format", "srt"),
max_subtitle_words=getattr(thread, "max_subtitle_words", 50),
# Save Options
save_mode=thread.save_option,
output_folder=output_folder,
save_chapters_separately=getattr(thread, "save_chapters_separately", False),
merge_chapters_at_end=getattr(thread, "merge_chapters_at_end", True),
separate_chapters_format=getattr(thread, "separate_chapters_format", "wav"),
save_as_project=getattr(thread, "save_as_project", False),
# Timing
silence_between_chapters=getattr(thread, "silence_duration", 2.0),
chapter_intro_delay=getattr(thread, "chapter_intro_delay", 0.0),
# Content Processing
replace_single_newlines=getattr(thread, "replace_single_newlines", False),
read_title_intro=getattr(thread, "read_title_intro", False),
read_closing_outro=getattr(thread, "read_closing_outro", True),
auto_prefix_chapter_titles=getattr(thread, "auto_prefix_chapter_titles", True),
normalize_chapter_opening_caps=thread.normalize_chapter_opening_caps,
# Metadata
metadata_tags=getattr(thread, "metadata_tags", {}) or {},
# Artifacts
cover_image_path=getattr(thread, "cover_image_path", None),
cover_image_mime=getattr(thread, "cover_image_mime", None),
# Feature configs
pronunciation=pronunciation,
epub3_export=epub3_export,
chapter_chunk=ChapterChunkConfig(), # PyQt doesn't use chapter overrides from GUI
)
class PyQtEvents:
"""PyQt implementation of ConversionEvents protocol.
Wraps a ConversionThread to provide logging, progress, and cancellation.
"""
def __init__(self, thread: Any):
self._thread = thread
def log(self, message: str, level: str = "info") -> None:
"""Log a message via signal."""
self._thread.log_updated.emit((message, _level_to_color(level)))
def progress(self, pct: int, etr: str) -> None:
"""Update progress via signal."""
self._thread.progress_updated.emit(pct, etr)
def check_cancelled(self) -> None:
"""Check if conversion was cancelled.
Raises:
ConversionCancelled: If cancellation was requested
"""
if self._thread.cancel_requested:
raise ConversionCancelled("Conversion cancelled by user")
class PyQtPipelineProvider:
"""PyQt implementation of PipelineProvider protocol.
Wraps the existing backend from ConversionThread.
"""
def __init__(self, backend: Any):
self._backend = backend
def get(self, provider: str, language: str, use_gpu: bool) -> Any:
"""Get a TTS backend instance.
For PyQt, this returns the pre-initialized backend.
"""
return self._backend
def dispose_all(self) -> None:
"""Dispose all backend resources."""
pass # PyQt manages backend lifecycle in thread
class PyQtVoiceResolver:
"""PyQt implementation of VoiceResolver protocol.
Wraps load_voice_cached from the ConversionThread.
"""
def __init__(self, thread: Any):
self._thread = thread
def resolve(self, voice_spec: str) -> ResolvedVoice:
"""Resolve a voice spec into a loaded voice."""
from abogen.domain.voice_resolution import resolve_fallback_voice_spec
# Use thread's load_voice_cached method
loaded_voice = self._thread.load_voice_cached(voice_spec, self._thread.backend)
return ResolvedVoice(
provider="kokoro",
resolved_spec=voice_spec,
voice=loaded_voice,
speed=self._thread.speed,
supertonic_steps=getattr(self._thread, "supertonic_total_steps", 5),
)
def _level_to_color(level: str) -> str:
"""Map log level to PyQt color string."""
colors = {
"info": "grey",
"warning": "orange",
"error": "red",
"debug": "grey",
}
return colors.get(level, "grey")
+172 -264
View File
@@ -5,12 +5,8 @@ import tempfile
import platform
import base64
import re
import logging
from abogen.pyqt.queue_manager_gui import QueueManager
from abogen.pyqt.queued_item import QueuedItem
_log = logging.getLogger("abogen.gui")
import abogen.hf_tracker as hf_tracker
import hashlib # Added for cache path generation
from PyQt6.QtWidgets import (
@@ -73,31 +69,27 @@ from abogen.utils import (
LoadPipelineThread,
)
from abogen.subtitle_utils import clean_text
from abogen.domain.text_utils import calculate_text_length
from abogen.subtitle_utils import (
clean_text,
calculate_text_length,
)
from abogen.pyqt.conversion import ConversionThread, VoicePreviewThread, PlayAudioThread, ChapterOptionsDialog, TimestampDetectionDialog
from abogen.pyqt.book_handler import HandlerDialog
from abogen.domain.enums import Language
from abogen.constants import (
PROGRAM_NAME,
VERSION,
GITHUB_URL,
PROGRAM_DESCRIPTION,
LANGUAGE_DESCRIPTIONS,
VOICES_INTERNAL,
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
COLORS,
SUBTITLE_FORMATS,
)
from abogen.tts_plugin.utils import get_voices
import threading
from abogen.pyqt.voice_formula_gui import VoiceFormulaDialog
from abogen.voice_profiles import load_profiles, resolve_profile_language
from abogen.domain.settings_core import all_settings_defaults
from plugins.kokoro.engine import language_for_code, language_for_voice_id
# Module-level default cache for use outside __init__
_DEFAULTS = all_settings_defaults()
from abogen.voice_profiles import load_profiles
# Import ctypes for Windows-specific taskbar icon
if platform.system() == "Windows":
@@ -137,28 +129,6 @@ class ThreadSafeLogSignal(QObject):
self.log_signal.emit(message)
_UPDATE_CHECK_URL = "https://raw.githubusercontent.com/denizsafak/abogen/refs/heads/main/abogen/VERSION"
_UPDATE_CHECK_TIMEOUT = 8 # seconds; bounds offline/DNS hangs so the GUI never blocks
class _UpdateCheckThread(QThread):
"""Fetch the remote VERSION file off the GUI thread."""
succeeded = pyqtSignal(str)
failed = pyqtSignal(str)
def run(self):
import urllib.request
try:
with urllib.request.urlopen(
_UPDATE_CHECK_URL, timeout=_UPDATE_CHECK_TIMEOUT
) as response:
self.succeeded.emit(response.read().decode().strip())
except Exception as exc: # offline, DNS hang, HTTP error, ...
self.failed.emit(str(exc))
class IconProvider(QFileIconProvider):
def icon(self, fileInfo):
return super().icon(fileInfo)
@@ -424,7 +394,11 @@ class InputBox(QLabel):
# Re-enable subtitle and replace newlines controls when cleared
window = self.window()
if hasattr(window, "subtitle_combo"):
window.subtitle_combo.setEnabled(True)
# Only enable if language supports it
current_lang = getattr(window, "selected_lang", "a")
window.subtitle_combo.setEnabled(
current_lang in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION
)
if hasattr(window, "replace_newlines_combo"):
window.replace_newlines_combo.setEnabled(True)
@@ -863,7 +837,7 @@ class WordSubstitutionsDialog(QDialog):
self,
)
instructions.setStyleSheet(
f"padding: 10px; background-color: {COLORS['GREY_BACKGROUND']}; border-radius: 5px;"
"padding: 10px; background-color: #f0f0f0; border-radius: 5px;"
)
instructions.setWordWrap(True)
layout.addWidget(instructions)
@@ -937,10 +911,9 @@ class abogen(QWidget):
def __init__(self):
super().__init__()
self.config = load_config()
_d = all_settings_defaults()
self.apply_theme(self.config.get("theme", _d["theme"]))
self.apply_theme(self.config.get("theme", "system"))
migrate_subtitle_format(self.config)
self.check_updates = self.config.get("check_updates", _d["check_updates"])
self.check_updates = self.config.get("check_updates", True)
self.save_option = self.config.get("save_option", "Save next to input file")
self.selected_output_folder = self.config.get("selected_output_folder", None)
self.selected_file = self.selected_file_type = self.selected_book_path = None
@@ -948,7 +921,7 @@ class abogen(QWidget):
None # Add new variable to track the displayed file path
)
# Max log lines
self.log_window_max_lines = self.config.get("log_window_max_lines", _d["log_window_max_lines"])
self.log_window_max_lines = self.config.get("log_window_max_lines", 2000)
self.selected_chapters = set()
self.last_opened_book_path = None # Track the last opened book path
self.last_output_path = None
@@ -963,28 +936,40 @@ class abogen(QWidget):
self.selected_voice = None
self.selected_lang = None
else:
self.selected_voice = self.config.get("selected_voice", _d["selected_voice"])
self.selected_lang = language_for_voice_id(self.selected_voice)
self.selected_voice = self.config.get("selected_voice", "af_heart")
self.selected_lang = self.selected_voice[0] if self.selected_voice else None
self.is_converting = False
self.subtitle_mode = self.config.get("subtitle_mode", _d["subtitle_mode"])
self.max_subtitle_words = self.config.get("max_subtitle_words", _d["max_subtitle_words"])
self.silence_duration = self.config.get("silence_duration", _d.get("silence_between_chapters", 2.0))
self.selected_format = self.config.get("selected_format", _d["selected_format"])
self.separate_chapters_format = self.config.get("separate_chapters_format", _d["separate_chapters_format"])
self.use_gpu = self.config.get("use_gpu", _d["use_gpu"])
self.replace_single_newlines = self.config.get("replace_single_newlines", _d.get("replace_single_newlines", True))
self.use_silent_gaps = self.config.get("use_silent_gaps", _d["use_silent_gaps"])
self.subtitle_speed_method = self.config.get("subtitle_speed_method", _d["subtitle_speed_method"])
self.use_spacy_segmentation = self.config.get("use_spacy_segmentation", _d["use_spacy_segmentation"])
self.read_title_intro = self.config.get("read_title_intro", _d.get("read_title_intro", False))
self.read_closing_outro = self.config.get("read_closing_outro", _d.get("read_closing_outro", True))
self.subtitle_mode = self.config.get("subtitle_mode", "Sentence")
self.max_subtitle_words = self.config.get(
"max_subtitle_words", 50
) # Default max words per subtitle
self.silence_duration = self.config.get(
"silence_duration", 2.0
) # Default silence duration
self.selected_format = self.config.get("selected_format", "wav")
self.separate_chapters_format = self.config.get(
"separate_chapters_format", "wav"
) # Format for individual chapter files
self.use_gpu = self.config.get(
"use_gpu", True # Load GPU setting with default True
)
self.replace_single_newlines = self.config.get("replace_single_newlines", True)
self.use_silent_gaps = self.config.get("use_silent_gaps", True)
self.subtitle_speed_method = self.config.get("subtitle_speed_method", "tts")
self.use_spacy_segmentation = self.config.get("use_spacy_segmentation", True)
# Word substitution settings
self.word_substitutions_enabled = self.config.get("word_substitutions_enabled", _d["word_substitutions_enabled"])
self.word_substitutions_list = self.config.get("word_substitutions_list", _d["word_substitutions_list"])
self.case_sensitive_substitutions = self.config.get("case_sensitive_substitutions", _d["case_sensitive_substitutions"])
self.replace_all_caps = self.config.get("replace_all_caps", _d["replace_all_caps"])
self.replace_numerals = self.config.get("replace_numerals", _d["replace_numerals"])
self.fix_nonstandard_punctuation = self.config.get("fix_nonstandard_punctuation", _d["fix_nonstandard_punctuation"])
self.word_substitutions_enabled = self.config.get(
"word_substitutions_enabled", False
)
self.word_substitutions_list = self.config.get("word_substitutions_list", "")
self.case_sensitive_substitutions = self.config.get(
"case_sensitive_substitutions", False
)
self.replace_all_caps = self.config.get("replace_all_caps", False)
self.replace_numerals = self.config.get("replace_numerals", False)
self.fix_nonstandard_punctuation = self.config.get(
"fix_nonstandard_punctuation", False
)
self._pending_close_event = None
self.gpu_ok = False # Initialize GPU availability status
@@ -1011,13 +996,8 @@ class abogen(QWidget):
self.queued_items = []
self.current_queue_index = 0
from abogen.utils import timed_log
import logging
_startup_log = logging.getLogger("abogen.startup")
with timed_log("GUI initUI (widget building)", logger=_startup_log):
self.initUI()
self.speed_slider.setValue(int(self.config.get("speed", _d["speed"]) * 100))
self.initUI()
self.speed_slider.setValue(int(self.config.get("speed", 1.00) * 100))
self.update_speed_label()
# Set initial selection: prefer profile, else voice
idx = -1
@@ -1031,17 +1011,13 @@ class abogen(QWidget):
if self.selected_profile_name:
from abogen.voice_profiles import load_profiles
with timed_log("voice profile load", logger=_startup_log):
entry = load_profiles().get(self.selected_profile_name, {})
entry = load_profiles().get(self.selected_profile_name, {})
if isinstance(entry, dict):
self.mixed_voice_state = entry.get("voices", [])
self.selected_lang = resolve_profile_language(entry)
self.selected_lang = entry.get("language")
else:
self.mixed_voice_state = entry
self.selected_lang = (
language_for_voice_id(entry[0]) if entry and entry[0] else Language.EN_US
)
self.update_subtitle_options_availability()
self.selected_lang = entry[0][0] if entry and entry[0] else None
if self.save_option == "Choose output folder" and self.selected_output_folder:
self.save_path_label.setText(self.selected_output_folder)
self.save_path_row_widget.show()
@@ -1207,7 +1183,6 @@ class abogen(QWidget):
"Sentence + Comma: Subtitles will be generated for each sentence and comma.\n"
"Sentence + Highlighting: Subtitles with word-by-word karaoke highlighting.\n"
"1+ word: Subtitles will be generated for each word(s).\n\n"
"Word-count and highlighting modes are only available for English.\n"
"Supported languages for subtitle generation:\n"
+ "\n".join(
f'"{lang}" => {LANGUAGE_DESCRIPTIONS.get(lang, lang)}'
@@ -1786,9 +1761,8 @@ class abogen(QWidget):
def update_subtitle_options_availability(self):
"""
Update the enabled state of subtitle options based on the selected
language and input type. Subtitle generation works for every language,
but word-count and highlighting modes are only available for English.
Update the enabled state of subtitle options based on the selected language.
For non-English languages, only sentence-based and line-based modes are supported.
"""
# Check if current file is a subtitle file
is_subtitle_input = False
@@ -1797,14 +1771,16 @@ class abogen(QWidget):
):
is_subtitle_input = True
if self.selected_lang not in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION:
self.subtitle_combo.setEnabled(False)
self.subtitle_format_combo.setEnabled(False)
return
# Only enable subtitle_combo if it's NOT a subtitle input
self.subtitle_combo.setEnabled(not is_subtitle_input)
self.subtitle_format_combo.setEnabled(True)
is_english = self.selected_lang in (
Language.EN_US,
Language.EN_GB,
)
is_english = self.selected_lang in ["a", "b"]
# Items to keep enabled for non-English
allowed_modes = ["Disabled", "Line", "Sentence", "Sentence + Comma"]
@@ -1819,7 +1795,10 @@ class abogen(QWidget):
if is_english:
item.setEnabled(True)
else:
item.setEnabled(text in allowed_modes)
if text in allowed_modes:
item.setEnabled(True)
else:
item.setEnabled(False)
# If current selection is disabled, switch to a valid one
current_text = self.subtitle_combo.currentText()
@@ -1838,7 +1817,7 @@ class abogen(QWidget):
def on_voice_changed(self, index):
voice = self.voice_combo.itemData(index)
self.selected_voice, self.selected_lang = voice, language_for_voice_id(voice)
self.selected_voice, self.selected_lang = voice, voice[0]
self.config["selected_voice"] = voice
save_config(self.config)
# Enable/disable subtitle options based on language
@@ -1855,12 +1834,10 @@ class abogen(QWidget):
# set mixed voices and language
if isinstance(entry, dict):
self.mixed_voice_state = entry.get("voices", [])
self.selected_lang = resolve_profile_language(entry)
self.selected_lang = entry.get("language")
else:
self.mixed_voice_state = entry
self.selected_lang = (
language_for_voice_id(entry[0]) if entry and entry[0] else Language.EN_US
)
self.selected_lang = entry[0][0] if entry and entry[0] else None
self.selected_voice = None
self.config["selected_profile_name"] = pname
self.config.pop("selected_voice", None)
@@ -1870,7 +1847,7 @@ class abogen(QWidget):
else:
self.mixed_voice_state = None
self.selected_profile_name = None
self.selected_voice, self.selected_lang = data, language_for_voice_id(data)
self.selected_voice, self.selected_lang = data, data[0]
self.config["selected_voice"] = data
if "selected_profile_name" in self.config:
del self.config["selected_profile_name"]
@@ -1881,9 +1858,8 @@ class abogen(QWidget):
from abogen.voice_profiles import load_profiles
entry = load_profiles().get(profile_name, {})
enable = (
resolve_profile_language(entry) in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION
)
lang = entry.get("language") if isinstance(entry, dict) else None
enable = lang in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION
self.subtitle_combo.setEnabled(enable)
self.subtitle_format_combo.setEnabled(enable)
@@ -1897,7 +1873,7 @@ class abogen(QWidget):
for pname in load_profiles().keys():
self.voice_combo.addItem(profile_icon, pname, f"profile:{pname}")
# re-add voices
for v in get_voices("kokoro"):
for v in VOICES_INTERNAL:
icon = QIcon()
flag_path = get_resource_path("abogen.assets.flags", f"{v[0]}.png")
if flag_path and os.path.exists(flag_path):
@@ -2184,7 +2160,7 @@ class abogen(QWidget):
)
# CHECK GLOBAL OVERRIDE SETTING
if not self.config.get("queue_override_settings", _DEFAULTS["queue_override_settings"]):
if not self.config.get("queue_override_settings", False):
self.selected_lang = queued_item.lang_code
self.speed_slider.setValue(int(queued_item.speed * 100))
@@ -2258,25 +2234,26 @@ class abogen(QWidget):
self.current_queue_index = 0 # Reset for next time
def get_voice_formula(self) -> str:
from abogen.voice_formulas import pairs_to_formula
if self.mixed_voice_state:
return pairs_to_formula(self.mixed_voice_state) or ""
formula_components = [
f"{name}*{weight}" for name, weight in self.mixed_voice_state
]
return " + ".join(filter(None, formula_components))
else:
return self.selected_voice
def get_selected_lang(self, voice_formula) -> Language:
def get_selected_lang(self, voice_formula) -> str:
if self.selected_profile_name:
from abogen.voice_profiles import load_profiles
entry = load_profiles().get(self.selected_profile_name, {})
selected_lang = resolve_profile_language(entry)
selected_lang = entry.get("language")
else:
selected_lang = language_for_voice_id(self.selected_voice)
selected_lang = self.selected_voice[0] if self.selected_voice else None
# fallback: extract from formula if missing
if not selected_lang:
m = re.search(r"\b([a-z])", voice_formula)
selected_lang = language_for_code(m.group(1)) if m else Language.EN_US
selected_lang = m.group(1) if m else None
return selected_lang
def get_actual_subtitle_mode(self) -> str:
@@ -2339,9 +2316,9 @@ class abogen(QWidget):
file_size_str = "Unknown"
# pipeline_loaded_callback remains unchanged
def pipeline_loaded_callback(backend, error):
def pipeline_loaded_callback(np_module, kpipeline_class, error):
if error:
self.update_log((f"Error loading TTS backend: {error}", "red"))
self.update_log((f"Error loading numpy or KPipeline: {error}", "red"))
prevent_sleep_end()
return
@@ -2364,7 +2341,8 @@ class abogen(QWidget):
self.selected_output_folder,
subtitle_mode=actual_subtitle_mode,
output_format=self.selected_format,
backend=backend,
np_module=np_module,
kpipeline_class=kpipeline_class,
start_time=self.start_time,
total_char_count=self.char_count,
use_gpu=self.gpu_ok,
@@ -2425,9 +2403,6 @@ class abogen(QWidget):
self.conversion_thread.merge_chapters_at_end = getattr(
self, "merge_chapters_at_end", True
)
# Pass intro/outro settings
self.conversion_thread.read_title_intro = self.read_title_intro
self.conversion_thread.read_closing_outro = self.read_closing_outro
self.conversion_thread.progress_updated.connect(self.update_progress)
self.conversion_thread.log_updated.connect(self.update_log)
self.conversion_thread.conversion_finished.connect(
@@ -2451,11 +2426,7 @@ class abogen(QWidget):
self.gpu_ok = gpu_ok
self.update_log((gpu_msg, gpu_ok))
self.update_log("Loading modules...")
lang_code = self.selected_lang or Language.EN_US
load_thread = LoadPipelineThread(
pipeline_loaded_callback, lang_code=lang_code, use_gpu=gpu_ok
)
load_thread = LoadPipelineThread(pipeline_loaded_callback)
load_thread.start()
threading.Thread(target=gpu_and_load, daemon=True).start()
@@ -2466,7 +2437,7 @@ class abogen(QWidget):
return
# Check if override was active (this determines which settings were ACTUALLY used)
override_active = self.config.get("queue_override_settings", _DEFAULTS["queue_override_settings"])
override_active = self.config.get("queue_override_settings", False)
# If override is ON, capture the global settings that were used for processing
if override_active:
@@ -2783,12 +2754,12 @@ class abogen(QWidget):
from abogen.voice_profiles import load_profiles
entry = load_profiles().get(self.selected_profile_name, {})
lang_to_cache = resolve_profile_language(entry)
lang_to_cache = entry.get("language")
else:
lang_to_cache = self.selected_lang
if not lang_to_cache and self.mixed_voice_state:
lang_to_cache = (
language_for_voice_id(self.mixed_voice_state[0][0])
self.mixed_voice_state[0][0][0]
if self.mixed_voice_state and self.mixed_voice_state[0][0]
else None
)
@@ -2892,18 +2863,18 @@ class abogen(QWidget):
)
self.loading_movie.start()
lang = self.selected_lang or Language.EN_US
load_thread = LoadPipelineThread(
self._on_pipeline_loaded_for_preview, lang_code=lang, use_gpu=self.gpu_ok
)
def pipeline_loaded_callback(np_module, kpipeline_class, error):
self._on_pipeline_loaded_for_preview(np_module, kpipeline_class, error)
load_thread = LoadPipelineThread(pipeline_loaded_callback)
load_thread.start()
def _on_pipeline_loaded_for_preview(self, backend, error):
def _on_pipeline_loaded_for_preview(self, np_module, kpipeline_class, error):
# stop loading animation and restore icon on error
if error:
self.loading_movie.stop()
self._show_error_message_box(
"Loading Error", f"Error loading TTS backend: {error}"
"Loading Error", f"Error loading numpy or KPipeline: {error}"
)
self.btn_preview.setIcon(self.play_icon)
self.btn_preview.setEnabled(True)
@@ -2924,24 +2895,24 @@ class abogen(QWidget):
from abogen.voice_profiles import load_profiles
entry = load_profiles().get(self.selected_profile_name, {})
lang = resolve_profile_language(entry)
lang = entry.get("language")
else:
lang = self.selected_lang
if not lang and self.mixed_voice_state:
lang = (
language_for_voice_id(self.mixed_voice_state[0][0])
self.mixed_voice_state[0][0][0]
if self.mixed_voice_state and self.mixed_voice_state[0][0]
else None
)
else:
lang = language_for_voice_id(self.selected_voice)
lang = self.selected_voice[0]
voice = self.selected_voice
# use same gpu/cpu logic as in conversion
gpu_msg, gpu_ok = get_gpu_acceleration(self.use_gpu)
self.preview_thread = VoicePreviewThread(
backend, lang, voice, speed, gpu_ok
np_module, kpipeline_class, lang, voice, speed, gpu_ok
)
self.preview_thread.finished.connect(self._play_preview_audio)
self.preview_thread.error.connect(self._preview_error)
@@ -3195,25 +3166,14 @@ class abogen(QWidget):
save_config(self.config)
def cleanup_conversion_thread(self):
# Stop conversion thread (bounded wait so closing never hangs)
# Stop conversion thread
if (
hasattr(self, "conversion_thread")
and self.conversion_thread is not None
and self.conversion_thread.isRunning()
):
_log.info("Close: stopping conversion thread")
start = time.perf_counter()
self.conversion_thread.cancel()
if not self.conversion_thread.wait(2000):
_log.warning("Close: conversion thread did not stop in 2s, terminating")
self.conversion_thread.terminate()
self.conversion_thread.wait(1000)
_log.info(
"Close: conversion thread stopped in %.2fs",
time.perf_counter() - start,
)
else:
_log.info("Close: no running conversion thread")
self.conversion_thread.wait()
def cleanup_preview_threads(self):
# Stop preview generation thread
@@ -3222,13 +3182,8 @@ class abogen(QWidget):
and self.preview_thread is not None
and self.preview_thread.isRunning()
):
_log.info("Close: terminating preview thread")
start = time.perf_counter()
self.preview_thread.terminate()
self.preview_thread.wait(1000)
_log.info(
"Close: preview thread stopped in %.2fs", time.perf_counter() - start
)
self.preview_thread.wait()
# Stop audio playback thread
if (
@@ -3236,13 +3191,8 @@ class abogen(QWidget):
and self.play_audio_thread is not None
and self.play_audio_thread.isRunning()
):
_log.info("Close: stopping audio playback thread")
start = time.perf_counter()
self.play_audio_thread.stop()
self.play_audio_thread.wait(1000)
_log.info(
"Close: audio thread stopped in %.2fs", time.perf_counter() - start
)
self.play_audio_thread.wait()
# Cleanup pygame mixer if initialized
try:
@@ -3253,7 +3203,6 @@ class abogen(QWidget):
pass
def closeEvent(self, event):
_log.info("Close: window close requested (converting=%s)", self.is_converting)
if self.is_converting:
box = QMessageBox(self)
box.setIcon(QMessageBox.Icon.Warning)
@@ -3266,12 +3215,10 @@ class abogen(QWidget):
)
box.setDefaultButton(QMessageBox.StandardButton.No)
if box.exec() == QMessageBox.StandardButton.Yes:
_log.info("Close: user confirmed exit during conversion")
self.cleanup_conversion_thread()
self.cleanup_preview_threads()
event.accept()
else:
_log.info("Close: user cancelled exit")
event.ignore()
else:
self.cleanup_conversion_thread()
@@ -3462,7 +3409,7 @@ class abogen(QWidget):
app.installEventFilter(app._dark_titlebar_event_filter)
# Save config if changed
if self.config.get("theme", _DEFAULTS["theme"]) != theme:
if self.config.get("theme", "system") != theme:
self.config["theme"] = theme
save_config(self.config)
@@ -3484,7 +3431,7 @@ class abogen(QWidget):
]
# Get current theme from config, default to "system"
current_theme = self.config.get("theme", _DEFAULTS["theme"])
current_theme = self.config.get("theme", "system")
for value, text in theme_options:
theme_action = QAction(text, self)
theme_action.setCheckable(True)
@@ -3615,27 +3562,6 @@ class abogen(QWidget):
# Add separator
menu.addSeparator()
# Add title intro option
self.title_intro_action = QAction("Read title intro before first chapter", self)
self.title_intro_action.setCheckable(True)
self.title_intro_action.setChecked(self.read_title_intro)
self.title_intro_action.triggered.connect(
lambda checked: self.toggle_read_title_intro(checked)
)
menu.addAction(self.title_intro_action)
# Add closing outro option
self.closing_outro_action = QAction("Read closing outro after last chapter", self)
self.closing_outro_action.setCheckable(True)
self.closing_outro_action.setChecked(self.read_closing_outro)
self.closing_outro_action.triggered.connect(
lambda checked: self.toggle_read_closing_outro(checked)
)
menu.addAction(self.closing_outro_action)
# Add separator
menu.addSeparator()
# Add "Pre-download models and voices for offline use" option
predownload_action = QAction(
"Pre-download models and voices for offline use", self
@@ -3647,7 +3573,7 @@ class abogen(QWidget):
disable_kokoro_action = QAction("Disable Kokoro's internet access", self)
disable_kokoro_action.setCheckable(True)
disable_kokoro_action.setChecked(
self.config.get("disable_kokoro_internet", _DEFAULTS["disable_kokoro_internet"])
self.config.get("disable_kokoro_internet", False)
)
disable_kokoro_action.triggered.connect(
lambda checked: self.toggle_kokoro_internet_access(checked)
@@ -3657,7 +3583,7 @@ class abogen(QWidget):
# Add check for updates option
check_updates_action = QAction("Check for updates at startup", self)
check_updates_action.setCheckable(True)
check_updates_action.setChecked(self.config.get("check_updates", _DEFAULTS["check_updates"]))
check_updates_action.setChecked(self.config.get("check_updates", True))
check_updates_action.triggered.connect(self.toggle_check_updates)
menu.addAction(check_updates_action)
@@ -3712,16 +3638,6 @@ class abogen(QWidget):
self.config["use_spacy_segmentation"] = enabled
save_config(self.config)
def toggle_read_title_intro(self, enabled):
self.read_title_intro = enabled
self.config["read_title_intro"] = enabled
save_config(self.config)
def toggle_read_closing_outro(self, enabled):
self.read_closing_outro = enabled
self.config["read_closing_outro"] = enabled
save_config(self.config)
def restart_app(self):
import sys
@@ -4000,9 +3916,7 @@ Categories=AudioVideo;Audio;Utility;
initial_state = entry.get("voices", [])
else:
initial_state = entry
self.selected_lang = (
language_for_voice_id(entry[0]) if entry and entry[0] else Language.EN_US
)
self.selected_lang = entry[0][0] if entry and entry[0] else None
dialog = VoiceFormulaDialog(
self, initial_state=initial_state, selected_profile=selected_profile
)
@@ -4099,85 +4013,75 @@ Categories=AudioVideo;Audio;Utility;
self.check_for_updates_startup()
def check_for_updates_startup(self):
# Network I/O runs in a worker thread: urlopen without a timeout on
# the GUI thread froze the whole app when offline (DNS/connect can
# hang for minutes). Results return via signals on the GUI thread.
thread = getattr(self, "_update_check_thread", None)
if thread is not None:
try:
if thread.isRunning():
return
except RuntimeError:
pass # previous thread already finished/deleted
import urllib.request
def show_update_message(remote_version, local_version):
msg_box = QMessageBox(self)
msg_box.setIcon(QMessageBox.Icon.Information)
msg_box.setWindowTitle("Update Available")
msg_box.setText(
f"A new version of {PROGRAM_NAME} is available! ({local_version} > {remote_version})"
)
msg_box.setInformativeText(
f"If you installed via pip, update by running:\n"
f"pip install --upgrade {PROGRAM_NAME}\n\n"
f"If you're using the Windows portable version, run 'WINDOWS_INSTALL.bat' again.\n\n"
"Alternatively, visit the GitHub repository for more information. "
"Would you like to view the changelog?"
)
msg_box.setStandardButtons(
QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No
)
msg_box.setDefaultButton(QMessageBox.StandardButton.Yes)
if msg_box.exec() == QMessageBox.StandardButton.Yes:
try:
QDesktopServices.openUrl(QUrl(GITHUB_URL + "/releases/latest"))
except Exception:
pass
# Reset flag to track if we should show "no updates" message
show_result = (
hasattr(self, "_show_update_check_result")
and self._show_update_check_result
)
self._show_update_check_result = False
self._update_check_thread = _UpdateCheckThread(self)
self._update_check_thread.succeeded.connect(
lambda remote_raw: self._on_update_check_done(remote_raw, show_result)
)
self._update_check_thread.failed.connect(
lambda err: self._on_update_check_failed(err, show_result)
)
self._update_check_thread.finished.connect(
self._update_check_thread.deleteLater
)
self._update_check_thread.start()
def _on_update_check_done(self, remote_raw, show_result):
remote_version = remote_raw.strip()
local_version = VERSION
try:
remote_num = int("".join(remote_version.split(".")))
local_num = int("".join(local_version.split(".")))
except ValueError:
return
if remote_num > local_num:
# Use QTimer to ensure UI is ready, then show update message.
QTimer.singleShot(
1000,
lambda: self._show_update_message(remote_version, local_version),
)
elif show_result:
QMessageBox.information(
self,
"Up to Date",
f"You are running the latest version of {PROGRAM_NAME} ({local_version}).",
)
update_url = "https://raw.githubusercontent.com/denizsafak/abogen/refs/heads/main/abogen/VERSION"
with urllib.request.urlopen(update_url) as response:
remote_raw = response.read().decode().strip()
local_raw = VERSION
def _on_update_check_failed(self, err, show_result):
if show_result:
QMessageBox.warning(
self,
"Update Check Failed",
f"Could not check for updates:\n{err}",
)
# Parse version numbers
remote_version = remote_raw
local_version = local_raw
def _show_update_message(self, remote_version, local_version):
msg_box = QMessageBox(self)
msg_box.setIcon(QMessageBox.Icon.Information)
msg_box.setWindowTitle("Update Available")
msg_box.setText(
f"A new version of {PROGRAM_NAME} is available! ({local_version} > {remote_version})"
)
msg_box.setInformativeText(
f"If you installed via pip, update by running:\n"
f"pip install --upgrade {PROGRAM_NAME}\n\n"
f"If you're using the Windows portable version, run 'WINDOWS_INSTALL.bat' again.\n\n"
"Alternatively, visit the GitHub repository for more information. "
"Would you like to view the changelog?"
)
msg_box.setStandardButtons(
QMessageBox.StandardButton.Yes | QMessageBox.StandardButton.No
)
msg_box.setDefaultButton(QMessageBox.StandardButton.Yes)
if msg_box.exec() == QMessageBox.StandardButton.Yes:
try:
QDesktopServices.openUrl(QUrl(GITHUB_URL + "/releases/latest"))
except Exception:
pass
remote_num = int("".join(remote_version.split(".")))
local_num = int("".join(local_version.split(".")))
except ValueError as ve:
return
if remote_num > local_num:
# Use QTimer to ensure UI is ready, then show update message.
QTimer.singleShot(
1000, lambda: show_update_message(remote_version, local_version)
)
elif show_result:
# Show "no updates" message if manually checking
QMessageBox.information(
self,
"Up to Date",
f"You are running the latest version of {PROGRAM_NAME} ({local_version}).",
)
except Exception as e:
if show_result:
QMessageBox.warning(
self,
"Update Check Failed",
f"Could not check for updates:\n{str(e)}",
)
pass
def clear_cache_files(self):
"""Clear cache files created by the program."""
@@ -4280,6 +4184,8 @@ Categories=AudioVideo;Audio;Utility;
def set_max_log_lines(self):
"""Open a dialog to set the maximum lines in the log window."""
from PyQt6.QtWidgets import QInputDialog
value, ok = QInputDialog.getInt(
self,
"Max Lines in Log Window",
@@ -4301,7 +4207,9 @@ Categories=AudioVideo;Audio;Utility;
def set_max_subtitle_words(self):
"""Open a dialog to set the maximum words per subtitle"""
current_value = self.config.get("max_subtitle_words", _DEFAULTS["max_subtitle_words"])
from PyQt6.QtWidgets import QInputDialog
current_value = self.config.get("max_subtitle_words", 50)
value, ok = QInputDialog.getInt(
self,
@@ -4329,7 +4237,7 @@ Categories=AudioVideo;Audio;Utility;
def set_silence_between_chapters(self):
"""Open a dialog to set the silence duration between chapters"""
current_value = self.config.get("silence_duration", _DEFAULTS.get("silence_between_chapters", 2.0))
current_value = self.config.get("silence_duration", 2.0)
dlg = QInputDialog(self)
dlg.setWindowTitle("Silence Duration (seconds)")
+95 -103
View File
@@ -1,124 +1,129 @@
import logging
import os
import sys
import platform
import atexit
import signal
from abogen.utils import get_resource_path, load_config, prevent_sleep_end
# Initialise global shutdown handling (atexit, signals, Qt) as early as possible.
from abogen import shutdown # noqa: F401
shutdown.register_shutdown()
from abogen.utils import get_resource_path, setup_console_logging, timed_log # noqa: E402
_log = logging.getLogger("abogen.startup")
setup_console_logging()
# Fix PyTorch DLL loading issue ([WinError 1114]) on Windows before importing PyQt6
if platform.system() == "Windows":
with timed_log("PyTorch DLLs (Windows)", logger=_log):
import ctypes
from importlib.util import find_spec
import ctypes
from importlib.util import find_spec
try:
if (
(spec := find_spec("torch"))
and spec.origin
and os.path.exists(
dll_path := os.path.join(os.path.dirname(spec.origin), "lib", "c10.dll")
)
):
ctypes.CDLL(os.path.normpath(dll_path))
except Exception:
pass
try:
if (
(spec := find_spec("torch"))
and spec.origin
and os.path.exists(
dll_path := os.path.join(os.path.dirname(spec.origin), "lib", "c10.dll")
)
):
ctypes.CDLL(os.path.normpath(dll_path))
except Exception:
pass
# Qt platform plugin detection (fixes #59)
with timed_log("Qt platform plugin detection", logger=_log):
try:
from PyQt6.QtCore import QLibraryInfo
try:
from PyQt6.QtCore import QLibraryInfo
# Get the path to the plugins directory
plugins = QLibraryInfo.path(QLibraryInfo.LibraryPath.PluginsPath)
# Get the path to the plugins directory
plugins = QLibraryInfo.path(QLibraryInfo.LibraryPath.PluginsPath)
# Normalize path to use the OS-native separators and absolute path
platform_dir = os.path.normpath(os.path.join(plugins, "platforms"))
# Normalize path to use the OS-native separators and absolute path
platform_dir = os.path.normpath(os.path.join(plugins, "platforms"))
# Ensure we work with an absolute path for clarity
platform_dir = os.path.abspath(platform_dir)
# Ensure we work with an absolute path for clarity
platform_dir = os.path.abspath(platform_dir)
if os.path.isdir(platform_dir):
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = platform_dir
_log.info("QT_QPA_PLATFORM_PLUGIN_PATH set to: %s", platform_dir)
else:
_log.warning("PyQt6 platform plugins not found at %s", platform_dir)
except ImportError:
_log.warning("PyQt6 not installed.")
if os.path.isdir(platform_dir):
os.environ["QT_QPA_PLATFORM_PLUGIN_PATH"] = platform_dir
print("QT_QPA_PLATFORM_PLUGIN_PATH set to:", platform_dir)
else:
print("PyQt6 platform plugins not found at", platform_dir)
except ImportError:
print("PyQt6 not installed.")
# Pre-load "libxcb-cursor" on Linux (fixes #101)
if platform.system() == "Linux":
with timed_log("libxcb-cursor preload (Linux)", logger=_log):
arch = platform.machine().lower()
lib_filename = {"x86_64": "libxcb-cursor-amd64.so.0", "amd64": "libxcb-cursor-amd64.so.0", "aarch64": "libxcb-cursor-arm64.so.0", "arm64": "libxcb-cursor-arm64.so.0"}.get(arch)
if lib_filename:
import ctypes
try:
# Try to load the system libxcb-cursor.so.0 first
ctypes.CDLL('libxcb-cursor.so.0', mode=ctypes.RTLD_GLOBAL)
except OSError:
# System lib not available, load the bundled version
lib_path = get_resource_path('abogen.libs', lib_filename)
if lib_path:
try:
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
except OSError:
# If it fails (e.g. wrong glibc version on very old systems),
# we simply ignore it and hope the system has the library.
pass
arch = platform.machine().lower()
lib_filename = {"x86_64": "libxcb-cursor-amd64.so.0", "amd64": "libxcb-cursor-amd64.so.0", "aarch64": "libxcb-cursor-arm64.so.0", "arm64": "libxcb-cursor-arm64.so.0"}.get(arch)
if lib_filename:
import ctypes
try:
# Try to load the system libxcb-cursor.so.0 first
ctypes.CDLL('libxcb-cursor.so.0', mode=ctypes.RTLD_GLOBAL)
except OSError:
# System lib not available, load the bundled version
lib_path = get_resource_path('abogen.libs', lib_filename)
if lib_path:
try:
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
except OSError:
# If it fails (e.g. wrong glibc version on very old systems),
# we simply ignore it and hope the system has the library.
pass
# Set application ID for Windows taskbar icon
if platform.system() == "Windows":
with timed_log("Windows AppUserModelID", logger=_log):
try:
from abogen.constants import PROGRAM_NAME, VERSION
import ctypes
try:
from abogen.constants import PROGRAM_NAME, VERSION
import ctypes
app_id = f"{PROGRAM_NAME}.{VERSION}"
ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID(app_id)
except Exception as e:
_log.warning("Failed to set AppUserModelID: %s", e)
app_id = f"{PROGRAM_NAME}.{VERSION}"
ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID(app_id)
except Exception as e:
print("Warning: failed to set AppUserModelID:", e)
with timed_log("PyQt6 imports", logger=_log):
from PyQt6.QtWidgets import QApplication
from PyQt6.QtGui import QIcon
from PyQt6.QtCore import (
QLibraryInfo,
qInstallMessageHandler,
QtMsgType,
)
from PyQt6.QtWidgets import QApplication
from PyQt6.QtGui import QIcon
from PyQt6.QtCore import (
QLibraryInfo,
qInstallMessageHandler,
QtMsgType,
)
# Add the directory to Python path
sys.path.insert(0, os.path.join(os.path.dirname(__file__)))
# Set Hugging Face Hub environment variables
with timed_log("config load + HF env setup", logger=_log):
os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1" # Disable Hugging Face telemetry
os.environ["HF_HUB_ETAG_TIMEOUT"] = "10" # Metadata request timeout (seconds)
os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = "10" # File download timeout (seconds)
os.environ["HF_HUB_DISABLE_SYMLINKS_WARNING"] = "1" # Disable symlinks warning
from abogen.utils import load_config
if load_config().get("disable_kokoro_internet", False):
_log.info("Kokoro's internet access is disabled.")
os.environ["HF_HUB_OFFLINE"] = "1" # Disable Hugging Face Hub internet access
os.environ["HF_HUB_DISABLE_TELEMETRY"] = "1" # Disable Hugging Face telemetry
os.environ["HF_HUB_ETAG_TIMEOUT"] = "10" # Metadata request timeout (seconds)
os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = "10" # File download timeout (seconds)
os.environ["HF_HUB_DISABLE_SYMLINKS_WARNING"] = "1" # Disable symlinks warning
if load_config().get("disable_kokoro_internet", False):
print("INFO: Kokoro's internet access is disabled.")
os.environ["HF_HUB_OFFLINE"] = "1" # Disable Hugging Face Hub internet access
with timed_log("GUI module import (abogen.pyqt.gui)", logger=_log):
from abogen.pyqt.gui import abogen
from abogen.constants import PROGRAM_NAME, VERSION
from abogen.pyqt.gui import abogen
from abogen.constants import PROGRAM_NAME, VERSION
# Set environment variables for AMD ROCm
os.environ["MIOPEN_FIND_MODE"] = "FAST"
os.environ["MIOPEN_CONV_PRECISE_ROCM_TUNING"] = "0"
# Reset sleep states
atexit.register(prevent_sleep_end)
# Also handle signals (Ctrl+C, kill, etc.)
def _cleanup_sleep(signum, frame):
prevent_sleep_end()
sys.exit(0)
signal.signal(signal.SIGINT, _cleanup_sleep)
signal.signal(signal.SIGTERM, _cleanup_sleep)
# Ensure sys.stdout and sys.stderr are valid in GUI mode
if sys.stdout is None:
sys.stdout = open(os.devnull, "w")
if sys.stderr is None:
sys.stderr = open(os.devnull, "w")
# Enable MPS GPU acceleration on Mac Apple Silicon
if platform.system() == "Darwin" and platform.processor() == "arm":
os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"
@@ -131,8 +136,6 @@ def qt_message_handler(mode, context, message):
return # Suppress this specific message
if "setGrabPopup called with a parent, QtWaylandClient" in message:
return
if "Failed to register with host portal" in message:
return
if mode == QtMsgType.QtWarningMsg:
print(f"Qt Warning: {message}")
@@ -161,11 +164,7 @@ if platform.system() == "Linux":
def main():
"""Main entry point for console usage."""
with timed_log("QApplication creation", logger=_log):
app = QApplication(sys.argv)
# Qt shutdown hook must be connected AFTER QApplication exists
shutdown.install_qt_hook()
app = QApplication(sys.argv)
# Set application icon using get_resource_path from utils
icon_path = get_resource_path("abogen.assets", "icon.ico")
@@ -179,17 +178,10 @@ def main():
except AttributeError:
pass
with timed_log("main window construction", logger=_log):
ex = abogen()
with timed_log("window show", logger=_log):
ex.show()
_log.info("App startup complete. Showing window.")
rc = app.exec()
# Restore the default Qt message handler BEFORE interpreter shutdown.
# A Python message handler invoked during Qt teardown segfaults (SIGSEGV).
qInstallMessageHandler(None)
sys.exit(rc)
ex = abogen()
ex.show()
sys.exit(app.exec())
if __name__ == "__main__":
main()
main()
+4 -5
View File
@@ -21,8 +21,7 @@ from PyQt6.QtWidgets import (
)
from PyQt6.QtCore import QThread, pyqtSignal
from abogen.constants import COLORS
from abogen.tts_plugin.utils import get_voices
from abogen.constants import COLORS, VOICES_INTERNAL
from abogen.spacy_utils import SPACY_MODELS
import abogen.hf_tracker
@@ -115,7 +114,7 @@ class PreDownloadWorker(QThread):
self._voices_success = False
return
voice_list = get_voices("kokoro")
voice_list = VOICES_INTERNAL
for idx, voice in enumerate(voice_list, start=1):
if self._cancelled:
self._voices_success = False
@@ -463,14 +462,14 @@ class PreDownloadDialog(QDialog):
try:
from huggingface_hub import try_to_load_from_cache
for voice in get_voices("kokoro"):
for voice in VOICES_INTERNAL:
if not try_to_load_from_cache(
repo_id="hexgrad/Kokoro-82M", filename=f"voices/{voice}.pt"
):
missing.append(voice)
except Exception:
# If HF missing, report all as missing
return False, list(get_voices("kokoro"))
return False, list(VOICES_INTERNAL)
return (len(missing) == 0), missing
def _check_kokoro_model(self) -> bool:
+1 -1
View File
@@ -523,7 +523,7 @@ class QueueManager(QDialog):
return attrs
def add_files_from_paths(self, file_paths):
from abogen.domain.text_utils import calculate_text_length
from abogen.subtitle_utils import calculate_text_length
from PyQt6.QtWidgets import QMessageBox
import os
+8 -10
View File
@@ -28,10 +28,11 @@ from PyQt6.QtWidgets import (
from PyQt6.QtCore import Qt, QTimer, QPoint, QRect, QSize
from PyQt6.QtGui import QPixmap, QIcon, QAction
from abogen.constants import (
VOICES_INTERNAL,
SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION,
LANGUAGE_DESCRIPTIONS,
COLORS,
)
from abogen.tts_plugin.utils import get_voices
import re
import platform
from abogen.utils import get_resource_path
@@ -178,7 +179,7 @@ class VoiceMixer(QWidget):
layout.addWidget(QLabel(name), alignment=Qt.AlignmentFlag.AlignCenter)
# Voice name label with gender icon
is_female = self.voice_name in get_voices("kokoro") and self.voice_name[1] == "f"
is_female = self.voice_name in VOICES_INTERNAL and self.voice_name[1] == "f"
# Icons layout (flag and gender)
icons_layout = QHBoxLayout()
@@ -771,7 +772,7 @@ class VoiceFormulaDialog(QDialog):
def add_voices(self, initial_state):
first_enabled_voice = None
for voice in get_voices("kokoro"):
for voice in VOICES_INTERNAL:
language_code = voice[0] # First character is the language code
matching_voice = next(
(item for item in initial_state if item[0] == voice), None
@@ -948,9 +949,7 @@ class VoiceFormulaDialog(QDialog):
lang = state.get("language") if isinstance(state, dict) else None
# apply language selection
if lang:
from abogen.voice_profiles import resolve_profile_language
i = self.language_combo.findData(resolve_profile_language(state))
i = self.language_combo.findData(lang)
if i >= 0:
self.language_combo.blockSignals(True)
self.language_combo.setCurrentIndex(i)
@@ -1572,10 +1571,9 @@ class VoiceFormulaDialog(QDialog):
parent.selected_profile_name = None
lang = self.language_combo.currentData()
parent.selected_lang = lang
if hasattr(parent, "update_subtitle_options_availability"):
parent.update_subtitle_options_availability()
else:
parent.subtitle_combo.setEnabled(True)
parent.subtitle_combo.setEnabled(
lang in SUPPORTED_LANGUAGES_FOR_SUBTITLE_GENERATION
)
# Reset start flag and trigger preview
self._started = False
parent.preview_voice()
-162
View File
@@ -1,162 +0,0 @@
"""Graceful shutdown — process-level hooks and orchestration.
Responsibilities:
- Install atexit/signal/Qt hooks
- Stop WebUI ConversionService (worker thread)
- Restore sleep prevention
- Terminate child processes (ffmpeg, etc.)
- Delegate GPU/engine/UI cleanup to application.cleanup
App-layer cleanup (GPU, engines, UI callbacks) lives in application/cleanup.py.
Per-conversion cleanup lives in run_conversion() finally block.
"""
from __future__ import annotations
import atexit
import logging
import signal
import sys
import time
from typing import Callable
_log = logging.getLogger("abogen.shutdown")
_CLEANUP_FUNCS: list[Callable[[], None]] = []
_EXECUTED = False
def register_cleanup(fn: Callable[[], None]) -> None:
"""Register a cleanup function to run on shutdown."""
_CLEANUP_FUNCS.append(fn)
def _run_cleanups() -> None:
global _EXECUTED
if _EXECUTED:
return
_EXECUTED = True
_log.info("Shutdown: starting %d cleanup hook(s)", len(_CLEANUP_FUNCS))
for fn in _CLEANUP_FUNCS:
start = time.perf_counter()
try:
fn()
except Exception:
pass
_log.info(
"Shutdown: %s done in %.2fs", fn.__name__, time.perf_counter() - start
)
_log.info("Shutdown: all cleanups finished")
# ---- Process-level cleanup functions ----
def _stop_conversion_service() -> None:
"""Stop WebUI ConversionService worker thread."""
try:
from abogen.webui.service import get_service
svc = get_service()
if svc is not None:
svc.shutdown()
except Exception:
pass
def _restore_sleep() -> None:
"""Restore system sleep prevention (caffeinate/systemd-inhibit/Windows)."""
try:
from abogen.utils import prevent_sleep_end
prevent_sleep_end()
except Exception:
pass
def _terminate_subprocesses() -> None:
"""Terminate all child processes (ffmpeg, etc.)."""
try:
import psutil
except Exception:
return
try:
current = psutil.Process()
for child in current.children(recursive=True):
try:
child.terminate()
except Exception:
pass
gone, alive = psutil.wait_procs(current.children(recursive=True), timeout=3)
for proc in alive:
try:
proc.kill()
except Exception:
pass
except Exception:
pass
def _app_cleanup() -> None:
"""Delegate to application-layer cleanup (engines, GPU, UI callbacks)."""
try:
from abogen.application.cleanup import cleanup
cleanup()
except Exception:
pass
# Register in execution order
register_cleanup(_stop_conversion_service)
register_cleanup(_app_cleanup)
register_cleanup(_restore_sleep)
register_cleanup(_terminate_subprocesses)
def register_shutdown() -> None:
"""Install process-wide shutdown hooks (atexit, signals, Qt)."""
if register_shutdown._registered:
return
register_shutdown._registered = True
atexit.register(_run_cleanups)
# POSIX signals
for sig in (signal.SIGINT, signal.SIGTERM):
try:
signal.signal(sig, _on_signal)
except Exception:
pass
install_qt_hook()
def install_qt_hook() -> None:
"""Connect Qt aboutToQuit to cleanup. Must run AFTER QApplication is created."""
try:
from PyQt6.QtWidgets import QApplication
app = QApplication.instance()
if app is not None and not getattr(app, "_abogen_cleanup_connected", False):
app.aboutToQuit.connect(_run_cleanups)
app._abogen_cleanup_connected = True
_log.info("Shutdown: Qt aboutToQuit hook connected")
except Exception:
pass
register_shutdown._registered = False
def _on_signal(signum: int, _frame) -> None:
_log.info("Shutdown: signal %s received", signum)
_run_cleanups()
sys.exit(0)
def request_shutdown() -> None:
"""Programmatically trigger cleanup (e.g., from GUI closeEvent)."""
_log.info("Shutdown: cleanup requested")
_run_cleanups()
__all__ = ["register_shutdown", "install_qt_hook", "request_shutdown", "register_cleanup"]
+5 -6
View File
@@ -6,9 +6,10 @@ from dataclasses import dataclass
from functools import lru_cache
from typing import Any, Dict, Optional, Tuple
# spaCy is intentionally NOT imported at module level: importing it pulls in
# thinc -> torch, which costs seconds of startup time. It is imported lazily
# inside _load_spacy_model below.
try: # pragma: no cover - optional dependency
import spacy
except Exception: # pragma: no cover - spaCy unavailable at runtime
spacy = None
# Lazy spaCy type hints to avoid a hard dependency at import time.
Language = Any # type: ignore[assignment]
@@ -36,9 +37,7 @@ _DEFAULT_MODEL = os.environ.get("ABOGEN_SPACY_MODEL", "en_core_web_sm")
@lru_cache(maxsize=1)
def _load_spacy_model(model: str = _DEFAULT_MODEL) -> Optional[Language]:
try: # pragma: no cover - optional dependency
import spacy
except Exception: # pragma: no cover - spaCy unavailable at runtime
if spacy is None:
logger.debug("spaCy is not installed; skipping contraction disambiguation")
return None
+27 -28
View File
@@ -2,25 +2,24 @@
Lazy-loaded spaCy utilities for sentence segmentation.
"""
from abogen.domain.enums import Language
# Cached spaCy module and models (lazy loaded)
_spacy = None
_nlp_cache = {}
# Language code to spaCy model mapping
SPACY_MODELS = {
Language.EN_US: "en_core_web_sm",
Language.EN_GB: "en_core_web_sm",
Language.ES: "es_core_news_sm",
Language.FR: "fr_core_news_sm",
Language.IT: "it_core_news_sm",
Language.PT_BR: "pt_core_news_sm",
Language.ZH: "zh_core_web_sm",
Language.JA: "ja_core_news_sm",
Language.HI: "xx_sent_ud_sm",
"a": "en_core_web_sm", # American English
"b": "en_core_web_sm", # British English
"e": "es_core_news_sm", # Spanish
"f": "fr_core_news_sm", # French
"i": "it_core_news_sm", # Italian
"p": "pt_core_news_sm", # Brazilian Portuguese
"z": "zh_core_web_sm", # Mandarin Chinese
"j": "ja_core_news_sm", # Japanese
"h": "xx_sent_ud_sm", # Hindi (multi-language model)
}
def _load_spacy():
"""Lazy load spaCy module."""
global _spacy
@@ -34,12 +33,13 @@ def _load_spacy():
return _spacy
def get_spacy_model(language: Language, log_callback=None):
def get_spacy_model(lang_code, log_callback=None):
"""
Get or load a spaCy model for the given language.
Get or load a spaCy model for the given language code.
Downloads the model automatically if not available.
Args:
language: Language enum value.
lang_code: Language code (a, b, e, f, etc.)
log_callback: Optional function to log messages
Returns:
@@ -47,26 +47,25 @@ def get_spacy_model(language: Language, log_callback=None):
"""
def log(msg, is_error=False):
# Prefer GUI log callback when provided to avoid spamming stdout.
if log_callback:
color = "red" if is_error else "grey"
try:
log_callback((msg, color))
except Exception:
# Fallback to printing if callback misbehaves
print(msg)
else:
print(msg)
if not isinstance(language, Language):
raise TypeError(
f"language must be Language enum, got {type(language).__name__}: {language!r}"
)
# Check if model is cached
if lang_code in _nlp_cache:
return _nlp_cache[lang_code]
if language in _nlp_cache:
return _nlp_cache[language]
model_name = SPACY_MODELS.get(language)
# Check if language is supported
model_name = SPACY_MODELS.get(lang_code)
if not model_name:
log(f"\nspaCy: No model mapping for language '{language}'...")
log(f"\nspaCy: No model mapping for language '{lang_code}'...")
return None
# Lazy load spaCy
@@ -90,7 +89,7 @@ def get_spacy_model(language: Language, log_callback=None):
if "parser" not in nlp.pipe_names and "sentencizer" not in nlp.pipe_names:
nlp.add_pipe("sentencizer")
_nlp_cache[language] = nlp
_nlp_cache[lang_code] = nlp
return nlp
except OSError:
# Model not found, attempt download
@@ -107,7 +106,7 @@ def get_spacy_model(language: Language, log_callback=None):
if "parser" not in nlp.pipe_names and "sentencizer" not in nlp.pipe_names:
nlp.add_pipe("sentencizer")
_nlp_cache[language] = nlp
_nlp_cache[lang_code] = nlp
log(f"spaCy model '{model_name}' downloaded and loaded")
return nlp
except Exception as e:
@@ -121,19 +120,19 @@ def get_spacy_model(language: Language, log_callback=None):
return None
def segment_sentences(text, language: Language, log_callback=None):
def segment_sentences(text, lang_code, log_callback=None):
"""
Segment text into sentences using spaCy.
Args:
text: Text to segment
language: Language enum value
lang_code: Language code
log_callback: Optional function to log messages
Returns:
List of sentence strings, or None if spaCy unavailable
"""
nlp = get_spacy_model(language, log_callback)
nlp = get_spacy_model(lang_code, log_callback)
if nlp is None:
return None
+2 -2
View File
@@ -4,7 +4,7 @@ import json
import os
from typing import Any, Dict, List, Optional
from abogen.constants import KOKORO_CODE_LABELS
from abogen.constants import LANGUAGE_DESCRIPTIONS
from abogen.utils import get_user_config_path
_CONFIG_WRAPPER_KEY = "abogen_speaker_configs"
@@ -163,4 +163,4 @@ def list_configs() -> List[Dict[str, Any]]:
def describe_language(code: str) -> str:
code = (code or "a").lower()
return KOKORO_CODE_LABELS.get(code, code.upper())
return LANGUAGE_DESCRIPTIONS.get(code, code.upper())
+199 -19
View File
@@ -1,7 +1,7 @@
import re
import platform
from abogen.utils import detect_encoding, load_config
from abogen.constants import SAMPLE_VOICE_TEXTS
from abogen.domain.enums import Language
# Pre-compile frequently used regex patterns for better performance
_METADATA_TAG_PATTERN = re.compile(r"<<METADATA_[^:]+:[^>]*>>")
@@ -23,6 +23,13 @@ _VTT_NOTE_PATTERN = re.compile(r"NOTE\s*\n.*?(?=\n\n|$)", re.DOTALL)
_DOUBLE_NEWLINE_SPLIT_PATTERN = re.compile(r"\n\s*\n")
_VTT_TIMESTAMP_PATTERN = re.compile(r"([\d:.]+)\s*-->\s*([\d:.]+)")
_TIMESTAMP_ONLY_PATTERN = re.compile(r"^(\d{1,2}:\d{2}:\d{2}(?:[.,]\d{1,3})?)$")
_WINDOWS_ILLEGAL_CHARS_PATTERN = re.compile(r'[<>:"/\\|?*]')
_CONTROL_CHARS_PATTERN = re.compile(r"[\x00-\x1f]")
_LINUX_CONTROL_CHARS_PATTERN = re.compile(
r"[\x01-\x1f]"
) # Linux: exclude \x00 for separate handling
_MACOS_ILLEGAL_CHARS_PATTERN = re.compile(r"[:]")
_LINUX_ILLEGAL_CHARS_PATTERN = re.compile(r"[/\x00]")
def clean_subtitle_text(text):
@@ -34,6 +41,17 @@ def clean_subtitle_text(text):
return text.strip()
def calculate_text_length(text):
# Use pre-compiled patterns for better performance
# Ignore chapter markers, voice markers, and metadata patterns in a single pass
text = _CHAPTER_MARKER_PATTERN.sub("", text)
text = _VOICE_MARKER_PATTERN.sub("", text)
text = _METADATA_TAG_PATTERN.sub("", text)
# Ignore newlines and leading/trailing spaces
text = text.replace("\n", "").strip()
# Calculate character count
char_count = len(text)
return char_count
def clean_text(text, *args, **kwargs):
@@ -378,27 +396,189 @@ def parse_ass_file(file_path):
return subtitles
def get_sample_voice_text(language):
"""Get sample voice text for a language.
def get_sample_voice_text(lang_code):
return SAMPLE_VOICE_TEXTS.get(lang_code, SAMPLE_VOICE_TEXTS["a"])
def sanitize_name_for_os(name, is_folder=True):
"""
Sanitize a filename or folder name based on the operating system.
Args:
language: Language enum value or string (for backward compatibility).
name: The name to sanitize
is_folder: Whether this is a folder name (default: True)
Returns:
Sanitized name safe for the current OS
"""
if isinstance(language, str):
try:
language = Language.from_str(language)
except (ValueError, AttributeError):
language = Language.EN_US
return SAMPLE_VOICE_TEXTS.get(language, SAMPLE_VOICE_TEXTS[Language.EN_US])
if not name:
return "audiobook"
system = platform.system()
if system == "Windows":
# Windows illegal characters: < > : " / \ | ? *
# Also can't end with space or dot
# Use pre-compiled pattern for better performance
sanitized = _WINDOWS_ILLEGAL_CHARS_PATTERN.sub("_", name)
# Remove control characters (0-31)
sanitized = _CONTROL_CHARS_PATTERN.sub("_", sanitized)
# Remove trailing spaces and dots
sanitized = sanitized.rstrip(". ")
# Windows reserved names (CON, PRN, AUX, NUL, COM1-9, LPT1-9)
reserved = (
["CON", "PRN", "AUX", "NUL"]
+ [f"COM{i}" for i in range(1, 10)]
+ [f"LPT{i}" for i in range(1, 10)]
)
if sanitized.upper() in reserved or sanitized.upper().split(".")[0] in reserved:
sanitized = f"_{sanitized}"
elif system == "Darwin": # macOS
# macOS illegal characters: : (colon is converted to / by the system)
# Also can't start with dot (hidden file) for folders typically
# Use pre-compiled pattern for better performance
sanitized = _MACOS_ILLEGAL_CHARS_PATTERN.sub("_", name)
# Remove control characters
sanitized = _CONTROL_CHARS_PATTERN.sub("_", sanitized)
# Avoid leading dot for folders (creates hidden folders)
if is_folder and sanitized.startswith("."):
sanitized = "_" + sanitized[1:]
else: # Linux and others
# Linux illegal characters: / and null character
# Though / is illegal, most other chars are technically allowed
# Use pre-compiled pattern for better performance
sanitized = _LINUX_ILLEGAL_CHARS_PATTERN.sub("_", name)
# Remove other control characters for safety (excluding \x00 which is already handled)
sanitized = _LINUX_CONTROL_CHARS_PATTERN.sub("_", sanitized)
# Avoid leading dot for folders (creates hidden folders)
if is_folder and sanitized.startswith("."):
sanitized = "_" + sanitized[1:]
# Ensure the name is not empty after sanitization
if not sanitized or sanitized.strip() == "":
sanitized = "audiobook"
# Limit length to 255 characters (common limit across filesystems)
if len(sanitized) > 255:
sanitized = sanitized[:255].rstrip(". ")
return sanitized
# Backward-compatible re-exports — canonical location is domain/output_paths.py
from abogen.domain.output_paths import sanitize_name_for_os # noqa: E402, F401
def validate_voice_name(voice_name):
"""Validate voice name against VOICES_INTERNAL list (case-insensitive).
Handles both single voices and formulas like 'af_heart*0.5 + am_echo*0.5'.
# Backward-compatible re-exports — canonical location is domain/voice_markers.py
from abogen.domain.voice_markers import ( # noqa: E402, F401
validate_voice_name,
split_text_by_voice_markers,
_VOICE_MARKER_PATTERN,
_VOICE_MARKER_SEARCH_PATTERN,
)
Args:
voice_name: Voice name or formula string to validate
Returns:
Tuple of (is_valid, invalid_voice_name):
- is_valid: True if all voices in the name/formula are valid
- invalid_voice_name: The first invalid voice found, or None if all valid
"""
from abogen.constants import VOICES_INTERNAL
# Create case-insensitive lookup set (done once per call)
voice_lookup_lower = {v.lower() for v in VOICES_INTERNAL}
voice_name = voice_name.strip()
# Check if it's a formula (contains *)
if "*" in voice_name:
# Extract voice names from formula
voices = voice_name.split("+")
for term in voices:
if "*" in term:
base_voice = term.split("*")[0].strip()
# Case-insensitive comparison
if base_voice.lower() not in voice_lookup_lower:
return False, base_voice
return True, None
else:
# Single voice - case-insensitive comparison
if voice_name.lower() not in voice_lookup_lower:
return False, voice_name
return True, None
def split_text_by_voice_markers(text, default_voice):
"""Split text by voice markers, returning list of (voice, text) tuples.
IMPORTANT: Returns the last voice used so it can persist across chapters.
Voice names are normalized to lowercase to match VOICES_INTERNAL.
Args:
text: Text potentially containing <<VOICE:name>> markers
default_voice: Voice to use if no markers found or before first marker
Returns:
Tuple of (segments_list, last_voice_used, valid_count, invalid_count):
- segments_list: List of (voice_name, segment_text) tuples
- last_voice_used: The voice that should continue into next chapter
- valid_count: Number of valid voice markers processed
- invalid_count: Number of invalid voice markers skipped
"""
from abogen.constants import VOICES_INTERNAL
voice_splits = list(_VOICE_MARKER_SEARCH_PATTERN.finditer(text))
if not voice_splits:
# No voice markers, return entire text with default voice
return [(default_voice, text)], default_voice, 0, 0
segments = []
current_voice = default_voice
valid_markers = 0
invalid_markers = 0
# Text before first marker uses default voice
first_start = voice_splits[0].start()
if first_start > 0:
intro_text = text[:first_start].strip()
if intro_text:
segments.append((current_voice, intro_text))
# Process each voice marker
for idx, match in enumerate(voice_splits):
voice_name = match.group(1).strip()
start = match.end()
end = voice_splits[idx + 1].start() if idx + 1 < len(voice_splits) else len(text)
segment_text = text[start:end].strip()
# Validate voice name
is_valid, invalid_voice = validate_voice_name(voice_name)
if is_valid:
# Normalize to lowercase to match canonical form
# Handle both single voices and formulas
if "*" in voice_name:
# Normalize each voice in the formula
normalized_parts = []
for part in voice_name.split("+"):
part = part.strip()
if "*" in part:
voice_part, weight = part.split("*", 1)
# Find the canonical (lowercase) voice name
voice_part_lower = voice_part.strip().lower()
canonical_voice = next(
(v for v in VOICES_INTERNAL if v.lower() == voice_part_lower),
voice_part.strip()
)
normalized_parts.append(f"{canonical_voice}*{weight.strip()}")
current_voice = " + ".join(normalized_parts)
else:
# Find the canonical (lowercase) voice name
voice_name_lower = voice_name.lower()
current_voice = next(
(v for v in VOICES_INTERNAL if v.lower() == voice_name_lower),
voice_name
)
valid_markers += 1
else:
# Invalid voice - stay with previous voice
invalid_markers += 1
if segment_text:
segments.append((current_voice, segment_text))
# Return segments, last voice, and counts
return segments, current_voice, valid_markers, invalid_markers
+1 -2
View File
@@ -16,8 +16,7 @@ import markdown # type: ignore[import]
from bs4 import BeautifulSoup, NavigableString # type: ignore[import]
from ebooklib import epub # type: ignore[import]
from .utils import clean_text, detect_encoding
from .domain.text_utils import calculate_text_length
from .utils import calculate_text_length, clean_text, detect_encoding
logger = logging.getLogger(__name__)
-170
View File
@@ -1,170 +0,0 @@
"""TTS Plugin Architecture - Public API.
This package defines the frozen Plugin API for the TTS Plugin Architecture.
All public interfaces are fully defined but contain no business logic.
Public modules:
- types: Core domain value objects (AudioFormat, Duration, VoiceSelection, etc.)
- errors: Error hierarchy (EngineError and subtypes)
- manifest: Plugin manifest types (PluginManifest, EngineManifest, etc.)
- engine: Engine and EngineSession protocols
- capabilities: Optional capability interfaces (VoiceLister, PreviewGenerator, etc.)
- host_context: HostContext dataclass
- plugin: Plugin contract (create_engine function signature)
- loader: Plugin discovery and loading
- plugin_manager: Plugin management and engine creation
- utils: Direct utility functions (get_voices, create_pipeline, etc.)
Usage:
from abogen.tts_plugin import (
# Types
AudioFormat,
Duration,
VoiceSelection,
ParameterValues,
SynthesisRequest,
SynthesizedAudio,
EngineConfig,
# Errors
EngineError,
ModelNotFoundError,
ModelLoadError,
NetworkError,
InvalidInputError,
ConfigurationError,
CancelledError,
InternalError,
# Manifest
PluginManifest,
EngineManifest,
VoiceSourceManifest,
VoiceManifest,
ParameterManifest,
AudioFormatManifest,
EnumOption,
RequirementManifest,
GpuRequirement,
ModelManifest,
# Engine
Engine,
EngineSession,
# Capabilities
VoiceLister,
PreviewGenerator,
StreamingSynthesizer,
CancelableSession,
# Host Context
HostContext,
HttpClient,
# Plugin Manager
get_plugin_manager,
reset_plugin_manager,
# Utils
get_voices,
get_default_voice,
is_plugin_registered,
resolve_voice_to_plugin,
create_pipeline,
)
"""
from abogen.tts_plugin.capabilities import (
CancelableSession,
PreviewGenerator,
StreamingSynthesizer,
VoiceLister,
)
from abogen.tts_plugin.engine import Engine, EngineSession
from abogen.tts_plugin.errors import (
CancelledError,
ConfigurationError,
EngineError,
InternalError,
InvalidInputError,
ModelLoadError,
ModelNotFoundError,
NetworkError,
)
from abogen.tts_plugin.host_context import HttpClient, HostContext
from abogen.tts_plugin.manifest import (
AudioFormatManifest,
EngineManifest,
EnumOption,
GpuRequirement,
ModelManifest,
ParameterManifest,
PluginManifest,
RequirementManifest,
VoiceManifest,
VoiceSourceManifest,
)
from abogen.tts_plugin.types import (
AudioFormat,
Duration,
EngineConfig,
ParameterValues,
SynthesisRequest,
SynthesizedAudio,
VoiceSelection,
)
# Plugin Manager and Utils
from abogen.tts_plugin.plugin_manager import get_plugin_manager, reset_plugin_manager
from abogen.tts_plugin.utils import (
create_pipeline,
get_default_voice,
get_voices,
is_plugin_registered,
resolve_voice_to_plugin,
)
__all__ = [
# Types
"AudioFormat",
"Duration",
"VoiceSelection",
"ParameterValues",
"SynthesisRequest",
"SynthesizedAudio",
"EngineConfig",
# Errors
"EngineError",
"ModelNotFoundError",
"ModelLoadError",
"NetworkError",
"InvalidInputError",
"ConfigurationError",
"CancelledError",
"InternalError",
# Manifest
"PluginManifest",
"EngineManifest",
"VoiceSourceManifest",
"VoiceManifest",
"ParameterManifest",
"AudioFormatManifest",
"EnumOption",
"RequirementManifest",
"GpuRequirement",
"ModelManifest",
# Engine
"Engine",
"EngineSession",
# Capabilities
"VoiceLister",
"PreviewGenerator",
"StreamingSynthesizer",
"CancelableSession",
# Host Context
"HostContext",
"HttpClient",
# Plugin Manager
"get_plugin_manager",
"reset_plugin_manager",
# Utils
"get_voices",
"get_default_voice",
"is_plugin_registered",
"resolve_voice_to_plugin",
"create_pipeline",
]

Some files were not shown because too many files have changed in this diff Show More