docs: archive historical plans, remove duplicates

- Move migration-roadmap.md, epub3_upgrade_plan.md, entities_step_overhaul_plan.md to docs/archive/ - Remove duplicate tts-plugin-architecture.md
2026-07-18 13:40:27 +02:00 · 2026-07-12 16:20:30 +03:00
parent 096ea58d74
commit f151a1ae0d
3 changed files with 689 additions and 0 deletions
@@ -0,0 +1,208 @@
+# EPUB 3 Upgrade Plan
+
+## Overview
+Elevate Abogen to produce rich EPUB 3 packages with synchronized narration, configurable TTS chunking, and groundwork for multi-speaker voice assignment. This document records the objectives, architectural adjustments, data model changes, UI flows, and implementation phases required to deliver the upgrade.
+
+## Goals
+- Generate EPUB 3 output that preserves source metadata and embeds audio narration via media overlays.
+- Allow users to choose the chunking granularity (paragraph vs. sentence) used for TTS synthesis and media-overlay alignment.
+- Introduce speaker assignments for every chunk, starting with a single narrator but paving the way for multi-speaker control.
+- Prototype practical, lightweight strategies for detecting likely speakers and estimating their dialogue frequency.
+
+## Non-goals / Out-of-scope
+- Full multi-speaker editing UI (beyond gating the option).
+- Automatic voice-casting or LLM-based dialogue attribution.
+- Desktop GUI resurrection (web UI remains primary).
+
+## Current Architecture Snapshot
+| Area | Notes |
+| --- | --- |
+| Text ingestion | `abogen/text_extractor.py` outputs `ExtractionResult` with chapter-level text.
+| Job prep UI | `web/routes.py` builds `PendingJob` objects and renders chapter selection.
+| Audio pipeline | `web/conversion_runner.py` creates per-job audio artifacts; chunking is effectively paragraph-level.
+| Metadata | `ExtractionResult.metadata` feeds into FF metadata and output tagging, but not yet into EPUB packaging.
+
+## Feature 1 – EPUB 3 Output with Narration
+### Requirements
+- Preserve original EPUB metadata (Dublin Core entries, TOC, cover art).
+- Package synthesized audio and SMIL media overlays aligned to chosen chunk granularity.
+- Provide EPUB as an additional selectable output alongside current audio/subtitle formats.
+
+### Proposed Components
+1. **`abogen/epub3/exporter.py`** (new module)
+   - Responsibilities: build XHTML spine with IDs, generate overlay SMIL files, write OPF manifest/spine, assemble zip package.
+   - Status: **Implemented** — `build_epub3_package` emits EPUB 3 archives with media overlays driven by chunk metadata.
+   - Dependencies: reuse `ebooklib` for reading source metadata; use `zipfile` for packaging; optional `lxml` for DOM manipulation.
+2. **`EPUB3PackageBuilder` class**
+   - Inputs: extraction payload, chunk collection (with IDs, speaker mapping, timing metadata), audio asset paths, source metadata.
+   - Outputs: path to generated EPUB.
+3. **Metadata preservation**
+   - Copy from source `ExtractionResult.metadata` and EPUB navigation if available.
+   - Ensure custom fields (e.g., chapter count) survive.
+4. **Media overlay generation**
+   - Create one SMIL per content doc or per chapter, depending on chunk count.
+   - `<par>` nodes reference chunk IDs and audio clip times.
+5. **Configuration surface**
+   - Add “EPUB 3 (audio + text)” to output format selector (or a dedicated toggle under project settings).
+
+### Data Flow
+```
+extract_from_path -> Chapter payload
+              |-> chunker (sentence/paragraph)
+              |-> chunk IDs + audio segments (timestamps from runner)
+Conversion runner -> audio files + timing index
+EPUB3PackageBuilder -> manifest, spine, SMIL, zip
+```
+
+### Open Questions
+- Should we embed audio inside the EPUB or link externally? (Plan: embed to comply with spec.)
+- How to handle very large audio assets? Consider splitting per chapter to keep file sizes manageable.
+
+## Feature 2 – Configurable Chunking
+### Requirements
+- Users select chunking level (paragraph or sentence) before audio generation.
+- Pipeline produces stable, unique IDs for each chunk regardless of level.
+- Provide chunk metadata (text, speaker, offsets) to both TTS and EPUB exporter.
+
+### Proposed Architecture
+1. **Chunk Model**
+   ```python
+   @dataclass
+   class Chunk:
+       id: str
+       chapter_index: int
+       order: int
+       level: Literal["paragraph", "sentence"]
+       text: str
+       speaker_id: str
+       approx_characters: int
+   ```
+2. **Chunker Service (`abogen/chunking.py`)**
+   - Accepts chapter text and desired level.
+   - Uses spaCy (already bundled via `en-core-web-sm`) for sentence segmentation; fallback to regex when model unavailable.
+   - Emits `Chunk` objects with deterministic IDs (e.g., `chap{chapter_index:04d}_para{paragraph_idx:03d}_sent{sentence_idx:03d}`).
+3. **Integration points**
+   - `web/routes.py` -> apply chunker when building `PendingJob` instead of storing raw paragraphs only.
+   - `PendingJob` / `Job` dataclasses -> include `chunks` list and `chunk_level` enum.
+   - `conversion_runner` -> iterate over `chunks` when synthesizing audio, producing per-chunk audio and capturing actual duration for overlay.
+4. **Settings persistence**
+   - Extend config with `chunking_level` default; expose in UI (radio buttons or select).
+
+### Testing
+- Unit tests for chunk splitting across languages, punctuation, abbreviations.
+- Property-based tests ensuring concatenated chunks reproduce original text (except whitespace normalization).
+
+## Feature 3 – Speaker Assignment Foundations
+### Requirements
+- Every chunk must carry a `speaker_id` (default `narrator`).
+- UI offers new option: “Single Speaker” (proceeds) vs. “Multi-Speaker (Coming Soon)” (blocks and shows message).
+- Data model anticipates future multi-speaker support.
+
+### Implementation Outline
+1. **Data Model Changes**
+   - `Chunk.speaker_id` default `"narrator"`.
+   - `PendingJob` & `Job` store `speakers` metadata (dictionary of speaker descriptors).
+   - `JobResult` optionally includes `chunk_speakers.json` artifact for downstream use.
+2. **UI Adjustments**
+   - On upload form (`index.html` / JS), add selector for speaker mode.
+   - If “Multi-Speaker” chosen, display tooltip/modal: “Coming soon; please choose Single Speaker to continue.” disable submission.
+   - In `prepare_job.html`, display speaker info column (read-only for now).
+3. **Serialization**
+   - Update JSON API routes to include speaker data.
+   - Update queue/job detail templates to show chunk level & speaker summary.
+
+### Testing
+- Add web route tests ensuring multi-speaker path blocks progression.
+- Verify job persistence includes `speaker_id` fields.
+
+## Feature 4 – Speaker Detection Strategies
+### Objectives
+Build groundwork for lightweight, deterministic speaker inference to inform future multi-speaker mode.
+
+### User Stories
+1. **As a producer**, I can run an automated analysis on a book to see the list of likely speakers and how often they talk, so I can decide where multiple voices make sense.
+   - _Acceptance_: System outputs a JSON report containing speaker IDs/names, occurrence counts, representative excerpts, and confidence tier. Report stored with job artifacts and downloadable from job detail page.
+2. **As a producer**, I can set a minimum occurrence threshold so that infrequent speakers automatically fall back to the narrator voice.
+   - _Acceptance_: Analysis respects configurable threshold; speakers below it are tagged as `default_narrator` in the report.
+3. **As a developer/operator**, I can trigger the analysis via CLI or background task without blocking the main conversion pipeline.
+   - _Acceptance_: Command `abogen analyze-speakers <input>` (or background queue hook) runs in isolation, returns exit code 0 on success, emits metrics/logs for CI.
+
+### Strategy Ideas
+1. **Quotation-bound heuristic**
+   - Split paragraphs on dialogue quotes.
+   - Use verb cues ("said", "asked") to associate names preceding/following quotes.
+2. **Name detection via NER**
+   - Use spaCy’s entity recognition to spot `PERSON` entities inside dialogue spans.
+   - Maintain frequency counts per name.
+3. **Speaker dictionary**
+   - Pre-build mapping of common narrator cues ("he said", "Mary replied") to propagate speaker assignment across adjacent sentences.
+4. **Pronoun fallback with gender hints**
+   - Map pronouns to most recent speaker mention; degrade gracefully when ambiguous.
+5. **Thresholding mechanism**
+   - After counting occurrences, expose a threshold slider (future UI) to decide when to allocate unique voices vs. default narrator.
+6. **Diagnostics**
+   - Provide summary report: top N speaker candidates, counts, unresolved dialogue segments.
+
+### Implementation Staging
+1. **Phase 1 – Analysis Engine (Backend)**
+   - Build `speaker_analysis.py` module implementing heuristics, returning structured results.
+   - Add CLI entry point `abogen-speaker-analyze` for standalone runs.
+   - Persist analysis artifacts (`speakers.json`, `speaker_excerpts.csv`) alongside job data when invoked post-extraction.
+   - Tests: unit tests for heuristic functions; snapshot tests for sample novels.
+2. **Phase 2 – Configuration & Thresholding**
+   - Extend settings UI with optional “speaker analysis threshold” control (numeric).
+   - Update analysis module to accept threshold; mark low-frequency speakers as narrator.
+   - Emit summary digest (top speakers, narrator fallback count) in job logs.
+3. **Phase 3 – UI Surfacing**
+   - Display analysis summary on job detail page (charts/table).
+   - Offer download link for raw JSON/CSV artifacts.
+   - Provide warning banner when analysis confidence is low (e.g., high unmatched dialogue percentage).
+4. **Phase 4 – Integration Hooks**
+   - Wire analysis output into chunk speaker assignments (without yet enabling multi-speaker playback).
+   - Store mapping in `Job.speakers` metadata for future voice routing.
+
+### Technical Notes
+- Reuse spaCy `en_core_web_sm` for entity recognition; allow pluggable models per language.
+- Maintain rolling context window to resolve pronouns (e.g., last two named speakers).
+- Provide instrumentation (timings, counts) to assess heuristic accuracy on sample corpora.
+- Design analysis output schema versioning (`speaker_analysis_version`) to support iterative improvements.
+
+## UI & Configuration Updates
+| Screen | Update |
+| --- | --- |
+| Upload form (`index.html`) | Add chunking level selector and speaker mode buttons. |
+| Prepare job (`prepare_job.html`) | Display chunk level, IDs, speaker column; allow future editing hooks. |
+| Settings modal | Persist defaults for chunking level and speaker mode. |
+
+## Data Model Checklist
+- [x] Update `PendingJob` and `Job` dataclasses with `chunk_level`, `chunks`, `speakers` metadata.
+- [x] Ensure serialization persists these fields in queue state file.
+- [x] Persist chunk timing metadata from TTS (start/end timestamps).
+
+## Testing Strategy
+- Unit tests for chunker and speaker heuristics.
+- Integration tests: enqueue job with sentence-level chunking, assert chunk IDs and speaker metadata.
+- Regression tests: ensure existing paragraph-level jobs still succeed.
+- Acceptance tests for EPUB exporter: validate manifest, spine, and SMIL structure against schema (use `epubcheck` in CI if feasible).
+
+## Migration & Compat
+- Bump state version in `ConversionService` when augmenting job schema; include migration logic for legacy queues.
+- Provide CLI flag to reprocess older jobs without speaker metadata.
+- Document new dependencies (e.g., `lxml`, optional spaCy models for languages beyond English).
+
+## Implementation Phases
+1. **Foundation** – Introduce chunk model, chunker service, speaker defaults.
+2. **Pipeline integration** – Update job lifecycle and TTS runner to work with chunks.
+3. **EPUB exporter** – Build packaging module, connect to pipeline.
+4. **UI polish** – Expose settings, guard multi-speaker path, surface diagnostics.
+5. **Speaker analysis tool** – Prototype heuristics and reporting.
+
+## Open Questions
+- How to handle non-EPUB inputs (PDF/TXT) when exporting EPUB 3? (Possible: generate synthetic XHTML with normalized chapters.)
+- Storage impact of embedding per-chunk audio – do we need compression or streaming strategies?
+- Internationalization: sentence segmentation quality varies; need language-specific models.
+
+## Next Steps
+- Review plan with stakeholders for scope confirmation.
+- Break down Phase 1 into actionable tickets (chunker, data model migration, UI toggle).
+- Estimate resource requirements for EPUB packaging and testing (including epubcheck integration).