From d430196d6ff1127caa438a3fa452766e31005f58 Mon Sep 17 00:00:00 2001 From: Dymas Date: Mon, 12 Oct 2026 23:08:57 +0200 Subject: [PATCH] Add subtitle-only backfill for existing watchlist videos --- CHANGELOG.md | 6 ++ README.md | 5 +- VERSION | 2 +- app.py | 51 +++++++++++--- app_support.py | 94 ++++++++++++++++++++++++- provider_downloader.py | 51 ++++++++++++++ queue_jobs.py | 48 ++++++++++--- queue_page.py | 4 +- test_app.py | 151 +++++++++++++++++++++++++++++++++++++++++ watchlist_page.py | 8 ++- 10 files changed, 395 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 70ef4b6..33503f8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,11 @@ # Changelog +## 0.53.17 - 2026-10-12 + +- Added **Subtitles only** to the Watchlist **Download all** menu. It backfills provider-supplied English external subtitle sidecars for matching existing videos in Downloads or the configured Jellyfin library, including unpadded season folders and episode-offset names. +- Subtitle-only jobs stage and place sidecars without downloading or changing video files, replacing existing subtitles, changing watchlist download state, covering pending video episodes, or sending video-download completion notifications. Queue logs identify skipped episodes and unavailable tracks; configured Jellyfin library refreshes run when a subtitle lands there. +- Checked all three bundled anime providers against upstream; no updates were available. + ## 0.53.16 - 2026-10-12 - Manual Jellyfin handoff now selects only eligible watchlist entries with the expected nonempty video files present in Downloads. Recorded download counts alone no longer permit an incomplete series to move. diff --git a/README.md b/README.md index 8e93c58..fa1bbeb 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,7 @@ Kaizoku is a local web app for searching, tracking, and downloading anime from A - Fall back across the other configured providers when an episode or stream cannot be resolved on the primary provider. - Save files with Jellyfin-friendly layout: `TV/Series Name/Season 01/Series Name - S01E01.mp4`, including Season 00 for specials. - Save English external subtitles when the provider exposes usable subtitle tracks. +- Backfill missing English subtitle sidecars for existing watchlist videos with **Download all → Subtitles only**, without downloading the video again. - Manage focused watchlists for `Watching`, `Planned`, `Finished`, and `Dropped`, with provider source tags and title links back to the original provider page. - Periodically refresh selected watchlists with a single visible advancing progress bar and auto-download newly available episodes. - Sync watchlist download flags from completed video files on the filesystem; subtitles, metadata, and partial downloads do not count as downloaded episodes or movies. @@ -123,7 +124,9 @@ Provider bridge subprocesses time out after 120 seconds by default. Watchlist re Provider server aliases that resolve to the same URL and request headers are downloaded only once, avoiding duplicate CDN requests and unnecessary rate limiting. -When the selected source exposes English external subtitle tracks, Kaizoku saves the first successfully fetched track alongside the video as `.eng.vtt`, `.eng.srt`, `.eng.ass`, or `.eng.ssa`. Subtitle requests use provider/track headers, session cookies, retries, and the media HTTP fallback. Subtitles move into the library with each completed video and keep matching filenames after custom naming, season/episode offsets, or filename collisions. Subtitle failures do not fail an otherwise completed video; the queue log reports failed tracks or that no English external track was exposed. Burned-in subtitles cannot be saved as a separate file, and Kaizoku does not extract embedded or HLS-playlist-only subtitle tracks. The bundled AnimePahe adapter currently exposes no external subtitle tracks. These changes apply to new downloads and do not backfill subtitles for existing videos. +When the selected source exposes English external subtitle tracks, Kaizoku saves the first successfully fetched track alongside the video as `.eng.vtt`, `.eng.srt`, `.eng.ass`, or `.eng.ssa`. Subtitle requests use provider/track headers, session cookies, retries, and the media HTTP fallback. Subtitles move into the library with each completed video and keep matching filenames after custom naming, season/episode offsets, or filename collisions. Subtitle failures do not fail an otherwise completed video; the queue log reports failed tracks or that no English external track was exposed. Burned-in subtitles cannot be saved as a separate file, and Kaizoku does not extract embedded or HLS-playlist-only subtitle tracks. The bundled AnimePahe adapter currently exposes no external subtitle tracks. + +For older videos, open the series card in Watchlist and choose **Download all → Subtitles only**. Kaizoku checks the selected season in its Downloads folder and the configured Jellyfin TV or movie folder (including `Season 1` and `Season 01` layouts), queues only existing videos without English or untagged subtitle sidecars, and tries available sub and dub provider streams for English external tracks. It saves a sidecar with the exact video basename, such as `Old Show - S01E01.eng.srt`, without changing the video, existing subtitles, watchlist download state, or sending a video-download notification. TV filenames must identify an episode using `S01E01`, `Episode 1`, or ` - 01` style numbering; movie folders must contain one video. The watchlist library name, season, and episode offset must match where the older files live. A job with no available external English tracks reports a failure in the Queue log; partially available tracks are saved and skipped episodes are listed there. If **Refresh after handoff** is enabled and a subtitle lands in Jellyfin, Kaizoku requests the same configured library refresh. Subtitles already embedded or burned into a video cannot be extracted by this action. If the primary provider cannot list, resolve, or download a requested episode, Kaizoku searches the selected provider's title on the remaining providers and tries the matching episode there. A custom library name is used only for output naming. Fallback requires a unique normalized title match, retaining season, part, and cour labels; equivalent notation such as `Season 2` and `2nd Season` is accepted. Shared words or a matching title prefix are insufficient. Ambiguous results, different seasons, and titles that cannot be verified (including alternate translations) are skipped. If no matching source succeeds, the job fails while preserving completed episodes. This prevents unavailable later-season episodes from being silently replaced with season 1 episodes; it does not identify or repair previously downloaded incorrect files. Existing finalization code moves staged files into the configured library layout, preserving data already present in production download folders. If a library move fails, the queue reports failure and leaves remaining completed files in the staging folder shown in the job log for manual recovery. Resolve the storage/permission problem and recover those files before retrying the download. diff --git a/VERSION b/VERSION index 5d014cf..e761fb5 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.53.16 +0.53.17 diff --git a/app.py b/app.py index 203d060..aea42dd 100644 --- a/app.py +++ b/app.py @@ -2952,17 +2952,47 @@ def download_watchlist_item(show_id, mode): runtime = ensure_runtime() item = runtime["watchlist"].get(show_id) normalized_mode = str(mode or "").strip().lower() - if normalized_mode not in MODE_CHOICES: - raise ValueError("Mode must be sub or dub") + subtitles_only = normalized_mode == "subtitles" + if normalized_mode not in MODE_CHOICES and not subtitles_only: + raise ValueError("Mode must be sub, dub, or subtitles") query = str(item.get("auto_download_source_name") or item.get("title") or item.get("show_id") or "").strip() if not query: raise ValueError("Watchlist item is missing a searchable title.") - episodes = episode_list(item["show_id"], normalized_mode) - if not episodes: - raise ValueError(f"No {normalized_mode} episodes are currently available for {item['title']}.") - episode_spec = episodes[0] if len(episodes) == 1 else f"{episodes[0]}-{episodes[-1]}" + subtitle_roots = [] + if subtitles_only: + subtitle_roots = [str(watchlist_source_season_dir(item, runtime["config"]))] + jellyfin_dir = jellyfin_target_season_dir(item, runtime["config"]) + if jellyfin_dir is not None and str(jellyfin_dir) not in subtitle_roots: + subtitle_roots.append(str(jellyfin_dir)) + if normalize_media_type(item.get("media_type")) == "tv": + # Older libraries commonly use "Season 1" rather than "Season 01". + season_number = int(normalize_season(item.get("auto_download_series"))) + for root in tuple(subtitle_roots): + unpadded = str(Path(root).parent / f"Season {season_number}") + if unpadded not in subtitle_roots: + subtitle_roots.append(unpadded) + available = fetch_show_episode_snapshot(item["show_id"])["episodes"] + episodes = list(dict.fromkeys((available.get("sub") or []) + (available.get("dub") or []))) + episodes = [ + episode for episode in episodes + if app_support.existing_video_for_episode( + subtitle_roots, episode, item.get("auto_download_offset"), item.get("media_type") or "tv", + require_missing_subtitle=True, + ) is not None + ] + if not episodes: + raise ValueError( + "No existing episodes missing subtitle sidecars were found in Downloads or the configured Jellyfin folder. " + "Check the watchlist library name, season, and episode offset." + ) + episode_spec = " ".join(str(episode) for episode in episodes) + else: + episodes = episode_list(item["show_id"], normalized_mode) + if not episodes: + raise ValueError(f"No {normalized_mode} episodes are currently available for {item['title']}.") + episode_spec = episodes[0] if len(episodes) == 1 else f"{episodes[0]}-{episodes[-1]}" payload = { "show_id": item["show_id"], @@ -2973,16 +3003,19 @@ def download_watchlist_item(show_id, mode): "media_type": item.get("media_type") or "tv", "season": normalize_season(item.get("auto_download_series")), "episode_offset": item.get("auto_download_offset"), - "mode": normalized_mode, + "mode": "sub" if subtitles_only else normalized_mode, "quality": item.get("auto_download_quality") or runtime["config"].get("auto_download_quality") or runtime["config"]["quality"], "episodes": episode_spec, "download_dir": runtime["config"]["download_dir"], } + if subtitles_only: + payload["subtitles_only"] = True + payload["subtitle_roots"] = subtitle_roots job = runtime["download_queue"].add(payload) return { - "message": f"Queued {item['title']} for {normalized_mode} download.", + "message": f"Queued {item['title']} for {'subtitles only' if subtitles_only else normalized_mode + ' download'}.", "job": job, "item": item, } @@ -3216,6 +3249,8 @@ def sync_downloaded_job_to_watchlist(job): ) def prepare_download_job(job): + if job.get("subtitles_only"): + return job config = ensure_runtime()["config"] if normalize_result_index(job.get("result_index")) is None: result_index = infer_download_job_result_index(job, config) diff --git a/app_support.py b/app_support.py index 5b46b33..bf598fd 100644 --- a/app_support.py +++ b/app_support.py @@ -1124,7 +1124,7 @@ def subtitle_filename_parts(path): """Separate downloaded English subtitle tags from the media basename.""" if path.suffix.lower() not in SUBTITLE_FILE_SUFFIXES: return path.stem, "" - match = re.search(r"(?:\.(?:(?:en|eng)(?:-[a-z0-9]+)?|forced|sdh|cc|default))+$", path.stem, re.IGNORECASE) + match = re.search(r"(?:\.(?:(?:en|eng|english)(?:-[a-z0-9]+)?|forced|sdh|cc|default))+$", path.stem, re.IGNORECASE) return (path.stem[:match.start()], match.group()) if match else (path.stem, "") @@ -1133,7 +1133,10 @@ def extract_episode_from_filename(path): match = re.search(r"\bS[0-9]+E([0-9][0-9A-Za-z.\-]*)\b", stem, re.IGNORECASE) if match: return match.group(1) - match = re.search(r"\bEpisode\s+([0-9][0-9A-Za-z.\-]*)\s*$", stem) + match = re.search(r"\bEpisode\s+([0-9]+(?:\.[0-9]+)?)\b", stem, re.IGNORECASE) + if match: + return match.group(1) + match = re.search(r"\s[-–]\s([0-9]{1,3}(?:\.[0-9]+)?)(?:\s+\[[^]]+\]|\s+\([^)]*\))*$", stem) return match.group(1) if match else None @@ -1147,6 +1150,76 @@ def episode_match_key(value): return f"{number}{suffix}" if number > 0 else text +def existing_video_for_episode(roots, episode, offset=None, media_type="tv", require_missing_subtitle=False): + """Find an existing episode video in configured library folders, in priority order.""" + wanted = episode_match_key(apply_episode_offset(episode, offset)) + for root in roots: + directory = Path(root).expanduser() + if not directory.is_dir(): + continue + videos = sorted( + (path for path in directory.iterdir() + if path.is_file() and path.suffix.lower() in VIDEO_FILE_SUFFIXES and path.stat().st_size > 0), + key=lambda path: path.name.lower(), + ) + if require_missing_subtitle: + videos = [path for path in videos if existing_subtitle_for_video(path) is None] + if media_type == "movie" and len(videos) == 1: + return videos[0] + matches = [path for path in videos if episode_match_key(extract_episode_from_filename(path)) == wanted] + if len(matches) == 1: + return matches[0] + return None + + +def existing_subtitle_for_video(video): + """Check for an English or untagged sidecar without touching the video.""" + for path in video.parent.iterdir(): + if path.is_file() and path.suffix.lower() in SUBTITLE_FILE_SUFFIXES: + stem, tags = subtitle_filename_parts(path) + if stem == video.stem and (not tags or re.search(r"\.(?:en|eng|english)(?:-|\.|$)", tags, re.I)): + return path + return None + + +def finalize_subtitle_files(job): + """Copy staged subtitles beside existing videos without replacing either file.""" + staging_dir = job_staging_dir(job) + roots = [Path(root).expanduser() for root in job.get("subtitle_roots") or []] + saved = [] + for source in sorted(staging_dir.iterdir()): + if not source.is_file() or source.suffix.lower() not in SUBTITLE_FILE_SUFFIXES: + continue + stem, tags = subtitle_filename_parts(source) + if not tags or not re.search(r"\.(?:en|eng|english)(?:-|\.|$)", tags, re.I): + continue + video = next( + (path for root in roots if root.is_dir() for path in root.iterdir() + if path.stem == stem and path.suffix.lower() in VIDEO_FILE_SUFFIXES and path.is_file() + and path.stat().st_size > 0 and existing_subtitle_for_video(path) is None), + None, + ) + if video is None: + continue + destination = video.with_name(source.name) + try: + with source.open("rb") as incoming: + with destination.open("xb") as output: + try: + shutil.copyfileobj(incoming, output) + except Exception: + destination.unlink(missing_ok=True) + raise + except FileExistsError: + continue + saved.append(str(destination)) + source.unlink() + if not saved: + raise RuntimeError("No new English subtitles were saved for existing videos") + cleanup_empty_staging_dirs(staging_dir, remove_root=True) + return saved + + def cleanup_empty_staging_dirs(staging_dir, remove_root=False): for path in sorted(staging_dir.rglob("*"), key=lambda item: len(item.parts), reverse=True): if path.is_dir(): @@ -1259,7 +1332,12 @@ def build_job(payload, config): if quality not in QUALITY_CHOICES: raise ValueError("Unknown quality") - return { + subtitles_only = bool(payload.get("subtitles_only")) + subtitle_roots = [str(Path(root).expanduser()) for root in payload.get("subtitle_roots") or []] + if subtitles_only and not subtitle_roots: + raise ValueError("Subtitle jobs require an existing video folder") + + job = { "id": uuid4().hex, "show_id": str(payload.get("show_id") or "").strip(), "title": title, @@ -1283,6 +1361,10 @@ def build_job(payload, config): "pid": None, "log": [], } + if subtitles_only: + job["subtitles_only"] = True + job["subtitle_roots"] = subtitle_roots + return job def normalize_result_index(value): @@ -1322,4 +1404,10 @@ def command_for_job(job, backend="kaizoku", download_path=None): "--output-dir", str(download_path or job_staging_dir(job)), ] + if job.get("subtitles_only"): + command.append("--subtitles-only") + command.extend(["--episode-offset", str(job.get("episode_offset") or "")]) + command.extend(["--media-type", str(job.get("media_type") or "tv")]) + for root in job.get("subtitle_roots") or []: + command.extend(["--existing-dir", str(root)]) return command diff --git a/provider_downloader.py b/provider_downloader.py index a418f95..32f8b03 100755 --- a/provider_downloader.py +++ b/provider_downloader.py @@ -18,6 +18,7 @@ from urllib.parse import urljoin, urlparse import urllib.request from pathlib import Path +from app_support import existing_video_for_episode from title_matching import normalize_identity_title_key from provider_bridge import PROVIDERS, bridge @@ -858,6 +859,50 @@ def provider_stream_candidates(provider, episode_id, mode, quality): return [] +def backfill_subtitles(args, provider, provider_show_id, source_title, wanted, output_dir): + """Fetch external subtitle tracks for videos already present in the library.""" + saved_count = 0 + skipped_count = 0 + for episode_index, ep in enumerate(wanted, start=1): + number = ep.get("number") + video = existing_video_for_episode(args.existing_dir, number, args.episode_offset, + args.media_type, require_missing_subtitle=True) + if video is None: + print(f"Episode {number}: no matching existing video; skipped.", flush=True) + skipped_count += 1 + continue + staged_video_name = output_dir / video.name + found = False + emit_progress(phase="resolve", message=f"Finding English subtitles for episode {number}.", + episode=number, episode_index=episode_index, episode_total=len(wanted)) + for active_provider, _active_show_id, active_ep in provider_episode_candidates(provider, provider_show_id, source_title, ep): + for mode in ("sub", "dub"): + try: + streams = provider_stream_candidates(active_provider, active_ep.get("id"), mode, args.quality) + except Exception as exc: + print(f"Episode {number} on {active_provider} ({mode}): {concise_error(exc)}", file=sys.stderr) + continue + for stream in streams: + saved = download_subtitles(stream.get("subtitles") or [], staged_video_name, headers=stream.get("headers")) + if saved: + for subtitle in saved: + print(f"Saved subtitle for episode {number}: {subtitle.name}", flush=True) + emit_progress(phase="subtitle_done", message=f"Subtitle for episode {number} staged.", + episode=number, episode_index=episode_index, episode_total=len(wanted)) + saved_count += 1 + found = True + break + if found: + break + if found: + break + if not found: + print(f"Episode {number}: no usable English external subtitle track found.", flush=True) + skipped_count += 1 + print(f"Subtitle backfill: {saved_count} staged, {skipped_count} skipped.", flush=True) + return 0 if saved_count else 1 + + def main(): parser = argparse.ArgumentParser() parser.add_argument("--provider", default="anikoto") @@ -867,6 +912,10 @@ def main(): parser.add_argument("--mode", choices=("sub", "dub"), default="sub") parser.add_argument("--quality", default="best") parser.add_argument("--output-dir", required=True) + parser.add_argument("--subtitles-only", action="store_true") + parser.add_argument("--existing-dir", action="append", default=[]) + parser.add_argument("--episode-offset", default="") + parser.add_argument("--media-type", choices=("tv", "movie"), default="tv") args = parser.parse_args() provider, provider_show_id = parse_show_id(args.show_id, args.provider) @@ -886,6 +935,8 @@ def main(): print(f"Provider: {provider}") print(f"Title: {series_title}") print(f"Episodes: {', '.join(str(ep.get('number')) for ep in wanted)}") + if args.subtitles_only: + raise SystemExit(backfill_subtitles(args, provider, provider_show_id, source_title, wanted, output_dir)) exit_code = 0 for episode_index, ep in enumerate(wanted, start=1): number = ep.get("number") diff --git a/queue_jobs.py b/queue_jobs.py index d4da0d0..dec4713 100644 --- a/queue_jobs.py +++ b/queue_jobs.py @@ -26,6 +26,7 @@ from app_support import ( command_for_job, debug_log, finalize_library_files, + finalize_subtitle_files, job_output_dir, job_staging_dir, job_stdout_enabled, @@ -1504,6 +1505,8 @@ class DownloadQueue: str(job.get("quality") or "").strip().lower(), str(job.get("episodes") or "").strip(), str(job.get("download_dir") or "").strip(), + bool(job.get("subtitles_only")), + tuple(job.get("subtitle_roots") or ()), ) def _find_reusable_failed_job_locked(self, job): @@ -1691,6 +1694,8 @@ class DownloadQueue: covered = set() for row in rows: job = self._job_from_row(row) if isinstance(row, sqlite3.Row) else {"episodes": row[0]} + if job.get("subtitles_only"): + continue episode_spec = job.get("episodes") if job.get("status") in {"pending", "running"} and "retry_episodes" in job: episode_spec = job.get("retry_episodes") @@ -1733,7 +1738,8 @@ class DownloadQueue: ).fetchall() for row in rows: job = self._job_from_row(row) - if job.get("started_at") or job.get("completed_episodes") or job.get("skip_watchlist_sync"): + if (job.get("started_at") or job.get("completed_episodes") or job.get("skip_watchlist_sync") + or job.get("subtitles_only")): continue job.update( anime_name=item["auto_download_name"], @@ -1905,6 +1911,8 @@ class DownloadQueue: print(f"[download:{job_id}:{backend}] {text}", flush=True) def _finalize_progress_episode(self, job, progress): + if job.get("subtitles_only"): + return [] if str((progress or {}).get("phase") or "").strip().lower() != "done": return [] episode = (progress or {}).get("episode") @@ -2026,7 +2034,8 @@ class DownloadQueue: target_dir = job_output_dir(job) try: staging_dir.mkdir(parents=True, exist_ok=True) - target_dir.mkdir(parents=True, exist_ok=True) + if not job.get("subtitles_only"): + target_dir.mkdir(parents=True, exist_ok=True) attempts, selected_methods, unavailable_methods = self._command_attempts(job, staging_dir) except Exception as exc: self._fail_job_startup(job, exc) @@ -2165,7 +2174,10 @@ class DownloadQueue: if exit_code == 0: try: moved_files = list(job.get("_moved_files") or []) - moved_files.extend(finalize_library_files(job, allow_empty=bool(moved_files))) + if job.get("subtitles_only"): + moved_files.extend(finalize_subtitle_files(job)) + else: + moved_files.extend(finalize_library_files(job, allow_empty=bool(moved_files))) except Exception as exc: finalize_error = exc no_files_downloaded = "No downloaded files were found in the staging folder" in str(exc) @@ -2180,7 +2192,7 @@ class DownloadQueue: continue else: successful_backend = backend - if self.watchlist_sync_fn is not None: + if self.watchlist_sync_fn is not None and not job.get("subtitles_only"): try: synced_watchlist_item = self.watchlist_sync_fn(job) except Exception as exc: @@ -2200,22 +2212,23 @@ class DownloadQueue: job.setdefault("log", []).append("Canceled.") elif finalize_error: job["status"] = "failed" - job.setdefault("progress", {})["message"] = "Download failed during library finalization." + job.setdefault("progress", {})["message"] = "Subtitle placement failed." if job.get("subtitles_only") else "Download failed during library finalization." message = str(finalize_error).strip() if "No downloaded files were found in the staging folder" in message: job.setdefault("log", []).append( f"{job.get('download_backend') or 'Downloader'} exited without downloading any files. The selected result or episodes may not be downloadable." ) else: - job.setdefault("log", []).append(f"Library rename failed: {finalize_error}") + job.setdefault("log", []).append(f"Library placement failed: {finalize_error}") # Keep completed files available for recovery after a failed move. cleanup_staging = no_files_downloaded elif exit_code == 0: job["status"] = "done" + completed_message = "Subtitles completed." if job.get("subtitles_only") else "Download completed." job["progress"] = { **(job.get("progress") or {}), "phase": "done", - "message": "Download completed.", + "message": completed_message, "percent": 100, } if successful_backend and successful_backend != attempts[0]["backend"]: @@ -2224,10 +2237,10 @@ class DownloadQueue: saved_line = f"Saved: {path}" if saved_line not in (job.get("log") or []): job.setdefault("log", []).append(saved_line) - job.setdefault("log", []).append("Download completed.") + job.setdefault("log", []).append(completed_message) if watchlist_sync_error: job.setdefault("log", []).append(f"Watchlist sync failed: {watchlist_sync_error}") - elif self.watchlist_sync_fn is not None: + elif self.watchlist_sync_fn is not None and not job.get("subtitles_only"): if job.get("skip_watchlist_sync"): job.setdefault("log", []).append("Watchlist sync skipped for detached job.") else: @@ -2247,7 +2260,22 @@ class DownloadQueue: self._save_job_locked(job) if cleanup_staging: self._cleanup_staging_dir(job) - elif exit_code == 0 and not finalize_error and not canceled: + elif exit_code == 0 and not finalize_error and not canceled and job.get("subtitles_only"): + active_config = self.config_getter() or {} + jellyfin_roots = [ + os.path.abspath(os.path.expanduser(str(active_config.get(key)))) + for key in ("jellyfin_tv_dir", "jellyfin_movie_dir") if active_config.get(key) + ] + if active_config.get("jellyfin_api_refresh_enabled") and any( + os.path.commonpath((os.path.abspath(path), root)) == root + for path in moved_files for root in jellyfin_roots + ): + try: + refresh_jellyfin_library(active_config) + self._append_log(job, "Jellyfin library refresh requested for new subtitles.") + except Exception as exc: + self._append_log(job, f"Jellyfin library refresh failed: {exc}") + elif exit_code == 0 and not finalize_error and not canceled and not job.get("subtitles_only"): download_title = ( str((synced_watchlist_item or {}).get("title") or "").strip() or str(job.get("title") or "").strip() diff --git a/queue_page.py b/queue_page.py index fc33ce5..2e61404 100644 --- a/queue_page.py +++ b/queue_page.py @@ -479,7 +479,9 @@ QUEUE_HTML = r""" const season = String(rawSeason).padStart(2, "0"); const seasonFolder = `Season ${season}`; const target = job.target_dir || `${job.download_dir}/${libraryName}/${seasonFolder}`; - return `${libraryName} · S${season} · ${job.mode} · ${job.quality} · episodes ${job.episodes} · ${target}`; + const task = job.subtitles_only ? "subtitles only" : `${job.mode} · ${job.quality}`; + const location = job.subtitles_only ? (job.subtitle_roots || []).join("; ") : target; + return `${libraryName} · S${season} · ${task} · episodes ${job.episodes} · ${location}`; } function buildJobLog(job) { diff --git a/test_app.py b/test_app.py index b5cb17a..a46dcdf 100644 --- a/test_app.py +++ b/test_app.py @@ -840,6 +840,19 @@ class QueueApiTests(unittest.TestCase): self.assertEqual(covered, {"1", "2", "3"}) + def test_subtitle_jobs_do_not_cover_video_downloads_or_reuse_failed_video_jobs(self): + base = {"show_id": "show-42", "query": "Older Show", "title": "Older Show", + "mode": "sub", "episodes": "1", "download_dir": "/tmp/example"} + video = self.queue.add(base) + with self.queue.lock: + failed = self.queue._find(video["id"]) + failed["status"] = "failed" + self.queue._save_job_locked(failed) + subtitle = self.queue.add({**base, "subtitles_only": True, + "subtitle_roots": ["/tmp/example/tv/Older Show/Season 01"]}) + self.assertNotEqual(subtitle["id"], video["id"]) + self.assertEqual(self.queue.covered_episodes("show-42", "sub", ["1"]), set()) + def test_add_reuses_matching_failed_job_instead_of_creating_duplicate(self): payload = { "show_id": "show-42", @@ -1316,6 +1329,47 @@ class DownloadQueueWorkerFailureTests(unittest.TestCase): self.assertEqual(send_webhook.call_args.args[2]["category"], "finished") self.assertEqual(send_webhook.call_args.args[2]["moved_files"], ["/tmp/example/Queue Show - S01E01.mp4"]) + def test_subtitle_job_places_sidecar_without_marking_video_downloaded(self): + with tempfile.TemporaryDirectory() as directory: + library = Path(directory) / "jellyfin" / "Old Show" / "Season 1" + library.mkdir(parents=True) + video = library / "Old Show - S01E01.mkv" + video.write_bytes(b"old video") + sync = mock.Mock() + queue = APP.DownloadQueue( + lambda: {"mode": "sub", "quality": "best", "download_dir": directory, + "jellyfin_tv_dir": str(Path(directory) / "jellyfin"), + "jellyfin_api_refresh_enabled": True}, + watchlist_sync_fn=sync, start_worker=False, + ) + job = queue.add({"query": "Old Show", "title": "Old Show", "episodes": "1", + "download_dir": directory, "subtitles_only": True, + "subtitle_roots": [str(library)]}) + staging = queue_jobs.job_staging_dir(job) + + class FakeProc: + pid = 4321 + stdout = [] + + def wait(self): + (staging / "Old Show - S01E01.eng.vtt").write_bytes(b"English") + return 0 + + with mock.patch.object(queue_jobs.subprocess, "Popen", return_value=FakeProc()) as launch, mock.patch.object( + queue_jobs, "send_discord_webhook_event" + ) as webhook, mock.patch.object(queue_jobs, "refresh_jellyfin_library") as refresh: + queue._run_job(job) + + stored = queue._find(job["id"]) + self.assertEqual(stored["status"], "done") + self.assertIn("Subtitles completed.", stored["log"]) + self.assertEqual(video.read_bytes(), b"old video") + self.assertEqual((library / "Old Show - S01E01.eng.vtt").read_bytes(), b"English") + self.assertIn("--subtitles-only", launch.call_args.args[0]) + sync.assert_not_called() + webhook.assert_not_called() + refresh.assert_called_once() + def test_failed_batch_preserves_episode_moved_after_progress_done(self): with tempfile.TemporaryDirectory() as temp_root: queue = APP.DownloadQueue( @@ -5156,6 +5210,7 @@ class TemplateHelperTests(unittest.TestCase): self.assertIn('downloadPicker.classList.toggle("visible")', APP.WATCHLIST_HTML) self.assertIn('downloadWatchlistItem(item, "sub"', APP.WATCHLIST_HTML) self.assertIn('downloadWatchlistItem(item, "dub"', APP.WATCHLIST_HTML) + self.assertIn('downloadWatchlistItem(item, "subtitles"', APP.WATCHLIST_HTML) self.assertIn("grid-template-columns: repeat(2, minmax(0, 1fr));", APP.WATCHLIST_HTML) self.assertLess(APP.WATCHLIST_HTML.index(''), APP.WATCHLIST_HTML.index('')) self.assertLess(APP.WATCHLIST_HTML.index(''), APP.WATCHLIST_HTML.index('')) @@ -5217,6 +5272,102 @@ class AniDbCacheTests(unittest.TestCase): class SubtitleTests(unittest.TestCase): + def test_subtitle_backfill_matches_existing_library_video_and_never_downloads_video(self): + with tempfile.TemporaryDirectory() as directory: + library = Path(directory) / "Season 2" + library.mkdir() + video = library / "Older Show - S02E13.mkv" + video.write_bytes(b"original video") + staging = Path(directory) / "staging" + staging.mkdir() + args = mock.Mock(existing_dir=[str(library)], episode_offset="13", media_type="tv", quality="best") + episodes = [{"number": 1, "id": "ep1"}, {"number": 2, "id": "ep2"}] + streams = [{"url": "https://cdn.example/video", "subtitles": [ + {"url": "https://cdn.example/english.srt", "lang": "en"}]}] + with mock.patch.object(provider_downloader, "provider_episode_candidates", side_effect=lambda *_: [("anikoto", "show", episodes[0])]), mock.patch.object( + provider_downloader, "provider_stream_candidates", return_value=streams + ), mock.patch.object(provider_downloader, "fetch_bytes_with_retries", return_value=b"subtitle"), mock.patch.object( + provider_downloader, "download_episode" + ) as download_video: + status = provider_downloader.backfill_subtitles(args, "anikoto", "show", "Older Show", episodes, staging) + self.assertEqual(status, 0) + download_video.assert_not_called() + self.assertEqual(video.read_bytes(), b"original video") + self.assertEqual((staging / "Older Show - S02E13.eng.srt").read_bytes(), b"subtitle") + self.assertFalse((staging / video.name).exists()) + self.assertFalse((library / "Older Show - S02E14.eng.srt").exists()) + + def test_subtitle_finalizer_pairs_sidecar_without_overwriting_video_or_subtitle(self): + with tempfile.TemporaryDirectory() as directory: + library = Path(directory) / "Season 1" + library.mkdir() + video = library / "Old Show - S01E01.mkv" + video.write_bytes(b"original") + job = app_support.build_job( + {"query": "Old Show", "episodes": "1", "download_dir": directory, + "subtitles_only": True, "subtitle_roots": [str(library)]}, + {"mode": "sub", "quality": "best", "download_dir": directory}, + ) + staging = app_support.job_staging_dir(job) + staging.mkdir(parents=True) + source = staging / "Old Show - S01E01.eng.vtt" + source.write_bytes(b"new") + self.assertEqual(app_support.finalize_subtitle_files(job), [str(library / source.name)]) + self.assertEqual(video.read_bytes(), b"original") + self.assertEqual((library / source.name).read_bytes(), b"new") + staging.mkdir(parents=True) + source.write_bytes(b"replacement") + with self.assertRaisesRegex(RuntimeError, "No new English subtitles"): + app_support.finalize_subtitle_files(job) + self.assertEqual((library / source.name).read_bytes(), b"new") + app_support.shutil.rmtree(staging) + + def test_watchlist_subtitles_only_queues_only_existing_missing_episodes(self): + with tempfile.TemporaryDirectory() as directory: + library = Path(directory) / "tv" / "Old Show" / "Season 02" + library.mkdir(parents=True) + (library / "Old Show - S02E13.mkv").write_bytes(b"first") + (library / "Old Show - S02E14.mkv").write_bytes(b"second") + (library / "Old Show - S02E14.eng.srt").write_bytes(b"present") + item = {"show_id": "anikoto:old", "title": "Old Show", "media_type": "tv", + "auto_download_series": "2", "auto_download_offset": "13"} + queue = mock.Mock() + queue.add.return_value = {"id": "subtitle-job"} + runtime = {"watchlist": mock.Mock(get=mock.Mock(return_value=item)), + "download_queue": queue, + "config": {"download_dir": directory, "provider": "anikoto", "quality": "best"}} + with mock.patch.object(APP, "ensure_runtime", return_value=runtime), mock.patch.object( + APP, "fetch_show_episode_snapshot", return_value={"episodes": {"sub": [1, 2, 3], "dub": []}} + ): + result = APP.download_watchlist_item("anikoto:old", "subtitles") + self.assertIn("subtitles only", result["message"]) + payload = queue.add.call_args.args[0] + self.assertTrue(payload["subtitles_only"]) + self.assertEqual(payload["episodes"], "1") + self.assertEqual(payload["mode"], "sub") + self.assertEqual(payload["subtitle_roots"][0], str(library)) + + def test_watchlist_subtitles_only_finds_older_jellyfin_season_folder(self): + with tempfile.TemporaryDirectory() as directory: + library = Path(directory) / "jellyfin" / "Old Show" / "Season 1" + library.mkdir(parents=True) + (library / "Old Show - 01 [1080p].mkv").write_bytes(b"existing") + item = {"show_id": "anikoto:old", "title": "Old Show", "media_type": "tv", + "auto_download_series": "1"} + queue = mock.Mock() + runtime = {"watchlist": mock.Mock(get=mock.Mock(return_value=item)), + "download_queue": queue, + "config": {"download_dir": str(Path(directory) / "downloads"), + "jellyfin_tv_dir": str(Path(directory) / "jellyfin"), + "provider": "anikoto", "quality": "best"}} + with mock.patch.object(APP, "ensure_runtime", return_value=runtime), mock.patch.object( + APP, "fetch_show_episode_snapshot", return_value={"episodes": {"sub": [1], "dub": []}} + ): + APP.download_watchlist_item("anikoto:old", "subtitles") + payload = queue.add.call_args.args[0] + self.assertEqual(payload["episodes"], "1") + self.assertIn(str(library), payload["subtitle_roots"]) + def test_bridge_preserves_payload_tracks_and_subtitle_referer(self): script = r""" const Module = require("module"); diff --git a/watchlist_page.py b/watchlist_page.py index f22fa34..29d7fba 100644 --- a/watchlist_page.py +++ b/watchlist_page.py @@ -696,6 +696,9 @@ WATCHLIST_HTML = r""" .download-picker.visible { display: grid; } + .download-picker button:last-child { + grid-column: 1 / -1; + } .pager { display: flex; justify-content: space-between; @@ -1428,6 +1431,7 @@ WATCHLIST_HTML = r"""
+
`; @@ -1480,6 +1484,7 @@ WATCHLIST_HTML = r""" const autoDownloadBtn = actionButtons[3]; const subBtn = actionButtons[4]; const dubBtn = actionButtons[5]; + const subtitlesBtn = actionButtons[6]; const downloadPicker = card.querySelector(".download-picker"); const categorySelect = card.querySelector(".category-select"); const mediaTypeSelect = card.querySelector(".media-type-select"); @@ -1525,6 +1530,7 @@ WATCHLIST_HTML = r""" }); subBtn.addEventListener("click", () => downloadWatchlistItem(item, "sub", downloadPicker, downloadBtn)); dubBtn.addEventListener("click", () => downloadWatchlistItem(item, "dub", downloadPicker, downloadBtn)); + subtitlesBtn.addEventListener("click", () => downloadWatchlistItem(item, "subtitles", downloadPicker, downloadBtn)); removeBtn.addEventListener("click", () => removeWatchlistItem(item.show_id, item.title)); container.appendChild(card); applyThumbnail(card, item, false); @@ -1595,7 +1601,7 @@ WATCHLIST_HTML = r""" } async function downloadWatchlistItem(item, mode, picker = null, toggleBtn = null) { - setNotice(`Queueing ${item.title || "selected anime"} for ${mode} download...`); + setNotice(`Queueing ${item.title || "selected anime"} for ${mode === "subtitles" ? "subtitles only" : mode + " download"}...`); try { const data = await api("/api/watchlist/download-all", { method: "POST",