Continue after provider playlist errors

This commit is contained in:
Dymas
2026-09-07 22:07:46 +02:00
parent d39d783738
commit a5401873cb
5 changed files with 72 additions and 9 deletions
+4
View File
@@ -1,5 +1,9 @@
# Changelog
## 0.52.28 - 2026-09-07
- Fixed provider source fallback so playlist-resolution exceptions such as `HTTP 502` fail only the current server and the downloader continues with the remaining same-mode servers.
## 0.52.27 - 2026-09-07
- Preserved provider CDN session cookies from HLS playlist requests across segment and encryption-key downloads, including curl fallback requests.
+1 -1
View File
@@ -101,7 +101,7 @@ Keep `./.kaizoku` mounted for production instances. That directory contains the
## Download Flow
Kaizoku stores provider-backed show IDs as `provider:id`, for example `anikoto:some-show-slug`. Queue jobs resolve the episode source through `providers/bridge.js`, then `provider_downloader.py` downloads the media with `ffmpeg` into a staging directory. If a provider returns multiple servers for the requested sub or dub mode, Kaizoku tries the same-mode server sources in quality order before falling back to another provider. It does not use sub sources for dub downloads, or dub sources for sub downloads. If a provider returns a master HLS playlist, Kaizoku selects the highest-bandwidth variant before starting `ffmpeg`. When a media playlist uses extensionless, SnapCDN, or disguised CDN segments, Kaizoku skips direct ffmpeg and uses a StrawVerse-style segment downloader; if HLS preflight fails, it stays on that native path instead of falling through to ffmpeg. The native downloader fetches the media playlist, preserves provider CDN session cookies across playlist, segment, and encryption-key requests, downloads and concatenates segments itself with browser-like media headers and the provider referer, can fall back to curl-impersonate browser wrappers after protected CDN `HTTP 403` responses, strips short PNG wrappers when present, retries temporary HTTP failures such as `429 Too Many Requests`, optionally paces requests with `KAIZOKU_SEGMENT_DOWNLOAD_DELAY`, then remuxes the local transport stream to MP4. Direct ffmpeg attempts also have a timeout guard so stalled HLS inputs can fall back cleanly. Each episode is written as a temporary `.mp4.part` file and moved into the downloads library after that episode succeeds, so already-finished episodes from a larger batch survive if a later episode fails. Retrying that failed queue job requests only the remaining episodes while keeping the original episode range for display and watchlist sync. Finalization preserves episode numbers from staged `SxxEyy` or `Episode yy` filenames before applying configured season/episode offsets.
Kaizoku stores provider-backed show IDs as `provider:id`, for example `anikoto:some-show-slug`. Queue jobs resolve the episode source through `providers/bridge.js`, then `provider_downloader.py` downloads the media with `ffmpeg` into a staging directory. If a provider returns multiple servers for the requested sub or dub mode, Kaizoku tries every same-mode server source in quality order before falling back to another provider, including when resolving one server's playlist raises an HTTP error. It does not use sub sources for dub downloads, or dub sources for sub downloads. If a provider returns a master HLS playlist, Kaizoku selects the highest-bandwidth variant before starting `ffmpeg`. When a media playlist uses extensionless, SnapCDN, or disguised CDN segments, Kaizoku skips direct ffmpeg and uses a StrawVerse-style segment downloader; if HLS preflight fails, it stays on that native path instead of falling through to ffmpeg. The native downloader fetches the media playlist, preserves provider CDN session cookies across playlist, segment, and encryption-key requests, downloads and concatenates segments itself with browser-like media headers and the provider referer, can fall back to curl-impersonate browser wrappers after protected CDN `HTTP 403` responses, strips short PNG wrappers when present, retries temporary HTTP failures such as `429 Too Many Requests`, optionally paces requests with `KAIZOKU_SEGMENT_DOWNLOAD_DELAY`, then remuxes the local transport stream to MP4. Direct ffmpeg attempts also have a timeout guard so stalled HLS inputs can fall back cleanly. Each episode is written as a temporary `.mp4.part` file and moved into the downloads library after that episode succeeds, so already-finished episodes from a larger batch survive if a later episode fails. Retrying that failed queue job requests only the remaining episodes while keeping the original episode range for display and watchlist sync. Finalization preserves episode numbers from staged `SxxEyy` or `Episode yy` filenames before applying configured season/episode offsets.
If the primary provider cannot list, resolve, or download a requested episode, Kaizoku searches the same title on the remaining providers and tries the matching episode there. Existing finalization code moves staged files into the configured library layout, preserving data already present in production download folders.
+1 -1
View File
@@ -1 +1 @@
0.52.27
0.52.28
+8
View File
@@ -955,6 +955,7 @@ def main():
f"{source_name} ({stream_index}/{len(streams)})...",
flush=True,
)
try:
code = download_episode(
stream,
target,
@@ -962,6 +963,13 @@ def main():
episode_index=episode_index,
episode_total=len(wanted),
)
except Exception as exc:
last_error = exc
print(
f"Source {source_name} on {active_provider} failed: {exc}",
file=sys.stderr,
)
continue
if code != 0:
last_error = RuntimeError(
f"Source {source_name} on {active_provider} failed with exit code {code}"
+51
View File
@@ -4824,6 +4824,57 @@ https://shard-102.snapcdn.top/anime/show/episode/seg-f1-00083.css
self.assertEqual(raised.exception.code, 0)
self.assertEqual(calls, ["HD-1", "HD-2"])
def test_provider_downloader_tries_next_source_after_playlist_exception(self):
calls = []
def fake_bridge(command, provider, *args):
if command == "info":
return {"title": "Retry Exception Show"}
if command == "episodes":
return {"episodes": [{"number": 1, "id": "ep1"}]}
if command == "resolve-all":
return {
"sources": [
{"url": "https://bad.example/master.m3u8", "server": "StreamHG", "type": "dub"},
{"url": "https://good.example/master.m3u8", "server": "Earnvids", "type": "dub"},
]
}
raise AssertionError(f"unexpected bridge command {command}")
def fake_download(stream, target, episode_number=None, episode_index=None, episode_total=None):
calls.append(stream["server"])
if stream["server"] == "StreamHG":
raise RuntimeError("HTTP Error 502: Bad Gateway")
return 0
argv = [
"provider_downloader.py",
"--provider",
"anikoto",
"--show-id",
"anikoto:show-1",
"--title",
"Retry Exception Show",
"--episodes",
"1",
"--mode",
"dub",
"--quality",
"best",
"--output-dir",
"/tmp/kaizoku-provider-test",
]
with mock.patch.object(provider_downloader, "bridge", side_effect=fake_bridge), mock.patch.object(
provider_downloader, "download_episode", side_effect=fake_download
), mock.patch.object(provider_downloader, "download_subtitles", return_value=[]), mock.patch.object(
provider_downloader.sys, "argv", argv
):
with self.assertRaises(SystemExit) as raised:
provider_downloader.main()
self.assertEqual(raised.exception.code, 0)
self.assertEqual(calls, ["StreamHG", "Earnvids"])
def test_provider_bridge_resolve_does_not_fallback_from_dub_to_sub(self):
bridge_js = (ROOT / "providers" / "bridge.js").read_text(encoding="utf-8")