feat: defer runtime bootstrap and harden watchlist refresh

CHANGELOG.md

@@ -1,5 +1,51 @@
 # Changelog
 
+## 0.12.0 - 2026-05-13
+
+- Deferred runtime initialization until the server starts or a runtime-backed API/helper is first used, removing the earlier worker/thread startup and state writes from plain module import.
+- Persisted watchlist refresh history in SQLite, restore interrupted `pending` and `running` refresh jobs as `interrupted` after restart, and block duplicate queued refresh submissions.
+- Hardened thumbnail in-flight event cleanup so unexpected errors during cache updates do not leave later requests waiting on a stuck per-show event.
+- Return `400 Bad Request` for non-UTF-8 JSON payloads instead of surfacing them as internal server errors.
+- Expanded regression coverage for deferred bootstrap, duplicate refresh protection, interrupted refresh recovery, UTF-8 body validation, and thumbnail event cleanup.
+
+## 0.11.0 - 2026-05-13
+
+- Moved the large inline page templates and shared page-shell helpers into `web_templates.py`, reducing `app.py` to server and runtime logic.
+- Changed the shared SQLite file from `queue.sqlite3` to `state.sqlite3` and added automatic migration from the previous project-local database filename.
+- Made watchlist poster image routes cache-only so opening the watchlist no longer fans out synchronous remote cover lookups from the browser image tags.
+- Moved `Refresh All` onto a background watchlist-refresh worker with status polling so large watchlists no longer block a request thread until every show finishes updating.
+- Added an HTTP JSON body-size guard to reject oversized uploads before the server reads unbounded request bodies into memory.
+- Reworked the test harness to run against an isolated temporary state root and added handler-level coverage for thumbnail serving, background refresh status, and oversized POST rejection.
+
+## 0.10.0 - 2026-05-13
+
+- Fixed running-job cancellation so the worker now consistently finishes canceled downloads as `canceled` instead of drifting into failed or completed final states.
+- Fixed watchlist `Refresh All` to iterate every tracked show instead of silently stopping at the paged list limit.
+- Added loopback-only request protection by default, with `ANI_CLI_WEB_ALLOW_REMOTE=1` available when remote/LAN access is intentionally desired.
+- Extracted shared sidebar branding and page-link helpers to reduce repeated inline page chrome across Search, Config, and Watchlist.
+- Added focused regression tests for cancellation propagation, full watchlist refresh coverage, local-access helpers, and shared template helpers.
+
+## 0.9.0 - 2026-05-13
+
+- Reworked Search, Config, and Watchlist with a Tsuki-inspired visual refresh using violet accents, glassy surfaces, softer gradients, and calmer card styling.
+- Kept the existing ani-cli-web behavior and routing intact while unifying the refreshed design language across the whole web UI.
+
+## 0.8.16 - 2026-05-13
+
+- Unified the left sidebar layout across Search, Config, and Watchlist so the page chrome now uses the same panel width, spacing, and structure everywhere.
+- Kept the Config page compact by constraining its main settings content while restoring the shared sidebar pattern.
+
+## 0.8.15 - 2026-05-13
+
+- Removed the dependency status panel from the Search page so the main route stays focused on search, episode selection, and queue management.
+- Reworked the Config page into a centered compact settings layout instead of the earlier stretched split-screen view.
+
+## 0.8.14 - 2026-05-13
+
+- Added a standalone `/config` page for saved defaults.
+- Moved the Search page defaults section onto the new Config page and added Config navigation across Search and Watchlist.
+- Kept dependency status visible while moving saved folder, mode, and quality management into the dedicated settings view.
+
 ## 0.8.13 - 2026-05-12
 
 - Added a manual thumbnail upload button for watchlist cards that still have no poster, saving the selected image into the local thumbnail cache for that anime.

README.md

@@ -2,14 +2,16 @@
 
 A local web UI for a system-wide `ani-cli` install.
 
-Current version: `0.8.13`
+Current version: `0.12.0`
 
 ## Project Layout
 
 ```text
 ani-cli-web/
-  app.py          Web server, queue, watchlist, thumbnail cache, and post-download file naming.
+  app.py          Web server, runtime wiring, request handlers, and CLI entrypoint.
+  web_templates.py Shared Search, Config, and Watchlist page markup plus page-shell helpers.
   ani-cli-web     Launcher script.
+  test_app.py     Regression tests with isolated temp-state setup.
   VERSION         Project version.
   CHANGELOG.md    Change history.
   README.md       Project documentation.
@@ -41,16 +43,21 @@ By default the server binds to `127.0.0.1:8421`.
 
 ## Overview
 
-`ani-cli-web` keeps search and download management on `/` and exposes the watchlist on `/watchlist`.
+`ani-cli-web` keeps search and download management on `/`, saved defaults on `/config`, and the watchlist on `/watchlist`, all with the same sidebar navigation layout and a Tsuki-inspired visual style.
 
 The app currently includes:
 
 - Search against AllAnime with `sub` or `dub` mode selection.
 - Episode loading for the selected result.
 - Queueing downloads with editable library name, season, episode range, quality, and target folder.
+- A dedicated Config page for saved default folder, mode, and quality settings.
 - A paged SQLite-backed download queue with retry, cancel, remove, retry-failed, and clear-finished actions.
 - A paged SQLite-backed watchlist with summary cards, per-show refresh, bulk refresh, source links, and thumbnail tools.
+- Background `Refresh All` execution with status polling instead of a long blocking HTTP request.
+- Persistent watchlist refresh history that survives restarts and marks interrupted bulk refresh jobs clearly.
 - Local thumbnail caching plus manual thumbnail upload when automatic cover lookup fails.
+- A refreshed glassy dark UI inspired by the Tsuki frontend design language.
+- Shared page templates moved out of `app.py` plus broader regression tests for higher-risk queue, watchlist, and handler flows.
 - Project-local runtime state under `.ani-cli-web/`.
 
 ## Search Page
@@ -66,10 +73,34 @@ Workflow:
 
 The page also includes:
 
-- A Defaults panel for the saved download folder, mode, and quality.
-- A dependency status panel so you can see which required tools are available.
 - A live queue view that refreshes automatically.
 
+## Config Page
+
+The Config page at `/config` is where saved defaults now live.
+
+Use it to:
+
+1. Set the default download folder.
+2. Choose the default `sub` or `dub` search mode.
+3. Choose the default search quality used to prefill the Search page.
+
+Those saved defaults are written into the project-local `config.json` and loaded automatically when the Search page opens.
+
+The Config page also shows the current app version plus dependency status in a compact centered layout.
+
+## Access Guard
+
+By default, `ani-cli-web` only serves loopback clients such as `127.0.0.1` and `::1`.
+
+If you intentionally want LAN or remote access, set:
+
+```sh
+ANI_CLI_WEB_ALLOW_REMOTE=1
+```
+
+This keeps the queue, config, and watchlist mutation APIs safer by default when the app is rebound away from localhost.
+
 ## Download Queue
 
 The queue is stored in SQLite and shown 10 jobs at a time.
@@ -100,7 +131,7 @@ You can add entries in two ways:
 Watchlist features:
 
 1. `Refresh` updates one tracked show.
-2. `Refresh All` updates the entire watchlist.
+2. `Refresh All` runs in the background, rejects duplicate queueing while a refresh is already pending or running, and preserves interrupted job status across restarts.
 3. `Open` jumps back to the Search page with the title pre-filled.
 4. Clicking the anime title opens the AllManga source page at `bangumi/<show_id>` in a new tab.
 5. `Remove` deletes the watchlist entry and clears its cached poster file.
@@ -116,6 +147,7 @@ Current thumbnail behavior:
 - Cached thumbnails render from the local project cache when available.
 - The page adds a fresh nonce to thumbnail URLs to avoid stale browser-cached misses.
 - Missing visible covers are warmed up in small batches instead of one request per card all at once.
+- Thumbnail image tags only hit the local route for already-cached files, so opening the watchlist no longer triggers a per-card remote lookup burst.
 - AnimeSchedule is tried first for cover lookup.
 - If a season-labeled title fails as-is, lookup retries with simplified base-title queries.
 - AniDB is used as a fallback when AnimeSchedule does not resolve a cover.
@@ -140,7 +172,7 @@ ani-cli-web/.ani-cli-web/
 That folder currently holds:
 
 - `config.json`
-- `queue.sqlite3`
+- `state.sqlite3`
 - legacy `queue.json` migration input when present
 - legacy `watchlist.json` migration input when present
 - `thumbnails/`
@@ -148,6 +180,7 @@ That folder currently holds:
 - staging files for active downloads
 
 Legacy files from `~/.config/ani-cli-web/` and `~/.local/state/ani-cli-web/` are migrated into the project-local folder automatically on startup.
+The earlier project-local `queue.sqlite3` filename is also migrated automatically into `state.sqlite3`.
 
 ## Configuration
 
@@ -159,11 +192,15 @@ Environment variables:
 - `ANI_CLI_QUALITY`: initial quality such as `best`, `1080`, or `720`.
 - `ANI_CLI_WEB_HOST`: server host, default `127.0.0.1`.
 - `ANI_CLI_WEB_PORT`: server port, default `8421`.
+- `ANI_CLI_WEB_ALLOW_REMOTE`: allow non-loopback clients when set to `1`, `true`, `yes`, or `on`.
 - `ANI_CLI_WEB_DEBUG`: enable debug logging when set to `1`, `true`, `yes`, or `on`.
+- `ANI_CLI_WEB_STATE_ROOT`: override the project-local state directory path; useful for isolated test runs or custom storage locations.
 
 Saved defaults from the UI are written into the project-local `config.json`.
 
 ## Notes
 
-- Watchlist and queue data are stored in the same project-local SQLite database.
+- Importing `app.py` no longer boots the runtime worker threads automatically; runtime setup is deferred until startup or first use.
+- Watchlist and queue data are stored in the same project-local SQLite database file, `state.sqlite3`.
 - Runtime folders such as `.ani-cli-web/`, `downloads/`, and Python bytecode cache directories at any depth are ignored by git.
+- Focused regression tests live in `test_app.py`, run against an isolated temporary state root, and can be started with `python3 -m unittest test_app.py`.

VERSION

@@ -1 +1 @@
-0.8.13
+0.12.0

test_app.py

@@ -0,0 +1,282 @@
+#!/usr/bin/env python3
+import importlib.util
+import io
+import os
+import sys
+import tempfile
+import threading
+import unittest
+from http import HTTPStatus
+from pathlib import Path
+from unittest import mock
+
+
+ROOT = Path(__file__).resolve().parent
+MODULE_PATH = ROOT / "app.py"
+TEMP_STATE = tempfile.TemporaryDirectory()
+
+if str(ROOT) not in sys.path:
+    sys.path.insert(0, str(ROOT))
+
+os.environ.setdefault("ANI_CLI_WEB_DISABLE_WORKER", "1")
+os.environ["ANI_CLI_WEB_STATE_ROOT"] = TEMP_STATE.name
+
+SPEC = importlib.util.spec_from_file_location("ani_cli_web_app", MODULE_PATH)
+APP = importlib.util.module_from_spec(SPEC)
+assert SPEC.loader is not None
+SPEC.loader.exec_module(APP)
+IMPORT_RUNTIME_WAS_DEFERRED = (
+    APP.CONFIG is None and APP.WATCHLIST is None and APP.DOWNLOAD_QUEUE is None and APP.WATCHLIST_REFRESH is None
+)
+APP.initialize_runtime(start_workers=False)
+
+
+class DummyHandler:
+    def __init__(self, path, body=b"{}", content_length=None):
+        self.path = path
+        self.client_address = ("127.0.0.1", 8421)
+        self.headers = {}
+        if content_length is not None:
+            self.headers["Content-Length"] = str(content_length)
+        self.rfile = io.BytesIO(body)
+        self.json_payload = None
+        self.json_status = None
+        self.error_status = None
+        self.error_message = None
+        self.file_path = None
+
+    def ensure_client_access(self):
+        return APP.Handler.ensure_client_access(self)
+
+    def body_json(self):
+        return APP.Handler.body_json(self)
+
+    def json(self, payload, status=HTTPStatus.OK):
+        self.json_payload = payload
+        self.json_status = status
+
+    def error(self, status, message):
+        self.error_status = status
+        self.error_message = message
+
+    def file(self, path):
+        self.file_path = path
+
+    def html(self, _content):
+        raise AssertionError("HTML should not be served in this test")
+
+    def exception(self, exc):
+        return APP.Handler.exception(self, exc)
+
+
+class NetworkGuardTests(unittest.TestCase):
+    def test_runtime_bootstrap_is_deferred_until_initialize(self):
+        self.assertTrue(IMPORT_RUNTIME_WAS_DEFERRED)
+
+    def test_client_address_is_local_accepts_loopback(self):
+        self.assertTrue(APP.client_address_is_local("127.0.0.1"))
+        self.assertTrue(APP.client_address_is_local("::1"))
+        self.assertTrue(APP.client_address_is_local("localhost"))
+
+    def test_client_address_is_local_rejects_non_loopback(self):
+        self.assertFalse(APP.client_address_is_local("192.168.1.25"))
+        self.assertFalse(APP.client_address_is_local("10.0.0.8"))
+
+
+class DownloadQueueCancelTests(unittest.TestCase):
+    def test_cancel_updates_inflight_job_state(self):
+        queue = object.__new__(APP.DownloadQueue)
+        queue.lock = threading.RLock()
+        queue.current_job_id = "job-1"
+        queue.current_process = mock.Mock(pid=4321)
+        queue.current_job = {"id": "job-1", "status": "running", "cancel_requested": False}
+
+        saved = []
+        queue._save_job_locked = lambda job: saved.append(dict(job))
+        queue._find = lambda job_id: {"id": job_id, "status": "running", "cancel_requested": False}
+
+        with mock.patch.object(APP.os, "getpgid", return_value=4321), mock.patch.object(APP.os, "killpg") as killpg:
+            result = APP.DownloadQueue.cancel(queue, "job-1")
+
+        killpg.assert_called_once()
+        self.assertTrue(result["cancel_requested"])
+        self.assertTrue(queue.current_job["cancel_requested"])
+        self.assertEqual(saved[-1]["id"], "job-1")
+        self.assertTrue(saved[-1]["cancel_requested"])
+
+
+class WatchlistRefreshAllTests(unittest.TestCase):
+    def test_refresh_all_iterates_every_show_id(self):
+        store = object.__new__(APP.WatchlistStore)
+        calls = []
+        store.all_show_ids = lambda: ["show-a", "show-b", "show-c"]
+        store.refresh = lambda show_id: calls.append(show_id) or {"show_id": show_id}
+
+        result = APP.WatchlistStore.refresh_all(store)
+
+        self.assertEqual(calls, ["show-a", "show-b", "show-c"])
+        self.assertEqual(result["count"], 3)
+        self.assertEqual([item["show_id"] for item in result["items"]], calls)
+
+
+class WatchlistRefreshJobTests(unittest.TestCase):
+    def setUp(self):
+        with APP.WatchlistRefreshJobs(APP.WATCHLIST, start_worker=False)._connect() as conn:
+            conn.execute("DELETE FROM watchlist_refresh_jobs")
+
+    def test_refresh_job_tracks_completion_and_errors(self):
+        class FakeStore:
+            def all_show_ids(self):
+                return ["show-a", "show-b"]
+
+            def refresh(self, show_id):
+                if show_id == "show-a":
+                    return {"title": "Show A", "status": "updated"}
+                return {"title": "Show B", "status": "error"}
+
+        service = APP.WatchlistRefreshJobs(FakeStore(), start_worker=False)
+        started = service.start()
+        self.assertTrue(started["created"])
+
+        job = service._next_pending()
+        self.assertIsNotNone(job)
+        service._run_job(job)
+
+        status = service.status()["job"]
+        self.assertIsNotNone(status)
+        self.assertEqual(status["status"], "done")
+        self.assertEqual(status["completed"], 2)
+        self.assertEqual(status["errors"], 1)
+
+    def test_refresh_start_reuses_pending_job(self):
+        class FakeStore:
+            def all_show_ids(self):
+                return []
+
+            def refresh(self, _show_id):
+                return {"title": "Unused", "status": "updated"}
+
+        service = APP.WatchlistRefreshJobs(FakeStore(), start_worker=False)
+        first = service.start()
+        second = service.start()
+
+        self.assertTrue(first["created"])
+        self.assertFalse(second["created"])
+        self.assertEqual(second["job"]["id"], first["job"]["id"])
+        self.assertEqual(second["job"]["status"], "pending")
+
+    def test_refresh_history_marks_pending_jobs_interrupted_after_restart(self):
+        class FakeStore:
+            def all_show_ids(self):
+                return []
+
+            def refresh(self, _show_id):
+                return {"title": "Unused", "status": "updated"}
+
+        service = APP.WatchlistRefreshJobs(FakeStore(), start_worker=False)
+        started = service.start()
+        restarted = APP.WatchlistRefreshJobs(FakeStore(), start_worker=False)
+
+        status = restarted.status()["job"]
+        self.assertEqual(status["id"], started["job"]["id"])
+        self.assertEqual(status["status"], "interrupted")
+        self.assertIn("restart", status["message"].lower())
+
+
+class ThumbnailEventRecoveryTests(unittest.TestCase):
+    def test_ensure_thumbnail_clears_inflight_event_after_update_failure(self):
+        store = object.__new__(APP.WatchlistStore)
+        store.lock = threading.RLock()
+        store.thumbnail_events = {}
+        store.get = lambda _show_id: {
+            "show_id": "show-1",
+            "title": "Show One",
+            "thumbnail_path": None,
+            "thumbnail_checked_at": None,
+            "thumbnail_ready": False,
+            "animeschedule_route": None,
+            "animeschedule_title": None,
+            "anidb_aid": None,
+            "anidb_title": None,
+        }
+
+        class BrokenConn:
+            def __enter__(self):
+                return self
+
+            def __exit__(self, exc_type, exc, tb):
+                return False
+
+            def execute(self, _sql, _params):
+                raise RuntimeError("db write failed")
+
+        store._connect = lambda: BrokenConn()
+
+        with mock.patch.object(APP, "fetch_animeschedule_cover", return_value={"route": "route-1", "title": "Show One", "url": "https://example.com/cover.jpg"}), mock.patch.object(
+            APP, "cache_thumbnail_image", return_value="show-1.jpg"
+        ):
+            with self.assertRaises(RuntimeError):
+                APP.WatchlistStore.ensure_thumbnail(store, "show-1", force=False)
+
+        self.assertEqual(store.thumbnail_events, {})
+
+
+class HandlerRouteTests(unittest.TestCase):
+    def test_body_json_rejects_oversized_request(self):
+        handler = DummyHandler("/api/watchlist/upload-thumbnail", body=b"{}", content_length=APP.MAX_JSON_BODY_BYTES + 1)
+        with self.assertRaises(APP.HttpError) as ctx:
+            APP.Handler.body_json(handler)
+        self.assertEqual(ctx.exception.status, HTTPStatus.REQUEST_ENTITY_TOO_LARGE)
+
+    def test_body_json_rejects_non_utf8_payload(self):
+        handler = DummyHandler("/api/watchlist/upload-thumbnail", body=b"\xff", content_length=1)
+        with self.assertRaises(ValueError) as ctx:
+            APP.Handler.body_json(handler)
+        self.assertIn("utf-8", str(ctx.exception).lower())
+
+    def test_thumbnail_route_serves_cached_file_without_fetching(self):
+        thumb_path = APP.THUMBNAIL_ROOT / "show-1.jpg"
+        thumb_path.parent.mkdir(parents=True, exist_ok=True)
+        thumb_path.write_bytes(b"jpg")
+
+        handler = DummyHandler("/api/watchlist/thumb/show-1")
+        with mock.patch.object(APP.WATCHLIST, "get", return_value={"thumbnail_path": "show-1.jpg"}) as get_item, mock.patch.object(
+            APP.WATCHLIST, "ensure_thumbnail", side_effect=AssertionError("thumbnail fetch should not run")
+        ):
+            APP.Handler.do_GET(handler)
+
+        get_item.assert_called_once_with("show-1")
+        self.assertEqual(handler.file_path, thumb_path)
+        self.assertIsNone(handler.error_status)
+
+    def test_watchlist_refresh_status_route_returns_json(self):
+        handler = DummyHandler("/api/watchlist/refresh-status")
+        payload = {"job": {"id": "refresh-1", "status": "running"}, "running": True}
+        with mock.patch.object(APP.WATCHLIST_REFRESH, "status", return_value=payload):
+            APP.Handler.do_GET(handler)
+        self.assertEqual(handler.json_payload, payload)
+        self.assertEqual(handler.json_status, HTTPStatus.OK)
+
+    def test_update_all_post_returns_accepted_for_new_job(self):
+        handler = DummyHandler("/api/watchlist/update-all", body=b"{}", content_length=2)
+        payload = {"message": "Watchlist refresh started.", "job": {"id": "refresh-1"}, "created": True}
+        with mock.patch.object(APP.WATCHLIST_REFRESH, "start", return_value=payload):
+            APP.Handler.do_POST(handler)
+        self.assertEqual(handler.json_payload, payload)
+        self.assertEqual(handler.json_status, HTTPStatus.ACCEPTED)
+
+
+class TemplateHelperTests(unittest.TestCase):
+    def test_page_links_marks_active_page(self):
+        markup = APP.render_page_links("config")
+        self.assertIn('class="page-link active" href="/config"', markup)
+        self.assertIn('class="page-link" href="/watchlist"', markup)
+
+    def test_sidebar_brand_can_attach_optional_badge_id(self):
+        markup = APP.render_sidebar_brand("Shared note", "Search", badge_id="serverBadge")
+        self.assertIn('id="serverBadge"', markup)
+        self.assertIn("Shared note", markup)
+
+
+if __name__ == "__main__":
+    unittest.main()