Compare commits

..
92 Commits
Author SHA1 Message Date
Assclaw 38169b881e Support original-language movie requests and fix repair dialog layout
Magent CI/CD / verify (push) Successful in 1m54s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Skipped
2026-09-11 18:05:18 +12:00
Assclaw 52c85daae3 Add reviewed duplicate account consolidation and prevent duplicate imports
Magent CI/CD / verify (push) Successful in 11m16s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m33s
2026-09-11 16:38:53 +12:00
Assclaw df6fe58278 Normalize portal kinds before checking feature access
Magent CI/CD / verify (push) Successful in 11m13s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m49s
2026-09-11 12:32:51 +12:00
Assclaw ec0a866ef3 Add user feature permissions and unified account management
Magent CI/CD / verify (push) Canceled after 1m19s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-11 12:31:25 +12:00
Assclaw e2be8b3872 Remove changelog link from user menu
Magent CI/CD / verify (push) Successful in 11m52s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 5m59s
2026-09-10 17:12:30 +12:00
Assclaw b286ca3c42 Let users email personal reports on demand
Magent CI/CD / verify (push) Successful in 11m6s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m40s
2026-09-10 16:15:39 +12:00
Assclaw 6e473fd0a7 Unify user management and add reviewed identity repairs
Magent CI/CD / verify (push) Canceled after 9m3s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-10 16:06:30 +12:00
Assclaw 9856c7fb90 Collapse request discovery after selecting a title
Magent CI/CD / verify (push) Successful in 11m0s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m11s
2026-09-10 15:47:50 +12:00
Assclaw df651eb312 Use admin quality defaults throughout the request pipeline
Magent CI/CD / verify (push) Successful in 10m49s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m6s
2026-09-10 15:20:23 +12:00
Assclaw 77f2c1b42a Add reviewed resolution for missing user identity links
Magent CI/CD / verify (push) Successful in 10m59s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m29s
2026-09-10 15:02:04 +12:00
Assclaw b310e86f80 Align user directory headings with account rows
Magent CI/CD / verify (push) Successful in 11m6s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m47s
2026-09-10 13:46:11 +12:00
Assclaw 747a330b19 Center the user directory and group management tools in a dialog
Magent CI/CD / verify (push) Canceled after 9m23s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-10 13:36:37 +12:00
Assclaw b0f8c89db7 Add Grizzlyflix newsletters with curated editions and weekly delivery
Magent CI/CD / verify (push) Successful in 10m54s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m14s
2026-09-09 23:42:52 +12:00
Assclaw e014baadc3 Preserve monthly recap destinations through sign-in
Magent CI/CD / verify (push) Successful in 10m36s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m26s
2026-09-09 22:42:58 +12:00
Assclaw 1979e02cde Add opt-in monthly email recaps with scheduling and delivery history
Magent CI/CD / verify (push) Canceled after 3m55s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-09 22:39:22 +12:00
Assclaw 333a799e21 Add personal monthly viewing reports and CSV exports
Magent CI/CD / verify (push) Successful in 10m58s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m42s
2026-09-09 16:25:35 +12:00
Assclaw 437836243c Remove page numbering from navigation and page links
Magent CI/CD / verify (push) Successful in 10m28s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m0s
2026-09-08 16:36:38 +12:00
Assclaw e7e4c9eff3 Fix viewing-history artwork and add transcode playback metrics
Magent CI/CD / verify (push) Successful in 10m43s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m9s
2026-09-08 16:15:16 +12:00
Assclaw 12611a9819 Add admin review and confirmation of cross-service user IDs
Magent CI/CD / verify (push) Successful in 10m29s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 54s
2026-09-08 15:41:45 +12:00
Assclaw 2976145dd8 Add private Jellystat viewing stats to Magent beta
Magent CI/CD / verify (push) Successful in 10m31s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m16s
2026-09-08 11:25:05 +12:00
Assclaw a3b5759708 Enforce recipient-bound single-use invites and fix issue card layout; clean release tooling
Magent CI/CD / verify (push) Successful in 10m31s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Skipped
2026-09-07 19:59:51 +12:00
Assclaw 13edcb8136 Prevent non-admin issue responses exposing account identities
Magent CI/CD / verify (push) Successful in 10m34s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Skipped
2026-09-07 19:45:19 +12:00
Assclaw 131b5fc5c7 Add private Prometheus API metrics and Grafana performance dashboard
Magent CI/CD / verify (push) Successful in 10m24s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Skipped
2026-09-07 19:23:35 +12:00
Assclaw bd1f2cb1cb Make request status badges prominent and easier to scan
Magent CI/CD / verify (push) Successful in 10m32s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Skipped
2026-09-07 17:31:16 +12:00
Assclaw edca300d27 Add bulk enable invite access to user directory
Magent CI/CD / verify (push) Successful in 10m27s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Skipped
2026-09-07 16:19:12 +12:00
Assclaw 4034a8f72a Add post-login welcome and friendly how-it-works guide
Magent CI/CD / verify (push) Successful in 10m26s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Skipped
2026-09-07 16:08:37 +12:00
Assclaw 0637860b95 Show imported Seerr accounts while preserving linked-user deduplication
Magent CI/CD / verify (push) Successful in 10m42s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Skipped
2026-09-07 15:20:10 +12:00
Assclaw 697fc235ee Serve resilient launch cover independently of application upstream
Magent CI/CD / verify (push) Failing after 50m56s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Skipped
2026-09-07 13:46:09 +12:00
Assclaw 62ee07f92b Record verified coming-soon cutover and rollback procedure
Magent CI/CD / verify (push) Canceled after 14m20s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-07 13:31:44 +12:00
Assclaw 458ef53f47 Exclude production bootstrap credentials from source and build context
Magent CI/CD / verify (push) Canceled after 1m56s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-07 13:29:43 +12:00
Assclaw c2685f43a7 Prepare clean production setup and coming-soon cover
Magent CI/CD / verify (push) Successful in 11m48s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Skipped
2026-09-07 13:14:22 +12:00
Assclaw 98d8b197a9 Link invite trace accounts to admin user profiles
Magent CI/CD / verify (push) Successful in 10m26s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m54s
2026-09-06 22:47:16 +12:00
Assclaw 625f9ad7f0 Discover Sonarr episode downloads and keep live tracker updating
Magent CI/CD / verify (push) Canceled after 1m9s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-06 22:46:01 +12:00
Assclaw bd668715a3 Keep diagnostic health cards compact beside database details
Magent CI/CD / verify (push) Canceled after 6m1s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-06 22:39:59 +12:00
Assclaw 212ac560ec Tidy invite delivery choices and align email fields
Magent CI/CD / verify (push) Canceled after 1m33s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-06 22:38:17 +12:00
Assclaw a32928b1c5 Make verified repair acceptance prominent and simplify confirmation email
Magent CI/CD / verify (push) Canceled after 5m52s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-06 22:32:21 +12:00
Assclaw 74c49fad5b Compact request activity into a live summary and modal history 2026-09-06 22:20:33 +12:00
Assclaw 009bb35032 Track repair collection cycles and reconcile request availability
Magent CI/CD / verify (push) Successful in 10m47s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m49s
2026-09-06 22:06:30 +12:00
Assclaw 8d720de500 Streamline issue wizard and make selections explicit
Magent CI/CD / verify (push) Successful in 10m37s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 2m2s
2026-09-06 21:44:44 +12:00
Assclaw 1851fa9753 Show actual collector search activity in request pipeline
Magent CI/CD / verify (push) Successful in 10m55s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 2m4s
2026-09-06 21:14:42 +12:00
Assclaw dec1dd902c Unify page layouts, compact headers and shared UI styling 2026-09-06 19:52:39 +12:00
Assclaw 4d67567d4c Simplify navigation and modernize profile and sign-in
Magent CI/CD / verify (push) Successful in 10m55s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m58s
2026-09-06 18:24:18 +12:00
Assclaw b5e4c57e93 Simplify settings workspace and fix responsive admin controls
Magent CI/CD / verify (push) Successful in 11m34s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 17s
2026-09-06 17:42:28 +12:00
Assclaw f65e1b114c Apply Stitch media operations redesign
Magent CI/CD / verify (push) Successful in 10m46s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 14s
2026-09-01 22:58:48 +12:00
Assclaw 32bfa20ab7 Add wrong download repair option
Magent CI/CD / verify (push) Successful in 10m31s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 22s
2026-09-01 22:42:07 +12:00
Assclaw 0b59289a2e Automate repair completion confirmation
Magent CI/CD / verify (push) Canceled after 4m31s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-01 22:37:23 +12:00
Assclaw ded794a819 Add self-service profile email management
Magent CI/CD / verify (push) Canceled after 10m22s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-01 22:26:53 +12:00
Assclaw c6d449dc17 Move invite flow to client navigation 2026-09-01 22:20:48 +12:00
Assclaw 06d944c9d9 Add admin user email management
Magent CI/CD / verify (push) Successful in 10m59s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 20s
2026-09-01 22:09:20 +12:00
Assclaw 0ac53b7f59 Add quality-aware release picker modal
Magent CI/CD / verify (push) Canceled after 7m6s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-01 22:02:22 +12:00
Assclaw 2dbe11e6bc Rework invite creation as guided flow
Magent CI/CD / verify (push) Canceled after 8m52s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-01 21:53:27 +12:00
Assclaw 3aac40ba0f Keep issue dialog above site navigation
Magent CI/CD / verify (push) Successful in 10m42s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 16s
2026-09-01 21:33:02 +12:00
Assclaw b6c48a0be7 Clarify completed repair handoff
Magent CI/CD / verify (push) Canceled after 9m53s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-01 21:22:55 +12:00
Assclaw 3fc52f70c7 Show live repair progress on requests
Magent CI/CD / verify (push) Canceled after 4m53s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-01 21:17:50 +12:00
Assclaw 6391fbfd81 Monitor media before issue repair searches
Magent CI/CD / verify (push) Successful in 10m43s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 19s
2026-09-01 20:50:16 +12:00
Assclaw 0ed22dd315 Separate invites from profile navigation
Magent CI/CD / verify (push) Successful in 10m56s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 15s
2026-09-01 19:51:02 +12:00
Assclaw 7ed0f4b103 Add global admin user-view preview
Magent CI/CD / verify (push) Successful in 10m37s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 22s
2026-09-01 17:04:25 +12:00
Assclaw b3c41f6dea Add safe user preview for invites
Magent CI/CD / verify (push) Successful in 10m52s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 17s
2026-09-01 16:51:32 +12:00
Assclaw 976d24217b Fix top-level invites navigation
Magent CI/CD / verify (push) Successful in 10m26s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 15s
2026-09-01 15:28:28 +12:00
Assclaw c49a149cfd Refresh invite operations workspace
Magent CI/CD / verify (push) Successful in 10m26s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 17s
2026-09-01 15:12:37 +12:00
Assclaw 5de14b1cb7 Add admin issue deletion workflow
Magent CI/CD / verify (push) Successful in 10m47s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 22s
2026-09-01 14:58:58 +12:00
Assclaw 16876e1cf0 Advance issue status from repair actions
Magent CI/CD / verify (push) Successful in 10m30s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Canceled after 5s
2026-09-01 14:48:08 +12:00
Assclaw c7a56f2525 Add issue workflow progress tracker
Magent CI/CD / verify (push) Canceled after 3m55s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-09-01 14:44:17 +12:00
Assclaw 87a4aae246 Move issue history into sidebar modal
Magent CI/CD / verify (push) Successful in 10m35s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 16s
2026-09-01 14:28:27 +12:00
Assclaw e58614305e Streamline issue reporting and automate repairs
Magent CI/CD / verify (push) Successful in 10m41s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 19s
2026-09-01 13:44:13 +12:00
Assclaw 2adbed7259 Add issue resolution confirmation workflow
Magent CI/CD / verify (push) Successful in 10m35s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 12s
2026-09-01 11:40:43 +12:00
Assclaw 393b8c2a88 Add prominent watch button to ready requests
Magent CI/CD / verify (push) Successful in 10m37s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 15s
2026-09-01 11:16:43 +12:00
Assclaw b0eff9ffcf Clarify and live-refresh media indexing state
Magent CI/CD / verify (push) Successful in 10m43s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 26s
2026-09-01 09:41:15 +12:00
Assclaw ae6cee5d0b Explain remote service responses in plain English
Magent CI/CD / verify (push) Successful in 10m20s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 15s
2026-08-31 22:28:56 +12:00
Assclaw 906a777b95 Fix issue search result card layout
Magent CI/CD / verify (push) Successful in 10m42s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 21s
2026-08-31 22:14:33 +12:00
Assclaw ec8145a58a Add exact media replacement from issues
Magent CI/CD / verify (push) Canceled after 6m9s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-08-31 22:05:11 +12:00
Assclaw f8770cb44a Build guided media issue workflow
Magent CI/CD / verify (push) Successful in 10m42s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 19s
2026-08-31 21:46:43 +12:00
Assclaw 0e04d219a0 Reorganize admin configuration sections
Magent CI/CD / verify (push) Successful in 10m37s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 16s
2026-08-31 16:32:24 +12:00
Assclaw 3402e53c31 Protect advanced request diagnostics for non-admins
Magent CI/CD / verify (push) Successful in 11m8s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 19s
2026-08-31 16:09:17 +12:00
Assclaw ecf9b230c1 Separate new requests from request tracking
Magent CI/CD / verify (push) Canceled after 7m35s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-08-31 16:02:00 +12:00
Assclaw 8f810e0f36 Isolate sparse request recheck regression test
Magent CI/CD / verify (push) Successful in 10m11s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 14s
2026-08-31 15:23:49 +12:00
Assclaw 82d87d968e Fix sparse request metadata hydration 2026-08-31 15:21:47 +12:00
Assclaw 372f4a1bfc Fix Seerr transport payload forwarding
Magent CI/CD / verify (push) Canceled after 9m15s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-08-31 15:14:32 +12:00
Assclaw 547ed754e6 Handle Seerr CSRF on request creation
Magent CI/CD / verify (push) Canceled after 2m30s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-08-31 15:11:57 +12:00
Assclaw a55369190b Move request service icons beside card content
Magent CI/CD / verify (push) Canceled after 6m44s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-08-31 15:05:01 +12:00
Assclaw ee81749b43 Add Sonarr and Radarr request icons
Magent CI/CD / verify (push) Canceled after 5m33s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-08-31 14:59:23 +12:00
Assclaw 9dfea25d56 Redesign the request portal flow
Magent CI/CD / verify (push) Canceled after 5m59s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-08-31 14:53:15 +12:00
Assclaw 963506d098 Show live feedback for remote request actions
Magent CI/CD / verify (push) Successful in 10m51s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 15s
2026-08-30 21:23:20 +12:00
Assclaw 02245d365e Add live request pipeline recheck
Magent CI/CD / verify (push) Successful in 11m6s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 1m56s
2026-08-30 19:50:09 +12:00
Assclaw 2cbd9fe73f Hide landing page live update indicator
Magent CI/CD / verify (push) Successful in 11m5s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 22s
2026-08-30 19:24:56 +12:00
Assclaw 9db32481bd Resolve Arr metadata before adding media
Magent CI/CD / verify (push) Successful in 11m12s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 14s
2026-08-30 16:32:14 +12:00
Assclaw 3815dfea60 Move fleet health into admin settings and tidy landing page
Magent CI/CD / verify (push) Successful in 10m46s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 18s
2026-08-29 22:53:48 +12:00
Assclaw c073581639 Mark available downloads complete
Magent CI/CD / verify (push) Successful in 10m40s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 14s
2026-08-29 22:41:50 +12:00
Assclaw 06a000bb06 Balance manual results across missing seasons
Magent CI/CD / verify (push) Successful in 10m41s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 16s
2026-08-29 22:17:39 +12:00
Assclaw 391cd41d71 Route manual downloads through Arr collectors
Magent CI/CD / verify (push) Canceled after 3m43s
Magent CI/CD / deploy-prod (push) Canceled after 0s
Magent CI/CD / deploy-beta (push) Canceled after 0s
2026-08-29 22:13:46 +12:00
Assclaw 96fc43365f Add live download progress updates
Magent CI/CD / verify (push) Successful in 12m15s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 21s
2026-08-29 20:52:49 +12:00
Assclaw 655e2f8158 Start Magent beta overhaul
Magent CI/CD / verify (push) Successful in 13m40s
Magent CI/CD / deploy-prod (push) Skipped
Magent CI/CD / deploy-beta (push) Successful in 19s
2026-08-29 20:27:17 +12:00
275 changed files with 15572 additions and 30186 deletions
+1
View File
@@ -0,0 +1 @@
0803262237
+17 -41
View File
@@ -1,44 +1,20 @@
# Release builds accept only application sources and explicit build inputs.
# Local configuration, databases, backups, Git metadata and tool caches must
# never be sent to the builder, even if new directories are added to the repo.
**
!Dockerfile
!.dockerignore
!LICENSE
!backend/
!backend/requirements.txt
!backend/app/
!backend/app/**
!frontend/
!frontend/package.json
!frontend/package-lock.json
!frontend/next-env.d.ts
!frontend/next.config.js
!frontend/proxy.ts
!frontend/tsconfig.json
!frontend/app/
!frontend/app/**
!frontend/public/
!frontend/public/**
!docker/
!docker/supervisord.conf
!docker/requirements-runtime.txt
!data/
.git
.env
.env.*
.venv/
**/.pytest_cache/
stitch_magent_media_operations_redesign/
*.tar
*.tar.gz
*.zip
bootstrap-admin.json
release.tar
*.log
data/*
!data/branding/
!data/branding/**
# Defense in depth for accidental private/generated files under allowed paths.
**/.env
**/.env.*
**/__pycache__
frontend/node_modules/
frontend/.next/
backend/__pycache__/
**/__pycache__/
**/*.pyc
**/*.log
**/*.db
**/*.db-*
**/*.sqlite
**/*.sqlite3
**/bootstrap-admin.json
**/bootstrap-secrets.json
**/.magent-secrets-*
**/node_modules
**/.next
-37
View File
@@ -1,37 +0,0 @@
# Copy to .env for a fresh install; never replace an existing deployment's keys.
# See docs/PUBLIC_RELEASE.md. The localhost settings below are for local HTTP only.
# Never deploy the example secret placeholders.
APP_NAME=Magent
# Public Docker Hub template: choose a published prod-<commit> tag or sha256 digest.
# Intentionally no default: do not silently pull a mutable or incompatible image.
MAGENT_IMAGE=
MAGENT_BIND_ADDRESS=127.0.0.1
MAGENT_HTTP_PORT=3000
# For public hosting set BOTH URLs to your exact HTTPS origin (no trailing slash),
# for example https://magent.example.com, and AUTH_COOKIE_SECURE=true below.
CORS_ALLOW_ORIGIN=http://localhost:3000
MAGENT_APPLICATION_URL=http://localhost:3000
# Backend address is internal to the combined container, not a browser endpoint.
MAGENT_API_URL=http://127.0.0.1:8000
SQLITE_PATH=/app/data/magent.db
LOG_FILE=/app/data/magent.log
LOG_FORMAT=text
# Generate independent values as documented in docs/PUBLIC_RELEASE.md.
# Keep both unchanged when upgrading or restoring an offline data-volume backup.
JWT_SECRET=replace-with-at-least-32-random-characters
SETTINGS_ENCRYPTION_KEY=replace-with-a-valid-fernet-key
ADMIN_USERNAME=admin
# Recommended fresh install: generate a separate random setup token. Open /setup
# to create the administrator and connect your apps; remove this after finishing.
SETUP_TOKEN=replace-with-a-separate-random-setup-token
# Alternatively pre-create the first admin with a unique password (12+ chars).
# Leave blank to create the account using the setup wizard and SETUP_TOKEN.
ADMIN_PASSWORD=
# false is ONLY for local HTTP; public HTTPS deployments must use true.
AUTH_COOKIE_SECURE=false
AUTH_COOKIE_SAMESITE=strict
API_DOCS_ENABLED=false
+104
View File
@@ -0,0 +1,104 @@
name: Magent CI/CD
on:
push:
branches:
- beta
- main
- prod
workflow_dispatch:
concurrency:
group: magent-${{ github.ref }}
cancel-in-progress: true
jobs:
verify:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "24"
# Gitea cache restore/save stalls here; npm ci takes about 15 seconds.
- name: Install frontend dependencies
working-directory: frontend
run: npm ci
- name: Run backend quality gate
run: bash scripts/ci_backend_quality_gate.sh
- name: Build frontend
working-directory: frontend
run: npm run build
deploy-prod:
if: github.ref_name == 'prod'
needs: verify
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Configure SSH key
env:
PROD_SSH_PRIVATE_KEY: ${{ secrets.PROD_SSH_PRIVATE_KEY }}
PROD_SSH_KNOWN_HOSTS: ${{ secrets.PROD_SSH_KNOWN_HOSTS }}
run: |
set -euo pipefail
mkdir -p ~/.ssh
chmod 700 ~/.ssh
printf '%s' "$PROD_SSH_PRIVATE_KEY" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
if [ -n "${PROD_SSH_KNOWN_HOSTS:-}" ]; then
printf '%s\n' "$PROD_SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
chmod 644 ~/.ssh/known_hosts
fi
- name: Deploy to AMS-DEV01
env:
DEPLOY_HOST: ${{ secrets.PROD_SSH_HOST }}
DEPLOY_USER: ${{ secrets.PROD_SSH_USER }}
DEPLOY_PATH: ${{ secrets.PROD_DEPLOY_PATH }}
DEPLOY_SSH_OPTS: -o StrictHostKeyChecking=accept-new
run: bash scripts/deploy_ams_dev01.sh
deploy-beta:
if: github.ref_name == 'beta'
needs: verify
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Configure SSH key
env:
PROD_SSH_PRIVATE_KEY: ${{ secrets.PROD_SSH_PRIVATE_KEY }}
PROD_SSH_KNOWN_HOSTS: ${{ secrets.PROD_SSH_KNOWN_HOSTS }}
run: |
set -euo pipefail
mkdir -p ~/.ssh
chmod 700 ~/.ssh
printf '%s' "$PROD_SSH_PRIVATE_KEY" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
if [ -n "${PROD_SSH_KNOWN_HOSTS:-}" ]; then
printf '%s\n' "$PROD_SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
chmod 644 ~/.ssh/known_hosts
fi
- name: Deploy beta to AMS-DEV01
env:
DEPLOY_HOST: ${{ secrets.PROD_SSH_HOST }}
DEPLOY_USER: ${{ secrets.PROD_SSH_USER }}
PROD_DEPLOY_PATH: ${{ secrets.PROD_DEPLOY_PATH }}
DEPLOY_SSH_OPTS: -o StrictHostKeyChecking=accept-new
run: bash scripts/deploy_beta_ams_dev01.sh
+10 -17
View File
@@ -1,27 +1,20 @@
.env
.env.*
!.env.example
bootstrap-admin.json
.venv/
data/
!data/branding/
!data/branding/**
backend/__pycache__/
**/__pycache__/
*.pyc
**/.pytest_cache/
.coverage
coverage.xml
htmlcov/
backend/.pytest_cache/
frontend/node_modules/
frontend/.next/
*.tsbuildinfo
*.log
*.db
*.db-*
*.sqlite*
*.magent-backup
bootstrap-admin.json
bootstrap-secrets.json
.magent-secrets-*
data/*
!data/branding/
backend/data/
**/.pytest_cache/
.env.*
!.env.example
!.env.*.example
*.tar
*.tar.gz
*.zip
+22 -65
View File
@@ -1,12 +1,8 @@
FROM node:24-alpine@sha256:ebfe2f90462722a7a4de65e91990e97fe0d401c70e0e762c5b53302f905ec1c1 AS frontend-builder
FROM node:24-slim AS frontend-builder
WORKDIR /frontend
# GNU cp is needed only to collect third-party notices in the builder.
RUN apk add --no-cache coreutils
ENV NODE_ENV=production \
NEXT_TELEMETRY_DISABLED=1 \
BACKEND_INTERNAL_URL=http://127.0.0.1:8000 \
NEXT_PUBLIC_API_BASE=/api
@@ -17,80 +13,41 @@ COPY frontend/app ./app
COPY frontend/public ./public
COPY frontend/next-env.d.ts ./next-env.d.ts
COPY frontend/next.config.js ./next.config.js
COPY frontend/proxy.ts ./proxy.ts
COPY frontend/tsconfig.json ./tsconfig.json
# Keep dependency notices outside the traced bundle: file tracing deliberately
# omits many license files that still need to accompany redistributed packages.
RUN npm run build \
&& npm prune --omit=dev \
&& mkdir /licenses \
&& npm ls --omit=dev --all --json > /licenses/dependencies.json \
&& find node_modules -type f \
\( -iname 'license*' -o -iname 'copying*' -o -iname 'notice*' -o -iname 'copyright*' \) \
-exec cp --parents -t /licenses {} +
RUN npm run build
FROM python:3.14-alpine@sha256:016508ba505da24f7139765bc4bb669df4e88eb2f12eeadd571bf2f88d7533df AS runtime
FROM python:3.14-slim
WORKDIR /app
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
MAGENT_MANAGED_SECRETS=auto \
SQLITE_PATH=/app/data/magent.db \
API_DOCS_ENABLED=false \
NODE_ENV=production \
NEXT_TELEMETRY_DISABLED=1
NODE_ENV=production
# Keep curl for existing deployments that override the image healthcheck.
# Copy only Node's runtime binary: npm, headers and the NodeSource installer
# are build tools, not dependencies of the standalone frontend server.
RUN apk upgrade --no-cache \
&& apk add --no-cache curl libstdc++
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl gnupg supervisor \
&& curl -fsSL https://deb.nodesource.com/setup_24.x | bash - \
&& apt-get install -y --no-install-recommends nodejs \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
COPY --from=frontend-builder /usr/local/bin/node /usr/local/bin/node
COPY --from=frontend-builder /usr/local/LICENSE /usr/local/share/doc/nodejs/LICENSE
RUN node --version
COPY backend/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
ARG MAGENT_UID=1000
ARG MAGENT_GID=1000
RUN addgroup -g ${MAGENT_GID} magent \
&& adduser -D -u ${MAGENT_UID} -G magent -s /sbin/nologin magent \
&& install -d -o magent -g magent -m 0700 /app/data \
&& install -d -o magent -g magent -m 0755 /app/frontend/.next/cache
COPY backend/app ./app
COPY data/branding /app/data/branding
COPY backend/requirements.txt docker/requirements-runtime.txt /tmp/requirements/
RUN pip install --no-cache-dir --no-compile \
-r /tmp/requirements/requirements.txt \
-r /tmp/requirements/requirements-runtime.txt \
&& pip uninstall -y pip \
&& rm /tmp/requirements/requirements.txt /tmp/requirements/requirements-runtime.txt \
&& rmdir /tmp/requirements
COPY --chown=magent:magent backend/app ./app
COPY --chown=magent:magent data/branding /app/data/branding
# Next's traced standalone output excludes the full dev/build dependency tree.
COPY --chown=magent:magent --from=frontend-builder /frontend/.next/standalone /app/frontend
COPY --chown=magent:magent --from=frontend-builder /frontend/.next/static /app/frontend/.next/static
COPY --chown=magent:magent --from=frontend-builder /frontend/public /app/frontend/public
COPY --from=frontend-builder /frontend/.next /app/frontend/.next
COPY --from=frontend-builder /frontend/public /app/frontend/public
COPY --from=frontend-builder /frontend/node_modules /app/frontend/node_modules
COPY --from=frontend-builder /frontend/package.json /app/frontend/package.json
COPY --from=frontend-builder /frontend/next.config.js /app/frontend/next.config.js
COPY --from=frontend-builder /frontend/next-env.d.ts /app/frontend/next-env.d.ts
COPY --from=frontend-builder /frontend/tsconfig.json /app/frontend/tsconfig.json
COPY docker/supervisord.conf /etc/supervisor/conf.d/magent.conf
COPY LICENSE /usr/share/licenses/magent/LICENSE
COPY --from=frontend-builder /licenses /usr/share/licenses/magent/frontend
LABEL org.opencontainers.image.title="Magent" \
org.opencontainers.image.description="Self-hosted media requests, issues and viewing insights" \
org.opencontainers.image.licenses="MIT"
USER magent:magent
EXPOSE 3000 8000
HEALTHCHECK --interval=30s --timeout=5s --start-period=45s --retries=3 \
CMD curl --fail --silent --show-error --max-time 2 http://127.0.0.1:8000/health >/dev/null \
&& curl --fail --silent --show-error --max-time 2 http://127.0.0.1:3000/login >/dev/null \
|| exit 1
ENTRYPOINT ["python", "-m", "app.container_bootstrap"]
CMD ["/usr/local/bin/supervisord", "-c", "/etc/supervisor/conf.d/magent.conf"]
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/magent.conf"]
-21
View File
@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2026 Magent contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+61
View File
@@ -0,0 +1,61 @@
# Production
Magent runs as one combined frontend/API image: `rephl3xnz/magent`.
The root `Dockerfile` is the supported build entry point. Source releases come
from `main`; use `prod-<short-commit>` tags to identify an exact release.
## Live deployment
- Host: GRZ-DKR01 (`10.30.1.81`).
- Container and Compose service: `magent`; Compose project: `arrstack`.
- Compose file: `/home/zak/grizzlystack/arrstack/docker-compose.yml`.
- Persistent data: `/home/zak/grizzlystack/arrstack/magent/data``/app/data`.
- Public URL: `https://magent.grizzlyflix.co.nz`.
- Caddy runs on AMS-CAD01 and proxies production to `10.30.1.81:3002`.
- Beta remains separate on AMS-DEV01. Do not overwrite it or change its routes.
## Release checklist
1. Run the backend tests and frontend production build. Review only the intended
changes, then commit and push `main`.
2. Build from a clean source export using the root Dockerfile. Never include
`.env`, databases or bootstrap credentials in the build context.
3. Publish `rephl3xnz/magent:prod-<short-commit>` and `:latest` to Docker Hub.
Confirm their digests match.
4. Pull the new image before stopping production. Keep the old image under a
rollback tag and back up the current Compose configuration.
5. Briefly stop only `magent`, then back up its complete data directory so SQLite
and its WAL files are consistent. Protect backups: they contain private data.
6. Recreate only this service with `docker compose -p arrstack -f
/home/zak/grizzlystack/arrstack/docker-compose.yml up -d --no-deps --no-build magent`.
Confirm that Compose selects the intended image before running this command.
7. Check container health, the API `/health` endpoint, public login, the changed
feature, database integrity and account counts. Do not trigger bulk permission
changes, email sends or user imports as a deployment smoke test.
For rollback, select the saved image and recreate only Magent. Restore data only
if needed; doing so can discard activity since the backup. Never restore a whole
shared Compose or Caddy file without checking for unrelated changes first.
## Build metadata
`.build_number` and `backend/app/build_info.py` currently hold the same legacy
display build number as the frontend package files. `.env` should have exactly
one `BUILD_NUMBER` assignment, not a history of previous releases. Docker release
tags identify the deployed source commit independently of this display value.
`scripts/process1.ps1` is a local development workflow: it updates metadata,
runs tests, rebuilds local Docker, and can commit changes/send Discord messages.
It is **not** the production deployment command. Its build-number helper can be
tested safely with `powershell -File scripts/test_env_build_number.ps1`.
## Fresh instances and historical notes
`scripts/prepare_production_settings.py` exports only allowlisted connection and
SMTP settings for a fresh instance. Do not use it to replace a live database.
`docker-compose.production.yml` is the separate fresh-instance template, not the
live GRZ-DKR01 Compose file. `docker-compose.hub.yml` is the generic Docker Hub
template; `docker-compose.yml` builds locally; `docker-compose.beta.yml` serves beta.
The temporary AMS-DEV01 setup and coming-soon cutover are retained under
[archived cutover notes](docs/archive/production-cutover-2026-09-07.md).
+162 -93
View File
@@ -1,122 +1,191 @@
# Magent
Self-hosted media requests, viewing stats and issue management for Jellyfin,
Seerr, Sonarr, Radarr and related services. Magent combines a Python/FastAPI API,
a Next.js frontend and SQLite. Run the prebuilt non-root container or install
the Python and Node.js services directly—Portainer is optional.
Magent is a friendly, AI-assisted request tracker for Seerr + Arr services. It shows a clear timeline of where a request is stuck, explains what is happening in plain English, and offers safe actions to help fix issues.
## Install
## How it works
Start with the [installation guide](docs/INSTALLATION.md) to choose a method:
1) Requests are pulled from Seerr and stored locally.
2) Magent joins that request to Sonarr/Radarr, Prowlarr, qBittorrent, and Jellyfin using TMDB/TVDB IDs and download hashes.
3) A state engine normalizes noisy service statuses into a simple, user-friendly state.
4) The UI renders a timeline and a central status box for each request.
5) Optional AI triage summarizes the likely cause and safest next steps.
| Method | Instructions |
| --- | --- |
| Docker Compose or `docker run` | [Docker installation](docs/DOCKER.md) |
| Portainer | [Single-file stack](docs/PORTAINER.md) |
| Linux without Docker | [Native install and systemd services](docs/NATIVE_INSTALL.md) |
| Windows/macOS/Linux foreground | [Source installation and development](docs/LOCAL_DEVELOPMENT.md) |
## Core features
For a fresh **Docker Compose** install, download [compose.yml](compose.yml) into
its own deployment directory and run:
- Request search by title/year or request ID.
- Recent requests list with posters and status.
- Timeline view across Seerr, Arr, Prowlarr, qBittorrent, Jellyfin.
- Central status box with clear reason + next steps.
- Safe action buttons (search, resume, re-add, etc.).
- Admin settings for service URLs, API keys, profiles, and root folders.
- Health status for each service in the pipeline.
- Cache and sync controls (full sync, delta sync, scheduled syncs).
- Local database for speed and audit history.
- Users and access control (admin vs user, block access).
- Local account password changes via "My profile".
- Personal viewing stats from Jellystat: minutes, movies, episodes, streaks, and recent plays alongside requests. See [Jellystat setup](docs/jellystat-integration.md).
- Admin review and confirmation of account IDs across Jellyfin, Seerr, Jellystat and Magent. See [user identities](docs/user-identities.md).
- Docker-first deployment for easy hosting.
```sh
docker compose -f compose.yml -p magent pull
docker compose -f compose.yml -p magent up -d --wait --wait-timeout 120
docker compose -f compose.yml -p magent ps
docker compose -f compose.yml -p magent exec --user magent magent python -m app.container_bootstrap setup-token
## Quick start (Docker - primary)
Docker is the recommended way to run Magent. It includes the backend and frontend with sane defaults.
```bash
docker compose up --build
```
Keep that directory/project name and its persistent volume unchanged for updates.
This uses `rephl3xnz/magent:latest` with no environment variables, local source
build or Dockerfile. Portainer users can paste the same file into a Docker
Standalone stack instead. Native users follow their separate guide for explicit
environment configuration and service management.
Then open:
**Image availability:** the managed-install image is published on Docker Hub.
Only Linux/amd64 has been validated. `latest` is mutable; record the resolved
image digest before updating, or pin an immutable release tag.
- Frontend: http://localhost:3000
- Backend: http://localhost:8000
1. Wait for the container to become healthy and retrieve the token using the
command above (or the Portainer console instructions in its guide).
2. Open the Docker host's address on port 3000. Confirm the browser-facing URL
in setup and use the token to create the first administrator.
The **Get setup token** button shows the console instructions and lets you
copy the command; it never reveals the token to public visitors.
3. Connect your apps, choose preferences and finish setup. Optional apps can
be skipped. Save an encrypted backup afterwards.
### Docker setup steps
Keep the Compose security block unchanged. Database storage is fixed at
`/app/data/magent.db` and API docs are disabled in managed installs. CORS and
cookie security follow the confirmed URL. Use HTTPS before public access.
1) Create `.env` with your service URLs and API keys.
2) Run `docker compose up --build`.
3) Log in at http://localhost:3000.
4) Visit Settings to confirm service health.
See [all installation methods](docs/INSTALLATION.md),
[all environment options](docs/ENVIRONMENT.md),
[backup and restore](docs/installation-and-recovery.md) and
[advanced installation/upgrades](docs/PUBLIC_RELEASE.md).
Existing installations must retain their original data volume and signing/
encryption keys; this fresh-install template is not an automatic migration.
### Docker environment variables (sample)
## Build and test
The source tree contains everything needed to build the application:
```sh
docker compose -f compose.yml -f compose.build.yml -p magent up -d --build
```bash
JELLYSEERR_URL="http://localhost:5055"
JELLYSEERR_API_KEY="..."
SONARR_URL="http://localhost:8989"
SONARR_API_KEY="..."
SONARR_QUALITY_PROFILE_ID="1"
SONARR_ROOT_FOLDER="/tv"
RADARR_URL="http://localhost:7878"
RADARR_API_KEY="..."
RADARR_QUALITY_PROFILE_ID="1"
RADARR_ROOT_FOLDER="/movies"
PROWLARR_URL="http://localhost:9696"
PROWLARR_API_KEY="..."
QBIT_URL="http://localhost:8080"
QBIT_USERNAME="..."
QBIT_PASSWORD="..."
SQLITE_PATH="data/magent.db"
JWT_SECRET="replace-with-a-long-random-secret"
JWT_EXP_MINUTES="720"
ADMIN_USERNAME="set-a-real-admin-username"
ADMIN_PASSWORD="set-a-long-unique-admin-password"
```
For a disposable verification run, without touching an existing installation:
## Screenshots
```sh
docker build -t magent:review .
bash scripts/ci_container_smoke.sh magent:review
MAGENT_SMOKE_MANAGED=true bash scripts/ci_container_smoke.sh magent:review
```
Add screenshots here once available:
Unit checks require Python 3.14 and Node 24:
- `docs/screenshots/home.png`
- `docs/screenshots/request-timeline.png`
- `docs/screenshots/settings.png`
- `docs/screenshots/profile.png`
```sh
## Local development (secondary)
Use this only when you need to modify code locally.
### Backend (FastAPI)
```bash
cd backend
python -m venv .venv
. .venv/bin/activate
pip install -r backend/requirements-dev.txt
python -m unittest discover -s backend/tests -p 'test_*.py'
python scripts/check_environment_docs.py
cd frontend
npm ci
npm test
npm run lint
npm run format:check
npm run typecheck
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000
```
On Windows, activate `.venv\Scripts\Activate.ps1` instead. Do not point tests
at live services or use production credentials.
Environment variables (sample):
## How it is organised
```bash
$env:JELLYSEERR_URL="http://localhost:5055"
$env:JELLYSEERR_API_KEY="..."
$env:SONARR_URL="http://localhost:8989"
$env:SONARR_API_KEY="..."
$env:SONARR_QUALITY_PROFILE_ID="1"
$env:SONARR_ROOT_FOLDER="/tv"
$env:RADARR_URL="http://localhost:7878"
$env:RADARR_API_KEY="..."
$env:RADARR_QUALITY_PROFILE_ID="1"
$env:RADARR_ROOT_FOLDER="/movies"
$env:PROWLARR_URL="http://localhost:9696"
$env:PROWLARR_API_KEY="..."
$env:QBIT_URL="http://localhost:8080"
$env:QBIT_USERNAME="..."
$env:QBIT_PASSWORD="..."
$env:SQLITE_PATH="data/magent.db"
$env:JWT_SECRET="replace-with-a-long-random-secret"
$env:JWT_EXP_MINUTES="720"
$env:ADMIN_USERNAME="set-a-real-admin-username"
$env:ADMIN_PASSWORD="set-a-long-unique-admin-password"
```
- `backend/app/routers/`: authenticated API endpoints and administration.
- `backend/app/clients/`: media-service clients; `services/`: request states,
synchronisation, notifications, setup and encrypted backups.
- `backend/app/db.py` and `schema_migrations.py`: SQLite persistence/migrations.
- `frontend/app/`: pages and shared interface components; `frontend/proxy.ts`:
browser security headers and request nonces.
- `backend/tests/` and frontend `*.test.*`: synthetic regression tests.
- `Dockerfile` and `docker/`: multi-stage build and process supervision.
- `compose.yml`: prebuilt-image install; `compose.build.yml`: source override.
- `deploy/native/`: example Linux service units and backend/frontend configuration.
### Frontend (Next.js)
Requests are cached from Seerr, joined to collector/download/library evidence,
normalised into a user-facing state and displayed by the frontend. App settings
are stored in SQLite; sensitive settings are encrypted with installation-specific
keys. Integrations are optional and are configured through the setup wizard.
```bash
cd frontend
npm install
npm run dev
```
This `release` branch intentionally excludes internal deployment scripts,
environment files, runtime data, development reports and prior Git history.
It contains no workflow that automatically deploys or publishes an image.
Open http://localhost:3000
## Contributing and security
Admin panel: http://localhost:3000/admin
Keep changes focused, add regression tests and run the checks above. Never
commit tokens, database exports, backups or real user information.
See [SECURITY.md](SECURITY.md) for reporting guidance and deployment precautions.
Login uses the admin credentials above (or any other local user you create in SQLite).
Licensed under [MIT](LICENSE). Third-party dependency licences remain applicable.
## Public Hosting Notes
The frontend proxies `/api/*` to the backend container. Set:
- `NEXT_PUBLIC_API_BASE=/api` (browser uses same-origin)
- `BACKEND_INTERNAL_URL=http://backend:8000` (container-to-container)
If you prefer the browser to call the backend directly, set `NEXT_PUBLIC_API_BASE` to your public backend URL and ensure CORS is configured.
## Gitea CI/CD
This repo now includes a Gitea Actions workflow at `.gitea/workflows/ci-cd.yml`.
- Push to `beta`: runs the backend unit-test quality gate and a production frontend build.
- Push to `prod`: runs the same verification, then deploys to Docker on `AMS-DEV01`.
The deploy step ships tracked repository files over SSH, preserves the server's `.env` and `data/`, rebuilds with `docker compose up -d --build`, and smoke-tests:
- `http://127.0.0.1:8000/health`
- `http://127.0.0.1:3000/login`
Configure these Gitea Actions secrets before enabling the deploy job:
- `PROD_SSH_PRIVATE_KEY`: private key for the deployment account.
- `PROD_SSH_HOST`: target host, for example `AMS-DEV01`.
- `PROD_SSH_USER`: target user, for example `zak`.
- `PROD_DEPLOY_PATH`: target app path, for example `/home/zak/magent`.
- `PROD_SSH_KNOWN_HOSTS`: optional pinned `known_hosts` entry for stricter host verification.
## History endpoints
- `GET /requests/{id}/history?limit=10` recent snapshots
- `GET /requests/{id}/actions?limit=10` recent action logs
## Troubleshooting
### Login fails
- Make sure `ADMIN_USERNAME` and `ADMIN_PASSWORD` are set in `.env`.
- Confirm the backend is reachable: `http://localhost:8000/health` (or see container logs).
### Services show as down
- Check the URLs and API keys in Settings.
- Verify containers can reach each service (network/DNS).
### No recent requests
- Confirm Seerr credentials in Settings.
- Run a full sync from Settings -> Requests.
### Docker images not updating
- Run `docker compose up --build` again.
- If needed, run `docker compose down` first, then rebuild.
-28
View File
@@ -1,28 +0,0 @@
# Security
## Reporting a vulnerability
Do not post passwords, access tokens, encryption keys, database exports, backup
files or live exploit details in public issues, discussions or container logs.
Use the repository hosting platform's private vulnerability-reporting feature
if the release owner has enabled it. Otherwise contact the maintainer privately
through the platform where you obtained this release before sending sensitive
details. This repository does not currently advertise a dedicated reporting
address; the release owner must establish one before a broad public launch.
Include the image tag/digest, affected version, a minimal reproduction using
synthetic data, and the security impact. Remove deployment credentials and
personal data from attachments. Do not test against systems you do not own or
have permission to assess.
## Deployment precautions
Follow [the public installation guide](docs/PUBLIC_RELEASE.md): use HTTPS for
public access, independent random secrets, a protected persistent data volume,
and the exact browser-facing origin. Keep the original signing/encryption keys
when upgrading. Do not disable origin checks or run as root to work around a
deployment failure.
Use a reviewed immutable release image and retain a tested backup. Check the
release's declared architecture support and migration notes. The project has
not declared an LTS support window or a guaranteed security-response SLA.
+4
View File
@@ -0,0 +1,4 @@
__pycache__/
*.pyc
.venv/
.env
-58
View File
@@ -1,58 +0,0 @@
"""Shared HTTP request and error contracts."""
from typing import Any, Optional
from pydantic import BaseModel, ConfigDict, Field
class StrictRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
class ErrorResponse(BaseModel):
detail: str
COMMON_ERROR_RESPONSES: dict[int, dict[str, Any]] = {
400: {"model": ErrorResponse, "description": "Invalid request"},
401: {"model": ErrorResponse, "description": "Authentication required"},
403: {"model": ErrorResponse, "description": "Permission denied"},
404: {"model": ErrorResponse, "description": "Resource not found"},
409: {"model": ErrorResponse, "description": "Request conflict"},
429: {"model": ErrorResponse, "description": "Rate limit exceeded"},
500: {"model": ErrorResponse, "description": "Unexpected server error"},
502: {"model": ErrorResponse, "description": "Upstream service error"},
503: {"model": ErrorResponse, "description": "Service unavailable"},
}
class SignupRequest(StrictRequest):
invite_code: str = Field(min_length=1, max_length=256)
username: str = Field(min_length=1, max_length=100)
password: str = Field(min_length=1, max_length=1024)
email: Optional[str] = Field(default=None, max_length=320)
class ForgotPasswordRequest(StrictRequest):
identifier: Optional[str] = Field(default=None, max_length=320)
username: Optional[str] = Field(default=None, max_length=100)
email: Optional[str] = Field(default=None, max_length=320)
class PasswordResetRequest(StrictRequest):
token: str = Field(min_length=1, max_length=512)
new_password: str = Field(min_length=1, max_length=1024)
class ProfileEmailUpdateRequest(StrictRequest):
email: Optional[str] = Field(default=None, max_length=320)
class ChangePasswordRequest(StrictRequest):
current_password: str = Field(min_length=1, max_length=1024)
new_password: str = Field(min_length=1, max_length=1024)
def request_data(payload: BaseModel | dict[str, Any]) -> dict[str, Any]:
"""Keep direct service-level tests compatible while FastAPI validates HTTP input."""
return payload if isinstance(payload, dict) else payload.model_dump()
+1 -12
View File
@@ -5,7 +5,6 @@ from fastapi import Depends, HTTPException, Request, Response, status
from fastapi.security import OAuth2PasswordBearer
from .config import settings
from .installation_origin import managed_runtime
from .db import get_user_by_username, set_user_auth_provider, upsert_user_activity
from .network_security import request_trusts_forwarded_headers
from .security import TokenError, safe_decode_token, verify_password
@@ -48,14 +47,8 @@ def _cookie_settings() -> dict[str, Any]:
samesite = str(settings.auth_cookie_samesite or "lax").strip().lower()
if samesite not in {"lax", "strict", "none"}:
samesite = "lax"
secure = bool(settings.auth_cookie_secure)
if managed_runtime():
from .services.public_urls import magent_public_url
# Follow the persisted operator-selected URL immediately, including
# first login after setup; a restart is not required to protect cookies.
secure = magent_public_url().startswith("https://")
return {
"secure": secure,
"secure": bool(settings.auth_cookie_secure),
"httponly": True,
"samesite": samesite,
"domain": settings.auth_cookie_domain or None,
@@ -166,9 +159,6 @@ def _load_current_user_from_token(
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="User is blocked")
if _is_expired(user.get("expires_at")):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="User access has expired")
token_version = payload.get("ver")
if not isinstance(token_version, int) or token_version != int(user.get("auth_version") or 1):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Session has been revoked")
user = normalize_user_auth_provider(user)
from .feature_access import permissions
@@ -193,7 +183,6 @@ def _load_current_user_from_token(
"is_expired": bool(user.get("is_expired", False)),
"password_change_supported": bool(user.get("password_change_supported", False)),
"password_provider": user.get("password_provider"),
"auth_version": int(user.get("auth_version") or 1),
}
File diff suppressed because one or more lines are too long
+13 -19
View File
@@ -1,6 +1,6 @@
import re
from typing import Any, Dict, Optional
import httpx
import time
from .base import ApiClient, _operation_error_message
from ..services.operation_progress import finish_remote_call, start_remote_call
@@ -15,9 +15,9 @@ def _availability_message(result: Any) -> str:
or (isinstance(items, list) and len(items) > 0)
)
return (
"Jellyfin returned possible matches. Magent still needs to check the exact title and file."
"Grizzlyflix returned possible matches. Magent still needs to check the exact title and file."
if available
else "Jellyfin did not find this title in its library search."
else "Grizzlyflix did not find this title in its library search."
)
@@ -185,33 +185,23 @@ class JellyfinClient(ApiClient):
) -> Optional[Dict[str, Any]]:
if not self.base_url or not self.api_key:
return None
started_at = time.perf_counter()
operation_event_id = start_remote_call("Jellyfin", "Checking whether the title is available in Jellyfin…")
url = f"{self.base_url}/Items"
params = {
"SearchTerm": term,
"IncludeItemTypes": ",".join(item_types or []),
"Recursive": "true",
"Fields": "Path,MediaSources,ProviderIds,OriginalTitle,SortName",
"Fields": "Path,MediaSources",
"Limit": limit,
}
headers = self._emby_headers()
try:
async with httpx.AsyncClient(timeout=10.0) as client:
normalized = ' '.join(re.sub(r"[^\w\s]", ' ', term, flags=re.UNICODE).split())
terms = list(dict.fromkeys([term, normalized]))
if normalized != term and normalized.split():
terms.append(max(normalized.split(), key=len))
items = {}
for search_term in dict.fromkeys(terms):
if not search_term:
continue
response = await client.get(url, headers=headers, params={**params, "SearchTerm": search_term})
response.raise_for_status()
payload = response.json()
for item in payload.get('Items', []):
if isinstance(item, dict) and item.get('Id'):
items[item['Id']] = item
result = {'Items': list(items.values()), 'TotalRecordCount': len(items)}
response = await client.get(url, headers=headers, params=params)
response.raise_for_status()
result = response.json()
duration_ms = round((time.perf_counter() - started_at) * 1000, 2)
finish_remote_call(
operation_event_id,
success=True,
@@ -220,6 +210,7 @@ class JellyfinClient(ApiClient):
)
return result
except Exception as exc:
duration_ms = round((time.perf_counter() - started_at) * 1000, 2)
status_code = exc.response.status_code if isinstance(exc, httpx.HTTPStatusError) else None
finish_remote_call(
operation_event_id,
@@ -273,6 +264,7 @@ class JellyfinClient(ApiClient):
async def refresh_library(self, recursive: bool = True) -> None:
if not self.base_url or not self.api_key:
return None
started_at = time.perf_counter()
operation_event_id = start_remote_call("Jellyfin", "Asking Jellyfin to refresh its library…")
url = f"{self.base_url}/Library/Refresh"
headers = self._emby_headers()
@@ -281,6 +273,7 @@ class JellyfinClient(ApiClient):
async with httpx.AsyncClient(timeout=10.0) as client:
response = await client.post(url, headers=headers, params=params)
response.raise_for_status()
duration_ms = round((time.perf_counter() - started_at) * 1000, 2)
finish_remote_call(
operation_event_id,
success=True,
@@ -288,6 +281,7 @@ class JellyfinClient(ApiClient):
message="Jellyfin accepted the library refresh and is scanning for new media.",
)
except Exception as exc:
duration_ms = round((time.perf_counter() - started_at) * 1000, 2)
status_code = exc.response.status_code if isinstance(exc, httpx.HTTPStatusError) else None
finish_remote_call(
operation_event_id,
+10
View File
@@ -1,6 +1,7 @@
from typing import Any, Dict, Optional
import httpx
import logging
import time
from .base import ApiClient, _operation_error_message
from ..services.operation_progress import finish_remote_call, start_remote_call
@@ -88,6 +89,7 @@ class QBittorrentClient(ApiClient):
async def _get(self, path: str, params: Optional[Dict[str, Any]] = None) -> Optional[Any]:
if not self.base_url:
return None
started_at = time.perf_counter()
operation_event_id = start_remote_call("qBittorrent", "Checking qBittorrent for matching downloads…")
try:
async with httpx.AsyncClient(timeout=10.0) as client:
@@ -95,6 +97,7 @@ class QBittorrentClient(ApiClient):
response = await client.get(f"{self.base_url}{path}", params=params)
response.raise_for_status()
result = response.json()
duration_ms = round((time.perf_counter() - started_at) * 1000, 2)
finish_remote_call(
operation_event_id,
success=True,
@@ -103,6 +106,7 @@ class QBittorrentClient(ApiClient):
)
return result
except Exception as exc:
duration_ms = round((time.perf_counter() - started_at) * 1000, 2)
status_code = exc.response.status_code if isinstance(exc, httpx.HTTPStatusError) else None
finish_remote_call(
operation_event_id,
@@ -115,6 +119,7 @@ class QBittorrentClient(ApiClient):
async def _get_text(self, path: str, params: Optional[Dict[str, Any]] = None) -> Optional[str]:
if not self.base_url:
return None
started_at = time.perf_counter()
operation_event_id = start_remote_call("qBittorrent")
try:
async with httpx.AsyncClient(timeout=10.0) as client:
@@ -122,6 +127,7 @@ class QBittorrentClient(ApiClient):
response = await client.get(f"{self.base_url}{path}", params=params)
response.raise_for_status()
result = response.text.strip()
duration_ms = round((time.perf_counter() - started_at) * 1000, 2)
finish_remote_call(
operation_event_id,
success=True,
@@ -130,6 +136,7 @@ class QBittorrentClient(ApiClient):
)
return result
except Exception as exc:
duration_ms = round((time.perf_counter() - started_at) * 1000, 2)
status_code = exc.response.status_code if isinstance(exc, httpx.HTTPStatusError) else None
finish_remote_call(
operation_event_id,
@@ -142,12 +149,14 @@ class QBittorrentClient(ApiClient):
async def _post_form(self, path: str, data: Dict[str, Any]) -> None:
if not self.base_url:
return None
started_at = time.perf_counter()
operation_event_id = start_remote_call("qBittorrent")
try:
async with httpx.AsyncClient(timeout=10.0) as client:
await self._login(client)
response = await client.post(f"{self.base_url}{path}", data=data)
response.raise_for_status()
duration_ms = round((time.perf_counter() - started_at) * 1000, 2)
finish_remote_call(
operation_event_id,
success=True,
@@ -155,6 +164,7 @@ class QBittorrentClient(ApiClient):
message=_torrent_action_message(path),
)
except Exception as exc:
duration_ms = round((time.perf_counter() - started_at) * 1000, 2)
status_code = exc.response.status_code if isinstance(exc, httpx.HTTPStatusError) else None
finish_remote_call(
operation_event_id,
+1 -1
View File
@@ -26,7 +26,7 @@ class RadarrClient(ApiClient):
return await self.get("/api/v3/qualityprofile")
async def get_queue(self, movie_id: int) -> Optional[Dict[str, Any]]:
return await self.get("/api/v3/queue", params={"movieIds": movie_id, "pageSize": 1000})
return await self.get("/api/v3/queue", params={"movieId": movie_id})
async def search_releases(self, movie_id: int) -> Optional[Any]:
return await self.get(
-3
View File
@@ -66,9 +66,6 @@ class SonarrClient(ApiClient):
timeout_seconds=90.0,
)
async def search_episode_releases(self, episode_id: int) -> Optional[Any]:
return await self.get('/api/v3/release', params={'episodeId': episode_id}, timeout_seconds=90.0)
async def search(self, series_id: int) -> Optional[Dict[str, Any]]:
return await self.post("/api/v3/command", payload={"name": "SeriesSearch", "seriesId": series_id})
+2 -30
View File
@@ -1,20 +1,9 @@
import re
from typing import Optional
from pydantic import AliasChoices, Field
from pydantic_settings import BaseSettings, SettingsConfigDict
from .build_info import BUILD_NUMBER, CHANGELOG
_BANNER_COLOR_PATTERN = re.compile(r"^#[0-9a-f]{6}$")
def normalize_banner_color(value: object) -> Optional[str]:
color = str(value or "").strip().lower()
return color if _BANNER_COLOR_PATTERN.fullmatch(color) else None
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_prefix="")
app_name: str = "Magent"
@@ -24,12 +13,7 @@ class Settings(BaseSettings):
default="DELETE", validation_alias=AliasChoices("SQLITE_JOURNAL_MODE")
)
jwt_secret: str = Field(default="", validation_alias=AliasChoices("JWT_SECRET"))
jwt_exp_minutes: int = Field(default=120, validation_alias=AliasChoices("JWT_EXP_MINUTES"))
jwt_issuer: str = Field(default="magent", validation_alias=AliasChoices("JWT_ISSUER"))
jwt_audience: str = Field(default="magent-web", validation_alias=AliasChoices("JWT_AUDIENCE"))
settings_encryption_key: Optional[str] = Field(
default=None, validation_alias=AliasChoices("SETTINGS_ENCRYPTION_KEY")
)
jwt_exp_minutes: int = Field(default=720, validation_alias=AliasChoices("JWT_EXP_MINUTES"))
api_docs_enabled: bool = Field(default=False, validation_alias=AliasChoices("API_DOCS_ENABLED"))
auth_rate_limit_window_seconds: int = Field(
default=60, validation_alias=AliasChoices("AUTH_RATE_LIMIT_WINDOW_SECONDS")
@@ -51,7 +35,6 @@ class Settings(BaseSettings):
)
admin_username: str = Field(default="admin", validation_alias=AliasChoices("ADMIN_USERNAME"))
admin_password: str = Field(default="", validation_alias=AliasChoices("ADMIN_PASSWORD"))
setup_token: str = Field(default="", validation_alias=AliasChoices("SETUP_TOKEN"))
auth_cookie_name: str = Field(
default="magent_auth", validation_alias=AliasChoices("AUTH_COOKIE_NAME")
)
@@ -59,7 +42,7 @@ class Settings(BaseSettings):
default=False, validation_alias=AliasChoices("AUTH_COOKIE_SECURE")
)
auth_cookie_samesite: str = Field(
default="strict", validation_alias=AliasChoices("AUTH_COOKIE_SAMESITE")
default="lax", validation_alias=AliasChoices("AUTH_COOKIE_SAMESITE")
)
auth_cookie_domain: Optional[str] = Field(
default=None, validation_alias=AliasChoices("AUTH_COOKIE_DOMAIN")
@@ -68,7 +51,6 @@ class Settings(BaseSettings):
default="magent_logged_in", validation_alias=AliasChoices("AUTH_STATE_COOKIE_NAME")
)
log_level: str = Field(default="INFO", validation_alias=AliasChoices("LOG_LEVEL"))
log_format: str = Field(default="text", validation_alias=AliasChoices("LOG_FORMAT"))
log_file: str = Field(default="data/magent.log", validation_alias=AliasChoices("LOG_FILE"))
log_file_max_bytes: int = Field(
default=20_000_000, validation_alias=AliasChoices("LOG_FILE_MAX_BYTES")
@@ -85,7 +67,6 @@ class Settings(BaseSettings):
requests_sync_ttl_minutes: int = Field(
default=1440, validation_alias=AliasChoices("REQUESTS_SYNC_TTL_MINUTES")
)
requests_stage_refresh_minutes: int = Field(default=15, ge=1, le=1440, validation_alias=AliasChoices("REQUESTS_STAGE_REFRESH_MINUTES"))
requests_poll_interval_seconds: int = Field(
default=300, validation_alias=AliasChoices("REQUESTS_POLL_INTERVAL_SECONDS")
)
@@ -126,15 +107,6 @@ class Settings(BaseSettings):
site_banner_tone: str = Field(
default="info", validation_alias=AliasChoices("SITE_BANNER_TONE")
)
site_banner_background_color: Optional[str] = Field(
default=None, validation_alias=AliasChoices("SITE_BANNER_BACKGROUND_COLOR")
)
site_banner_border_color: Optional[str] = Field(
default=None, validation_alias=AliasChoices("SITE_BANNER_BORDER_COLOR")
)
site_login_message: Optional[str] = Field(
default=None, validation_alias=AliasChoices("SITE_LOGIN_MESSAGE")
)
site_login_show_jellyfin_login: bool = Field(
default=True, validation_alias=AliasChoices("SITE_LOGIN_SHOW_JELLYFIN_LOGIN")
)
-258
View File
@@ -1,258 +0,0 @@
"""Persistent secrets for fresh image-only container installations.
Runs before importing application settings. Existing environment-managed
deployments are unchanged. Secrets are never printed during normal startup.
"""
import base64
import binascii
from contextlib import closing
import json
import os
from pathlib import Path
import re
import secrets
import sqlite3
import stat
import sys
import tempfile
from urllib.parse import urlsplit
from .installation_origin import normalize_application_origin
DATA_DIRECTORY = Path("/app/data")
STATE_FILENAME = "bootstrap-secrets.json"
SECRET_NAMES = ("JWT_SECRET", "SETTINGS_ENCRYPTION_KEY", "SETUP_TOKEN")
MAX_STATE_BYTES = 4096
class BootstrapError(ValueError):
"""An operator-actionable error that never includes a secret value."""
def managed_mode(environment: dict) -> bool:
value = environment.get("MAGENT_MANAGED_SECRETS", "false").strip().lower()
if value == "auto":
# Existing explicitly keyed installations retain their environment and
# JWT-derived encryption behaviour. Fresh image-only installs opt in.
return not bool(environment.get("JWT_SECRET", "").strip())
if value not in {"true", "false", "1", "0", "yes", "no", ""}:
raise BootstrapError("MAGENT_MANAGED_SECRETS must be auto, true or false.")
return value in {"true", "1", "yes"}
def _data_paths(environment: dict, directory: Path) -> tuple[Path, Path]:
directory = directory.absolute()
if not directory.is_dir() or any(part.is_symlink() for part in (directory, *directory.parents)):
raise BootstrapError("Managed installation requires a real, writable /app/data volume; symlinks are not allowed.")
if os.name == "posix":
metadata = directory.stat()
if metadata.st_uid != os.geteuid() or stat.S_IMODE(metadata.st_mode) & 0o022:
raise BootstrapError("Managed data volume must belong to the runtime user and not be writable by other users.")
database = directory / "magent.db"
configured = Path(environment.get("SQLITE_PATH") or str(database)).absolute()
if configured != database:
raise BootstrapError("Managed installation requires SQLITE_PATH=/app/data/magent.db; retain manual keys for custom paths.")
if os.path.lexists(database) and (database.is_symlink() or not database.is_file()):
raise BootstrapError("Managed database must be a regular file, not a symlink or directory.")
return directory / STATE_FILENAME, database
def _read_state(path: Path) -> dict:
try:
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0) | getattr(os, "O_NONBLOCK", 0))
with os.fdopen(descriptor, "rb") as handle:
metadata = os.fstat(handle.fileno())
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > MAX_STATE_BYTES:
raise BootstrapError("Managed secrets file must be a small regular file.")
if os.name == "posix" and (
metadata.st_uid != os.geteuid() or stat.S_IMODE(metadata.st_mode) != 0o600
):
raise BootstrapError("Managed secrets file must belong to the runtime user with permissions 0600.")
state = json.loads(handle.read(MAX_STATE_BYTES + 1))
except FileNotFoundError:
raise
except (OSError, ValueError, UnicodeError) as exc:
if isinstance(exc, BootstrapError):
raise
raise BootstrapError("Cannot read managed secrets. Restore the original file; keys will not be regenerated.") from None
if not isinstance(state, dict) or set(state) != {"version", *SECRET_NAMES} or type(state["version"]) is not int or state["version"] != 1:
raise BootstrapError("Invalid managed secrets format. Restore the original file; keys will not be regenerated.")
for key in SECRET_NAMES:
if not isinstance(state[key], str):
raise BootstrapError("Invalid managed secret values. Restore the original file.")
for key in ("JWT_SECRET", "SETUP_TOKEN"):
if not re.fullmatch(r"[A-Za-z0-9_-]{64}", state[key]) or len(set(state[key])) < 2:
raise BootstrapError("Invalid managed token. Restore the original file.")
try:
decoded = base64.b64decode(state["SETTINGS_ENCRYPTION_KEY"], altchars=b"-_", validate=True)
except (ValueError, binascii.Error):
raise BootstrapError("Invalid managed encryption key. Restore the original file.") from None
if len(decoded) != 32 or base64.urlsafe_b64encode(decoded).decode() != state["SETTINGS_ENCRYPTION_KEY"]:
raise BootstrapError("Invalid managed encryption key. Restore the original file.")
if state["JWT_SECRET"] == state["SETUP_TOKEN"]:
raise BootstrapError("Managed signing and setup tokens must be independent.")
return state
def _sync_directory(directory: Path) -> None:
if os.name == "posix":
descriptor = os.open(directory, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(descriptor)
finally:
os.close(descriptor)
def _create_state(path: Path) -> dict:
state = {
"version": 1,
"JWT_SECRET": secrets.token_urlsafe(48),
"SETTINGS_ENCRYPTION_KEY": base64.urlsafe_b64encode(secrets.token_bytes(32)).decode(),
"SETUP_TOKEN": secrets.token_urlsafe(48),
}
descriptor, temporary_name = tempfile.mkstemp(prefix=".magent-secrets-", dir=path.parent)
temporary = Path(temporary_name)
try:
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
json.dump(state, handle, separators=(",", ":"))
handle.flush()
os.fsync(handle.fileno())
try:
# Publish an entirely written file without replacing another
# initializer's state. Both callers subsequently read the winner.
os.link(temporary, path)
_sync_directory(path.parent)
except FileExistsError:
pass
finally:
temporary.unlink(missing_ok=True)
return _read_state(path)
def _saved_origin(database: Path) -> str:
if not database.exists():
return ""
try:
with closing(sqlite3.connect(database.as_uri() + "?mode=ro", uri=True)) as connection:
if not connection.execute("SELECT 1 FROM sqlite_master WHERE type='table' AND name='settings'").fetchone():
return ""
row = connection.execute("SELECT value FROM settings WHERE key='magent_application_url'").fetchone()
return str(row[0] or "") if row else ""
except sqlite3.Error:
raise BootstrapError("Cannot read the saved application address. Check the existing database; no keys were changed.") from None
def _configure_origin(environment: dict, database: Path) -> None:
value = environment.get("MAGENT_APPLICATION_URL", "")
saved = _saved_origin(database)
if saved:
value = saved
if not value:
# No network address is trusted automatically. The token-authorized
# first-admin transaction will save the explicitly confirmed origin.
environment.setdefault("CORS_ALLOW_ORIGIN", "http://localhost:3000")
environment.setdefault("AUTH_COOKIE_SECURE", "false")
return
try:
parsed = urlsplit(value)
valid = (
bool(value) and not any(c.isspace() or ord(c) < 33 or ord(c) == 127 for c in value)
and parsed.scheme in {"http", "https"} and parsed.hostname
and parsed.username is None and parsed.password is None and not parsed.path
and "?" not in value and "#" not in value and "\\" not in value and "*" not in value
and (parsed.port is None or 1 <= parsed.port <= 65535)
)
except ValueError:
valid = False
if not valid:
raise BootstrapError("Set MAGENT_APPLICATION_URL to the exact http(s) browser origin, with no path or trailing slash.")
if not saved and environment.get("CORS_ALLOW_ORIGIN") not in (None, "", value):
raise BootstrapError("CORS_ALLOW_ORIGIN must match MAGENT_APPLICATION_URL for a managed install.")
value = normalize_application_origin(value)
environment["MAGENT_APPLICATION_URL"] = value
environment["CORS_ALLOW_ORIGIN"] = value
secure = environment.get("AUTH_COOKIE_SECURE", "").strip().lower()
if not secure:
environment["AUTH_COOKIE_SECURE"] = str(parsed.scheme == "https").lower()
elif secure not in {"true", "false", "1", "0"}:
raise BootstrapError("AUTH_COOKIE_SECURE must be true or false.")
elif parsed.scheme == "https" and secure in {"false", "0"}:
raise BootstrapError("HTTPS managed installations require AUTH_COOKIE_SECURE=true.")
elif parsed.scheme == "http" and secure in {"true", "1"}:
raise BootstrapError("Secure cookies require an HTTPS application URL.")
def prepare_environment(environment: dict, directory: Path = DATA_DIRECTORY) -> dict:
prepared = dict(environment)
if not managed_mode(prepared):
return prepared
if not prepared.get("JWT_SECRET", "").strip():
prepared.pop("JWT_SECRET", None)
path, database = _data_paths(prepared, directory)
_configure_origin(prepared, database)
if prepared.get("API_DOCS_ENABLED", "false").strip().lower() not in {"", "false", "0"}:
raise BootstrapError("API_DOCS_ENABLED is fixed to false for managed installations.")
try:
state = _read_state(path)
except FileNotFoundError:
# Never add independent encryption to an existing JWT-derived database
# or invent replacement keys after a lost secrets file.
if any(os.path.lexists(str(database) + suffix) for suffix in ("", "-wal", "-shm", "-journal")):
raise BootstrapError("Existing database has no managed secrets file. Restore its original keys or use the existing manual deployment.") from None
if any(prepared.get(key) for key in SECRET_NAMES):
raise BootstrapError("Fresh managed installs generate their own keys. Remove manual key variables or disable managed mode.") from None
state = _create_state(path)
for key in SECRET_NAMES:
if prepared.get(key) and prepared[key] != state[key]:
raise BootstrapError(f"{key} conflicts with the persistent managed value. Keys will not be replaced.")
prepared[key] = state[key]
prepared["SQLITE_PATH"] = str(database)
prepared["API_DOCS_ENABLED"] = "false"
prepared["MAGENT_MANAGED_SECRETS"] = "true"
prepared["MAGENT_RUNTIME_MANAGED"] = "1"
return prepared
def setup_token(environment: dict, directory: Path = DATA_DIRECTORY) -> str:
if not managed_mode(environment):
raise BootstrapError("Managed secrets are disabled. Use the SETUP_TOKEN from your deployment configuration.")
path, database = _data_paths(environment, directory)
state = _read_state(path) # This read-only command never generates keys.
if database.is_symlink() or not database.is_file():
raise BootstrapError("Database is not initialized. Wait for the container to become healthy.")
try:
with closing(sqlite3.connect(database.as_uri() + "?mode=ro", uri=True)) as connection:
row = connection.execute("SELECT completed FROM installation_setup WHERE id = 1").fetchone()
admin = connection.execute("SELECT 1 FROM users WHERE LOWER(role) = 'admin' LIMIT 1").fetchone()
except sqlite3.Error:
raise BootstrapError("Cannot verify setup state. No setup token will be displayed.") from None
if row is None or row[0] != 0 or admin is not None:
raise BootstrapError("Initial administrator setup is no longer available. Sign in with the existing administrator.")
return state["SETUP_TOKEN"]
def main() -> int:
try:
if sys.argv[1:] == ["setup-token"]:
print(setup_token(dict(os.environ)))
return 0
if len(sys.argv) < 2:
raise BootstrapError("Pass the container startup command, or setup-token from the operator console.")
environment = prepare_environment(dict(os.environ))
if managed_mode(environment):
print("Managed installation secrets loaded. For first setup, run in the container console: "
"python -m app.container_bootstrap setup-token", flush=True)
os.execvpe(sys.argv[1], sys.argv[1:], environment)
except (BootstrapError, OSError):
# Never include unexpected I/O details or environment values in logs.
error = sys.exc_info()[1]
message = str(error) if isinstance(error, BootstrapError) else "Cannot access managed installation files or start the runtime. Check volume permissions and original keys."
print(f"Magent startup: {message}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
+220 -496
View File
File diff suppressed because it is too large Load Diff
+1 -2
View File
@@ -1,14 +1,13 @@
"""Live account permissions. Invite access uses the existing users column."""
from .db import _connect
FEATURES = ("stats", "requests", "new_requests", "issues", "invites", "ignore_profile_limits")
FEATURES = ("stats", "requests", "new_requests", "issues", "invites")
def permissions(user: dict) -> dict[str, bool]:
if user.get("role") == "admin":
return dict.fromkeys(FEATURES, True)
values = dict.fromkeys(FEATURES, True)
values["ignore_profile_limits"] = False
values["invites"] = bool(user.get("invite_management_enabled", False))
with _connect() as conn:
rows = conn.execute("""SELECT p.feature, p.enabled FROM user_feature_permissions p
-32
View File
@@ -1,32 +0,0 @@
"""Origin validation shared by first-install setup and container startup."""
import os
from urllib.parse import urlsplit
def managed_runtime() -> bool:
# Set by the entrypoint, never by an HTTP header or a database setting.
return os.environ.get("MAGENT_RUNTIME_MANAGED") == "1"
def normalize_application_origin(value: str) -> str:
if not isinstance(value, str) or not value or any(
c.isspace() or ord(c) < 33 or ord(c) == 127 or c in '<>"\\*?#' for c in value
):
raise ValueError("Enter an exact http(s) site address without a path, credentials, query or fragment.")
try:
parsed = urlsplit(value)
if (parsed.scheme not in {"http", "https"} or not parsed.hostname
or parsed.username is not None or parsed.password is not None
or parsed.path not in {"", "/"} or parsed.netloc.endswith(":")):
raise ValueError
port = parsed.port
if port is not None and not 1 <= port <= 65535:
raise ValueError
host = parsed.hostname.encode("idna").decode("ascii").lower()
if ":" in host:
host = f"[{host}]"
suffix = f":{port}" if port is not None and port != (443 if parsed.scheme == "https" else 80) else ""
return f"{parsed.scheme}://{host}{suffix}"
except (ValueError, UnicodeError):
raise ValueError("Enter an exact http(s) site address without a path, credentials, query or fragment.") from None
+8 -41
View File
@@ -2,8 +2,6 @@ import contextvars
import json
import logging
import os
import re
from datetime import datetime, timezone
from logging.handlers import RotatingFileHandler
from typing import Any, Mapping, Optional
from urllib.parse import parse_qs
@@ -29,9 +27,6 @@ _SENSITIVE_KEYWORDS = (
"token",
)
_MAX_BODY_BYTES = 4096
_SENSITIVE_PATH_PATTERNS = (
re.compile(r"(/auth/invites/)[^/]+", re.IGNORECASE),
)
class RequestContextFilter(logging.Filter):
@@ -40,22 +35,6 @@ class RequestContextFilter(logging.Filter):
return True
class JsonLogFormatter(logging.Formatter):
"""Stable JSON output for production log collectors."""
def format(self, record: logging.LogRecord) -> str:
payload: dict[str, Any] = {
"timestamp": datetime.fromtimestamp(record.created, timezone.utc).isoformat(),
"level": record.levelname,
"logger": record.name,
"request_id": getattr(record, "request_id", "-"),
"message": record.getMessage(),
}
if record.exc_info:
payload["exception"] = self.formatException(record.exc_info)
return json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
def bind_request_id(request_id: str) -> contextvars.Token[str]:
return REQUEST_ID_CONTEXT.set(request_id or "-")
@@ -68,13 +47,6 @@ def current_request_id() -> str:
return REQUEST_ID_CONTEXT.get("-")
def sanitize_path(path: str) -> str:
sanitized = str(path or "")
for pattern in _SENSITIVE_PATH_PATTERNS:
sanitized = pattern.sub(r"\1[REDACTED]", sanitized)
return sanitized
def _is_sensitive_key(key: str) -> bool:
lowered = key.strip().lower()
return any(marker in lowered for marker in _SENSITIVE_KEYWORDS)
@@ -83,7 +55,10 @@ def _is_sensitive_key(key: str) -> bool:
def _redact_scalar(value: Any) -> Any:
if value is None or isinstance(value, (int, float, bool)):
return value
return "[REDACTED]"
text = str(value)
if len(text) <= 4:
return "***"
return f"{text[:2]}***{text[-2:]}"
def sanitize_value(value: Any, *, key_hint: Optional[str] = None, depth: int = 0) -> Any:
@@ -167,7 +142,6 @@ def configure_logging(
log_file_backup_count: int = 10,
log_http_client_level: Optional[str] = "INFO",
log_background_sync_level: Optional[str] = "INFO",
log_format: Optional[str] = "text",
) -> None:
level_name = (log_level or "INFO").upper()
level = getattr(logging, level_name, logging.INFO)
@@ -187,20 +161,13 @@ def configure_logging(
backupCount=max(1, int(log_file_backup_count or 10)),
encoding="utf-8",
)
try:
os.chmod(log_path, 0o600)
except OSError:
pass
handlers.append(file_handler)
context_filter = RequestContextFilter()
if str(log_format or "text").strip().lower() == "json":
formatter: logging.Formatter = JsonLogFormatter()
else:
formatter = logging.Formatter(
fmt="%(asctime)s | %(levelname)s | %(name)s | request_id=%(request_id)s | %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
formatter = logging.Formatter(
fmt="%(asctime)s | %(levelname)s | %(name)s | request_id=%(request_id)s | %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
for handler in handlers:
handler.addFilter(context_filter)
handler.setFormatter(formatter)
+27 -111
View File
@@ -6,16 +6,13 @@ import uuid
from typing import Awaitable, Callable
from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.exception_handlers import request_validation_exception_handler
from fastapi.responses import JSONResponse
from fastapi.middleware.cors import CORSMiddleware
from .config import settings
from .db import has_admin_user, init_db
from .routers.requests import (
router as requests_router,
startup_warmup_requests_cache,
run_local_request_stage_loop,
run_requests_delta_loop,
run_daily_requests_full_sync,
run_daily_db_cleanup,
@@ -34,10 +31,6 @@ from .routers.insights import router as insights_router
from .routers.identities import router as identities_router
from .routers.recaps import router as recaps_router
from .routers.newsletters import router as newsletters_router
from .routers.backups import router as backups_router
from .routers.setup import router as setup_router
from .services.backups import apply_pending_restore
from .services.setup import initialize_setup_state, is_setup_required, setup_token_configured
from .services.jellyfin_sync import run_daily_jellyfin_sync
from .services.issue_resolution import run_issue_confirmation_loop
from .services.email_recaps import run_email_recap_loop
@@ -53,17 +46,14 @@ from .logging_config import (
configure_logging,
reset_request_id,
sanitize_headers,
sanitize_path,
sanitize_value,
summarize_http_body,
)
from .runtime import get_runtime_settings
from .metrics import record_api, start_metrics
from .request_limits import InstallationBodyLimitMiddleware
from .secret_storage import validate_secret_storage_configuration
from .services.request_origins import ConfiguredOriginCORSMiddleware, can_claim_initial_origin, is_allowed_request_origin
logger = logging.getLogger(__name__)
_background_tasks: list[asyncio.Task[None]] = []
_background_started = False
app = FastAPI(
title=settings.app_name,
@@ -73,29 +63,12 @@ app = FastAPI(
)
app.add_middleware(
ConfiguredOriginCORSMiddleware,
CORSMiddleware,
allow_origins=[settings.cors_allow_origin],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
app.add_middleware(InstallationBodyLimitMiddleware)
@app.exception_handler(RequestValidationError)
async def installation_validation_error(request: Request, exc: RequestValidationError):
if request.url.path.rstrip("/") == "/setup/bootstrap" or request.url.path.startswith("/admin/backups"):
# Pydantic SecretStr masks parsed values, but FastAPI's default 422 body
# includes rejected raw input. Never echo tokens/passwords/passphrases.
return JSONResponse(
status_code=422,
content={"detail": [
{key: error[key] for key in ("type", "loc", "msg") if key in error}
for error in exc.errors()
]},
headers={"Cache-Control": "no-store"},
)
return await request_validation_exception_handler(request, exc)
@app.middleware("http")
@@ -108,36 +81,22 @@ async def log_requests_and_add_security_headers(request: Request, call_next):
operation_token = begin_operation(
operation_id,
label=request.headers.get("X-Magent-Operation-Label"),
path=sanitize_path(request.url.path),
path=request.url.path,
)
request.state.request_id = request_id
if request.method.upper() not in {"GET", "HEAD", "OPTIONS"}:
origin = str(request.headers.get("origin") or "")
initial_origin_claim = (
request.method.upper() == "POST" and request.url.path == "/setup/bootstrap"
and can_claim_initial_origin()
)
if origin and not is_allowed_request_origin(origin) and not initial_origin_claim:
record_api(request, 403, 0.0)
if operation_id and operation_token is not None:
finish_operation(operation_id, success=False, status_code=403)
reset_operation(operation_token)
reset_request_id(token)
return JSONResponse(
status_code=403,
content={"detail": "Cross-origin state change rejected"},
headers={"X-Request-ID": request_id},
)
started_at = time.perf_counter()
body_summary = {
"content_type": (request.headers.get("content-type") or "").split(";", 1)[0],
"declared_bytes": request.headers.get("content-length"),
}
body = await request.body()
body_summary = summarize_http_body(body, request.headers.get("content-type"))
async def receive() -> dict:
return {"type": "http.request", "body": body, "more_body": False}
request._receive = receive
logger.info(
"request started method=%s path=%s query_keys=%s client=%s headers=%s body=%s",
"request started method=%s path=%s query=%s client=%s headers=%s body=%s",
request.method,
sanitize_path(request.url.path),
sorted(set(request.query_params.keys())),
request.url.path,
sanitize_value(dict(request.query_params)),
request.client.host if request.client else "-",
sanitize_headers(
{
@@ -164,7 +123,7 @@ async def log_requests_and_add_security_headers(request: Request, call_next):
logger.exception(
"request failed method=%s path=%s duration_ms=%s",
request.method,
sanitize_path(request.url.path),
request.url.path,
duration_ms,
)
if operation_id and operation_token is not None:
@@ -180,7 +139,6 @@ async def log_requests_and_add_security_headers(request: Request, call_next):
response.headers.setdefault("X-Frame-Options", "DENY")
response.headers.setdefault("Referrer-Policy", "no-referrer")
response.headers.setdefault("Permissions-Policy", "geolocation=(), microphone=(), camera=()")
response.headers.setdefault("Strict-Transport-Security", "max-age=31536000; includeSubDomains")
# Keep API responses non-executable and non-embeddable by default.
if request.url.path not in {"/docs", "/redoc"} and not request.url.path.startswith("/openapi"):
response.headers.setdefault(
@@ -190,7 +148,7 @@ async def log_requests_and_add_security_headers(request: Request, call_next):
logger.info(
"request completed method=%s path=%s status=%s duration_ms=%s response_headers=%s",
request.method,
sanitize_path(request.url.path),
request.url.path,
response.status_code,
duration_ms,
sanitize_headers(
@@ -244,14 +202,14 @@ def _launch_background_task(name: str, coroutine_factory: Callable[[], Awaitable
def _log_security_configuration_warnings() -> None:
jwt_secret = str(settings.jwt_secret or "").strip()
if len(jwt_secret) < 32 or jwt_secret == "change-me":
if not jwt_secret or jwt_secret == "change-me":
logger.warning(
"security configuration warning: JWT_SECRET is missing, short, or still set to the default value"
"security configuration warning: JWT_SECRET is unset or still set to the default value"
)
admin_password = str(settings.admin_password or "")
if admin_password == "adminadmin":
if not admin_password or admin_password == "adminadmin":
logger.warning(
"security configuration warning: ADMIN_PASSWORD is still set to the bootstrap default"
"security configuration warning: ADMIN_PASSWORD is unset or still set to the bootstrap default"
)
if bool(settings.api_docs_enabled):
logger.warning(
@@ -259,24 +217,14 @@ def _log_security_configuration_warnings() -> None:
)
def _enforce_secret_configuration() -> None:
jwt_secret = str(settings.jwt_secret or "").strip()
if len(jwt_secret) < 32 or jwt_secret == "change-me":
raise RuntimeError(
"JWT_SECRET must be a strong, non-default value of at least 32 characters before startup."
)
validate_secret_storage_configuration()
def _enforce_secure_startup_configuration() -> None:
_enforce_secret_configuration()
jwt_secret = str(settings.jwt_secret or "").strip()
if not jwt_secret or jwt_secret == "change-me":
raise RuntimeError("JWT_SECRET must be set to a strong, non-default value before startup.")
admin_password = str(settings.admin_password or "")
if not has_admin_user() and (not admin_password or admin_password == "adminadmin"):
if is_setup_required() and setup_token_configured():
return
raise RuntimeError(
"First startup requires a strong SETUP_TOKEN (at least 32 characters) for the setup wizard, "
"or a secure ADMIN_PASSWORD, until an admin account exists."
"A secure ADMIN_PASSWORD is required on first startup until an admin account exists."
)
@@ -290,14 +238,9 @@ async def startup() -> None:
log_file_backup_count=settings.log_file_backup_count,
log_http_client_level=settings.log_http_client_level,
log_background_sync_level=settings.log_background_sync_level,
log_format=settings.log_format,
)
logger.info("startup begin app=%s build=%s", settings.app_name, settings.site_build_number)
_log_security_configuration_warnings()
_enforce_secret_configuration()
# Restore offline, before any schema migration, database reader or worker.
apply_pending_restore()
initialize_setup_state()
init_db()
_enforce_secure_startup_configuration()
runtime = get_runtime_settings()
@@ -308,7 +251,6 @@ async def startup() -> None:
log_file_backup_count=runtime.log_file_backup_count,
log_http_client_level=runtime.log_http_client_level,
log_background_sync_level=runtime.log_background_sync_level,
log_format=runtime.log_format,
)
logger.info(
"runtime settings applied log_level=%s log_file=%s log_file_max_bytes=%s log_file_backup_count=%s log_http_client_level=%s log_background_sync_level=%s request_source=%s",
@@ -320,42 +262,18 @@ async def startup() -> None:
runtime.log_background_sync_level,
runtime.requests_data_source,
)
app.state.on_setup_complete = _start_background_tasks
await _start_background_tasks()
logger.info("startup complete")
async def _start_background_tasks() -> None:
global _background_started
if _background_started:
return
if is_setup_required():
logger.info("Background imports and automation paused until setup is complete")
return
if os.environ.get("BACKGROUND_TASKS_ENABLED", "true").lower() == "false":
logger.info("Background imports and automation disabled by configuration")
logger.info("Background imports and automation paused for initial setup")
return
_background_started = True
_launch_background_task("jellyfin-sync", run_daily_jellyfin_sync)
_launch_background_task("requests-warmup", startup_warmup_requests_cache)
_launch_background_task("request-local-stages", run_local_request_stage_loop)
_launch_background_task("requests-delta-loop", run_requests_delta_loop)
_launch_background_task("requests-full-sync", run_daily_requests_full_sync)
_launch_background_task("db-cleanup", run_daily_db_cleanup)
_launch_background_task("issue-confirmation", run_issue_confirmation_loop)
_launch_background_task("email-recaps", run_email_recap_loop)
_launch_background_task("newsletters", run_newsletter_loop)
@app.on_event("shutdown")
async def shutdown() -> None:
global _background_started
for task in _background_tasks:
task.cancel()
if _background_tasks:
await asyncio.gather(*_background_tasks, return_exceptions=True)
_background_tasks.clear()
_background_started = False
logger.info("startup complete")
app.include_router(requests_router)
@@ -374,5 +292,3 @@ app.include_router(insights_router)
app.include_router(identities_router)
app.include_router(recaps_router)
app.include_router(newsletters_router)
app.include_router(backups_router)
app.include_router(setup_router)
-50
View File
@@ -1,50 +0,0 @@
"""Bound security-sensitive request bodies before JSON/multipart parsing."""
from starlette.exceptions import HTTPException
from starlette.responses import JSONResponse
from starlette.types import ASGIApp, Message, Receive, Scope, Send
# Encrypted backup limit is 32 MiB. Allow a bounded margin for the multipart
# envelope; count streamed chunks as well as checking the untrusted header.
RESTORE_BODY_LIMIT = 34 * 1024 * 1024
BOOTSTRAP_BODY_LIMIT = 16 * 1024
class InstallationBodyLimitMiddleware:
def __init__(self, app: ASGIApp) -> None:
self.app = app
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
if scope["type"] != "http" or scope.get("method") != "POST":
await self.app(scope, receive, send)
return
path = scope.get("path", "").rstrip("/")
limit = {
"/admin/backups/restore": RESTORE_BODY_LIMIT,
"/admin/backups/export": BOOTSTRAP_BODY_LIMIT,
"/setup/bootstrap": BOOTSTRAP_BODY_LIMIT,
}.get(path)
if limit is None:
await self.app(scope, receive, send)
return
headers = dict(scope.get("headers", []))
try:
length = int(headers.get(b"content-length", b"0"))
except ValueError:
length = -1
if length < 0 or length > limit:
await JSONResponse({"detail": "Request body is too large or has an invalid length."}, status_code=413)(scope, receive, send)
return
received = 0
async def bounded_receive() -> Message:
nonlocal received
message = await receive()
if message["type"] == "http.request":
received += len(message.get("body", b""))
if received > limit:
raise HTTPException(status_code=413, detail="Request body is too large.")
return message
await self.app(scope, bounded_receive, send)
+73 -89
View File
@@ -20,8 +20,7 @@ from ..auth import (
normalize_user_auth_provider,
resolve_user_auth_provider,
)
from ..config import normalize_banner_color, settings as env_settings
from ..api_models import COMMON_ERROR_RESPONSES
from ..config import settings as env_settings
from ..network_security import validate_notification_target_url
from ..db import (
delete_setting,
@@ -36,9 +35,12 @@ from ..db import (
get_user_by_id,
get_user_by_username,
get_user_request_stats,
create_user_if_missing,
set_user_jellyseerr_id,
set_setting,
set_user_blocked,
delete_user_data_by_username,
delete_user_by_username,
delete_user_activity_by_username,
set_user_auto_search_enabled,
set_auto_search_enabled_for_non_admin_users,
set_user_email,
@@ -47,7 +49,6 @@ from ..db import (
set_user_profile_id,
set_user_expires_at,
set_user_password,
increment_user_auth_version,
sync_jellyfin_password_state,
set_user_role,
run_integrity_check,
@@ -58,6 +59,7 @@ from ..db import (
cleanup_history,
update_request_cache_title,
repair_request_cache_titles,
delete_non_admin_users,
list_user_profiles,
get_user_profile,
create_user_profile,
@@ -67,11 +69,9 @@ from ..db import (
get_signup_invite_by_id,
create_signup_invite,
update_signup_invite,
rotate_signup_invite_code,
delete_signup_invite,
get_signup_invite_by_code,
disable_signup_invites_by_creator,
delete_non_admin_users, # noqa: F401 - retained for compatibility with maintenance tooling/tests
)
from ..runtime import get_runtime_settings
from ..clients.sonarr import SonarrClient
@@ -80,8 +80,12 @@ from ..clients.jellyfin import JellyfinClient
from ..clients.jellyseerr import JellyseerrClient
from ..services.jellyfin_sync import sync_jellyfin_users
from ..services.user_cache import (
build_jellyseerr_candidate_map,
extract_jellyseerr_user_email,
find_matching_jellyseerr_user,
get_cached_jellyfin_users,
get_cached_jellyseerr_users,
match_jellyseerr_user_id,
save_jellyfin_users_cache,
save_jellyseerr_users_cache,
clear_user_import_caches,
@@ -104,12 +108,7 @@ from ..logging_config import configure_logging
from ..routers import requests as requests_router
from ..routers.branding import save_branding_image
router = APIRouter(
prefix="/admin",
tags=["admin"],
dependencies=[Depends(require_admin)],
responses=COMMON_ERROR_RESPONSES,
)
router = APIRouter(prefix="/admin", tags=["admin"], dependencies=[Depends(require_admin)])
events_router = APIRouter(prefix="/admin/events", tags=["admin"])
logger = logging.getLogger(__name__)
SELF_SERVICE_INVITE_MASTER_ID_KEY = "self_service_invite_master_id"
@@ -175,11 +174,6 @@ NOTIFICATION_URL_SETTING_KEYS = {
"magent_notify_webhook_url",
}
BANNER_COLOR_SETTING_KEYS = {
"site_banner_background_color",
"site_banner_border_color",
}
SETTING_KEYS: List[str] = [
"jellystat_base_url",
"jellystat_api_key",
@@ -247,7 +241,6 @@ SETTING_KEYS: List[str] = [
"qbittorrent_username",
"qbittorrent_password",
"log_level",
"log_format",
"log_file",
"log_file_max_bytes",
"log_file_backup_count",
@@ -255,7 +248,6 @@ SETTING_KEYS: List[str] = [
"log_background_sync_level",
"requests_sync_ttl_minutes",
"requests_poll_interval_seconds",
"requests_stage_refresh_minutes",
"requests_delta_sync_interval_minutes",
"requests_full_sync_time",
"requests_cleanup_time",
@@ -267,9 +259,6 @@ SETTING_KEYS: List[str] = [
"site_banner_enabled",
"site_banner_message",
"site_banner_tone",
"site_banner_background_color",
"site_banner_border_color",
"site_login_message",
"site_login_show_jellyfin_login",
"site_login_show_local_login",
"site_login_show_forgot_password",
@@ -680,12 +669,6 @@ async def list_settings() -> Dict[str, Any]:
@router.put("/settings")
async def update_settings(payload: Dict[str, Any]) -> Dict[str, Any]:
from ..installation_origin import managed_runtime, normalize_application_origin
if managed_runtime() and "magent_application_url" in payload:
try:
payload = {**payload, "magent_application_url": normalize_application_origin(payload["magent_application_url"])}
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
updates = 0
touched_logging = False
changed_keys: List[str] = []
@@ -700,14 +683,6 @@ async def update_settings(payload: Dict[str, Any]) -> Dict[str, Any]:
changed_keys.append(key)
continue
value_to_store = str(value).strip() if isinstance(value, str) else str(value)
if key == "requests_stage_refresh_minutes":
try:
interval = int(value_to_store)
except (TypeError, ValueError) as exc:
raise HTTPException(status_code=400, detail="Local stage refresh must be a whole number from 1 to 1440 minutes") from exc
if not 1 <= interval <= 1440:
raise HTTPException(status_code=400, detail="Local stage refresh must be from 1 to 1440 minutes")
value_to_store = str(interval)
if key == "issue_confirmation_contact_attempts":
try:
attempts = int(value_to_store)
@@ -728,11 +703,6 @@ async def update_settings(payload: Dict[str, Any]) -> Dict[str, Any]:
value_to_store = value_to_store.lower()
if value_to_store not in {"days", "weeks", "months"}:
raise HTTPException(status_code=400, detail="Confirmation interval unit must be days, weeks, or months")
if key in BANNER_COLOR_SETTING_KEYS:
normalized_color = normalize_banner_color(value_to_store)
if not normalized_color:
raise HTTPException(status_code=400, detail=f"{key.replace('_', ' ')} must be a six-digit hex colour such as #ffc857")
value_to_store = normalized_color
if key in URL_SETTING_KEYS and value_to_store:
try:
value_to_store = _normalize_service_url(value_to_store)
@@ -748,7 +718,7 @@ async def update_settings(payload: Dict[str, Any]) -> Dict[str, Any]:
set_setting(key, value_to_store)
updates += 1
changed_keys.append(key)
if key in {"log_level", "log_format", "log_file", "log_file_max_bytes", "log_file_backup_count", "log_http_client_level", "log_background_sync_level"}:
if key in {"log_level", "log_file", "log_file_max_bytes", "log_file_backup_count", "log_http_client_level", "log_background_sync_level"}:
touched_logging = True
if touched_logging:
runtime = get_runtime_settings()
@@ -759,7 +729,6 @@ async def update_settings(payload: Dict[str, Any]) -> Dict[str, Any]:
log_file_backup_count=runtime.log_file_backup_count,
log_http_client_level=runtime.log_http_client_level,
log_background_sync_level=runtime.log_background_sync_level,
log_format=runtime.log_format,
)
logger.info("Admin updated settings: count=%s keys=%s", updates, changed_keys)
return {"status": "ok", "updated": updates}
@@ -788,7 +757,7 @@ async def test_email_settings(request: Request) -> Dict[str, Any]:
result = await send_test_email(recipient_email=recipient_email)
except RuntimeError as exc:
raise HTTPException(status_code=502, detail=str(exc)) from exc
logger.info("Admin triggered SMTP test")
logger.info("Admin triggered SMTP test: recipient=%s", result.get("recipient_email"))
return {"status": "ok", **result}
@@ -911,10 +880,28 @@ async def jellyseerr_users_sync() -> Dict[str, Any]:
if not jellyseerr_users:
return {"status": "ok", "matched": 0, "skipped": 0, "total": 0}
from ..services.jellyfin_sync import sync_jellyfin_users
imported = await sync_jellyfin_users()
return {"status": "ok", "matched": len(jellyseerr_users), "skipped": 0, "imported": imported, "total": len(jellyseerr_users)}
candidate_to_id = build_jellyseerr_candidate_map(jellyseerr_users)
updated = 0
skipped = 0
users = get_all_users()
for user in users:
if user.get("jellyseerr_user_id") is not None:
skipped += 1
continue
username = user.get("username") or ""
matched_id = match_jellyseerr_user_id(username, candidate_to_id)
matched_seerr_user = find_matching_jellyseerr_user(username, jellyseerr_users)
matched_email = extract_jellyseerr_user_email(matched_seerr_user)
if matched_id is not None:
set_user_jellyseerr_id(username, matched_id)
if matched_email:
set_user_email(username, matched_email)
updated += 1
else:
skipped += 1
return {"status": "ok", "matched": updated, "skipped": skipped, "total": len(users)}
def _pick_jellyseerr_username(user: Dict[str, Any]) -> Optional[str]:
for key in ("email", "username", "displayName", "name"):
@@ -935,9 +922,33 @@ async def jellyseerr_users_resync() -> Dict[str, Any]:
if not jellyseerr_users:
return {"status": "ok", "imported": 0, "cleared": 0}
from ..services.jellyfin_sync import sync_jellyfin_users
imported = await sync_jellyfin_users()
return {"status": "ok", "imported": imported, "cleared": 0}
cleared = delete_non_admin_users()
imported = 0
for user in jellyseerr_users:
user_id = user.get("id") or user.get("userId") or user.get("Id")
try:
user_id = int(user_id)
except (TypeError, ValueError):
continue
username = _pick_jellyseerr_username(user)
if not username:
continue
email = extract_jellyseerr_user_email(user)
created = create_user_if_missing(
username,
"jellyseerr-user",
role="user",
email=email,
auth_provider="jellyseerr",
jellyseerr_user_id=user_id,
)
if created:
imported += 1
else:
set_user_jellyseerr_id(username, user_id)
if email:
set_user_email(username, email)
return {"status": "ok", "imported": imported, "cleared": cleared}
@router.post("/requests/sync")
async def requests_sync() -> Dict[str, Any]:
@@ -1316,12 +1327,12 @@ async def user_system_action(username: str, payload: Dict[str, Any]) -> Dict[str
result["jellyseerr"] = {"status": "error", "detail": _http_error_detail(exc)}
if action == "remove":
deletion = delete_user_data_by_username(username)
deleted = bool(deletion.get("deleted"))
deleted = delete_user_by_username(username)
activity_deleted = delete_user_activity_by_username(username)
result["local"] = {
"status": "ok" if deleted else "not_found",
"deleted": bool(deleted),
"data_cleanup": deletion,
"activity_deleted": activity_deleted,
}
if any(
@@ -1583,7 +1594,6 @@ async def update_user_password(username: str, payload: Dict[str, Any]) -> Dict[s
except Exception as exc:
raise HTTPException(status_code=502, detail=f"Jellyfin password update failed: {exc}") from exc
sync_jellyfin_password_state(username, new_password_clean)
increment_user_auth_version(username)
return {"status": "ok", "username": username, "provider": "jellyfin"}
raise HTTPException(
status_code=400,
@@ -1927,11 +1937,6 @@ async def send_invite_email(payload: Dict[str, Any]) -> Dict[str, Any]:
role=invite.get('role'), max_uses=1, enabled=bool(invite.get('enabled')),
expires_at=invite.get('expires_at'), recipient_email=recipient_email,
)
if not invite:
raise HTTPException(status_code=404, detail='Invite not found')
invite = rotate_signup_invite_code(int(invite['id']), _generate_invite_code())
if not invite:
raise HTTPException(status_code=409, detail='Invite is unavailable')
try:
result = await send_templated_email(
@@ -1945,8 +1950,9 @@ async def send_invite_email(payload: Dict[str, Any]) -> Dict[str, Any]:
except Exception as exc:
raise HTTPException(status_code=502, detail=str(exc)) from exc
logger.info(
"Admin sent invite email template: template=%s invite_id=%s username=%s",
"Admin sent invite email template: template=%s recipient=%s invite_id=%s username=%s",
template_key,
result.get("recipient_email"),
invite.get("id") if invite else None,
user.get("username") if user else None,
)
@@ -2012,14 +2018,15 @@ async def create_invite(payload: Dict[str, Any], current_user: Dict[str, Any] =
except Exception as exc:
email_error = str(exc)
logger.info(
"Admin created invite: invite_id=%s label=%s profile_id=%s role=%s max_uses=%s enabled=%s has_recipient=%s send_email=%s",
"Admin created invite: invite_id=%s code=%s label=%s profile_id=%s role=%s max_uses=%s enabled=%s recipient_email=%s send_email=%s",
invite.get("id"),
invite.get("code"),
invite.get("label"),
invite.get("profile_id"),
invite.get("role"),
invite.get("max_uses"),
invite.get("enabled"),
bool(invite.get("recipient_email")),
invite.get("recipient_email"),
send_email,
)
return {
@@ -2042,11 +2049,7 @@ async def edit_invite(invite_id: int, payload: Dict[str, Any]) -> Dict[str, Any]
existing = get_signup_invite_by_id(invite_id)
if not existing:
raise HTTPException(status_code=404, detail="Invite not found")
requested_code = _normalize_optional_text(payload.get("code"))
if requested_code and not requested_code.startswith("••••") and requested_code != "Protected invite":
code = _normalize_invite_code(requested_code)
else:
code = str(existing.get("code") or "")
code = _normalize_invite_code(_normalize_optional_text(payload.get("code")) or existing["code"])
profile_id = _parse_optional_profile_id(payload.get("profile_id"))
enabled = payload.get("enabled")
if not isinstance(enabled, bool):
@@ -2080,10 +2083,6 @@ async def edit_invite(invite_id: int, payload: Dict[str, Any]) -> Dict[str, Any]
email_error = None
if send_email:
try:
rotated = rotate_signup_invite_code(invite_id, _generate_invite_code())
if not rotated:
raise ValueError("Invite is unavailable")
invite = rotated
email_result = await send_templated_email(
"invited",
invite=invite,
@@ -2093,14 +2092,15 @@ async def edit_invite(invite_id: int, payload: Dict[str, Any]) -> Dict[str, Any]
except Exception as exc:
email_error = str(exc)
logger.info(
"Admin updated invite: invite_id=%s label=%s profile_id=%s role=%s max_uses=%s enabled=%s has_recipient=%s send_email=%s",
"Admin updated invite: invite_id=%s code=%s label=%s profile_id=%s role=%s max_uses=%s enabled=%s recipient_email=%s send_email=%s",
invite.get("id"),
invite.get("code"),
invite.get("label"),
invite.get("profile_id"),
invite.get("role"),
invite.get("max_uses"),
invite.get("enabled"),
bool(invite.get("recipient_email")),
invite.get("recipient_email"),
send_email,
)
return {
@@ -2116,22 +2116,6 @@ async def edit_invite(invite_id: int, payload: Dict[str, Any]) -> Dict[str, Any]
}
@router.post("/invites/{invite_id}/rotate")
async def rotate_invite(
invite_id: int,
current_user: Dict[str, Any] = Depends(require_admin),
) -> Dict[str, Any]:
invite = rotate_signup_invite_code(invite_id, _generate_invite_code())
if not invite:
raise HTTPException(status_code=409, detail="Invite is unavailable")
logger.info(
"Admin rotated invite: invite_id=%s actor=%s",
invite_id,
current_user.get("username"),
)
return {"status": "ok", "invite": invite}
@router.delete("/invites/{invite_id}")
async def remove_invite(invite_id: int) -> Dict[str, Any]:
deleted = delete_signup_invite(invite_id)
+115 -120
View File
@@ -1,8 +1,11 @@
from ..feature_guards import require_invites
from datetime import datetime, timedelta, timezone
from collections import defaultdict, deque
import logging
import secrets
import string
import time
from threading import Lock
import httpx
from fastapi import APIRouter, HTTPException, status, Depends, Request, Response
@@ -25,7 +28,6 @@ from ..db import (
list_signup_invites,
create_signup_invite,
update_signup_invite,
rotate_signup_invite_code,
delete_signup_invite,
reserve_signup_invite_use,
release_signup_invite_use,
@@ -37,10 +39,6 @@ from ..db import (
get_global_request_total,
get_setting,
sync_jellyfin_password_state,
increment_user_auth_version,
get_rate_limit_status,
record_rate_limit_event,
clear_rate_limit_events,
)
from ..runtime import get_runtime_settings
from ..clients.jellyfin import JellyfinClient
@@ -60,15 +58,6 @@ from ..auth import (
set_auth_cookies,
)
from ..config import settings
from ..api_models import (
COMMON_ERROR_RESPONSES,
ChangePasswordRequest,
ForgotPasswordRequest,
PasswordResetRequest,
ProfileEmailUpdateRequest,
SignupRequest,
request_data,
)
from ..network_security import request_trusts_forwarded_headers
from ..services.user_cache import (
build_jellyseerr_candidate_map,
@@ -90,7 +79,7 @@ from ..services.password_reset import (
verify_password_reset_token,
)
router = APIRouter(prefix="/auth", tags=["auth"], responses=COMMON_ERROR_RESPONSES)
router = APIRouter(prefix="/auth", tags=["auth"])
logger = logging.getLogger(__name__)
SELF_SERVICE_INVITE_MASTER_ID_KEY = "self_service_invite_master_id"
STREAM_TOKEN_TTL_SECONDS = 120
@@ -98,6 +87,14 @@ PASSWORD_RESET_GENERIC_MESSAGE = (
"If an account exists for that username or email, a password reset link has been sent."
)
_LOGIN_RATE_LOCK = Lock()
_LOGIN_ATTEMPTS_BY_IP: dict[str, deque[float]] = defaultdict(deque)
_LOGIN_ATTEMPTS_BY_USER: dict[str, deque[float]] = defaultdict(deque)
_RESET_RATE_LOCK = Lock()
_RESET_ATTEMPTS_BY_IP: dict[str, deque[float]] = defaultdict(deque)
_RESET_ATTEMPTS_BY_IDENTIFIER: dict[str, deque[float]] = defaultdict(deque)
def _require_recipient_email(value: object) -> str:
normalized = normalize_delivery_email(value)
if normalized:
@@ -148,6 +145,12 @@ def _password_reset_rate_key_identifier(identifier: str) -> str:
return (identifier or "").strip().lower()[:256] or "<empty>"
def _prune_attempts(bucket: deque[float], now: float, window_seconds: int) -> None:
cutoff = now - window_seconds
while bucket and bucket[0] < cutoff:
bucket.popleft()
def _pick_preferred_ci_user_match(users: list[dict], requested_username: str) -> dict | None:
if not users:
return None
@@ -169,33 +172,56 @@ def _pick_preferred_ci_user_match(users: list[dict], requested_username: str) ->
def _record_login_failure(request: Request, username: str) -> None:
now = time.monotonic()
window = max(int(settings.auth_rate_limit_window_seconds or 60), 1)
ip_key = _auth_client_ip(request)
user_key = _login_rate_key_user(username)
record_rate_limit_event("login-ip", ip_key)
record_rate_limit_event("login-user", user_key)
logger.warning("login failure recorded")
with _LOGIN_RATE_LOCK:
ip_bucket = _LOGIN_ATTEMPTS_BY_IP[ip_key]
user_bucket = _LOGIN_ATTEMPTS_BY_USER[user_key]
_prune_attempts(ip_bucket, now, window)
_prune_attempts(user_bucket, now, window)
ip_bucket.append(now)
user_bucket.append(now)
logger.warning("login failure recorded username=%s client=%s", user_key, ip_key)
def _clear_login_failures(request: Request, username: str) -> None:
ip_key = _auth_client_ip(request)
user_key = _login_rate_key_user(username)
clear_rate_limit_events("login-ip", ip_key)
clear_rate_limit_events("login-user", user_key)
with _LOGIN_RATE_LOCK:
_LOGIN_ATTEMPTS_BY_IP.pop(ip_key, None)
_LOGIN_ATTEMPTS_BY_USER.pop(user_key, None)
def _enforce_login_rate_limit(request: Request, username: str) -> None:
now = time.monotonic()
window = max(int(settings.auth_rate_limit_window_seconds or 60), 1)
max_ip = max(int(settings.auth_rate_limit_max_attempts_ip or 20), 1)
max_user = max(int(settings.auth_rate_limit_max_attempts_user or 10), 1)
ip_key = _auth_client_ip(request)
user_key = _login_rate_key_user(username)
ip_exceeded, ip_retry = get_rate_limit_status("login-ip", ip_key, window, max_ip)
user_exceeded, user_retry = get_rate_limit_status("login-user", user_key, window, max_user)
exceeded = ip_exceeded or user_exceeded
retry_after = max(ip_retry if ip_exceeded else 1, user_retry if user_exceeded else 1)
with _LOGIN_RATE_LOCK:
ip_bucket = _LOGIN_ATTEMPTS_BY_IP[ip_key]
user_bucket = _LOGIN_ATTEMPTS_BY_USER[user_key]
_prune_attempts(ip_bucket, now, window)
_prune_attempts(user_bucket, now, window)
exceeded = len(ip_bucket) >= max_ip or len(user_bucket) >= max_user
retry_after = 1
if exceeded:
retry_candidates = []
if ip_bucket:
retry_candidates.append(max(1, int(window - (now - ip_bucket[0]))))
if user_bucket:
retry_candidates.append(max(1, int(window - (now - user_bucket[0]))))
if retry_candidates:
retry_after = max(retry_candidates)
if exceeded:
logger.warning(
"login rate limit exceeded retry_after=%s", retry_after,
"login rate limit exceeded username=%s client=%s retry_after=%s",
user_key,
ip_key,
retry_after,
)
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
@@ -205,28 +231,48 @@ def _enforce_login_rate_limit(request: Request, username: str) -> None:
def _record_password_reset_attempt(request: Request, identifier: str) -> None:
now = time.monotonic()
window = max(int(settings.password_reset_rate_limit_window_seconds or 300), 1)
ip_key = _auth_client_ip(request)
identifier_key = _password_reset_rate_key_identifier(identifier)
record_rate_limit_event("reset-ip", ip_key)
record_rate_limit_event("reset-identifier", identifier_key)
logger.info("password reset rate event recorded")
with _RESET_RATE_LOCK:
ip_bucket = _RESET_ATTEMPTS_BY_IP[ip_key]
identifier_bucket = _RESET_ATTEMPTS_BY_IDENTIFIER[identifier_key]
_prune_attempts(ip_bucket, now, window)
_prune_attempts(identifier_bucket, now, window)
ip_bucket.append(now)
identifier_bucket.append(now)
logger.info("password reset rate event recorded identifier=%s client=%s", identifier_key, ip_key)
def _enforce_password_reset_rate_limit(request: Request, identifier: str) -> None:
now = time.monotonic()
window = max(int(settings.password_reset_rate_limit_window_seconds or 300), 1)
max_ip = max(int(settings.password_reset_rate_limit_max_attempts_ip or 6), 1)
max_identifier = max(int(settings.password_reset_rate_limit_max_attempts_identifier or 3), 1)
ip_key = _auth_client_ip(request)
identifier_key = _password_reset_rate_key_identifier(identifier)
ip_exceeded, ip_retry = get_rate_limit_status("reset-ip", ip_key, window, max_ip)
identifier_exceeded, identifier_retry = get_rate_limit_status(
"reset-identifier", identifier_key, window, max_identifier
)
exceeded = ip_exceeded or identifier_exceeded
retry_after = max(ip_retry if ip_exceeded else 1, identifier_retry if identifier_exceeded else 1)
with _RESET_RATE_LOCK:
ip_bucket = _RESET_ATTEMPTS_BY_IP[ip_key]
identifier_bucket = _RESET_ATTEMPTS_BY_IDENTIFIER[identifier_key]
_prune_attempts(ip_bucket, now, window)
_prune_attempts(identifier_bucket, now, window)
exceeded = len(ip_bucket) >= max_ip or len(identifier_bucket) >= max_identifier
retry_after = 1
if exceeded:
retry_candidates = []
if ip_bucket:
retry_candidates.append(max(1, int(window - (now - ip_bucket[0]))))
if identifier_bucket:
retry_candidates.append(max(1, int(window - (now - identifier_bucket[0]))))
if retry_candidates:
retry_after = max(retry_candidates)
if exceeded:
logger.warning(
"password reset rate limit exceeded retry_after=%s", retry_after,
"password reset rate limit exceeded identifier=%s client=%s retry_after=%s",
identifier_key,
ip_key,
retry_after,
)
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
@@ -354,7 +400,6 @@ def _auth_success_response(response: Response, token: str, user_payload: dict) -
def _public_invite_payload(invite: dict, profile: dict | None = None) -> dict:
return {
"code": invite.get("code"),
"code_available": bool(invite.get("code_available")),
"email_bound": bool(invite.get("recipient_email")),
"label": invite.get("label"),
"description": invite.get("description"),
@@ -448,7 +493,6 @@ def _serialize_self_invite(invite: dict) -> dict:
return {
"id": invite.get("id"),
"code": invite.get("code"),
"code_available": bool(invite.get("code_available")),
"label": invite.get("label"),
"description": invite.get("description"),
"profile_id": invite.get("profile_id"),
@@ -532,7 +576,6 @@ def _serialize_self_service_master_invite(invite: dict | None) -> dict | None:
return {
"id": invite.get("id"),
"code": invite.get("code"),
"code_available": bool(invite.get("code_available")),
"label": invite.get("label"),
"description": invite.get("description"),
"profile_id": invite.get("profile_id"),
@@ -621,9 +664,7 @@ async def login(
detail="This account uses external sign-in. Use the external sign-in option.",
)
_assert_user_can_login(user)
token = create_access_token(
user["username"], user["role"], auth_version=int(user.get("auth_version") or 1)
)
token = create_access_token(user["username"], user["role"])
_clear_login_failures(request, form_data.username)
set_last_login(user["username"])
logger.info(
@@ -667,9 +708,7 @@ async def jellyfin_login(
matched_email = extract_jellyseerr_user_email(matched_seerr_user)
_assert_user_can_login(user)
if user and _has_valid_jellyfin_cache(user, password):
token = create_access_token(
canonical_username, "user", auth_version=int(user.get("auth_version") or 1)
)
token = create_access_token(canonical_username, "user")
_clear_login_failures(request, username)
set_last_login(canonical_username)
logger.info(
@@ -694,13 +733,6 @@ async def jellyfin_login(
if not isinstance(auth_response, dict) or not auth_response.get("User"):
_record_login_failure(request, username)
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid Jellyfin credentials")
from ..services.jellyfin_identity import user_for_identity
identity_owner = user_for_identity(auth_response['User'].get('Id'), runtime.jellyfin_base_url)
if identity_owner:
preferred_match = identity_owner
user = identity_owner
canonical_username = identity_owner['username']
_assert_user_can_login(user)
if not preferred_match:
create_user_if_missing(
canonical_username,
@@ -736,10 +768,7 @@ async def jellyfin_login(
matched_id = match_jellyseerr_user_id(canonical_username, candidate_map)
if matched_id is not None:
set_user_jellyseerr_id(canonical_username, matched_id)
refreshed_user = get_user_by_username(canonical_username) or user or {}
token = create_access_token(
canonical_username, "user", auth_version=int(refreshed_user.get("auth_version") or 1)
)
token = create_access_token(canonical_username, "user")
_clear_login_failures(request, username)
set_last_login(canonical_username)
logger.info(
@@ -786,13 +815,8 @@ async def jellyseerr_login(
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid Seerr credentials")
jellyseerr_user_id = _extract_jellyseerr_user_id(auth_response)
jellyseerr_email = _extract_jellyseerr_response_email(auth_response)
id_matches = [row for row in get_all_users() if jellyseerr_user_id is not None and row.get('jellyseerr_user_id') == jellyseerr_user_id]
if len(id_matches) > 1:
raise HTTPException(409, 'Multiple Magent accounts claim this Seerr identity. Ask an administrator to repair the links.')
ci_matches = get_users_by_username_ci(form_data.username)
preferred_match = id_matches[0] if id_matches else _pick_preferred_ci_user_match(ci_matches, form_data.username)
if preferred_match and preferred_match.get('jellyseerr_user_id') not in (None, jellyseerr_user_id):
raise HTTPException(409, 'The account name and authenticated identity disagree. Ask an administrator to repair the links.')
preferred_match = _pick_preferred_ci_user_match(ci_matches, form_data.username)
canonical_username = str(preferred_match.get("username") or form_data.username) if preferred_match else form_data.username
if not preferred_match:
create_user_if_missing(
@@ -815,10 +839,7 @@ async def jellyseerr_login(
set_user_jellyseerr_id(canonical_username, jellyseerr_user_id)
if jellyseerr_email:
set_user_email(canonical_username, jellyseerr_email)
refreshed_user = get_user_by_username(canonical_username) or user or {}
token = create_access_token(
canonical_username, "user", auth_version=int(refreshed_user.get("auth_version") or 1)
)
token = create_access_token(canonical_username, "user")
_clear_login_failures(request, form_data.username)
set_last_login(canonical_username)
logger.info(
@@ -840,10 +861,7 @@ async def me(current_user: dict = Depends(get_current_user)) -> dict:
@router.post("/logout")
async def logout(
response: Response, current_user: dict = Depends(get_current_user)
) -> dict:
increment_user_auth_version(str(current_user.get("username") or ""))
async def logout(response: Response) -> dict:
clear_auth_cookies(response)
return {"status": "ok"}
@@ -854,7 +872,6 @@ async def stream_token(current_user: dict = Depends(get_current_user)) -> dict:
current_user["username"],
current_user["role"],
expires_seconds=STREAM_TOKEN_TTL_SECONDS,
auth_version=int(current_user.get("auth_version") or 1),
)
return {
"stream_token": token,
@@ -878,8 +895,7 @@ async def invite_details(code: str) -> dict:
@router.post("/signup")
async def signup(payload: SignupRequest, response: Response) -> dict:
payload = request_data(payload)
async def signup(payload: dict, response: Response) -> dict:
if not isinstance(payload, dict):
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid payload")
invite_code = str(payload.get("invite_code") or "").strip()
@@ -895,7 +911,11 @@ async def signup(payload: SignupRequest, response: Response) -> dict:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc
if get_user_by_username(username):
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail="User already exists")
logger.info("signup attempt username=%s", username)
logger.info(
"signup attempt username=%s invite_code=%s",
username,
invite_code,
)
invite = get_signup_invite_by_code(invite_code)
if not invite:
@@ -1007,7 +1027,7 @@ async def signup(payload: SignupRequest, response: Response) -> dict:
auto_search_enabled=auto_search_enabled,
profile_id=int(profile_id) if profile_id is not None else None,
expires_at=expires_at,
invited_by_code=f"invite:{invite.get('id')}",
invited_by_code=invite.get("code"),
)
except Exception as exc:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(exc)) from exc
@@ -1034,18 +1054,15 @@ async def signup(payload: SignupRequest, response: Response) -> dict:
# Welcome email delivery is best-effort and must not break signup.
logger.warning("Welcome email send skipped for %s: %s", username, exc)
_assert_user_can_login(created_user)
refreshed_user = get_user_by_username(username) or created_user or {}
token = create_access_token(
username, role, auth_version=int(refreshed_user.get("auth_version") or 1)
)
token = create_access_token(username, role)
set_last_login(username)
logger.info(
"signup success username=%s role=%s auth_provider=%s profile_id=%s invite_id=%s",
"signup success username=%s role=%s auth_provider=%s profile_id=%s invite_code=%s",
username,
role,
created_user.get("auth_provider") if created_user else auth_provider,
created_user.get("profile_id") if created_user else None,
invite.get("id"),
invite.get("code"),
)
return _auth_success_response(
response,
@@ -1064,8 +1081,7 @@ async def signup(payload: SignupRequest, response: Response) -> dict:
@router.post("/password/forgot")
async def forgot_password(payload: ForgotPasswordRequest, request: Request) -> dict:
payload = request_data(payload)
async def forgot_password(payload: dict, request: Request) -> dict:
if not isinstance(payload, dict):
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid payload")
identifier = payload.get("identifier") or payload.get("username") or payload.get("email")
@@ -1082,7 +1098,8 @@ async def forgot_password(payload: ForgotPasswordRequest, request: Request) -> d
)
client_ip = _auth_client_ip(request)
logger.info("password reset requested")
safe_identifier = identifier.strip().lower()[:256]
logger.info("password reset requested identifier=%s client=%s", safe_identifier, client_ip)
try:
reset_result = await request_password_reset(
identifier,
@@ -1091,17 +1108,24 @@ async def forgot_password(payload: ForgotPasswordRequest, request: Request) -> d
)
if reset_result.get("issued"):
logger.info(
"password reset issued username=%s provider=%s",
"password reset issued username=%s provider=%s recipient=%s client=%s",
reset_result.get("username"),
reset_result.get("auth_provider"),
reset_result.get("recipient_email"),
client_ip,
)
else:
logger.info(
"password reset request completed with no eligible account",
"password reset request completed with no eligible account identifier=%s client=%s",
safe_identifier,
client_ip,
)
except Exception as exc:
logger.warning(
"password reset email dispatch failed detail=%s", type(exc).__name__,
"password reset email dispatch failed identifier=%s client=%s detail=%s",
safe_identifier,
client_ip,
str(exc),
)
return {"status": "ok", "message": PASSWORD_RESET_GENERIC_MESSAGE}
@@ -1117,8 +1141,7 @@ async def password_reset_verify(token: str) -> dict:
@router.post("/password/reset")
async def password_reset(payload: PasswordResetRequest) -> dict:
payload = request_data(payload)
async def password_reset(payload: dict) -> dict:
if not isinstance(payload, dict):
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid payload")
token = payload.get("token")
@@ -1181,10 +1204,7 @@ async def profile(current_user: dict = Depends(get_current_user)) -> dict:
@router.put("/profile/email")
async def update_profile_email(
payload: ProfileEmailUpdateRequest, current_user: dict = Depends(get_current_user)
) -> dict:
payload = request_data(payload)
async def update_profile_email(payload: dict, current_user: dict = Depends(get_current_user)) -> dict:
if not isinstance(payload, dict):
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid payload")
username = str(current_user.get("username") or "").strip()
@@ -1339,13 +1359,8 @@ async def update_profile_invite(
_require_self_service_invite_access(current_user)
existing = _get_owned_invite(invite_id, current_user)
requested_code = payload.get("code")
if (
isinstance(requested_code, str)
and requested_code.strip()
and not requested_code.strip().startswith("••••")
and requested_code.strip() != "Protected invite"
):
requested_code = payload.get("code", existing.get("code"))
if isinstance(requested_code, str) and requested_code.strip():
code = _normalize_invite_code(requested_code)
else:
code = str(existing.get("code") or "").strip()
@@ -1400,10 +1415,6 @@ async def update_profile_invite(
email_error = None
if send_email:
try:
rotated = rotate_signup_invite_code(invite_id, _generate_invite_code())
if not rotated:
raise ValueError("Invite is unavailable")
invite = rotated
email_result = await send_templated_email(
"invited",
invite=invite,
@@ -1427,18 +1438,6 @@ async def update_profile_invite(
}
@router.post("/profile/invites/{invite_id}/rotate")
async def rotate_profile_invite(
invite_id: int, current_user: dict = Depends(get_current_user)
) -> dict:
_require_self_service_invite_access(current_user)
_get_owned_invite(invite_id, current_user)
invite = rotate_signup_invite_code(invite_id, _generate_invite_code())
if not invite:
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail="Invite is unavailable")
return {"status": "ok", "invite": _serialize_self_invite(invite)}
@router.delete("/profile/invites/{invite_id}")
async def delete_profile_invite(invite_id: int, current_user: dict = Depends(get_current_user)) -> dict:
_require_self_service_invite_access(current_user)
@@ -1450,10 +1449,7 @@ async def delete_profile_invite(invite_id: int, current_user: dict = Depends(get
@router.post("/password")
async def change_password(
payload: ChangePasswordRequest, current_user: dict = Depends(get_current_user)
) -> dict:
payload = request_data(payload)
async def change_password(payload: dict, current_user: dict = Depends(get_current_user)) -> dict:
current_password = payload.get("current_password") if isinstance(payload, dict) else None
new_password = payload.get("new_password") if isinstance(payload, dict) else None
if not isinstance(current_password, str) or not isinstance(new_password, str):
@@ -1523,7 +1519,6 @@ async def change_password(
# Keep Magent's password hash and Jellyfin auth cache aligned with Jellyfin.
sync_jellyfin_password_state(username, new_password_clean)
increment_user_auth_version(username)
logger.info("password change completed username=%s provider=jellyfin", username)
return {"status": "ok", "provider": "jellyfin"}
-85
View File
@@ -1,85 +0,0 @@
"""Administrator-only encrypted backup downloads and staged restores."""
from typing import Literal
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
from fastapi.responses import Response
from pydantic import BaseModel, ConfigDict, Field, SecretStr
from starlette.concurrency import run_in_threadpool
from ..auth import require_admin
from ..db import get_rate_limit_status, record_rate_limit_event
from ..services import backups
def _no_store(response: Response) -> None:
response.headers["Cache-Control"] = "no-store"
response.headers["Pragma"] = "no-cache"
router = APIRouter(
prefix="/admin/backups", tags=["backups"],
dependencies=[Depends(require_admin), Depends(_no_store)],
)
class ExportRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
passphrase: SecretStr = Field(min_length=12, max_length=1024)
include_cache: bool = False
def _rate_limit(user: dict) -> None:
key = str(user["username"])
exceeded, retry = get_rate_limit_status("backups", key, 300, 3)
if exceeded:
raise HTTPException(429, "Too many backup operations; try again shortly", headers={"Retry-After": str(retry)})
record_rate_limit_event("backups", key)
@router.get("")
def status() -> dict:
return backups.backup_status()
@router.post("/export")
def export(payload: ExportRequest, user: dict = Depends(require_admin)) -> Response:
_rate_limit(user)
try:
content, filename = backups.create_backup(payload.passphrase.get_secret_value(), payload.include_cache)
except backups.BackupError as exc:
raise HTTPException(400, str(exc)) from exc
return Response(content, media_type="application/octet-stream", headers={
"Content-Disposition": f'attachment; filename="{filename}"',
"Cache-Control": "no-store", "Pragma": "no-cache",
})
@router.post("/restore", status_code=202)
async def restore(
file: UploadFile = File(...),
passphrase: str = Form(..., min_length=12, max_length=1024),
confirmation: Literal["RESTORE"] = Form(...),
user: dict = Depends(require_admin),
) -> dict:
_rate_limit(user)
try:
if file.size is not None and file.size > backups.MAX_UPLOAD_BYTES:
raise HTTPException(413, "Backup exceeds the 32 MiB upload limit")
metadata = await run_in_threadpool(backups.stage_restore, file.file, passphrase)
except backups.BackupError as exc:
raise HTTPException(400, str(exc)) from exc
finally:
await file.close()
return {
"status": "staged", "restart_required": True, "backup": metadata,
"message": "Backup validated. Restart Magent to apply it. Current data remains active until restart.",
}
@router.delete("/restore")
def cancel() -> dict:
try:
backups.cancel_restore()
except backups.BackupError as exc:
raise HTTPException(409, str(exc)) from exc
return {"status": "cancelled"}
+6 -24
View File
@@ -1,9 +1,8 @@
import os
import warnings
from io import BytesIO
from typing import Any, Dict
from fastapi import APIRouter, HTTPException, UploadFile
from fastapi import APIRouter, HTTPException, UploadFile, File
from fastapi.responses import FileResponse
from PIL import Image, ImageDraw, ImageFont
@@ -16,10 +15,6 @@ _BUNDLED_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "as
_BUNDLED_LOGO_PATH = os.path.join(_BUNDLED_DIR, "logo.png")
_BUNDLED_FAVICON_PATH = os.path.join(_BUNDLED_DIR, "favicon.ico")
_BRANDING_SOURCE = os.getenv("BRANDING_SOURCE", "bundled").lower()
_MAX_UPLOAD_BYTES = 5 * 1024 * 1024
_MAX_IMAGE_PIXELS = 25_000_000
_ALLOWED_IMAGE_TYPES = {"image/png", "image/jpeg", "image/webp"}
_ALLOWED_IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp"}
def _ensure_branding_dir() -> None:
@@ -115,27 +110,14 @@ async def branding_favicon() -> FileResponse:
async def save_branding_image(file: UploadFile) -> Dict[str, Any]:
content_type = str(file.content_type or "").lower()
extension = os.path.splitext(str(file.filename or ""))[1].lower()
if content_type not in _ALLOWED_IMAGE_TYPES or extension not in _ALLOWED_IMAGE_EXTENSIONS:
raise HTTPException(status_code=400, detail="Upload a PNG, JPEG, or WebP image.")
content = await file.read(_MAX_UPLOAD_BYTES + 1)
if not file.content_type or not file.content_type.startswith("image/"):
raise HTTPException(status_code=400, detail="Please upload an image file.")
content = await file.read()
if not content:
raise HTTPException(status_code=400, detail="Uploaded file is empty.")
if len(content) > _MAX_UPLOAD_BYTES:
raise HTTPException(status_code=413, detail="Image is too large (maximum 5 MB).")
try:
with warnings.catch_warnings():
warnings.simplefilter("error", Image.DecompressionBombWarning)
candidate = Image.open(BytesIO(content))
if candidate.format not in {"PNG", "JPEG", "WEBP"}:
raise ValueError("Unsupported image format")
if candidate.width * candidate.height > _MAX_IMAGE_PIXELS:
raise Image.DecompressionBombError("Image pixel limit exceeded")
candidate.verify()
image = Image.open(BytesIO(content))
image.load()
except (OSError, ValueError, Image.DecompressionBombError, Image.DecompressionBombWarning) as exc:
image = Image.open(BytesIO(content))
except OSError as exc:
raise HTTPException(status_code=400, detail="Image file could not be read.") from exc
_ensure_branding_dir()
+1 -1
View File
@@ -3,7 +3,7 @@ import re
import mimetypes
import logging
from typing import Optional
from fastapi import APIRouter, HTTPException
from fastapi import APIRouter, HTTPException, Response
from fastapi.responses import FileResponse, RedirectResponse
import httpx
+3 -5
View File
@@ -6,7 +6,6 @@ from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Query, Response
from pydantic import Field, field_validator
from ..services.public_urls import magent_public_url
from ..auth import get_current_user, require_admin
from ..runtime import get_runtime_settings
from ..services import newsletters as service, newsletter_store as store, newsletter_catalog as catalog
@@ -20,7 +19,7 @@ class Settings(StrictPayload):
weekday: int = Field(ge=0, le=6)
hour: int = Field(ge=0, le=23)
limit_titles: int = Field(ge=1, le=24)
public_url: str = Field(default="", max_length=500)
public_url: str = Field(max_length=500)
intro: str = Field(default='', max_length=2000)
revision: int = Field(ge=1)
_url = field_validator('public_url')(RecapSettings.origin_only.__func__)
@@ -115,11 +114,10 @@ def overview(offset: int = Query(default=0, ge=0, le=1_000_000), user: dict = De
@router.put('/admin/newsletters')
def settings(payload: Settings, user: dict = Depends(require_admin)):
try:
public_url = magent_public_url(payload.public_url or store.settings()['public_url'])
ready, detail = service.delivery_ready(public_url)
ready, detail = service.delivery_ready(payload.public_url)
if payload.enabled and not ready:
raise service.NewsletterError(detail)
return store.save_settings({**payload.model_dump(), "public_url": public_url}, datetime.now(timezone.utc))
return store.save_settings(payload.model_dump(), datetime.now(timezone.utc))
except (service.NewsletterError, store.Conflict) as exc:
fail(exc)
+1 -7
View File
@@ -11,7 +11,6 @@ import httpx
from fastapi import APIRouter, Depends, HTTPException, Query
from ..auth import get_current_user
from ..api_models import COMMON_ERROR_RESPONSES
from ..clients.jellyfin import JellyfinClient
from ..db import (
add_portal_item_activity,
@@ -35,12 +34,7 @@ from ..services.issue_resolution import (
from ..services.notifications import send_portal_notification
from ..runtime import get_runtime_settings
router = APIRouter(
prefix="/portal",
tags=["portal"],
dependencies=[Depends(get_current_user), Depends(require_portal_access)],
responses=COMMON_ERROR_RESPONSES,
)
router = APIRouter(prefix="/portal", tags=["portal"], dependencies=[Depends(get_current_user), Depends(require_portal_access)])
logger = logging.getLogger(__name__)
PORTAL_KINDS = {"request", "issue", "feature"}
+4 -5
View File
@@ -6,8 +6,7 @@ from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException, Query, Response
from pydantic import BaseModel, ConfigDict, Field, field_validator
from ..services.public_urls import magent_public_url
from ..auth import require_admin
from ..auth import get_current_user, require_admin
from ..feature_guards import require_stats
from ..services import email_recaps as recaps, recap_store as store
@@ -32,7 +31,7 @@ class RecapSettings(StrictPayload):
enabled: bool
day: int = Field(ge=1, le=28)
hour: int = Field(ge=0, le=23)
public_url: str = Field(default="", max_length=500)
public_url: str = Field(max_length=500)
@field_validator("public_url")
@classmethod
@@ -115,9 +114,9 @@ def settings(payload: RecapSettings, user: dict = Depends(require_admin)) -> dic
# Validate against the proposed URL without writing any partial settings.
ready, detail = recaps.smtp_email_config_ready()
runtime = recaps.get_runtime_settings()
if not magent_public_url(payload.public_url or store.settings()["public_url"]) or not ready or not recaps.worker_enabled() or not runtime.jellystat_base_url or not runtime.jellystat_api_key:
if not payload.public_url or not ready or not recaps.worker_enabled() or not runtime.jellystat_base_url or not runtime.jellystat_api_key:
raise HTTPException(409, "Set the public address, enable SMTP email and connect Jellystat before starting the schedule." if ready else detail)
return store.save_settings({**payload.model_dump(), "public_url": magent_public_url(payload.public_url or store.settings()["public_url"])}, datetime.now(timezone.utc))
return store.save_settings(payload.model_dump(), datetime.now(timezone.utc))
@router.get("/admin/email-recaps/preview")
File diff suppressed because it is too large Load Diff
-92
View File
@@ -1,92 +0,0 @@
"""Initial install bootstrap and authenticated setup wizard endpoints."""
from inspect import isawaitable
from fastapi import APIRouter, Depends, HTTPException, Request, Response
from pydantic import Field, SecretStr
from ..api_models import COMMON_ERROR_RESPONSES, StrictRequest
from ..auth import _extract_client_ip, require_admin
from ..services import setup as setup_service
from ..installation_origin import normalize_application_origin
from ..services.request_origins import can_claim_initial_origin
router = APIRouter(prefix="/setup", tags=["setup"], responses=COMMON_ERROR_RESPONSES)
class BootstrapRequest(StrictRequest):
setup_token: SecretStr = Field(min_length=1, max_length=1024)
username: str = Field(min_length=1, max_length=100)
password: SecretStr = Field(min_length=1, max_length=1024)
application_url: str | None = Field(default=None, max_length=2048)
class SetupProgress(StrictRequest):
step: setup_service.SetupStep
@router.get("/status")
def public_status(response: Response) -> dict:
response.headers["Cache-Control"] = "no-store"
return setup_service.get_public_setup_status()
@router.post("/bootstrap", status_code=201)
def bootstrap(payload: BootstrapRequest, request: Request) -> dict:
status = setup_service.get_public_setup_status()
if not status["needs_admin"]:
raise HTTPException(status_code=409, detail="Initial administrator setup is no longer available.")
retry_after = setup_service.consume_bootstrap_attempt(_extract_client_ip(request))
if retry_after is not None:
raise HTTPException(
status_code=429,
detail="Too many setup attempts. Try again later.",
headers={"Retry-After": str(retry_after)},
)
try:
application_url = payload.application_url
if application_url is not None:
application_url = normalize_application_origin(application_url)
origin = request.headers.get("origin", "")
if not origin or application_url != normalize_application_origin(origin):
raise HTTPException(status_code=403, detail="The site address must match the address open in your browser.")
elif can_claim_initial_origin():
raise HTTPException(status_code=400, detail="Confirm the application URL to create the administrator.")
setup_service.bootstrap_administrator(
payload.setup_token.get_secret_value(), payload.username, payload.password.get_secret_value(),
application_url=application_url,
)
except setup_service.InvalidSetupTokenError as exc:
raise HTTPException(status_code=403, detail=str(exc)) from exc
except setup_service.SetupUnavailableError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
return {"status": "created", "username": payload.username.strip()}
@router.get("/state", dependencies=[Depends(require_admin)])
def get_state() -> dict:
return setup_service.get_setup_state()
@router.put("/state", dependencies=[Depends(require_admin)])
def update_state(payload: SetupProgress) -> dict:
return setup_service.update_setup_step(payload.step)
@router.post("/complete", dependencies=[Depends(require_admin)])
async def finish_setup(request: Request) -> dict:
try:
state = setup_service.complete_setup()
except setup_service.SetupUnavailableError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
# Startup owns worker lifecycle. Its callback must be idempotent so retries
# after a network interruption cannot start duplicate import/automation jobs.
callback = getattr(request.app.state, "on_setup_complete", None)
if callback is not None:
result = callback()
if isawaitable(result):
await result
return state
-5
View File
@@ -5,7 +5,6 @@ from fastapi import APIRouter, Depends
from ..auth import get_current_user
from ..build_info import BUILD_NUMBER, CHANGELOG
from ..config import normalize_banner_color
from ..runtime import get_runtime_settings
router = APIRouter(prefix="/site", tags=["site"])
@@ -16,7 +15,6 @@ _BANNER_TONES = {"info", "warning", "error", "maintenance"}
def _build_site_info(include_changelog: bool) -> Dict[str, Any]:
runtime = get_runtime_settings()
banner_message = (runtime.site_banner_message or "").strip()
login_message = (runtime.site_login_message or "").strip()
tone = (runtime.site_banner_tone or "info").strip().lower()
if tone not in _BANNER_TONES:
tone = "info"
@@ -26,11 +24,8 @@ def _build_site_info(include_changelog: bool) -> Dict[str, Any]:
"enabled": bool(runtime.site_banner_enabled and banner_message),
"message": banner_message,
"tone": tone,
"backgroundColor": normalize_banner_color(runtime.site_banner_background_color),
"borderColor": normalize_banner_color(runtime.site_banner_border_color),
},
"login": {
"message": login_message,
"showJellyfinLogin": bool(runtime.site_login_show_jellyfin_login),
"showLocalLogin": bool(runtime.site_login_show_local_login),
"showForgotPassword": bool(runtime.site_login_show_forgot_password),
-1
View File
@@ -17,7 +17,6 @@ _INT_FIELDS = {
"log_file_backup_count",
"requests_sync_ttl_minutes",
"requests_poll_interval_seconds",
"requests_stage_refresh_minutes",
"requests_delta_sync_interval_minutes",
"requests_cleanup_days",
"issue_confirmation_contact_attempts",
-116
View File
@@ -1,116 +0,0 @@
"""Transactional, versioned SQLite schema migrations for Magent."""
from __future__ import annotations
import sqlite3
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Callable
MigrationStep = Callable[[sqlite3.Connection], None]
@dataclass(frozen=True)
class Migration:
version: int
name: str
apply: MigrationStep
def _column_names(conn: sqlite3.Connection, table: str) -> set[str]:
return {str(row[1]) for row in conn.execute(f'PRAGMA table_info("{table}")').fetchall()}
def _add_column(conn: sqlite3.Connection, table: str, definition: str) -> None:
column = definition.split(maxsplit=1)[0].strip('"')
if column not in _column_names(conn, table):
conn.execute(f'ALTER TABLE "{table}" ADD COLUMN {definition}')
def _migration_001_legacy_columns_and_indexes(conn: sqlite3.Connection) -> None:
for definition in (
"email TEXT",
"last_login_at TEXT",
"is_blocked INTEGER NOT NULL DEFAULT 0",
"auth_provider TEXT NOT NULL DEFAULT 'local'",
"jellyfin_password_hash TEXT",
"last_jellyfin_auth_at TEXT",
"jellyseerr_user_id INTEGER",
"auto_search_enabled INTEGER NOT NULL DEFAULT 1",
"invite_management_enabled INTEGER NOT NULL DEFAULT 0",
"profile_id INTEGER",
"expires_at TEXT",
"invited_by_code TEXT",
"invited_at TEXT",
"auth_version INTEGER NOT NULL DEFAULT 1",
):
_add_column(conn, "users", definition)
for definition in ("recipient_email TEXT", "code_hint TEXT"):
_add_column(conn, "signup_invites", definition)
for definition in (
"related_item_id INTEGER",
"workflow_request_status TEXT",
"workflow_media_status TEXT",
"issue_type TEXT",
"issue_resolved_at TEXT",
"metadata_json TEXT",
):
_add_column(conn, "portal_items", definition)
_add_column(conn, "requests_cache", "requested_by_id INTEGER")
statements = (
"CREATE INDEX IF NOT EXISTS idx_portal_items_workflow ON portal_items "
"(kind, workflow_request_status, workflow_media_status, updated_at DESC, id DESC)",
"CREATE INDEX IF NOT EXISTS idx_portal_items_related_item ON portal_items "
"(related_item_id, updated_at DESC, id DESC)",
"CREATE INDEX IF NOT EXISTS idx_users_profile_id ON users (profile_id)",
"CREATE INDEX IF NOT EXISTS idx_users_expires_at ON users (expires_at)",
"CREATE INDEX IF NOT EXISTS idx_users_username_nocase ON users (username COLLATE NOCASE)",
"CREATE INDEX IF NOT EXISTS idx_users_email_nocase ON users (email COLLATE NOCASE)",
"CREATE INDEX IF NOT EXISTS idx_requests_cache_requested_by_id ON requests_cache (requested_by_id)",
"CREATE INDEX IF NOT EXISTS idx_requests_cache_requested_by_id_created_at ON requests_cache "
"(requested_by_id, created_at DESC, request_id DESC)",
)
for statement in statements:
conn.execute(statement)
MIGRATIONS = (
Migration(1, "legacy_columns_and_indexes", _migration_001_legacy_columns_and_indexes),
)
def run_schema_migrations(conn: sqlite3.Connection) -> list[int]:
conn.execute(
"""
CREATE TABLE IF NOT EXISTS schema_migrations (
version INTEGER PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
applied_at TEXT NOT NULL
)
"""
)
applied = {int(row[0]) for row in conn.execute("SELECT version FROM schema_migrations")}
completed: list[int] = []
for migration in MIGRATIONS:
if migration.version in applied:
continue
savepoint = f"magent_migration_{migration.version}"
conn.execute(f"SAVEPOINT {savepoint}")
try:
migration.apply(conn)
conn.execute(
"INSERT INTO schema_migrations (version, name, applied_at) VALUES (?, ?, ?)",
(migration.version, migration.name, datetime.now(timezone.utc).isoformat()),
)
conn.execute(f"RELEASE SAVEPOINT {savepoint}")
except Exception:
conn.execute(f"ROLLBACK TO SAVEPOINT {savepoint}")
conn.execute(f"RELEASE SAVEPOINT {savepoint}")
raise
completed.append(migration.version)
return completed
-74
View File
@@ -1,74 +0,0 @@
import base64
import hashlib
from typing import Optional
from cryptography.fernet import Fernet, InvalidToken
from .config import settings
ENCRYPTED_PREFIX = "enc:v1:"
SENSITIVE_SETTING_KEYS = frozenset(
{
"jellystat_api_key", "magent_ssl_certificate_pem", "magent_ssl_private_key_pem",
"magent_notify_email_smtp_password", "magent_notify_discord_webhook_url",
"magent_notify_telegram_bot_token", "magent_notify_push_token",
"magent_notify_push_user_key", "magent_notify_webhook_url", "jellyseerr_api_key",
"jellyfin_api_key", "sonarr_api_key", "radarr_api_key", "bazarr_api_key",
"prowlarr_api_key", "qbittorrent_password", "discord_webhook_url",
}
)
def _fernet_key() -> bytes:
configured = str(settings.settings_encryption_key or "").strip()
if configured:
try:
decoded = base64.urlsafe_b64decode(configured.encode("ascii"))
except Exception as exc:
raise RuntimeError("SETTINGS_ENCRYPTION_KEY must be a valid Fernet key") from exc
if len(decoded) != 32:
raise RuntimeError("SETTINGS_ENCRYPTION_KEY must decode to exactly 32 bytes")
return configured.encode("ascii")
jwt_secret = str(settings.jwt_secret or "").strip()
if len(jwt_secret) < 32 or jwt_secret == "change-me":
raise RuntimeError(
"SETTINGS_ENCRYPTION_KEY is required when JWT_SECRET is not a strong migration key"
)
derived = hashlib.sha256(("magent-settings-v1:" + jwt_secret).encode("utf-8")).digest()
return base64.urlsafe_b64encode(derived)
def is_sensitive_setting(key: str) -> bool:
return str(key or "").strip().lower() in SENSITIVE_SETTING_KEYS
def validate_secret_storage_configuration() -> None:
"""Validate the configured or JWT-derived Fernet key without touching stored data."""
Fernet(_fernet_key())
def encrypt_setting_value(key: str, value: Optional[str]) -> Optional[str]:
if value is None or not is_sensitive_setting(key):
return value
text = str(value)
if text.startswith(ENCRYPTED_PREFIX):
return text
token = Fernet(_fernet_key()).encrypt(text.encode("utf-8")).decode("ascii")
return ENCRYPTED_PREFIX + token
def decrypt_setting_value(key: str, value: Optional[str]) -> Optional[str]:
if value is None or not is_sensitive_setting(key):
return value
text = str(value)
if not text.startswith(ENCRYPTED_PREFIX):
return text
try:
return Fernet(_fernet_key()).decrypt(
text[len(ENCRYPTED_PREFIX) :].encode("ascii")
).decode("utf-8")
except InvalidToken as exc:
raise RuntimeError(
f"Stored secret '{key}' cannot be decrypted with the configured key"
) from exc
+8 -51
View File
@@ -1,5 +1,4 @@
from datetime import datetime, timedelta, timezone
import uuid
from typing import Any, Dict, Optional
from passlib.context import CryptContext
@@ -8,15 +7,9 @@ from jwt import InvalidTokenError
from .config import settings
_pwd_context = CryptContext(
schemes=["argon2", "pbkdf2_sha256"],
deprecated=["pbkdf2_sha256"],
argon2__memory_cost=65536,
argon2__time_cost=3,
argon2__parallelism=4,
)
_pwd_context = CryptContext(schemes=["pbkdf2_sha256"], deprecated="auto")
_ALGORITHM = "HS256"
MIN_PASSWORD_LENGTH = 12
MIN_PASSWORD_LENGTH = 8
PASSWORD_POLICY_MESSAGE = f"Password must be at least {MIN_PASSWORD_LENGTH} characters."
@@ -25,17 +18,7 @@ def hash_password(password: str) -> str:
def verify_password(plain_password: str, hashed_password: str) -> bool:
try:
return _pwd_context.verify(plain_password, hashed_password)
except (TypeError, ValueError):
return False
def verify_and_update_password(plain_password: str, hashed_password: str) -> tuple[bool, Optional[str]]:
try:
return _pwd_context.verify_and_update(plain_password, hashed_password)
except (TypeError, ValueError):
return False, None
return _pwd_context.verify(plain_password, hashed_password)
def validate_password_policy(password: str) -> str:
@@ -51,58 +34,32 @@ def _create_token(
*,
expires_at: datetime,
token_type: str = "access",
auth_version: int = 1,
) -> str:
issued_at = datetime.now(timezone.utc)
payload: Dict[str, Any] = {
"sub": subject,
"role": role,
"typ": token_type,
"exp": expires_at,
"iat": issued_at,
"jti": uuid.uuid4().hex,
"iss": settings.jwt_issuer,
"aud": settings.jwt_audience,
"ver": max(1, int(auth_version or 1)),
}
return jwt.encode(payload, settings.jwt_secret, algorithm=_ALGORITHM)
def create_access_token(
subject: str,
role: str,
expires_minutes: Optional[int] = None,
*,
auth_version: int = 1,
) -> str:
def create_access_token(subject: str, role: str, expires_minutes: Optional[int] = None) -> str:
if not settings.jwt_secret:
raise ValueError("JWT_SECRET is not configured")
minutes = expires_minutes or settings.jwt_exp_minutes
expires = datetime.now(timezone.utc) + timedelta(minutes=minutes)
return _create_token(subject, role, expires_at=expires, token_type="access", auth_version=auth_version)
return _create_token(subject, role, expires_at=expires, token_type="access")
def create_stream_token(
subject: str,
role: str,
expires_seconds: int = 120,
*,
auth_version: int = 1,
) -> str:
def create_stream_token(subject: str, role: str, expires_seconds: int = 120) -> str:
expires = datetime.now(timezone.utc) + timedelta(seconds=max(30, int(expires_seconds or 120)))
return _create_token(subject, role, expires_at=expires, token_type="sse", auth_version=auth_version)
return _create_token(subject, role, expires_at=expires, token_type="sse")
def decode_token(token: str) -> Dict[str, Any]:
if not settings.jwt_secret:
raise ValueError("JWT_SECRET is not configured")
return jwt.decode(
token,
settings.jwt_secret,
algorithms=[_ALGORITHM],
audience=settings.jwt_audience,
issuer=settings.jwt_issuer,
options={"require": ["exp", "iat", "jti", "iss", "aud", "sub", "typ", "ver"]},
)
return jwt.decode(token, settings.jwt_secret, algorithms=[_ALGORITHM])
class TokenError(Exception):
-21
View File
@@ -1,21 +0,0 @@
"""Shared Sonarr/Radarr configuration helpers."""
from typing import Any
class RootFolderNotFoundError(ValueError):
pass
async def resolve_root_folder_path(client: Any, root_folder: str, service_name: str) -> str:
configured = str(root_folder or "").strip()
if not configured.isdigit():
return configured
folders = await client.get_root_folders()
if isinstance(folders, list):
for folder in folders:
if isinstance(folder, dict) and folder.get("id") == int(configured):
path = str(folder.get("path") or "").strip()
if path:
return path
raise RootFolderNotFoundError(f"{service_name} root folder id {configured} not found")
-647
View File
@@ -1,647 +0,0 @@
"""Encrypted, portable backups and restart-only SQLite restores.
Restore is deliberately a two-step operation: the authenticated request validates
and stages it, then a single backend process applies it before opening the DB.
A durable journal and a private rollback copy protect interrupted installations.
"""
from __future__ import annotations
from contextlib import closing, contextmanager
from datetime import datetime, timezone
import hashlib
import io
import json
import os
from pathlib import Path, PurePosixPath
import re
import secrets
import shutil
import sqlite3
import stat
import tempfile
import threading
import time
from typing import Any, BinaryIO, Iterator
import uuid
import zipfile
import zlib
from cryptography.exceptions import InvalidTag
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from cryptography.hazmat.primitives.kdf.scrypt import Scrypt
from pydantic import TypeAdapter
from ..config import Settings, settings
from ..db import _db_path
from ..installation_origin import managed_runtime, normalize_application_origin
from ..schema_migrations import MIGRATIONS
from ..secret_storage import SENSITIVE_SETTING_KEYS, decrypt_setting_value, encrypt_setting_value
FORMAT_VERSION = 1
MAX_UPLOAD_BYTES = 32 * 1024 * 1024
MAX_EXPANDED_BYTES = 128 * 1024 * 1024
MAX_ENTRIES = 20_000
MAGIC = b"MAGENT-BACKUP\x00\x01"
_LOCK = threading.Lock()
_ASSET_NAME = re.compile(r"^[A-Za-z0-9_.-]+$")
_TMDB_SIZES = {"w92", "w154", "w185", "w342", "w500", "w780", "original"}
# Host identity, process controls and local file locations belong to the target.
_LOCAL_FIELDS = {
"sqlite_path", "sqlite_journal_mode", "jwt_secret", "settings_encryption_key",
"admin_username", "admin_password", "setup_token", "app_name", "cors_allow_origin",
"auth_cookie_name", "auth_cookie_secure", "auth_cookie_samesite", "auth_cookie_domain",
"auth_state_cookie_name", "jwt_issuer", "jwt_audience", "api_docs_enabled",
"log_file", "magent_application_port", "magent_api_port", "magent_bind_host",
"magent_proxy_trusted_proxies", "magent_proxy_trust_forwarded_headers",
"magent_ssl_bind_enabled", "magent_ssl_certificate_path", "magent_ssl_private_key_path",
"magent_ssl_certificate_pem", "magent_ssl_private_key_pem",
"site_build_number", "site_changelog", "magent_allow_private_notification_targets",
}
class BackupError(ValueError):
"""A safe-to-display backup validation or state error."""
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _assets_root() -> Path:
# Matches the image and branding routers, independently of SQLITE_PATH.
return Path.cwd() / "data"
def _control_root() -> Path:
return Path(_db_path()).absolute().parent / "backups"
def _private_dir(path: Path) -> None:
if path.is_symlink():
raise BackupError("Backup directories must not be symbolic links")
path.mkdir(parents=True, exist_ok=True, mode=0o700)
path.chmod(0o700)
def _write_private(path: Path, content: bytes) -> None:
with path.open("xb") as handle:
path.chmod(0o600)
handle.write(content)
handle.flush()
os.fsync(handle.fileno())
def _write_json(path: Path, data: dict) -> None:
temporary = path.with_name(path.name + ".tmp-" + uuid.uuid4().hex)
try:
_write_private(temporary, json.dumps(data, separators=(",", ":")).encode())
os.replace(temporary, path)
_sync_directory(path.parent)
finally:
temporary.unlink(missing_ok=True)
def _sync_directory(path: Path) -> None:
if os.name != "nt":
descriptor = os.open(path, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(descriptor)
finally:
os.close(descriptor)
def _sync_tree(path: Path) -> None:
for parent, _directories, files in os.walk(path, topdown=False):
for filename in files:
with (Path(parent) / filename).open("r+b") as handle:
os.fsync(handle.fileno())
_sync_directory(Path(parent))
@contextmanager
def _exclusive_operation() -> Iterator[None]:
if not _LOCK.acquire(blocking=False):
raise BackupError("Another backup or restore operation is in progress")
handle = None
locked = False
try:
root = _control_root()
_private_dir(root)
handle = (root / "operation.lock").open("a+b")
os.chmod(handle.name, 0o600)
# OS locks are released even if a process crashes; support the dev host too.
if os.name == "nt":
import msvcrt
handle.seek(0)
if not handle.read(1):
handle.write(b"0")
handle.flush()
handle.seek(0)
try:
msvcrt.locking(handle.fileno(), msvcrt.LK_NBLCK, 1)
except OSError as exc:
raise BackupError("Another backup or restore operation is in progress") from exc
else:
import fcntl
try:
fcntl.flock(handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
except OSError as exc:
raise BackupError("Another backup or restore operation is in progress") from exc
locked = True
yield
finally:
if handle is not None:
if locked:
if os.name == "nt":
import msvcrt
handle.seek(0)
msvcrt.locking(handle.fileno(), msvcrt.LK_UNLCK, 1)
else:
import fcntl
fcntl.flock(handle.fileno(), fcntl.LOCK_UN)
handle.close()
_LOCK.release()
def validate_passphrase(passphrase: str) -> None:
if not isinstance(passphrase, str) or not 12 <= len(passphrase) <= 1024:
raise BackupError("Use a backup passphrase between 12 and 1024 characters")
def _key(passphrase: str, salt: bytes) -> bytes:
validate_passphrase(passphrase)
return Scrypt(salt=salt, length=32, n=2**15, r=8, p=1).derive(passphrase.encode("utf-8"))
def _encrypt(content: bytes, passphrase: str) -> bytes:
salt, nonce = os.urandom(16), os.urandom(12)
header = MAGIC + salt + nonce
return header + AESGCM(_key(passphrase, salt)).encrypt(nonce, content, header)
def _decrypt(content: bytes, passphrase: str) -> bytes:
header_size = len(MAGIC) + 28
if len(content) > MAX_UPLOAD_BYTES:
raise BackupError("Backup exceeds the 32 MiB upload limit")
if len(content) < header_size + 16 or not content.startswith(MAGIC):
raise BackupError("This is not a supported encrypted Magent backup")
salt = content[len(MAGIC):len(MAGIC) + 16]
nonce = content[len(MAGIC) + 16:header_size]
try:
return AESGCM(_key(passphrase, salt)).decrypt(nonce, content[header_size:], content[:header_size])
except InvalidTag as exc:
raise BackupError("Incorrect passphrase or damaged backup") from exc
def _database_copy(source: Path, destination: Path) -> None:
if not source.is_file() or source.is_symlink():
raise BackupError("The configured database is unavailable or is a symbolic link")
deadline = time.monotonic() + 60
def progress(_status: int, _remaining: int, _total: int) -> None:
if time.monotonic() > deadline:
raise BackupError("Database is too busy to back up; try again shortly")
with closing(sqlite3.connect(source.as_uri() + "?mode=ro", uri=True)) as src:
with closing(sqlite3.connect(destination)) as dst:
destination.chmod(0o600)
src.backup(dst, pages=256, progress=progress, sleep=0.05)
dst.execute("PRAGMA journal_mode=DELETE")
def _portable_database(path: Path) -> None:
"""Materialize env-backed settings and remove source-specific encryption."""
with closing(sqlite3.connect(path)) as conn, conn:
conn.execute("PRAGMA secure_delete=ON")
# init_db recreates application-owned triggers after restoration; never
# distribute executable schema objects in a data backup.
for (trigger,) in conn.execute("SELECT name FROM sqlite_master WHERE type='trigger'").fetchall():
quoted = str(trigger).replace('"', '""')
conn.execute(f'DROP TRIGGER "{quoted}"')
overrides = dict(conn.execute("SELECT key, value FROM settings"))
for key, default in settings.model_dump().items():
if key in _LOCAL_FIELDS:
continue
value = overrides.get(key)
value = default if value is None else decrypt_setting_value(key, value)
conn.execute(
"INSERT INTO settings(key,value,updated_at) VALUES (?,?,?) "
"ON CONFLICT(key) DO UPDATE SET value=excluded.value,updated_at=excluded.updated_at",
(key, "" if value is None else str(value), _now()),
)
for key in _LOCAL_FIELDS:
conn.execute("DELETE FROM settings WHERE key=?", (key,))
# Future secret keys may not yet be exposed through Settings.
for key, value in conn.execute("SELECT key,value FROM settings").fetchall():
if key in SENSITIVE_SETTING_KEYS:
conn.execute("UPDATE settings SET value=? WHERE key=?", (decrypt_setting_value(key, value), key))
conn.commit()
conn.execute("VACUUM")
def _asset_allowed(name: str, include_cache: bool) -> bool:
parts = PurePosixPath(name).parts
if name in {"files/branding/logo.png", "files/branding/favicon.ico"}:
return True
return bool(
include_cache and len(parts) == 5 and parts[:3] == ("files", "artwork", "tmdb")
and parts[3] in _TMDB_SIZES and _ASSET_NAME.fullmatch(parts[4])
and parts[4] not in {".", ".."}
)
def _asset_files(include_cache: bool) -> Iterator[tuple[Path, str]]:
root = _assets_root()
for directory in ("branding", "artwork") if include_cache else ("branding",):
base = root / directory
if not base.exists():
continue
if base.is_symlink() or root.is_symlink():
raise BackupError("Asset directories must not be symbolic links")
for parent, directories, files in os.walk(base, followlinks=False):
if any((Path(parent) / name).is_symlink() for name in directories + files):
raise BackupError("Symbolic links are not supported in backup assets")
for filename in files:
path = Path(parent) / filename
archive_name = "files/" + path.relative_to(root).as_posix()
if _asset_allowed(archive_name, include_cache):
yield path, archive_name
def create_backup(passphrase: str, include_cache: bool = False) -> tuple[bytes, str]:
validate_passphrase(passphrase)
with _exclusive_operation(), tempfile.TemporaryDirectory(prefix="export-", dir=_control_root()) as temporary:
directory = Path(temporary)
directory.chmod(0o700)
database = directory / "database.sqlite3"
_database_copy(Path(_db_path()).absolute(), database)
_portable_database(database)
files = [(database, "database.sqlite3"), *_asset_files(include_cache)]
if len(files) > MAX_ENTRIES - 1 or sum(path.stat().st_size for path, _ in files) > MAX_EXPANDED_BYTES:
raise BackupError("Backup is too large; retry without the artwork cache")
archive_path = directory / "payload.zip"
manifest = {
"format_version": FORMAT_VERSION, "created_at": _now(),
"build": str(settings.site_build_number or "unknown"), "include_cache": include_cache,
"files": {},
}
with zipfile.ZipFile(archive_path, "w", compression=zipfile.ZIP_DEFLATED, compresslevel=6) as archive:
archive_path.chmod(0o600)
total = 0
for path, name in files:
digest = hashlib.sha256()
size = 0
with path.open("rb") as source, archive.open(name, "w") as destination:
while chunk := source.read(1024 * 1024):
total += len(chunk)
size += len(chunk)
if total > MAX_EXPANDED_BYTES:
raise BackupError("Backup is too large; retry without the artwork cache")
digest.update(chunk)
destination.write(chunk)
manifest["files"][name] = {"bytes": size, "sha256": digest.hexdigest()}
archive.writestr("manifest.json", json.dumps(manifest))
if archive_path.stat().st_size > MAX_UPLOAD_BYTES - 128:
raise BackupError("Backup exceeds the 32 MiB limit; retry without the artwork cache")
encrypted = _encrypt(archive_path.read_bytes(), passphrase)
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
return encrypted, f"magent-backup-{stamp}.magent-backup"
def _validate_database(path: Path, *, verify_settings_encryption: bool = False) -> None:
try:
with closing(sqlite3.connect(path.as_uri() + "?mode=ro", uri=True)) as conn:
conn.execute("PRAGMA trusted_schema=OFF")
deadline = time.monotonic() + 30
conn.set_progress_handler(lambda: int(time.monotonic() > deadline), 10_000)
if conn.execute("PRAGMA integrity_check").fetchall() != [("ok",)]:
raise BackupError("Backup database failed its integrity check")
schema = conn.execute("SELECT type,name,sql FROM sqlite_master").fetchall()
if len(schema) > 500 or any(
kind in {"trigger", "view"} or "VIRTUAL TABLE" in str(sql).upper()
for kind, _name, sql in schema
):
raise BackupError("Backup contains an unsupported database schema")
if conn.execute("PRAGMA foreign_key_check").fetchone() is not None:
raise BackupError("Backup database contains broken references")
required = {
"settings": {"key", "value", "updated_at"},
"users": {"id", "username", "password_hash", "role", "is_blocked", "auth_version"},
"signup_invites": {"id", "code", "enabled"},
"requests_cache": {"request_id", "payload_json"},
"schema_migrations": {"version", "name", "applied_at"},
"password_reset_tokens": {"id", "token_hash"},
}
for table, fields in required.items():
columns = {row[1] for row in conn.execute(f'PRAGMA table_info("{table}")')}
if not fields <= columns:
raise BackupError("Backup does not contain a compatible Magent database")
optional = {
"installation_setup": {"id", "completed", "step", "completed_at"},
"installation_setup_attempts": {"scope", "key_hash", "occurred_at"},
}
table_names = {name for kind, name, _sql in schema if kind == "table"}
for table, fields in optional.items():
if table in table_names:
columns = {row[1] for row in conn.execute(f'PRAGMA table_info("{table}")')}
if not fields <= columns:
raise BackupError("Backup setup state has an incompatible schema")
# An admin can stage a restore only after target initialization. Its
# schema is a trusted reference for *all* runtime columns, including
# versioned migrations that init_db will not rerun on a restored DB.
target = Path(_db_path()).absolute()
if target.is_file() and target != path:
with closing(sqlite3.connect(target.as_uri() + "?mode=ro", uri=True)) as reference:
tables = [row[0] for row in reference.execute("SELECT name FROM sqlite_master WHERE type='table'")]
for table in tables:
if table.startswith("sqlite_") or table in {"installation_setup", "installation_setup_attempts"}:
continue
quoted = str(table).replace('"', '""')
expected = {
row[1]: (row[2].upper(), bool(row[3]), row[5])
for row in reference.execute(f'PRAGMA table_info("{quoted}")')
}
actual = {
row[1]: (row[2].upper(), bool(row[3]), row[5])
for row in conn.execute(f'PRAGMA table_info("{quoted}")')
}
if expected != actual:
raise BackupError("Backup is missing database columns required by this installation")
versions = {int(row[0]) for row in conn.execute("SELECT version FROM schema_migrations")}
if versions != {migration.version for migration in MIGRATIONS}:
raise BackupError("Backup schema is incompatible; restore using the same Magent version")
if not conn.execute(
"SELECT 1 FROM users WHERE role='admin' AND is_blocked=0 AND password_hash IS NOT NULL LIMIT 1"
).fetchone():
raise BackupError("Backup must contain an active administrator account")
values = dict(conn.execute("SELECT key,value FROM settings"))
if _LOCAL_FIELDS.intersection(values):
raise BackupError("Backup contains host-specific configuration")
# Pydantic checks the types of portable settings without reading env values.
for key, value in values.items():
if verify_settings_encryption and key in SENSITIVE_SETTING_KEYS:
value = decrypt_setting_value(key, value)
if key in Settings.model_fields and value not in {None, ""}:
field = Settings.model_fields[key]
TypeAdapter(field.rebuild_annotation()).validate_python(value)
except (sqlite3.DatabaseError, TypeError, ValueError, RuntimeError) as exc:
if isinstance(exc, BackupError):
raise
raise BackupError("Backup database or configuration is invalid") from exc
def _extract_archive(payload: bytes, directory: Path) -> dict[str, Any]:
try:
with zipfile.ZipFile(io.BytesIO(payload)) as archive:
entries = archive.infolist()
if not entries or len(entries) > MAX_ENTRIES:
raise BackupError("Backup contains too many files")
names = [entry.filename for entry in entries]
if len(set(names)) != len(names) or "manifest.json" not in names or "database.sqlite3" not in names:
raise BackupError("Backup manifest is missing or contains duplicate files")
if sum(entry.file_size for entry in entries) > MAX_EXPANDED_BYTES:
raise BackupError("Expanded backup exceeds the 128 MiB limit")
for entry in entries:
parts = PurePosixPath(entry.filename).parts
mode = entry.external_attr >> 16
if (
entry.is_dir() or entry.filename.startswith("/") or "\\" in entry.filename
or str(PurePosixPath(entry.filename)) != entry.filename
or ":" in entry.filename or any(part in {".", ".."} for part in parts)
or (stat.S_IFMT(mode) not in {0, stat.S_IFREG}) or entry.flag_bits & 1
or entry.compress_type not in {zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED}
):
raise BackupError("Backup contains an unsafe archive entry")
if archive.getinfo("manifest.json").file_size > 4 * 1024 * 1024:
raise BackupError("Backup manifest is too large")
manifest = json.loads(archive.read("manifest.json"))
if (
not isinstance(manifest, dict) or manifest.get("format_version") != FORMAT_VERSION
or not isinstance(manifest.get("include_cache"), bool)
or not isinstance(manifest.get("created_at"), str) or len(manifest["created_at"]) > 64
or not isinstance(manifest.get("build"), str) or len(manifest["build"]) > 100
or not isinstance(manifest.get("files"), dict)
or set(manifest["files"]) != set(names) - {"manifest.json"}
):
raise BackupError("Backup manifest is invalid or unsupported")
extracted_bytes = 0
for entry in entries:
name = entry.filename
if name == "manifest.json":
continue
if name != "database.sqlite3" and not _asset_allowed(name, manifest["include_cache"]):
raise BackupError("Backup contains an unsupported file")
expected = manifest["files"][name]
if not isinstance(expected, dict) or expected.get("bytes") != entry.file_size:
raise BackupError("Backup file does not match its manifest")
target = directory.joinpath(*PurePosixPath(name).parts)
_private_dir(target.parent)
digest = hashlib.sha256()
with archive.open(entry) as source, target.open("xb") as destination:
target.chmod(0o600)
while chunk := source.read(1024 * 1024):
extracted_bytes += len(chunk)
if extracted_bytes > MAX_EXPANDED_BYTES:
raise BackupError("Expanded backup exceeds the 128 MiB limit")
digest.update(chunk)
destination.write(chunk)
destination.flush()
os.fsync(destination.fileno())
if digest.hexdigest() != expected.get("sha256"):
raise BackupError("Backup file failed its checksum")
_validate_database(directory / "database.sqlite3")
return manifest
except (zipfile.BadZipFile, KeyError, TypeError, ValueError, RuntimeError, zlib.error) as exc:
if isinstance(exc, BackupError):
raise
raise BackupError("Backup archive is invalid or damaged") from exc
def stage_restore(source: BinaryIO, passphrase: str) -> dict[str, Any]:
validate_passphrase(passphrase)
with _exclusive_operation():
root = _control_root()
pending = root / "pending"
if pending.exists():
raise BackupError("A restore is already staged; cancel it before uploading another")
payload = _decrypt(source.read(MAX_UPLOAD_BYTES + 1), passphrase)
destination_origin = None
if managed_runtime():
from .public_urls import magent_public_url
try:
destination_origin = normalize_application_origin(magent_public_url())
except ValueError:
raise BackupError("Configure a valid destination application address before restoring a backup") from None
with tempfile.TemporaryDirectory(prefix="validate-", dir=root) as temporary:
stage = Path(temporary)
stage.chmod(0o700)
manifest = _extract_archive(payload, stage)
with closing(sqlite3.connect(stage / "database.sqlite3")) as conn, conn:
conn.execute("PRAGMA secure_delete=ON")
for key, value in conn.execute("SELECT key,value FROM settings").fetchall():
if key in SENSITIVE_SETTING_KEYS:
if value and str(value).startswith("enc:v1:"):
raise BackupError("Backup settings are not portable")
conn.execute("UPDATE settings SET value=? WHERE key=?", (encrypt_setting_value(key, value), key))
if destination_origin is not None:
# The backup's hostname must not replace this installation's
# trusted browser origin or change its cookie policy.
conn.execute(
"INSERT INTO settings(key,value,updated_at) VALUES ('magent_application_url',?,?) "
"ON CONFLICT(key) DO UPDATE SET value=excluded.value,updated_at=excluded.updated_at",
(destination_origin, _now()),
)
# Do not revive reset links or existing browser sessions. Invites remain intact.
conn.execute("DELETE FROM password_reset_tokens")
conn.execute("UPDATE users SET auth_version=?", (secrets.randbelow(2**52) + 1_000_000,))
if not manifest["include_cache"]:
conn.execute("UPDATE artwork_cache_status SET poster_cached=0,backdrop_cached=0")
conn.commit()
# Remove plaintext secret remnants from replaced/free SQLite pages.
conn.execute("VACUUM")
metadata = {key: manifest[key] for key in ("created_at", "build", "include_cache")}
metadata["staged_at"] = _now()
_write_json(stage / "metadata.json", metadata)
# Stage survives reboot; it contains only secrets encrypted for this installation.
os.replace(stage, pending)
_sync_directory(root)
return metadata
def backup_status() -> dict[str, Any]:
root = _control_root()
pending_path = root / "pending" / "metadata.json"
last_path = root / "last-restore.json"
return {
"format_version": FORMAT_VERSION, "max_upload_bytes": MAX_UPLOAD_BYTES,
"max_expanded_bytes": MAX_EXPANDED_BYTES,
"include_cache_default": False,
"pending_restore": json.loads(pending_path.read_text()) if pending_path.is_file() else None,
"last_restore": json.loads(last_path.read_text()) if last_path.is_file() else None,
}
def cancel_restore() -> None:
with _exclusive_operation():
pending = _control_root() / "pending"
if pending.is_symlink():
raise BackupError("Invalid staged restore directory")
if pending.exists():
shutil.rmtree(pending)
def _replace_file(source: Path, target: Path) -> None:
_private_dir(target.parent)
temporary = target.with_name(target.name + ".restore-" + uuid.uuid4().hex)
try:
shutil.copyfile(source, temporary)
temporary.chmod(0o600)
with temporary.open("r+b") as handle:
os.fsync(handle.fileno())
os.replace(temporary, target)
_sync_directory(target.parent)
finally:
temporary.unlink(missing_ok=True)
def _replace_assets(source: Path, target: Path) -> None:
if target.is_symlink():
raise BackupError("Asset directories must not be symbolic links")
if target.exists():
shutil.rmtree(target)
if source.exists():
shutil.copytree(source, target, copy_function=shutil.copyfile)
for parent, _directories, files in os.walk(target):
Path(parent).chmod(0o700)
for filename in files:
(Path(parent) / filename).chmod(0o600)
_sync_tree(target)
if target.parent.exists():
_sync_directory(target.parent)
def _recover(journal: dict, root: Path) -> None:
rollback_name = journal.get("rollback_directory", "")
if not re.fullmatch(r"rollback-[0-9a-f]{32}", rollback_name):
raise BackupError("Restore recovery journal is invalid")
rollback = root / rollback_name
database = Path(_db_path()).absolute()
if journal["had_database"]:
_replace_file(rollback / "database.sqlite3", database)
else:
database.unlink(missing_ok=True)
for suffix in ("-wal", "-shm", "-journal"):
Path(str(database) + suffix).unlink(missing_ok=True)
for name in journal["asset_roots"]:
if name not in {"branding", "artwork"}:
raise BackupError("Restore recovery journal is invalid")
_replace_assets(rollback / "files" / name, _assets_root() / name)
_write_json(root / "last-restore.json", {
"status": "rolled_back", "restored_at": _now(), "rollback_directory": rollback.name,
"message": "An interrupted or failed restore was rolled back automatically.",
})
_write_json(root / "restore-journal.json", {**journal, "phase": "rolled_back"})
pending = root / "pending"
if pending.exists():
shutil.rmtree(pending)
(root / "restore-journal.json").unlink()
_sync_directory(root)
def apply_pending_restore() -> bool:
"""Call once before init_db, with no other backend processes using the DB."""
with _exclusive_operation():
root = _control_root()
journal_path = root / "restore-journal.json"
if journal_path.exists():
journal = json.loads(journal_path.read_text())
if journal.get("phase") in {"complete", "rolled_back"}:
if (root / "pending").exists():
shutil.rmtree(root / "pending")
journal_path.unlink()
_sync_directory(root)
return journal["phase"] == "complete"
_recover(journal, root)
return False
pending = root / "pending"
if not pending.exists():
return False
if pending.is_symlink():
raise BackupError("Invalid staged restore directory")
metadata = json.loads((pending / "metadata.json").read_text())
_validate_database(pending / "database.sqlite3", verify_settings_encryption=True)
database = Path(_db_path()).absolute()
rollback = root / ("rollback-" + uuid.uuid4().hex)
_private_dir(rollback)
# Ensure all disk-space/permission failures in backup happen before replacement.
if database.exists():
_database_copy(database, rollback / "database.sqlite3")
names = ["branding", "artwork"] if metadata["include_cache"] else ["branding"]
# Reject links anywhere before copying or deleting the controlled asset trees.
list(_asset_files(metadata["include_cache"]))
for name in names:
source = _assets_root() / name
if source.exists():
shutil.copytree(source, rollback / "files" / name)
_sync_tree(rollback)
journal = {"rollback_directory": rollback.name, "had_database": database.exists(), "asset_roots": names}
_write_json(journal_path, journal)
try:
for suffix in ("-wal", "-shm", "-journal"):
Path(str(database) + suffix).unlink(missing_ok=True)
_replace_file(pending / "database.sqlite3", database)
for name in names:
_replace_assets(pending / "files" / name, _assets_root() / name)
_write_json(root / "last-restore.json", {
"status": "restored", "restored_at": _now(), "rollback_directory": rollback.name,
"backup_created_at": metadata["created_at"],
})
_write_json(journal_path, {**journal, "phase": "complete"})
except Exception:
_recover(journal, root)
raise
shutil.rmtree(pending)
journal_path.unlink()
_sync_directory(root)
return True
+14 -20
View File
@@ -1,4 +1,4 @@
"""Reviewed consolidation of accounts sharing a verified Jellyfin ID, entirely within Magent."""
"""Reviewed consolidation of same-name Jellyfin accounts, entirely within Magent."""
import asyncio
import json
from contextlib import closing
@@ -30,19 +30,14 @@ def account_state(conn, ids):
('email_recap_subscriptions', 'user_id'), ('newsletter_subscriptions', 'user_id')]}
def identity_group(report, target):
identity = target['candidate_jellyfin_id']
return [row for row in report['rows'] if identity and row['candidate_jellyfin_id'] == identity]
def build_preview(report, local, runtime, state, user_id, keep_id=None):
target = next((row for row in report['rows'] if row['user']['id'] == user_id), None)
if not target:
raise HTTPException(404, 'This Magent account no longer exists. Run the check again.')
group = identity_group(report, target)
group = [row for row in report['rows'] if review.name_key(row['user']['username']) == review.name_key(target['user']['username'])]
ids = {row['user']['id'] for row in group}
if len(ids) < 2:
raise HTTPException(409, 'No duplicate identity group remains. Run the account check again.')
raise HTTPException(409, 'No same-name duplicate group remains. Run the account check again.')
jf_id = target['candidate_jellyfin_id']
source = source_key(runtime.jellyfin_base_url)
owned = {link['local_user_id'] for link in local['links'] if link['source'] == source and review.normalized_id(link['jellyfin_user_id']) == jf_id}
@@ -55,14 +50,14 @@ def build_preview(report, local, runtime, state, user_id, keep_id=None):
problems.append('Restore all three media-service connections before consolidating accounts.')
if not target['jellyfin'] or target['jellystat']['state'] != 'matched' or len(target['seerr']) != 1:
problems.append('One Jellyfin identity and one Seerr account must be verified against Jellystat.')
if target['jellyfin'] and review.name_key(target['jellyfin']['name']) != review.name_key(target['user']['username']):
problems.append('The current Jellyfin name does not match this duplicate group.')
if len([account for account in report['jellyfin_users'] if review.name_key(account['name']) == review.name_key(target['user']['username'])]) != 1:
problems.append('The name must identify exactly one current Jellyfin account.')
seerr_id = target['seerr'][0]['id'] if len(target['seerr']) == 1 else None
for row in group:
if row['basis'] not in {'confirmed_id', 'stored_jellyfin_id', 'stored_seerr_id'}:
problems.append('Every account needs a stored Jellyfin or Seerr ID; names alone cannot authorize consolidation.')
if any('different accounts' in issue or 'multiple distinct Jellyfin' in issue for issue in row['issues']):
problems.append('A name and stored identity disagree. Resolve that mapping before consolidation.')
if row['user']['role'] != 'user' or row['user']['auth_provider'] not in {'jellyfin', 'jellyseerr'}:
problems.append('Only non-admin Jellyfin or Seerr accounts can use duplicate consolidation.')
if row['user']['role'] != 'user' or row['user']['auth_provider'] != 'jellyfin':
problems.append('Only non-admin Jellyfin sign-in accounts can use duplicate consolidation.')
if not jf_id or row['candidate_jellyfin_id'] != jf_id or row['user']['jellyseerr_user_id'] not in (None, seerr_id):
problems.append('These rows do not all resolve to the same Jellyfin and Seerr identity.')
for link in local['links']:
@@ -81,12 +76,12 @@ def build_preview(report, local, runtime, state, user_id, keep_id=None):
problems.append('Another confirmation owns this identity.')
if any(row['user']['id'] not in ids and (row['candidate_jellyfin_id'] == jf_id or
(seerr_id is not None and row['user']['jellyseerr_user_id'] == seerr_id)) for row in report['rows']):
problems.append('An account outside this identity group also claims the identity.')
problems.append('An account outside this same-name group also claims the identity.')
accounts = [account for account in state['users'] if account['id'] in ids]
kept = next(account for account in accounts if account['id'] == keep_id)
overrides = {(entry['user_id'], entry['feature']): bool(entry['enabled']) for entry in state['user_feature_permissions']}
features = {key: all(bool(account['invite_management_enabled']) if key == 'invites' else
overrides.get((account['id'], key), key != 'ignore_profile_limits') for account in accounts) for key in FEATURES}
overrides.get((account['id'], key), True) for account in accounts) for key in FEATURES}
expiries = [account['expires_at'] for account in accounts if account['expires_at']]
try:
expiry = min(expiries, key=lambda value: db._parse_datetime_value(value).timestamp()) if expiries else None
@@ -110,8 +105,7 @@ async def prepare(user_id, keep_id=None):
target = next((row for row in local['users'] if row['id'] == user_id), None)
if not target:
raise HTTPException(404, 'Account not found.')
report_target = next(row for row in report['rows'] if row['user']['id'] == user_id)
ids = sorted(row['user']['id'] for row in identity_group(report, report_target))
ids = sorted(row['id'] for row in local['users'] if review.name_key(row['username']) == review.name_key(target['username']))
with closing(db._connect()) as conn:
conn.execute('BEGIN')
if review.digest(review.snapshot(conn)) != review.digest(local):
@@ -152,7 +146,7 @@ def consolidate(preview, report, local, runtime, state, admin):
for name in old_values:
new_value = review.name_key(values['username']) if column == 'requested_by_norm' else values['username']
conn.execute(f'UPDATE {table} SET {column}=? WHERE {column}=? COLLATE BINARY', (new_value, name))
activity = [dict(row) for row in conn.execute('SELECT * FROM user_activity') if row['username'] in names]
activity = [dict(row) for row in conn.execute('SELECT * FROM user_activity') if review.name_key(row['username']) == review.name_key(values['username'])]
for entry in activity:
conn.execute('DELETE FROM user_activity WHERE id=?', (entry['id'],))
for entry in activity:
@@ -171,7 +165,7 @@ def consolidate(preview, report, local, runtime, state, admin):
conn.execute('DELETE FROM user_identity_confirmations WHERE local_user_id=?', (identity,))
conn.execute('DELETE FROM users WHERE id=?', (identity,))
last_login = max((account['last_login_at'] for account in state['users'] if account['last_login_at']), default=None)
conn.execute('''UPDATE users SET auth_provider='jellyfin',username=?,jellyseerr_user_id=?,is_blocked=?,auto_search_enabled=?,
conn.execute('''UPDATE users SET username=?,jellyseerr_user_id=?,is_blocked=?,auto_search_enabled=?,
invite_management_enabled=?,expires_at=?,last_login_at=? WHERE id=?''',
(values['username'], values['seerr_user_id'], values['is_blocked'], values['auto_search_enabled'],
values['features']['invites'], values['expires_at'], last_login, keep))
+5 -33
View File
@@ -32,7 +32,7 @@ def worker_enabled() -> bool:
def delivery_ready() -> tuple[bool, str]:
config = store.settings()
if not config["public_url"]:
return False, "Set the application URL in Hosting & proxy for email links."
return False, "Set the public Magent address for email links."
ready, detail = smtp_email_config_ready()
if not ready:
return False, detail
@@ -142,40 +142,12 @@ def completed_month(month: str | None) -> str:
return period["month"]
async def illustrated_recap(report, account, public_url, unsubscribe_url, *, preview=False, **kwargs):
"""Embed only signed artwork from this account's report; missing art is optional."""
import base64
import re
from .insights_artwork import get_artwork
runtime = get_runtime_settings()
images = []
report = {**report, "top_titles": [dict(row) for row in report.get("top_titles", [])]}
async def picture(index, row):
match = re.fullmatch(r"/insights/artwork/([a-f0-9]{32})\?token=([0-9]+\.[a-f0-9]{64})", row.get("artwork_url") or "")
if not match:
return
try:
data, mime = await get_artwork(account, runtime, *match.groups())
cid = f"recap-title-{index}@magent"
row["email_artwork"] = f"data:{mime};base64,{base64.b64encode(data).decode()}" if preview else f"cid:{cid}"
images.append({"cid": cid, "data": data, "subtype": mime.split("/")[1]})
except Exception:
pass # An unavailable poster must never prevent a personal report.
await asyncio.gather(*(picture(i, row) for i, row in enumerate(report["top_titles"][:3])))
rendered = mail.render_recap(report, account["username"], public_url, unsubscribe_url, **kwargs)
if not preview:
rendered["inline_images"] = images
return rendered
async def preview(user: dict, month: str | None) -> dict:
account = current_account(user)
selected = completed_month(month)
config = store.settings()
if not config["public_url"]:
raise RecapError("Set the application URL in Hosting & proxy before previewing an email.")
raise RecapError("Save the public Magent address before previewing an email.")
try:
report = await asyncio.wait_for(get_monthly_report(account, selected), timeout=180)
except HistoryLimitError as exc:
@@ -184,8 +156,8 @@ async def preview(user: dict, month: str | None) -> dict:
raise RecapError("Your report is temporarily unavailable. Please try again shortly.", 502) from exc
if report["state"] != "ready":
raise RecapError("Connect Jellystat and link your Jellyfin account to preview your recap.")
return {"month": selected, "email": account.get("email"), **await illustrated_recap(
report, account, config["public_url"], config["public_url"] + "/profile#monthly-recaps", preview=True)}
return {"month": selected, "email": account.get("email"), **mail.render_recap(
report, account["username"], config["public_url"], config["public_url"] + "/profile#monthly-recaps")}
def queue_test(user: dict, month: str | None, request_id: str) -> dict:
@@ -228,7 +200,7 @@ async def process_delivery(delivery: dict) -> None:
if report["state"] != "ready" or (report["is_partial"] and delivery["kind"] != "on_demand"):
raise mail.DeliveryError("failed", "A complete personal report is not available.")
unsubscribe = f"{delivery['public_url']}/email-recaps#" + urlencode({"action": "unsubscribe", "token": sub["unsubscribe_token"]})
rendered = await illustrated_recap(report, account, delivery["public_url"], unsubscribe, test=delivery["kind"] == "test", requested=delivery["kind"] == "on_demand")
rendered = mail.render_recap(report, account["username"], delivery["public_url"], unsubscribe, test=delivery["kind"] == "test", requested=delivery["kind"] == "on_demand")
def before_data():
eligible_delivery(delivery)
+1 -12
View File
@@ -125,9 +125,6 @@ def request_summary(user: dict, start: datetime, end: datetime, *, end_exclusive
def summarize(history: list, libraries: list, start: datetime, end: datetime, *, end_exclusive: bool = False) -> dict:
library_types = {str(row.get("Id")): str(row.get("CollectionType") or "").lower() for row in libraries}
daily_seconds = defaultdict(float)
weekdays = [0.0] * 7
media_minutes = defaultdict(float)
longest_play = 0.0
clients = defaultdict(float)
methods = defaultdict(float)
transcoding = dict.fromkeys(("video_minutes", "audio_minutes", "hardware_video_minutes", "software_video_minutes",
@@ -160,9 +157,6 @@ def summarize(history: list, libraries: list, start: datetime, end: datetime, *,
episode_ids.add(str(episode_id))
elif media_type == "movie":
movie_ids.add(item_id)
weekdays[date.weekday()] += duration / 60
media_minutes[media_type] += duration / 60
longest_play = max(longest_play, duration / 60)
seconds += duration
daily_seconds[date.date().isoformat()] += duration
client = str(row.get("Client") or "Unknown player")[:200]
@@ -172,7 +166,7 @@ def summarize(history: list, libraries: list, start: datetime, end: datetime, *,
methods[method] += duration
name = str(row.get("NowPlayingItemName") or "Untitled")[:500]
series = str(row.get("SeriesName") or "")[:500]
title = titles.setdefault(item_id, {"title": series or name, "type": "series" if episode_id else media_type, "minutes": 0, "plays": 0, "artwork_item_id": artwork_item_id(row.get("NowPlayingItemId"))})
title = titles.setdefault(item_id, {"title": series or name, "type": "series" if episode_id else media_type, "minutes": 0, "plays": 0})
title["minutes"] += duration / 60
title["plays"] += 1
recent.append({"id": row_id, "title": name, "series": series, "type": media_type,
@@ -199,11 +193,6 @@ def summarize(history: list, libraries: list, start: datetime, end: datetime, *,
return {"summary": {"minutes": round(seconds / 60, 1), "plays": len(recent), "movies": len(movie_ids),
"episodes": len(episode_ids), "active_days": len(active_days),
"current_streak": current, "longest_streak": longest},
"patterns": {"average_play_minutes": round(seconds / 60 / len(recent), 1) if recent else 0,
"longest_play_minutes": round(longest_play, 1),
"weekend_percent": round(sum(weekdays[5:]) / (seconds / 60) * 100, 1) if seconds else 0,
"weekdays": [{"name": name, "minutes": round(weekdays[i], 1)} for i, name in enumerate(("Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"))],
"media": [{"name": name, "minutes": round(media_minutes[key], 1)} for key, name in (("movie", "Movies"), ("episode", "TV episodes"), ("other", "Other media"))]},
"daily": daily, "top_titles": top,
"clients": [{"name": name, "minutes": round(value / 60, 1)} for name, value in sorted(clients.items(), key=lambda pair: -pair[1])[:6]],
"methods": [{"name": name, "minutes": round(value / 60, 1)} for name, value in sorted(methods.items(), key=lambda pair: -pair[1])],
+10 -13
View File
@@ -35,19 +35,16 @@ def signature(user, runtime, media_id, expires):
def with_artwork(data, user, runtime):
expires = int(time.time()) + TOKEN_SECONDS
result = {**data}
for field in ("recent", "top_titles"):
rows = []
for play in data.get(field, []):
row = {**play}
media_id = row.pop("artwork_item_id", None)
row["artwork_url"] = None
if item_id(media_id) and settings.jwt_secret and runtime.jellyfin_base_url and runtime.jellyfin_api_key:
token = f"{expires}.{signature(user, runtime, media_id, expires)}"
row["artwork_url"] = f"/insights/artwork/{media_id}?token={token}"
rows.append(row)
result[field] = rows
return result
recent = []
for play in data.get("recent", []):
row = {**play}
media_id = row.pop("artwork_item_id", None)
row["artwork_url"] = None
if item_id(media_id) and settings.jwt_secret and runtime.jellyfin_base_url and runtime.jellyfin_api_key:
token = f"{expires}.{signature(user, runtime, media_id, expires)}"
row["artwork_url"] = f"/insights/artwork/{media_id}?token={token}"
recent.append(row)
return {**data, "recent": recent}
def verify_artwork_token(user, runtime, media_id, token):
+21 -8
View File
@@ -1025,12 +1025,16 @@ def _send_email_sync(*, recipient_email: str, subject: str, body_text: str, body
raise RuntimeError("SMTP email settings are incomplete.")
local_hostname = _derive_mail_hostname(from_address=from_address)
logger.info(
"smtp send started host=%s port=%s tls=%s ssl=%s auth=%s",
"smtp send started recipient=%s from=%s host=%s port=%s tls=%s ssl=%s auth=%s subject=%s ehlo=%s",
recipient_email,
from_address,
host,
port,
use_tls,
use_ssl,
bool(username and password),
subject,
local_hostname,
)
if delivery_warning:
logger.warning("smtp delivery warning host=%s detail=%s", host, delivery_warning)
@@ -1079,7 +1083,11 @@ def _send_email_sync(*, recipient_email: str, subject: str, body_text: str, body
message=message,
)
logger.info(
"smtp send accepted host=%s mode=ssl", host,
"smtp send accepted recipient=%s host=%s mode=ssl provider_message_id=%s provider_internal_id=%s",
recipient_email,
host,
receipt.get("provider_message_id"),
receipt.get("provider_internal_id"),
)
return receipt
@@ -1092,7 +1100,7 @@ def _send_email_sync(*, recipient_email: str, subject: str, body_text: str, body
logger.debug("smtp starttls negotiated host=%s port=%s", host, port)
if username and password:
smtp.login(username, password)
logger.debug("smtp login succeeded host=%s", host)
logger.debug("smtp login succeeded host=%s username=%s", host, username)
receipt = _send_via_smtp_session(
smtp,
from_address=from_address,
@@ -1100,7 +1108,11 @@ def _send_email_sync(*, recipient_email: str, subject: str, body_text: str, body
message=message,
)
logger.info(
"smtp send accepted host=%s mode=plain", host,
"smtp send accepted recipient=%s host=%s mode=plain provider_message_id=%s provider_internal_id=%s",
recipient_email,
host,
receipt.get("provider_message_id"),
receipt.get("provider_internal_id"),
)
return receipt
@@ -1141,7 +1153,7 @@ async def send_templated_email(
body_text=rendered["body_text"],
body_html=rendered["body_html"],
)
logger.info("Email template sent: template=%s", template_key)
logger.info("Email template sent: template=%s recipient=%s", template_key, resolved_email)
return {
"recipient_email": resolved_email,
"subject": rendered["subject"],
@@ -1173,7 +1185,7 @@ async def send_generic_email(
body_text=body_text.strip(),
body_html=body_html.strip(),
)
logger.info("Generic email sent")
logger.info("Generic email sent recipient=%s subject=%s", resolved_email, subject)
return {
"recipient_email": resolved_email,
"subject": subject.strip() or f"{env_settings.app_name} notification",
@@ -1272,7 +1284,7 @@ async def send_test_email(recipient_email: Optional[str] = None) -> Dict[str, st
body_text=body_text,
body_html=body_html,
)
logger.info("SMTP test email sent")
logger.info("SMTP test email sent: recipient=%s", resolved_email)
result = {"recipient_email": resolved_email, "subject": subject}
result.update(
{
@@ -1371,8 +1383,9 @@ async def send_password_reset_email(
body_html=body_html,
)
logger.info(
"Password reset email sent: username=%s provider=%s",
"Password reset email sent: username=%s recipient=%s provider=%s",
username,
resolved_email,
auth_provider,
)
result = {
+3 -3
View File
@@ -213,11 +213,11 @@ async def _contact_reporter(item: Dict[str, Any]) -> Dict[str, Any]:
sent = False
delivery_error: Optional[str] = None
if recipient:
subject = f"Ready to try again? Magent issue #{item['id']}"
subject = f"Ready to try again? Grizzlyflix issue #{item['id']}"
body_text = (
"Your repair looks ready to test.\n\n"
f"{item.get('title') or 'Your reported issue'}\n\n"
"Please try the affected content in Jellyfin. Is it fixed?\n\n"
"Please try the affected content in Grizzlyflix. Is it fixed?\n\n"
f"YES — it works: {issue_url}#yes\n"
f"NO — still broken: {issue_url}#no\n\n"
"Confirm your answer in Magent. You may need to sign in first.\n"
@@ -227,7 +227,7 @@ async def _contact_reporter(item: Dict[str, Any]) -> Dict[str, Any]:
body_html = (
'<div style="background:#111113;padding:24px 12px;font-family:Arial,sans-serif;color:#f4f4f5;">'
'<table role="presentation" style="max-width:560px;width:100%;margin:auto;background:#202023;border:1px solid #45454d;border-radius:18px;"><tr><td style="padding:28px;">'
'<p style="margin:0 0 24px;color:#c7baff;font-weight:bold;letter-spacing:2px;">MAGENT</p>'
'<p style="margin:0 0 24px;color:#c7baff;font-weight:bold;letter-spacing:2px;">GRIZZLYFLIX · MAGENT</p>'
'<h1 style="font-size:32px;line-height:1.2;margin:0 0 16px;color:#fff;">Ready to try again?</h1>'
'<p style="font-size:17px;line-height:1.6;color:#e4e4e7;">Your repair looks ready to test. Give the affected content a try, then let us know:</p>'
f'<p style="padding:16px;background:#131315;border-radius:10px;color:#fff;">{escape(str(item.get("title") or "Your reported issue"))}</p>'
-13
View File
@@ -36,16 +36,3 @@ def link_user(username: str, jellyfin_user_id: str, base_url: str | None) -> Non
"INSERT OR IGNORE INTO jellyfin_user_links (source, local_user_id, jellyfin_user_id) VALUES (?, ?, ?)",
(source_key(base_url), user["id"], str(jellyfin_user_id)),
)
def user_for_identity(jellyfin_user_id: str, base_url: str | None):
"""Resolve a verified upstream login to its existing local account."""
if not jellyfin_user_id or not base_url:
return None
with closing(db._connect()) as conn:
rows = conn.execute("SELECT local_user_id FROM jellyfin_user_links WHERE source=? AND lower(replace(jellyfin_user_id,'-',''))=?",
(source_key(base_url), str(jellyfin_user_id).replace('-', '').lower())).fetchall()
if len(rows) > 1:
from fastapi import HTTPException
raise HTTPException(409, 'Multiple accounts claim this Jellyfin ID. Ask an administrator to repair the links.')
return db.get_user_by_id(rows[0][0]) if rows else None
+35 -45
View File
@@ -1,9 +1,4 @@
import logging
from collections import Counter
from contextlib import closing
from .. import db
from .jellyfin_identity import source_key
from .identity_review import normalized_id, name_key
from fastapi import HTTPException
@@ -11,14 +6,18 @@ from ..clients.jellyfin import JellyfinClient
from ..db import (
create_user_if_missing,
get_user_by_username,
set_user_email,
set_user_auth_provider,
set_user_jellyseerr_id,
)
from ..runtime import get_runtime_settings
from .jellyfin_identity import link_user
from .user_cache import (
build_jellyseerr_candidate_map,
extract_jellyseerr_user_email,
find_matching_jellyseerr_user,
get_cached_jellyseerr_users,
match_jellyseerr_user_id,
save_jellyfin_users_cache,
)
@@ -37,52 +36,43 @@ async def sync_jellyfin_users() -> int:
# Jellyfin is the canonical source for local user objects; Seerr IDs are
# matched as enrichment when possible.
jellyseerr_users = get_cached_jellyseerr_users()
candidate_map = build_jellyseerr_candidate_map(jellyseerr_users or [])
imported = 0
name_counts = Counter(name_key(row.get('Name')) for row in users if isinstance(row, dict))
with closing(db._connect()) as conn:
links = [dict(zip(('local_id', 'jf_id'), row)) for row in conn.execute(
'SELECT local_user_id,jellyfin_user_id FROM jellyfin_user_links WHERE source=?', (source_key(runtime.jellyfin_base_url),))]
for user in users:
if not isinstance(user, dict):
continue
name, jf_id = user.get('Name'), normalized_id(user.get('Id'))
if not name or not jf_id or name_counts[name_key(name)] != 1:
name = user.get("Name")
if not name:
continue
matches = [row for row in (jellyseerr_users or []) if normalized_id(row.get('jellyfinUserId')) == jf_id]
if len(matches) > 1:
continue
matched = matches[0] if matches else None
matched_id = matched.get('id') if matched else None
owners = [row['local_id'] for row in links if normalized_id(row['jf_id']) == jf_id]
if len(owners) > 1:
continue
existing = db.get_user_by_id(owners[0]) if owners else None
if not existing and matched_id is not None:
candidates = [row for row in db.get_all_users() if row.get('jellyseerr_user_id') == matched_id]
if len(candidates) > 1:
continue
existing = candidates[0] if candidates else None
if not existing:
existing = get_user_by_username(name)
if existing:
existing_links = [normalized_id(row['jf_id']) for row in links if row['local_id'] == existing['id']]
if existing_links and any(value != jf_id for value in existing_links):
continue
if existing.get('role') == 'admin' or existing.get('auth_provider') == 'local':
continue
canonical = existing['username']
# Never overwrite a stored Seerr identity on name evidence.
if existing.get('jellyseerr_user_id') not in (None, matched_id):
continue
set_user_auth_provider(canonical, 'jellyfin')
matched_id = match_jellyseerr_user_id(name, candidate_map) if candidate_map else None
matched_seerr_user = find_matching_jellyseerr_user(name, jellyseerr_users or [])
matched_email = extract_jellyseerr_user_email(matched_seerr_user)
created = create_user_if_missing(
name,
"jellyfin-user",
role="user",
email=matched_email,
auth_provider="jellyfin",
jellyseerr_user_id=matched_id,
)
if created:
imported += 1
else:
canonical = name
if create_user_if_missing(canonical, 'jellyfin-user', auth_provider='jellyfin',
jellyseerr_user_id=matched_id, email=extract_jellyseerr_user_email(matched)):
imported += 1
if matched_id is not None:
set_user_jellyseerr_id(canonical, matched_id)
link_user(canonical, jf_id, runtime.jellyfin_base_url)
existing = get_user_by_username(name)
if (
existing
and str(existing.get("role") or "user").strip().lower() != "admin"
and str(existing.get("auth_provider") or "local").strip().lower() != "jellyfin"
):
set_user_auth_provider(name, "jellyfin")
if matched_id is not None:
set_user_jellyseerr_id(name, matched_id)
if matched_email:
set_user_email(name, matched_email)
if user.get("Id"):
local_user = get_user_by_username(name)
if local_user and local_user.get("auth_provider") == "jellyfin":
link_user(name, str(user["Id"]), runtime.jellyfin_base_url)
return imported
-55
View File
@@ -1,55 +0,0 @@
"""Manual collector decisions and short-lived, request-bound selection receipts."""
from datetime import datetime, timedelta, timezone
import hashlib
import jwt
from fastapi import HTTPException
from ..config import settings
def can_override(user):
return user.get('role') == 'admin' or (user.get('features') or {}).get('ignore_profile_limits') is True
def decision(item):
reasons = [str(r) for r in (item.get('rejections') or [])]
accepted = (item.get('approved') is True and not reasons and not item.get('rejected')
and not item.get('temporarilyRejected') and item.get('downloadAllowed') is not False)
# Unknown/operational rejections remain blocked. This permission only relaxes profile limits.
profile_only = bool(reasons) and all(any(term in reason.lower() for term in (
'quality profile', 'not wanted in profile', 'custom format', 'minimum score',
'quality is not', 'quality for', 'language', 'maximum size', 'minimum size',
'larger than', 'smaller than', 'size limit', 'release profile',
)) for reason in reasons)
override = not accepted and profile_only and item.get('downloadAllowed') is not False and not item.get('temporarilyRejected')
return accepted, override, reasons
def source_id(url):
return hashlib.sha256(str(url).rstrip('/').encode()).hexdigest()
def issue_selection(release, request_id, user, source, item_id):
return jwt.encode({'aud': 'manual-release', 'sub': user['username'], 'request': str(request_id),
'source': source_id(source), 'item': item_id, 'guid': release['guid'],
'indexer': release['indexerId'], 'title': release.get('title'),
'override': release['requiresOverride'], 'rejections': release['rejections'],
'exp': datetime.now(timezone.utc) + timedelta(minutes=10)},
settings.jwt_secret, algorithm='HS256')
def verify_selection(payload, request_id, user, source, item_id):
try:
receipt = jwt.decode(payload.get('selectionToken', ''), settings.jwt_secret,
algorithms=['HS256'], audience='manual-release')
except jwt.InvalidTokenError as exc:
raise HTTPException(409, 'This release selection expired or is invalid. Search again before downloading.') from exc
if (receipt.get('sub') != user.get('username') or receipt.get('request') != str(request_id)
or receipt.get('source') != source_id(source) or receipt.get('item') != item_id
or receipt.get('guid') != payload.get('guid') or receipt.get('indexer') != payload.get('indexerId')):
raise HTTPException(409, 'This release does not belong to this account and request. Search again.')
if receipt.get('override'):
if not can_override(user):
raise HTTPException(403, 'Ignore profile limits is disabled for your account.')
if payload.get('ignoreProfileLimits') is not True:
raise HTTPException(400, 'Explicitly confirm ignoring the profile limits for this release.')
return receipt
+9 -9
View File
@@ -16,12 +16,12 @@ def description(entry):
def render_confirmation(username, url):
intro = f"Hi {username}, confirm your email to receive new arrivals, featured picks and announcements from your media library."
return {'subject': 'Confirm your Magent newsletter subscription',
intro = f"Hi {username}, confirm your email to receive new arrivals, featured picks and announcements from Grizzlyflix."
return {'subject': 'Confirm your Grizzlyflix newsletter subscription',
'body_text': f'{intro}\n\nConfirm newsletter subscription: {url}\n\nThis link expires in 24 hours. If you did not request this, ignore this email.',
'body_html': document(title='Your next watch starts here.', intro=intro,
content='<p style="color:#bdb6c3;font-size:14px;line-height:1.7">A weekly look at new movies and TV updates, with posters and links to watch.</p>',
action='Confirm newsletter subscription', url=url, kicker='NEW IN YOUR LIBRARY',
action='Confirm newsletter subscription', url=url, kicker='NEW ON GRIZZLYFLIX',
footer='This link expires in 24 hours. If you did not request this, ignore this email.')}
@@ -58,17 +58,17 @@ def render(content, images, public_url, playback_url, unsubscribe_url, *, previe
body.append(f'''<table role="presentation" width="100%" cellpadding="0" cellspacing="0" style="table-layout:fixed;border-bottom:1px solid #363338"><tr>
<td width="92" valign="top" style="padding:18px 12px 18px 0">{poster}</td><td valign="top" style="padding:18px 0;overflow-wrap:anywhere">
<h3 style="margin:0 0 8px;font-size:16px;line-height:1.4;color:#eee8f2">{esc(entry['title'])}</h3><p style="font-size:12px;line-height:1.6;color:#a69fac;margin:0 0 10px">{esc(details)}</p>{copy}
<a href="{esc(watch, quote=True)}" style="display:inline-block;padding:8px 0;color:#c7bdff;text-decoration:none;font-size:13px;font-weight:bold">Watch on Jellyfin &#8599;</a></td></tr></table>''')
<a href="{esc(watch, quote=True)}" style="display:inline-block;padding:8px 0;color:#c7bdff;text-decoration:none;font-size:13px;font-weight:bold">Watch on Grizzlyflix &#8599;</a></td></tr></table>''')
lines += [entry['title'], details, watch, '']
if not titles:
body.append('<p style="font-size:14px;line-height:1.7;color:#bdb6c3">Your next discovery is waiting in your media library.</p>')
body.append('<p style="font-size:14px;line-height:1.7;color:#bdb6c3">Your next discovery is waiting in Grizzlyflix.</p>')
period = f"{content['period_start'][:10]} to {content['period_end'][:10]} · UTC"
footer = f'You subscribed to the Magent newsletter.<br>Arrivals recorded by Jellyfin · {esc(period)}<br><a href="{esc(unsubscribe_url, quote=True)}" style="color:#c7bdff">Unsubscribe from newsletters</a> · <a href="{esc(public_url + "/profile#newsletters", quote=True)}" style="color:#c7bdff">Email preferences</a>'
footer = f'You subscribed to the Grizzlyflix newsletter.<br>Arrivals recorded by Jellyfin · {esc(period)}<br><a href="{esc(unsubscribe_url, quote=True)}" style="color:#c7bdff">Unsubscribe from newsletters</a> · <a href="{esc(public_url + "/profile#newsletters", quote=True)}" style="color:#c7bdff">Email preferences</a>'
subject = ('[Test] ' if test else '') + content['subject']
return {'subject': subject, 'body_text': '\n'.join([subject, '', *lines, f'Browse Jellyfin: {playback_url}', '',
return {'subject': subject, 'body_text': '\n'.join([subject, '', *lines, f'Browse Grizzlyflix: {playback_url}', '',
f'Arrivals recorded by Jellyfin: {period}', f'Unsubscribe from newsletters: {unsubscribe_url}',
f'Email preferences: {public_url}/profile#newsletters']),
'body_html': document(title='Whats new in your library',
'body_html': document(title='Whats new on Grizzlyflix',
intro=('This is your test edition. ' if test else '') + 'New stories for your watchlist. Find your next movie or catch up on fresh episodes.',
content=''.join(body), action='Explore Jellyfin', url=playback_url, footer=footer, kicker='YOUR NEXT WATCH'),
content=''.join(body), action='Explore Grizzlyflix', url=playback_url, footer=footer, kicker='YOUR NEXT WATCH'),
'inline_images': attachments}
+2 -6
View File
@@ -10,7 +10,6 @@ from datetime import datetime, timedelta, timezone
from .. import db
from . import email_queue
from .recap_store import read_one, transaction
from .public_urls import magent_public_url
class Conflict(ValueError):
@@ -64,7 +63,6 @@ def init_schema(conn):
def settings() -> dict:
result = read_one('SELECT * FROM newsletter_settings WHERE id=1')
result['public_url'] = magent_public_url(result['public_url'])
result['enabled'] = bool(result['enabled'])
return result
@@ -81,7 +79,6 @@ def next_due(now: datetime, weekday: int, hour: int) -> datetime:
def save_settings(values: dict, now: datetime):
values = {**values, "public_url": magent_public_url(values.get("public_url", ""))}
with transaction() as conn:
old = dict(conn.execute('SELECT * FROM newsletter_settings WHERE id=1').fetchone())
if old['revision'] != values['revision']:
@@ -249,8 +246,7 @@ def enqueue_test(sub, identity, revision, request_id, public_url, now):
def enqueue_due(now):
with transaction() as conn:
config = dict(conn.execute('SELECT * FROM newsletter_settings WHERE id=1').fetchone())
config['public_url'] = magent_public_url(config['public_url'])
config = conn.execute('SELECT * FROM newsletter_settings WHERE id=1').fetchone()
rows = conn.execute("SELECT * FROM newsletter_editions WHERE state='scheduled' AND send_at<=?", (now,)).fetchall()
for raw in rows:
row = unpack(raw)
@@ -322,7 +318,7 @@ def complete_weekly(config, content, now: datetime, failure=''):
conn.execute('''INSERT OR IGNORE INTO newsletter_editions
(id,subject,intro,content_json,state,origin,weekly_key,send_at,created_at,updated_at,created_by)
VALUES (?,?,?,?,?,'weekly',?,?,?,?,?)''',
(identity, f"Whats new in your library · {due.strftime('%d %b %Y')}", config['intro'], json.dumps(content),
(identity, f"Whats new on Grizzlyflix · {due.strftime('%d %b %Y')}", config['intro'], json.dumps(content),
'skipped' if empty else 'scheduled', due.isoformat(), due.timestamp(), now.timestamp(), now.timestamp(), 'Weekly schedule'))
row = unpack(conn.execute('SELECT * FROM newsletter_editions WHERE weekly_key=?', (due.isoformat(),)).fetchone())
if not empty:
+3 -3
View File
@@ -32,7 +32,7 @@ def playback_url(runtime) -> str:
def delivery_ready(public_url=None):
config = store.settings()
if not (public_url if public_url is not None else config['public_url']):
return False, 'Set the application URL in Hosting & proxy for newsletter email links.'
return False, 'Set the public Magent address for newsletter email links.'
runtime = get_runtime_settings()
if not runtime.jellyfin_base_url or not runtime.jellyfin_api_key:
return False, 'Connect Jellyfin to collect new arrivals.'
@@ -133,7 +133,7 @@ async def create_draft(user, days):
end = datetime.now(timezone.utc)
config = store.settings()
content = await collect(end - timedelta(days=days), end, config['limit_titles'])
return store.create_edition(content, f"Whats new in your library · {end.strftime('%d %b %Y')}", config['intro'], user['username'], end.timestamp())
return store.create_edition(content, f"Whats new on Grizzlyflix · {end.strftime('%d %b %Y')}", config['intro'], user['username'], end.timestamp())
def require_edition(identity, revision=None):
@@ -150,7 +150,7 @@ async def preview(identity, revision):
runtime = get_runtime_settings()
config = store.settings()
if not config['public_url'] or not playback_url(runtime):
raise NewsletterError('Check the application URL in Hosting & proxy and the public playback URL in Jellyfin settings before previewing.')
raise NewsletterError('Set the public Magent and Jellyfin addresses before previewing.')
if row['content']['source'] != source_key(runtime.jellyfin_base_url) or row['content']['playback_url'] != playback_url(runtime):
raise NewsletterError('The Jellyfin connection or public address changed. Create a fresh draft.')
content = {**row['content'], 'subject': row['subject'], 'intro': row['intro']}
+1 -3
View File
@@ -18,7 +18,6 @@ from ..db import (
mark_password_reset_token_used,
set_user_auth_provider,
set_user_password,
increment_user_auth_version,
sync_jellyfin_password_state,
)
from ..runtime import get_runtime_settings
@@ -244,7 +243,7 @@ async def request_password_reset(
delete_expired_password_reset_tokens()
target = await _resolve_reset_target(identifier)
if not target:
logger.info("password reset requested with no eligible match")
logger.info("password reset requested with no eligible match identifier=%s", identifier.strip().lower()[:256])
return {"status": "ok", "issued": False}
token = secrets.token_urlsafe(32)
@@ -325,7 +324,6 @@ async def apply_password_reset(token: str, new_password: str) -> Dict[str, Any]:
raise ValueError("Password reset link is invalid or has expired.")
await client.set_user_password(user_id, new_password)
sync_jellyfin_password_state(username, new_password)
increment_user_auth_version(username)
if str(stored_user.get("auth_provider") or "").strip().lower() != "jellyfin":
set_user_auth_provider(username, "jellyfin")
mark_password_reset_token_used(token)
-32
View File
@@ -1,32 +0,0 @@
"""Configured public email links, independent of request Host/forwarded headers."""
from urllib.parse import urlsplit
from ..runtime import get_runtime_settings
from ..installation_origin import managed_runtime
def valid_public_url(value):
value = str(value or '').strip().rstrip('/')
try:
parsed = urlsplit(value)
if (parsed.scheme in {'http', 'https'} and parsed.hostname
and not (parsed.username or parsed.password or parsed.query or parsed.fragment)
and (parsed.port is None or parsed.port > 0)
and not any(c.isspace() or ord(c) < 33 or c in '<>"\\' for c in value)):
return value
except ValueError:
pass
return ''
def magent_public_url(legacy_url=''):
runtime = get_runtime_settings()
proxy = getattr(runtime, 'magent_proxy_base_url', None)
application = getattr(runtime, 'magent_application_url', None)
if managed_runtime():
return valid_public_url(application)
if getattr(runtime, 'magent_proxy_enabled', False) and str(proxy or '').strip():
return valid_public_url(proxy)
if str(application or '').strip():
return valid_public_url(application)
# Preserve pre-existing installations until Hosting & proxy has been configured.
return valid_public_url(legacy_url)
+2 -18
View File
@@ -88,26 +88,10 @@ def render_recap(report: dict, username: str, public_url: str, unsubscribe_url:
content = '<table role="presentation" class="email-metrics" width="100%" cellpadding="0" cellspacing="0" style="table-layout:fixed"><tr>' + ''.join(cells[:2]) + '</tr><tr>' + ''.join(cells[2:]) + '</tr></table>'
habit = f"{number(summary['active_days'])} days watched · {number(summary['longest_streak'])}-day longest run"
content += f'<p style="color:#e5e1e4;font-size:14px;line-height:1.7;margin:24px 0">{esc(habit)}</p>'
patterns = report.get("patterns", {})
if patterns:
detail = f"Average play: {number(patterns['average_play_minutes'])} min. Longest play: {number(patterns['longest_play_minutes'])} min. Weekend viewing: {number(patterns['weekend_percent'])}%."
lines.append(detail)
content += f'<p style="padding:18px;background:#242334;border-radius:12px;color:#d8cfff;line-height:1.8">{esc(detail)}</p>'
for heading, rows in (("Your week in viewing (UTC)", patterns["weekdays"]), ("Movies, TV and more", patterns["media"])):
peak = max(1, *(row["minutes"] for row in rows))
content += f'<h2 style="font-size:18px;color:#e5e1e4">{heading}</h2><table role="presentation" width="100%" cellspacing="0" cellpadding="0">'
for row in rows:
width = round(row["minutes"] / peak * 100)
content += f'<tr><td style="padding:8px 0;color:#bdb6c3;font-size:12px;width:100px">{esc(row["name"])}</td><td style="padding:8px"><table role="presentation" width="{width}%" cellspacing="0" cellpadding="0"><tr><td height="8" style="background:{"#8cdbdd" if width else "transparent"};border-radius:4px;font-size:0">&nbsp;</td></tr></table></td><td style="width:65px;color:#e0d8ff;font-size:12px;text-align:right">{number(row["minutes"])} min</td></tr>'
lines.append(f"{row['name']}: {number(row['minutes'])} minutes")
content += '</table>'
top = report.get("top_titles", [])[:3]
if top:
content += '<h2 style="font-size:18px;color:#e5e1e4;margin:24px 0 8px">Your most watched</h2>'
for item in top:
artwork = item.get("email_artwork", "")
if artwork.startswith(("cid:", "data:image/")):
content += f'<img src="{esc(artwork, quote=True)}" alt="{esc(item["title"], quote=True)}" width="80" style="display:block;border-radius:10px;margin-top:20px" />'
content += f'<p style="font-size:14px;line-height:1.6;color:#e5e1e4;margin:12px 0;overflow-wrap:anywhere">{esc(item["title"])}<br><span style="font-size:12px;color:#a69fac">{number(item["minutes"])} minutes · {number(item["plays"])} plays</span></p>'
else:
content += '<p style="font-size:14px;color:#bdb6c3;line-height:1.7">No viewing was recorded this month. Your requests are still included.</p>'
@@ -146,8 +130,8 @@ def send_email(recipient: str, rendered: dict, message_id: str, before_data=lamb
html_part = message.get_payload()[-1]
for attachment in rendered.get('inline_images', []):
html_part.add_related(
attachment['data'], maintype='image', subtype=attachment.get('subtype', 'jpeg'), cid=f"<{attachment['cid']}>",
filename=attachment['cid'].split('@')[0] + '.' + attachment.get('subtype', 'jpeg'), disposition='inline')
attachment['data'], maintype='image', subtype='jpeg', cid=f"<{attachment['cid']}>",
filename=attachment['cid'].split('@')[0] + '.jpg', disposition='inline')
payload = message.as_bytes()
smtp, stage = None, "connect"
try:
-4
View File
@@ -10,7 +10,6 @@ from datetime import datetime
from .. import db
from .monthly_reports import shift_month
from . import email_queue
from .public_urls import magent_public_url
def init_schema(conn: sqlite3.Connection) -> None:
@@ -74,7 +73,6 @@ def read_one(sql: str, args=()) -> dict | None:
def settings() -> dict:
row = read_one("SELECT * FROM email_recap_settings WHERE id = 1")
row["public_url"] = magent_public_url(row["public_url"])
return {key: (bool(value) if key == "enabled" else value) for key, value in row.items() if key != "id"}
@@ -84,7 +82,6 @@ def next_due(now: datetime, day: int, hour: int) -> datetime:
def save_settings(values: dict, now: datetime) -> dict:
values = {**values, "public_url": magent_public_url(values.get("public_url", ""))}
with transaction() as conn:
old = dict(conn.execute("SELECT * FROM email_recap_settings WHERE id = 1").fetchone())
changed = any(old[key] != values[key] for key in ("day", "hour", "public_url"))
@@ -177,7 +174,6 @@ def enqueue_test(sub: dict, month: str, request_id: str, public_url: str, now: f
def enqueue_due(now: datetime) -> int:
with transaction() as conn:
config = dict(conn.execute("SELECT * FROM email_recap_settings WHERE id=1").fetchone())
config["public_url"] = magent_public_url(config["public_url"])
if not config["enabled"] or not config["next_send_at"] or config["next_send_at"] > now.timestamp():
return 0
# After long downtime, send only the latest due recap; never backfill a pile of old emails.
-69
View File
@@ -58,72 +58,3 @@ async def original_profile(client, default_id):
if not isinstance(result, dict) or not isinstance(result.get("id"), int):
raise HTTPException(502, "Radarr could not prepare the original-language profile. Try again.")
return result["id"]
async def apply_original_to_movie(client, tmdb_id):
movies = await client.get_movie_by_tmdb_id(tmdb_id)
if not isinstance(movies, list):
raise HTTPException(502, "Radarr did not return the movie list.")
matches = [movie for movie in movies if movie.get('tmdbId') == tmdb_id]
if not matches:
return None
if len(matches) != 1:
raise HTTPException(409, "Radarr returned multiple movies for this identity.")
movie = matches[0]
profile_id = await original_profile(client, movie['qualityProfileId'])
if movie['qualityProfileId'] != profile_id:
movie['qualityProfileId'] = profile_id
await client.update_movie(movie)
verified = await client.get_movie(movie['id'])
if not verified or verified.get('qualityProfileId') != profile_id:
raise HTTPException(502, "Radarr did not save the original-language choice. Try again before searching.")
return profile_id
async def movie_search_outcome(client, movie_id, command, attempts=12, delay=2):
command_id = command.get('id') if isinstance(command, dict) else None
if not isinstance(command_id, int):
return {'status': 'searching', 'message': 'Search submitted; download confirmation is not available yet. Recheck the request shortly.'}
for attempt in range(attempts):
state = await client.get(f'/api/v3/command/{command_id}')
status = str((state or {}).get('status', '')).lower()
queue = await client.get_queue(movie_id)
records = queue.get('records', []) if isinstance(queue, dict) else queue or []
matching = [item for item in records if item.get('movieId') == movie_id]
if any(item.get('trackedDownloadStatus') in {'warning', 'error'} for item in matching):
return {'status': 'attention', 'message': 'Radarr found a download, but it reports a download or import problem. Open the pipeline details to review it.'}
if matching:
return {'status': 'downloading', 'message': 'Radarr has a download queued for this movie. The pipeline will track its progress.'}
if status in {'failed', 'aborted', 'cancelled'}:
return {'status': 'attention', 'message': 'Radarr could not complete the search. Check service health or try Search and choose a download.'}
if status == 'completed':
movie = await client.get_movie(movie_id)
if (movie or {}).get('hasFile'):
return {'status': 'complete', 'message': 'Radarr already has the movie file. Recheck the request for Jellyfin availability.'}
# Command completion precedes download-client queue refresh. Keep polling.
pass
if attempt + 1 < attempts:
await asyncio.sleep(delay)
return {'status': 'pending', 'message': 'The search was submitted, but a download is not confirmed yet. The download queue may still be updating. Close this window and recheck the request shortly.'}
async def series_search_outcome(client, series_id, commands, attempts=12, delay=2):
ids = [item.get('id') for item in commands if isinstance(item, dict) and isinstance(item.get('id'), int)]
if not ids:
return {'status': 'searching', 'message': 'Search submitted to Sonarr; no download is confirmed yet. Recheck the pipeline shortly.'}
for attempt in range(attempts):
states = await asyncio.gather(*(client.get(f'/api/v3/command/{identity}') for identity in ids))
queue = await client.get_queue(series_id)
records = queue.get('records', []) if isinstance(queue, dict) else queue or []
matching = [item for item in records if item.get('seriesId') == series_id]
if any(item.get('trackedDownloadStatus') in {'warning', 'error'} for item in matching):
return {'status': 'attention', 'message': 'Sonarr has a download with a reported problem. Review the pipeline details.'}
if matching:
return {'status': 'downloading', 'message': 'Sonarr has downloads queued for this show. The pipeline will track their progress.'}
statuses = {str((state or {}).get('status', '')).lower() for state in states}
if statuses & {'failed', 'aborted', 'cancelled'}:
return {'status': 'attention', 'message': 'A Sonarr search failed. Check the service or try Search and choose a download.'}
# Even completed commands can precede Sonarr's download queue refresh.
if attempt + 1 < attempts:
await asyncio.sleep(delay)
return {'status': 'pending', 'message': 'The search was submitted, but a download is not confirmed yet. The download queue may still be updating. Close this window and recheck the request shortly.'}
-63
View File
@@ -1,63 +0,0 @@
"""State-changing requests may originate only from explicitly configured sites.
The public Hosting & proxy URL can be stored in the database, while the CORS
environment setting still has its localhost default on an upgraded install.
Never infer a trusted origin from request Host or forwarded headers.
"""
from urllib.parse import urlsplit
from starlette.middleware.cors import CORSMiddleware
from ..config import settings
from ..installation_origin import managed_runtime
from .public_urls import magent_public_url, valid_public_url
def _origin(value: str, *, configured_url: bool = False) -> tuple[str, str, int] | None:
value = str(value or "")
if any(character.isspace() or ord(character) < 33 or ord(character) == 127 for character in value):
return None
if "?" in value or "#" in value:
return None
validated = valid_public_url(value)
if not validated:
return None
parsed = urlsplit(value)
if parsed.username is not None or parsed.password is not None:
return None
if not configured_url and parsed.path:
return None
return (
parsed.scheme.lower(),
parsed.hostname.lower(),
parsed.port or (443 if parsed.scheme == "https" else 80),
)
def is_allowed_request_origin(origin: str) -> bool:
candidate = _origin(origin)
if candidate is None:
return False
if managed_runtime():
# The operator confirms this address using the first-install token.
# No localhost fallback remains trusted after a managed installation.
return candidate == _origin(magent_public_url(), configured_url=True)
if candidate == _origin(str(settings.cors_allow_origin or "").rstrip("/")):
return True
return candidate == _origin(magent_public_url(), configured_url=True)
def can_claim_initial_origin() -> bool:
if not managed_runtime() or magent_public_url():
return False
from .setup import get_public_setup_status
return get_public_setup_status()["needs_admin"]
class ConfiguredOriginCORSMiddleware(CORSMiddleware):
"""Keep CORS response/preflight policy aligned with managed origin checks."""
def is_allowed_origin(self, origin: str) -> bool:
if managed_runtime():
return is_allowed_request_origin(origin)
return super().is_allowed_origin(origin)
-206
View File
@@ -1,206 +0,0 @@
"""Persistent, operator-authorized first-install setup.
Initialize the marker before the main schema: an existing users table identifies
an upgraded installation, while a new database must finish the setup wizard.
The marker and first administrator are protected by SQLite write transactions.
"""
from datetime import datetime, timezone
import hmac
from math import ceil
from time import time
from typing import Literal
from .. import db
from ..config import settings
from ..security import hash_password, validate_password_policy
from ..installation_origin import normalize_application_origin
SetupStep = Literal["administrator", "apps", "preferences", "review"]
SETUP_STEPS = ("administrator", "apps", "preferences", "review")
BOOTSTRAP_WINDOW_SECONDS = 15 * 60
BOOTSTRAP_IP_ATTEMPTS = 5
BOOTSTRAP_GLOBAL_ATTEMPTS = 30
class SetupUnavailableError(ValueError):
"""Setup has finished, or another administrator already exists."""
class InvalidSetupTokenError(ValueError):
"""The operator's setup token was absent or did not match."""
def initialize_setup_state() -> None:
"""Run once before init_db; subsequent calls preserve progress."""
with db._connect() as conn:
conn.execute("BEGIN IMMEDIATE")
existing_install = conn.execute(
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'users'"
).fetchone() is not None
conn.execute(
"""CREATE TABLE IF NOT EXISTS installation_setup (
id INTEGER PRIMARY KEY CHECK (id = 1),
completed INTEGER NOT NULL CHECK (completed IN (0, 1)),
step TEXT NOT NULL,
completed_at TEXT
)"""
)
conn.execute(
"""CREATE TABLE IF NOT EXISTS installation_setup_attempts (
scope TEXT NOT NULL,
key_hash TEXT NOT NULL,
occurred_at REAL NOT NULL
)"""
)
conn.execute(
"""INSERT OR IGNORE INTO installation_setup (id, completed, step, completed_at)
VALUES (1, ?, ?, ?)""",
(
int(existing_install),
"review" if existing_install else "administrator",
datetime.now(timezone.utc).isoformat() if existing_install else None,
),
)
def get_setup_state() -> dict:
with db._connect() as conn:
# Old databases and isolated callers without startup initialization are
# already installed. A missing marker must never open public bootstrap.
table = conn.execute(
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'installation_setup'"
).fetchone()
row = conn.execute(
"SELECT completed, step, completed_at FROM installation_setup WHERE id = 1"
).fetchone() if table else None
if row is None:
return {"completed": True, "step": "review", "completed_at": None}
return {"completed": bool(row[0]), "step": row[1], "completed_at": row[2]}
def is_setup_required() -> bool:
return not get_setup_state()["completed"]
def get_public_setup_status() -> dict:
required = is_setup_required()
return {"setup_required": required, "needs_admin": required and not db.has_admin_user()}
def setup_token_configured() -> bool:
"""Reject missing values and obvious examples, without claiming to measure entropy."""
token = str(getattr(settings, "setup_token", "") or "").strip()
placeholder = token.casefold().replace("_", "-")
return (
len(token) >= 32
and len(set(token)) > 1
and not placeholder.startswith(("replace-with-", "replace-me", "change-me", "changeme", "your-setup-token"))
)
def consume_bootstrap_attempt(client_ip: str) -> int | None:
"""Atomically reserve one attempt; return Retry-After when limited.
The IP is keyed using the existing HMAC helper, never stored in clear text.
A shared cap limits distributed attempts and expensive password hashing.
"""
now = time()
cutoff = now - BOOTSTRAP_WINDOW_SECONDS
limits = (
("setup-ip", db._rate_limit_key_hash(client_ip), BOOTSTRAP_IP_ATTEMPTS),
("setup-global", db._rate_limit_key_hash("bootstrap"), BOOTSTRAP_GLOBAL_ATTEMPTS),
)
with db._connect() as conn:
conn.execute("BEGIN IMMEDIATE")
conn.execute(
"DELETE FROM installation_setup_attempts WHERE occurred_at < ?",
(cutoff,),
)
retry_after = 0
for scope, key, maximum in limits:
count, oldest = conn.execute(
"""SELECT COUNT(*), MIN(occurred_at) FROM installation_setup_attempts
WHERE scope = ? AND key_hash = ? AND occurred_at >= ?""",
(scope, key, cutoff),
).fetchone()
if count >= maximum:
retry_after = max(retry_after, ceil(BOOTSTRAP_WINDOW_SECONDS - (now - oldest)), 1)
if retry_after:
return retry_after
conn.executemany(
"INSERT INTO installation_setup_attempts (scope, key_hash, occurred_at) VALUES (?, ?, ?)",
[(scope, key, now) for scope, key, _ in limits],
)
return None
def bootstrap_administrator(setup_token: str, username: str, password: str, *, application_url: str | None = None) -> None:
"""Claim fresh setup exactly once using the deployment's setup token."""
expected = str(getattr(settings, "setup_token", "") or "")
if not setup_token_configured() or not hmac.compare_digest(
setup_token.encode("utf-8"), expected.encode("utf-8")
):
raise InvalidSetupTokenError("Invalid setup token.")
username = username.strip()
if not username or len(username) > 100 or any(
character.isspace() or ord(character) < 32 or ord(character) == 127 for character in username
):
raise ValueError("Username must contain 1 to 100 characters without spaces or control characters.")
if len(password) > 1024:
raise ValueError("Password must contain no more than 1024 characters.")
password = validate_password_policy(password)
if application_url is not None:
application_url = normalize_application_origin(application_url)
if not is_setup_required() or db.has_admin_user():
raise SetupUnavailableError("Initial administrator setup is no longer available.")
password_hash = hash_password(password)
with db._connect() as conn:
conn.execute("BEGIN IMMEDIATE")
setup = conn.execute("SELECT completed FROM installation_setup WHERE id = 1").fetchone()
admin = conn.execute("SELECT 1 FROM users WHERE LOWER(role) = 'admin' LIMIT 1").fetchone()
if setup is None or setup[0] or admin:
raise SetupUnavailableError("Initial administrator setup is no longer available.")
if any(str(row[0]).strip().casefold() == username.casefold() for row in conn.execute("SELECT username FROM users")):
raise SetupUnavailableError("That username already exists.")
conn.execute(
"""INSERT INTO users (username, password_hash, role, auth_provider, created_at)
VALUES (?, ?, 'admin', 'local', ?)""",
(username, password_hash, datetime.now(timezone.utc).isoformat()),
)
conn.execute("UPDATE installation_setup SET step = 'apps' WHERE id = 1")
if application_url is not None:
conn.execute(
"""INSERT INTO settings (key, value, updated_at) VALUES ('magent_application_url', ?, ?)
ON CONFLICT(key) DO UPDATE SET value=excluded.value, updated_at=excluded.updated_at""",
(application_url, datetime.now(timezone.utc).isoformat()),
)
def update_setup_step(step: SetupStep) -> dict:
if step not in SETUP_STEPS:
raise ValueError("Invalid setup step.")
if not is_setup_required():
return get_setup_state()
with db._connect() as conn:
conn.execute(
"UPDATE installation_setup SET step = ? WHERE id = 1 AND completed = 0", (step,)
)
return get_setup_state()
def complete_setup() -> dict:
if not is_setup_required():
return get_setup_state()
with db._connect() as conn:
conn.execute("BEGIN IMMEDIATE")
if not conn.execute("SELECT 1 FROM users WHERE LOWER(role) = 'admin' LIMIT 1").fetchone():
raise SetupUnavailableError("Create an administrator before completing setup.")
conn.execute(
"""UPDATE installation_setup SET completed = 1, step = 'review', completed_at = ?
WHERE id = 1 AND completed = 0""",
(datetime.now(timezone.utc).isoformat(),),
)
return get_setup_state()
+37 -68
View File
@@ -32,7 +32,6 @@ from ..models import ActionOption, NormalizedState, RequestType, Snapshot, Timel
from .collector_search import read_search_status
from .media_repair import current_cycle_torrents, evaluate_media_repair
from .download_labels import label_episode_downloads
from .arr import RootFolderNotFoundError, resolve_root_folder_path
logger = logging.getLogger(__name__)
@@ -138,10 +137,12 @@ def jellyfin_item_matches_request(
request_provider_ids = extract_request_provider_ids(request_payload or {})
item_provider_ids = extract_request_provider_ids(item)
shared = set(request_provider_ids) & set(item_provider_ids)
if shared:
# Conflicting metadata must never fall through to title matching.
return all(request_provider_ids[key] == item_provider_ids[key] for key in shared)
provider_priority = ("tmdb", "tvdb", "imdb")
for key in provider_priority:
request_id = request_provider_ids.get(key)
item_id = item_provider_ids.get(key)
if request_id and item_id and request_id == item_id:
return True
request_title = _normalize_media_title(title)
if not request_title:
@@ -168,6 +169,11 @@ def jellyfin_item_matches_request(
if request_title in item_titles:
return True
if request_type == RequestType.tv:
for candidate in item_titles:
if candidate and (candidate.startswith(request_title) or request_title.startswith(candidate)):
return True
return False
@@ -370,38 +376,6 @@ def _episode_availability(episodes: Any) -> Dict[str, Any]:
}
def _unmonitored_season_options(series: Any, episodes: Any) -> List[Dict[str, int]]:
"""Describe regular Sonarr seasons that can be added to an existing request."""
if not isinstance(series, dict) or not isinstance(series.get("seasons"), list):
return []
episode_rows = [episode for episode in episodes if isinstance(episode, dict)] if isinstance(episodes, list) else []
options: List[Dict[str, int]] = []
for season in series["seasons"]:
if not isinstance(season, dict) or season.get("monitored") is not False:
continue
season_number = season.get("seasonNumber")
if not isinstance(season_number, int) or season_number <= 0:
continue
matching = [episode for episode in episode_rows if episode.get("seasonNumber") == season_number]
statistics = season.get("statistics") if isinstance(season.get("statistics"), dict) else {}
episode_count = statistics.get("totalEpisodeCount")
if not isinstance(episode_count, int):
episode_count = statistics.get("episodeCount")
if not isinstance(episode_count, int):
episode_count = len(matching)
available = statistics.get("episodeFileCount")
if not isinstance(available, int):
available = sum(1 for episode in matching if episode.get("hasFile") is True)
options.append(
{
"seasonNumber": season_number,
"episodeCount": max(0, episode_count),
"available": max(0, available),
}
)
return sorted(options, key=lambda item: item["seasonNumber"])
def _summarize_qbit(torrents: List[Dict[str, Any]]) -> Dict[str, Any]:
if not torrents:
return {"state": "idle", "message": "0 active downloads."}
@@ -582,9 +556,9 @@ def _build_repair_activity(
message = (
f"{collector} now reports the replacement file as collected. "
+ (
"It is also available in Jellyfin."
"It is also available in Grizzlyflix."
if jellyfin_found
else "Jellyfin is indexing the updated file now."
else "Grizzlyflix is indexing the updated file now."
)
)
state = "complete" if jellyfin_found else "indexing"
@@ -658,7 +632,7 @@ def _build_repair_activity(
"label": "Updated media available",
"state": available_step_state,
"detail": (
"The repaired title is available in Jellyfin."
"The repaired title is available in Grizzlyflix."
if jellyfin_found and collection_complete
else (
"The media server is indexing the replacement."
@@ -922,22 +896,22 @@ def _build_presentation(
available_label = "Partially available"
available_state = "partial"
available_state_label = "Partly ready"
available_summary = f"{available} of {total} episodes are ready to watch in Jellyfin."
available_summary = f"{available} of {total} episodes are ready to watch in Grizzlyflix."
elif jellyfin_found:
available_label = "Available to watch"
available_state = "complete"
available_state_label = "Ready"
available_summary = "This title is ready to watch in Jellyfin."
available_summary = "This title is ready to watch in Grizzlyflix."
elif arr_state == "available":
available_label = "Adding to Jellyfin"
available_label = "Adding to Grizzlyflix"
available_state = "active"
available_state_label = "Indexing"
available_summary = "The download is complete. Jellyfin is indexing this title now."
available_summary = "The download is complete. Grizzlyflix is indexing this title now."
else:
available_label = "Media server"
available_state = "waiting"
available_state_label = "Waiting"
available_summary = "This title has not reached Jellyfin yet."
available_summary = "This title has not reached Grizzlyflix yet."
display_download = dict(download)
if fully_available:
@@ -972,7 +946,6 @@ def _build_presentation(
"missing": missing,
"total": total,
"seasons": availability.get("seasons") or [],
"unmonitoredSeasons": arr_details.get("unmonitoredSeasons") or [],
"missingEpisodes": arr_details.get("missingEpisodes") or {},
},
{
@@ -1026,13 +999,13 @@ def _apply_repair_presentation(
search = (arr_details.get("search") or {}).get("state")
pipeline = {stage["id"]: stage for stage in snapshot.presentation["pipeline"]}
if imported:
label = "Replacement collected — updating Jellyfin"
meaning = "The replacement has been imported. Waiting for Jellyfin to index the updated file."
label = "Replacement collected — updating Grizzlyflix"
meaning = "The replacement has been imported. Waiting for Grizzlyflix to index the updated file."
snapshot.state = NormalizedState.importing
pipeline["download"].update(state="complete", summary="The replacement has been imported.", torrents=[], visible=False)
pipeline["available"].update(label="Updating Jellyfin", state="active", stateLabel="Indexing", summary=meaning)
pipeline["available"].update(label="Updating Grizzlyflix", state="active", stateLabel="Indexing", summary=meaning)
snapshot.presentation["nextStep"] = {
"title": "Wait for the updated file", "description": "This page will update when Jellyfin confirms the replacement.", "actionIds": [],
"title": "Wait for the updated file", "description": "This page will update when Grizzlyflix confirms the replacement.", "actionIds": [],
}
elif unavailable:
label = "Repair status temporarily unavailable"
@@ -1073,7 +1046,7 @@ def _apply_repair_presentation(
if has_unaffected and catalog_found and isinstance(jellyfin_item, dict) and jellyfin_item.get("Id"):
link = f"{public_url.rstrip('/')}/web/index.html#!/details?id={quote(str(jellyfin_item['Id']))}" if public_url else None
pipeline["available"].update(label="Partially available", state="partial", stateLabel="Repair in progress",
summary="Other collected episodes remain available. The selected episodes are being replaced." if not imported else "Other episodes remain available. Waiting for Jellyfin to index the repaired episodes.", link=link)
summary="Other collected episodes remain available. The selected episodes are being replaced." if not imported else "Other episodes remain available. Waiting for Grizzlyflix to index the repaired episodes.", link=link)
snapshot.raw["jellyfin"].update(partial=True, link=link)
@@ -1235,6 +1208,11 @@ async def build_snapshot(request_id: str) -> Snapshot:
arr_item = None
arr_queue = None
episodes = None
media_status = jelly_request.get("media", {}).get("status")
try:
media_status_code = int(media_status) if media_status is not None else None
except (TypeError, ValueError):
media_status_code = None
if snapshot.request_type == RequestType.tv:
tvdb_id = jelly_request.get("media", {}).get("tvdbId")
if tvdb_id:
@@ -1269,7 +1247,6 @@ async def build_snapshot(request_id: str) -> Snapshot:
"state": await read_search_status(sonarr, RequestType.tv, series_id, episodes)
}
arr_details["availability"] = _episode_availability(episodes)
arr_details["unmonitoredSeasons"] = _unmonitored_season_options(arr_item, episodes)
counts = arr_details["availability"]
arr_state = "available" if counts.get("total", 0) > 0 and not counts.get("missing") else "added"
missing_by_season = _missing_episode_numbers_by_season(episodes)
@@ -1386,15 +1363,11 @@ async def build_snapshot(request_id: str) -> Snapshot:
if runtime.radarr_quality_profile_id and runtime.radarr_root_folder:
radarr_client = RadarrClient(runtime.radarr_base_url, runtime.radarr_api_key)
if radarr_client.configured():
try:
root_folder = await resolve_root_folder_path(
radarr_client, runtime.radarr_root_folder, "Radarr"
)
except RootFolderNotFoundError as exc:
logger.warning("Skipping Jellyfin-to-Radarr sync: %s", exc)
root_folder = ""
root_folder = await _resolve_root_folder_path(
radarr_client, runtime.radarr_root_folder, "Radarr"
)
tmdb_id = jelly_request.get("media", {}).get("tmdbId")
if tmdb_id and root_folder:
if tmdb_id:
try:
await radarr_client.add_movie(
int(tmdb_id),
@@ -1409,15 +1382,11 @@ async def build_snapshot(request_id: str) -> Snapshot:
if runtime.sonarr_quality_profile_id and runtime.sonarr_root_folder:
sonarr_client = SonarrClient(runtime.sonarr_base_url, runtime.sonarr_api_key)
if sonarr_client.configured():
try:
root_folder = await resolve_root_folder_path(
sonarr_client, runtime.sonarr_root_folder, "Sonarr"
)
except RootFolderNotFoundError as exc:
logger.warning("Skipping Jellyfin-to-Sonarr sync: %s", exc)
root_folder = ""
root_folder = await _resolve_root_folder_path(
sonarr_client, runtime.sonarr_root_folder, "Sonarr"
)
tvdb_id = jelly_request.get("media", {}).get("tvdbId")
if tvdb_id and root_folder:
if tvdb_id:
try:
await sonarr_client.add_series(
int(tvdb_id),
-4
View File
@@ -1,4 +0,0 @@
-r requirements.txt
coverage==7.16.1
pip-audit==2.10.1
ruff==0.16.8
-2
View File
@@ -5,8 +5,6 @@ pydantic==2.12.5
pydantic-settings==2.14.2
PyJWT==2.13.0
passlib==1.7.4
argon2-cffi==25.1.0
cryptography==50.0.1
python-multipart==0.0.31
Pillow==12.3.0
prometheus-client==0.22.1
-24
View File
@@ -1,24 +0,0 @@
import unittest
from pydantic import ValidationError
from backend.app.api_models import PasswordResetRequest, SignupRequest
class ApiRequestModelTests(unittest.TestCase):
def test_signup_rejects_unknown_fields(self) -> None:
with self.assertRaises(ValidationError):
SignupRequest(
invite_code="invite",
username="viewer",
password="strong password",
unexpected="value",
)
def test_password_reset_preserves_password_whitespace_for_policy_validation(self) -> None:
request = PasswordResetRequest(token="token", new_password=" leading and trailing ")
self.assertEqual(request.new_password, " leading and trailing ")
if __name__ == "__main__":
unittest.main()
-24
View File
@@ -1,24 +0,0 @@
import unittest
from backend.app.services.arr import RootFolderNotFoundError, resolve_root_folder_path
class _ArrClient:
async def get_root_folders(self):
return [{"id": 7, "path": "/media/tv"}]
class ArrHelperTests(unittest.IsolatedAsyncioTestCase):
async def test_resolves_numeric_root_folder_id(self) -> None:
self.assertEqual(await resolve_root_folder_path(_ArrClient(), "7", "Sonarr"), "/media/tv")
async def test_preserves_configured_path(self) -> None:
self.assertEqual(await resolve_root_folder_path(_ArrClient(), "/media/movies", "Radarr"), "/media/movies")
async def test_rejects_missing_root_folder_id(self) -> None:
with self.assertRaises(RootFolderNotFoundError):
await resolve_root_folder_path(_ArrClient(), "8", "Sonarr")
if __name__ == "__main__":
unittest.main()
+40 -439
View File
@@ -2,29 +2,27 @@ import os
from types import SimpleNamespace
import tempfile
import unittest
from unittest.mock import AsyncMock, call, patch
from unittest.mock import AsyncMock, patch
import httpx
from fastapi import HTTPException
from passlib.context import CryptContext
from starlette.requests import Request
from backend.app import db
from backend.app.clients.base import _operation_error_message, _operation_result_message
from backend.app.clients.jellyfin import _availability_message
from backend.app.clients.qbittorrent import _torrent_result_message
from backend.app.auth import _load_current_user_from_token, require_admin
from backend.app.auth import require_admin
from backend.app.config import settings
from backend.app.network_security import request_trusts_forwarded_headers, validate_notification_target_url
from backend.app.models import ActionOption, NormalizedState, RequestType, Snapshot, TimelineHop
from backend.app.routers import auth as auth_router
from backend.app.routers import admin as admin_router
from backend.app.routers import branding as branding_router
from backend.app.routers import portal as portal_router
from backend.app.routers import requests as requests_router
from backend.app.routers import site as site_router
from backend.app.routers import status as status_router
from backend.app.security import PASSWORD_POLICY_MESSAGE, create_access_token, validate_password_policy
from backend.app.security import PASSWORD_POLICY_MESSAGE, validate_password_policy
from backend.app.services import password_reset
from backend.app.services import issue_resolution
from backend.app.services.operation_progress import (
@@ -41,7 +39,6 @@ from backend.app.services.snapshot import (
_build_repair_activity,
_episode_availability,
_torrent_progress,
_unmonitored_season_options,
)
@@ -73,16 +70,21 @@ class TempDatabaseMixin:
self._tempdir = tempfile.TemporaryDirectory(ignore_cleanup_errors=True)
self._original_sqlite_path = settings.sqlite_path
self._original_journal_mode = getattr(settings, "sqlite_journal_mode", "DELETE")
self._original_settings_encryption_key = settings.settings_encryption_key
settings.sqlite_path = os.path.join(self._tempdir.name, "test.db")
settings.sqlite_journal_mode = "DELETE"
settings.settings_encryption_key = "bWFnZW50LXNlY3VyaXR5LXRlc3Qta2V5LTMyLWJ5dGU="
auth_router._LOGIN_ATTEMPTS_BY_IP.clear()
auth_router._LOGIN_ATTEMPTS_BY_USER.clear()
auth_router._RESET_ATTEMPTS_BY_IP.clear()
auth_router._RESET_ATTEMPTS_BY_IDENTIFIER.clear()
db.init_db()
def tearDown(self) -> None:
settings.sqlite_path = self._original_sqlite_path
settings.sqlite_journal_mode = self._original_journal_mode
settings.settings_encryption_key = self._original_settings_encryption_key
auth_router._LOGIN_ATTEMPTS_BY_IP.clear()
auth_router._LOGIN_ATTEMPTS_BY_USER.clear()
auth_router._RESET_ATTEMPTS_BY_IP.clear()
auth_router._RESET_ATTEMPTS_BY_IDENTIFIER.clear()
self._tempdir.cleanup()
super_method = getattr(super(), "tearDown", None)
if callable(super_method):
@@ -95,204 +97,7 @@ class PasswordPolicyTests(unittest.TestCase):
validate_password_policy("short")
def test_validate_password_policy_trims_whitespace(self) -> None:
self.assertEqual(validate_password_policy(" password1234 "), "password1234")
class SecurityHardeningTests(TempDatabaseMixin, unittest.IsolatedAsyncioTestCase):
def setUp(self) -> None:
super().setUp()
self._jwt_secret = patch.object(
settings, "jwt_secret", "security-hardening-tests-secret-123456789"
)
self._jwt_secret.start()
self.addCleanup(self._jwt_secret.stop)
def test_sensitive_settings_are_encrypted_at_rest(self) -> None:
db.set_setting("jellyfin_api_key", "private-api-key")
with db._connect() as conn:
stored = conn.execute(
"SELECT value FROM settings WHERE key = ?", ("jellyfin_api_key",)
).fetchone()[0]
self.assertTrue(stored.startswith("enc:v1:"))
self.assertNotIn("private-api-key", stored)
self.assertEqual(db.get_setting("jellyfin_api_key"), "private-api-key")
def test_invites_are_hashed_and_rotation_invalidates_old_link(self) -> None:
created = db.create_signup_invite(code="TopSecretInvite42")
invite_id = int(created["id"])
with db._connect() as conn:
stored = conn.execute(
"SELECT code FROM signup_invites WHERE id = ?", (invite_id,)
).fetchone()[0]
self.assertTrue(stored.startswith("sha256:"))
self.assertNotIn("TOPSECRETINVITE42", stored.upper())
self.assertFalse(db.get_signup_invite_by_id(invite_id)["code_available"])
self.assertIsNotNone(db.get_signup_invite_by_code("TopSecretInvite42"))
rotated = db.rotate_signup_invite_code(invite_id, "ReplacementInvite99")
self.assertTrue(rotated["code_available"])
self.assertIsNone(db.get_signup_invite_by_code("TopSecretInvite42"))
self.assertIsNotNone(db.get_signup_invite_by_code("ReplacementInvite99"))
def test_legacy_invites_and_plaintext_settings_migrate_in_place(self) -> None:
created = db.create_signup_invite(code="TemporaryInvite77")
with db._connect() as conn:
conn.execute(
"UPDATE signup_invites SET code = ?, code_hint = NULL WHERE id = ?",
("Legacy-Code-77", int(created["id"])),
)
conn.execute(
"INSERT OR REPLACE INTO settings (key, value, updated_at) VALUES (?, ?, ?)",
("radarr_api_key", "legacy-plaintext-key", "2026-09-17T00:00:00+00:00"),
)
db.init_db()
migrated = db.get_signup_invite_by_code("Legacy-Code-77")
self.assertEqual(migrated["id"], created["id"])
self.assertEqual(db.get_setting("radarr_api_key"), "legacy-plaintext-key")
with db._connect() as conn:
invite_code = conn.execute(
"SELECT code FROM signup_invites WHERE id = ?", (int(created["id"]),)
).fetchone()[0]
stored_setting = conn.execute(
"SELECT value FROM settings WHERE key = 'radarr_api_key'"
).fetchone()[0]
self.assertTrue(invite_code.startswith("sha256:"))
self.assertTrue(stored_setting.startswith("enc:v1:"))
def test_legacy_password_hash_is_replaced_with_argon2(self) -> None:
password = "Example-password123!"
db.create_user("legacy", password)
legacy_hash = CryptContext(schemes=["pbkdf2_sha256"]).hash(password)
with db._connect() as conn:
conn.execute(
"UPDATE users SET password_hash = ? WHERE username = ?",
(legacy_hash, "legacy"),
)
self.assertIsNotNone(db.verify_user_password("legacy", password))
self.assertTrue(db.get_user_by_username("legacy")["password_hash"].startswith("$argon2"))
def test_auth_version_revokes_existing_token(self) -> None:
db.create_user("viewer", "Example-password123!")
user = db.get_user_by_username("viewer")
token = create_access_token(
"viewer", "user", auth_version=int(user["auth_version"])
)
self.assertEqual(_load_current_user_from_token(token)["username"], "viewer")
db.increment_user_auth_version("viewer")
with self.assertRaises(HTTPException) as context:
_load_current_user_from_token(token)
self.assertEqual(context.exception.status_code, 401)
async def test_request_mutations_require_owner_or_admin(self) -> None:
runtime = SimpleNamespace(
jellyseerr_base_url="http://seerr.test", jellyseerr_api_key="secret"
)
client = SimpleNamespace(
configured=lambda: True,
get_request=AsyncMock(
return_value={"id": 42, "requestedBy": {"username": "owner"}}
),
)
with patch.object(requests_router, "JellyseerrClient", return_value=client):
with self.assertRaises(HTTPException) as context:
await requests_router._ensure_request_mutation_access(
runtime, 42, {"username": "someone-else", "role": "user"}
)
self.assertEqual(context.exception.status_code, 403)
owned = await requests_router._ensure_request_mutation_access(
runtime, 42, {"username": "owner", "role": "user"}
)
self.assertEqual(owned["id"], 42)
self.assertIsNone(
await requests_router._ensure_request_mutation_access(
SimpleNamespace(), 42, {"username": "admin", "role": "admin"}
)
)
def test_account_deletion_removes_or_anonymizes_personal_data(self) -> None:
db.create_user(
"viewer", "Example-password123!", email="viewer@example.test"
)
user = db.get_user_by_username("viewer")
now = "2026-09-17T00:00:00+00:00"
db.upsert_request_cache(
42,
99,
"movie",
2,
"Example",
2026,
"viewer",
"viewer",
int(user["id"]),
now,
now,
'{"requestedBy":{"username":"viewer","email":"viewer@example.test"}}',
)
with db._connect() as conn:
conn.execute(
"INSERT INTO snapshots (request_id, state, created_at, payload_json) VALUES (?, ?, ?, ?)",
(
"42",
"available",
now,
'{"requestedBy":{"username":"viewer","email":"viewer@example.test"}}',
),
)
db.save_action("42", "created", "Created", "ok", "Created by viewer")
item = db.create_portal_item(
kind="issue",
title="Example",
description="Example",
created_by_username="viewer",
created_by_id=int(user["id"]),
)
result = db.delete_user_data_by_username("viewer")
self.assertTrue(result["deleted"])
self.assertIsNone(db.get_user_by_username("viewer"))
with db._connect() as conn:
request_row = conn.execute(
"SELECT requested_by, requested_by_id, payload_json FROM requests_cache WHERE request_id = 42"
).fetchone()
snapshot_json = conn.execute(
"SELECT payload_json FROM snapshots WHERE request_id = '42'"
).fetchone()[0]
action_message = conn.execute(
"SELECT message FROM actions WHERE request_id = '42'"
).fetchone()[0]
portal_owner = conn.execute(
"SELECT created_by_username, created_by_id FROM portal_items WHERE id = ?",
(item["id"],),
).fetchone()
self.assertEqual(request_row[0], "Deleted user")
self.assertIsNone(request_row[1])
self.assertNotIn("viewer", request_row[2].lower())
self.assertNotIn("viewer", snapshot_json.lower())
self.assertNotIn("viewer", action_message.lower())
self.assertTrue(portal_owner[0].startswith("deleted-user-"))
self.assertIsNone(portal_owner[1])
async def test_branding_upload_rejects_oversized_images_before_decode(self) -> None:
upload = SimpleNamespace(
filename="logo.png",
content_type="image/png",
read=AsyncMock(return_value=b"x" * (5 * 1024 * 1024 + 1)),
)
with self.assertRaises(HTTPException) as context:
await branding_router.save_branding_image(upload)
self.assertEqual(context.exception.status_code, 413)
upload.read.assert_awaited_once_with(5 * 1024 * 1024 + 1)
self.assertEqual(validate_password_policy(" password123 "), "password123")
class NetworkSecurityTests(unittest.TestCase):
@@ -448,7 +253,7 @@ class OperationMessageTests(unittest.TestCase):
)
self.assertEqual(
_availability_message({"TotalRecordCount": 0, "Items": []}),
"Jellyfin did not find this title in its library search.",
"Grizzlyflix did not find this title in its library search.",
)
def test_bazarr_subtitle_search_is_explained_in_plain_english(self) -> None:
@@ -493,9 +298,6 @@ class SiteInfoTests(unittest.TestCase):
site_banner_enabled=False,
site_banner_message="",
site_banner_tone="info",
site_banner_background_color=None,
site_banner_border_color=None,
site_login_message="",
site_login_show_jellyfin_login=True,
site_login_show_local_login=True,
site_login_show_forgot_password=True,
@@ -508,49 +310,6 @@ class SiteInfoTests(unittest.TestCase):
self.assertEqual(info["navigation"], {"showRequests": False})
def test_site_public_exposes_safe_banner_colours_and_login_message(self) -> None:
runtime = settings.model_copy(update={
"site_banner_enabled": True,
"site_banner_message": "Planned maintenance",
"site_banner_tone": "warning",
"site_banner_background_color": "#123ABC",
"site_banner_border_color": "red",
"site_login_message": "Use your Grizzlyflix account to sign in.",
})
with patch.object(site_router, "get_runtime_settings", return_value=runtime):
info = site_router._build_site_info(False)
self.assertEqual(info["banner"]["backgroundColor"], "#123abc")
self.assertIsNone(info["banner"]["borderColor"])
self.assertEqual(info["login"]["message"], "Use your Grizzlyflix account to sign in.")
class SiteSettingValidationTests(unittest.IsolatedAsyncioTestCase):
async def test_banner_colours_are_normalized_before_saving(self) -> None:
with patch.object(admin_router, "set_setting") as save:
result = await admin_router.update_settings({
"site_banner_background_color": "#A1B2C3",
"site_banner_border_color": "#010203",
})
self.assertEqual(result, {"status": "ok", "updated": 2})
self.assertEqual(
save.call_args_list,
[
call("site_banner_background_color", "#a1b2c3"),
call("site_banner_border_color", "#010203"),
],
)
async def test_banner_colours_reject_unsafe_css_values(self) -> None:
with self.assertRaises(HTTPException) as raised:
await admin_router.update_settings({
"site_banner_border_color": "red; background: url(example)",
})
self.assertEqual(raised.exception.status_code, 400)
class RequestCacheTests(unittest.TestCase):
def tearDown(self) -> None:
@@ -762,34 +521,6 @@ class RequestPresentationTests(unittest.TestCase):
self.assertEqual(availability["missing"], 1)
self.assertEqual(availability["total"], 2)
def test_unmonitored_seasons_are_offered_separately_from_collection_progress(self) -> None:
series = {
"seasons": [
{"seasonNumber": 0, "monitored": False},
{"seasonNumber": 7, "monitored": True},
{
"seasonNumber": 8,
"monitored": False,
"statistics": {"episodeCount": 16, "episodeFileCount": 2},
},
{"seasonNumber": 9, "monitored": False},
]
}
episodes = [
{"seasonNumber": 9, "episodeNumber": 1, "hasFile": True},
{"seasonNumber": 9, "episodeNumber": 2, "hasFile": False},
]
options = _unmonitored_season_options(series, episodes)
self.assertEqual(
options,
[
{"seasonNumber": 8, "episodeCount": 16, "available": 2},
{"seasonNumber": 9, "episodeCount": 2, "available": 1},
],
)
def test_presentation_hides_download_without_download_evidence(self) -> None:
snapshot = Snapshot(
request_id="3909",
@@ -868,7 +599,7 @@ class RequestPresentationTests(unittest.TestCase):
self.assertEqual(available_stage["state"], "complete")
self.assertEqual(available_stage["stateLabel"], "Ready")
self.assertEqual(available_stage["label"], "Available to watch")
self.assertEqual(available_stage["summary"], "This title is ready to watch in Jellyfin.")
self.assertEqual(available_stage["summary"], "This title is ready to watch in Grizzlyflix.")
self.assertEqual(available_stage["link"], "https://media.test/title/3909")
def test_partially_available_content_keeps_missing_download_attention(self) -> None:
@@ -940,10 +671,10 @@ class RequestPresentationTests(unittest.TestCase):
self.assertEqual(download_stage["state"], "complete")
self.assertEqual(available_stage["state"], "active")
self.assertEqual(available_stage["stateLabel"], "Indexing")
self.assertEqual(available_stage["label"], "Adding to Jellyfin")
self.assertEqual(available_stage["label"], "Adding to Grizzlyflix")
self.assertEqual(
available_stage["summary"],
"The download is complete. Jellyfin is indexing this title now.",
"The download is complete. Grizzlyflix is indexing this title now.",
)
@@ -1397,25 +1128,6 @@ class ArrAddPayloadTests(unittest.IsolatedAsyncioTestCase):
class CollectorManualDownloadTests(unittest.IsolatedAsyncioTestCase):
def setUp(self):
from backend.app.config import settings
secret = patch.object(settings, 'jwt_secret', 'manual-release-tests-secret-1234567890123456')
secret.start()
self.addCleanup(secret.stop)
access = patch.object(
requests_router,
"_ensure_request_mutation_access",
new=AsyncMock(return_value=None),
)
access.start()
self.addCleanup(access.stop)
def selection(self, payload, request_id, source):
payload['selectionToken'] = requests_router.manual_releases.issue_selection(
{**payload, 'requiresOverride': False, 'rejections': []}, request_id,
{'username': 'viewer'}, source, None)
return payload
@staticmethod
def _runtime() -> SimpleNamespace:
return SimpleNamespace(
@@ -1443,7 +1155,7 @@ class CollectorManualDownloadTests(unittest.IsolatedAsyncioTestCase):
{"id": 202, "seasonNumber": 2, "monitored": True, "hasFile": True},
]
),
search_episode_releases=AsyncMock(
search_releases=AsyncMock(
side_effect=[
[
{
@@ -1482,17 +1194,15 @@ class CollectorManualDownloadTests(unittest.IsolatedAsyncioTestCase):
"3909", user={"username": "viewer", "role": "user"}
)
sonarr.search_episode_releases.assert_any_await(101)
sonarr.search_episode_releases.assert_any_await(201)
sonarr.search_releases.assert_any_await(42, 1)
sonarr.search_releases.assert_any_await(42, 2)
self.assertEqual(result["collector"], "Sonarr")
self.assertEqual(len(result["releases"]), 2)
self.assertEqual(len(result["releases"]), 1)
self.assertTrue(result["releases"][0]["fullSeason"])
self.assertEqual(result["releases"][0]["seasonNumber"], 1)
self.assertEqual(result["releases"][0]["quality"], "WEBDL-1080p")
self.assertTrue(result["releases"][0]["bestPick"])
self.assertFalse(result["qualityFiltered"])
self.assertNotIn("selectionToken", result["releases"][1])
self.assertTrue(result["releases"][1]["requiresOverride"])
self.assertTrue(result["qualityFiltered"])
async def test_movie_manual_search_uses_radarr(self) -> None:
snapshot = Snapshot(
@@ -1582,14 +1292,14 @@ class CollectorManualDownloadTests(unittest.IsolatedAsyncioTestCase):
requests_router, "save_action"
):
result = await requests_router.action_grab(
"3909", self.selection(payload, "3909", self._runtime().sonarr_base_url), user={"username": "viewer", "role": "user"}
"3909", payload, user={"username": "viewer", "role": "user"}
)
sonarr.grab_release.assert_awaited_once_with("season-one", 7)
sonarr.push_release.assert_not_awaited()
self.assertEqual(result["response"], {"collector": "Sonarr", "queued": True})
async def test_stale_movie_release_requires_fresh_search(self) -> None:
async def test_stale_movie_release_still_routes_through_radarr_push(self) -> None:
snapshot = Snapshot(
request_id="4000",
title="Example Movie",
@@ -1625,12 +1335,15 @@ class CollectorManualDownloadTests(unittest.IsolatedAsyncioTestCase):
), patch.object(requests_router, "RadarrClient", return_value=radarr), patch.object(
requests_router, "save_action"
):
with self.assertRaises(HTTPException) as error:
await requests_router.action_grab(
"4000", self.selection(payload, "4000", self._runtime().radarr_base_url), user={"username": "viewer", "role": "user"})
self.assertEqual(error.exception.status_code, 409)
radarr.push_release.assert_not_awaited()
result = await requests_router.action_grab(
"4000", payload, user={"username": "viewer", "role": "user"}
)
radarr.push_release.assert_awaited_once()
pushed = radarr.push_release.await_args.args[0]
self.assertEqual(pushed["downloadUrl"], "http://prowlarr.test/download/1")
self.assertEqual(pushed["protocol"], "torrent")
self.assertEqual(result["response"], {"collector": "Radarr", "queued": True})
class DatabaseEmailTests(TempDatabaseMixin, unittest.TestCase):
@@ -1829,16 +1542,6 @@ class AuthFlowTests(TempDatabaseMixin, unittest.IsolatedAsyncioTestCase):
class MediaReplacementTests(TempDatabaseMixin, unittest.IsolatedAsyncioTestCase):
def setUp(self) -> None:
super().setUp()
access = patch.object(
requests_router,
"_ensure_request_mutation_access",
new=AsyncMock(return_value=None),
)
access.start()
self.addCleanup(access.stop)
def test_failed_repair_marks_linked_issue_as_blocked(self) -> None:
issue = {"id": 12, "status": "in_progress"}
with (
@@ -2132,103 +1835,6 @@ class MediaReplacementTests(TempDatabaseMixin, unittest.IsolatedAsyncioTestCase)
sonarr.search_episodes.assert_awaited_once_with([36899])
sonarr.search.assert_not_awaited()
async def test_add_seasons_monitors_series_and_searches_released_missing_episodes(self) -> None:
snapshot = Snapshot(
request_id="3580",
title="Suits",
request_type=RequestType.tv,
state=NormalizedState.available,
raw={"arr": {"item": {"id": 540}}},
)
refreshed = Snapshot(
request_id="3580",
title="Suits",
request_type=RequestType.tv,
state=NormalizedState.importing,
presentation={"pipeline": [{"id": "library", "unmonitoredSeasons": []}]},
)
original_series = {
"id": 540,
"monitored": True,
"qualityProfileId": 7,
"seasons": [
{"seasonNumber": 7, "monitored": True},
{"seasonNumber": 8, "monitored": False},
{"seasonNumber": 9, "monitored": False},
],
}
updated_series = {
**original_series,
"seasons": [
{"seasonNumber": 7, "monitored": True},
{"seasonNumber": 8, "monitored": True},
{"seasonNumber": 9, "monitored": True},
],
}
episodes = [
{
"id": 801,
"seasonNumber": 8,
"episodeNumber": 1,
"monitored": False,
"hasFile": False,
"airDateUtc": "2018-07-18T00:00:00Z",
},
{
"id": 802,
"seasonNumber": 8,
"episodeNumber": 2,
"monitored": False,
"hasFile": True,
"episodeFileId": 88,
},
{
"id": 901,
"seasonNumber": 9,
"episodeNumber": 1,
"monitored": False,
"hasFile": False,
"airDateUtc": "2019-07-17T00:00:00Z",
},
]
verified_episodes = [{**episode, "monitored": True} for episode in episodes]
sonarr = SimpleNamespace(
configured=lambda: True,
get_series=AsyncMock(side_effect=[original_series, updated_series]),
update_series=AsyncMock(return_value=updated_series),
get_episodes=AsyncMock(side_effect=[episodes, verified_episodes]),
monitor_episodes=AsyncMock(return_value={"monitored": True}),
search_episodes=AsyncMock(return_value={"id": 9001}),
)
runtime = SimpleNamespace(
jellyseerr_base_url=None,
jellyseerr_api_key=None,
sonarr_base_url="http://sonarr",
sonarr_api_key="secret",
)
with (
patch.object(requests_router, "get_runtime_settings", return_value=runtime),
patch.object(
requests_router,
"build_snapshot",
new=AsyncMock(side_effect=[snapshot, refreshed]),
),
patch.object(requests_router, "SonarrClient", return_value=sonarr),
patch.object(requests_router, "save_action"),
):
result = await requests_router.action_add_seasons(
"3580",
{"season_numbers": [8, 9]},
{"username": "viewer", "role": "user", "auto_search_enabled": True},
)
self.assertEqual(result["season_numbers"], [8, 9])
self.assertEqual(result["searched_episode_count"], 2)
self.assertTrue(result["snapshot"].presentation["pipeline"][0]["canAddSeasons"])
sonarr.update_series.assert_awaited_once_with(updated_series)
sonarr.monitor_episodes.assert_awaited_once_with([801, 802, 901], True)
sonarr.search_episodes.assert_awaited_once_with([801, 901])
async def test_missing_movie_search_monitors_movie_before_search(self) -> None:
snapshot = Snapshot(
request_id="3914",
@@ -2283,11 +1889,6 @@ class MediaReplacementTests(TempDatabaseMixin, unittest.IsolatedAsyncioTestCase)
patch.object(requests_router, "get_runtime_settings", return_value=runtime),
patch.object(requests_router, "build_snapshot", new=AsyncMock(return_value=snapshot)),
patch.object(requests_router, "BazarrClient", return_value=bazarr),
patch.object(
requests_router,
"_ensure_request_mutation_access",
new=AsyncMock(return_value=None),
),
patch.object(requests_router, "save_action"),
patch.object(requests_router, "get_portal_item", return_value={
"id": 12,
@@ -2400,28 +2001,28 @@ class InviteOperationalStateTests(TempDatabaseMixin, unittest.IsolatedAsyncioTes
async def test_invite_list_reports_automatic_operational_states(self) -> None:
ready = db.create_signup_invite(code="READY", recipient_email="ready@example.com")
disabled = db.create_signup_invite(code="DISABLED", enabled=False, recipient_email="off@example.com")
db.create_signup_invite(code="DISABLED", enabled=False, recipient_email="off@example.com")
used = db.create_signup_invite(code="USED", max_uses=1, recipient_email="used@example.com")
db.increment_signup_invite_use(int(used["id"]))
expired = db.create_signup_invite(
db.create_signup_invite(
code="EXPIRED",
expires_at="2000-01-01T00:00:00+00:00",
recipient_email="expired@example.com",
)
no_profile = db.create_signup_invite(
db.create_signup_invite(
code="NO-PROFILE",
profile_id=999,
recipient_email="profile@example.com",
)
payload = await admin_router.get_invites()
states = {invite["id"]: invite["operational_state"] for invite in payload["invites"]}
states = {invite["code"]: invite["operational_state"] for invite in payload["invites"]}
self.assertEqual(states[ready["id"]], "ready")
self.assertEqual(states[disabled["id"]], "disabled")
self.assertEqual(states[used["id"]], "exhausted")
self.assertEqual(states[expired["id"]], "expired")
self.assertEqual(states[no_profile["id"]], "profile_unavailable")
self.assertEqual(states[ready["code"]], "ready")
self.assertEqual(states["DISABLED"], "disabled")
self.assertEqual(states["USED"], "exhausted")
self.assertEqual(states["EXPIRED"], "expired")
self.assertEqual(states["NO-PROFILE"], "profile_unavailable")
self.assertEqual(payload["summary"]["total"], 5)
self.assertEqual(payload["summary"]["ready"], 1)
self.assertEqual(payload["summary"]["attention"], 4)
-336
View File
@@ -1,336 +0,0 @@
from contextlib import closing
import io
import json
from pathlib import Path
import sqlite3
import tempfile
import unittest
from unittest.mock import patch
import zipfile
from cryptography.fernet import Fernet
from fastapi import FastAPI
from fastapi.testclient import TestClient
from backend.app import db
from backend.app.auth import get_current_user
from backend.app.config import settings
from backend.app.routers import backups as backup_router
from backend.app.services import backups
PASSPHRASE = "test backup passphrase with spaces"
class BackupTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.root = Path(self.temp.name)
self.database = self.root / "magent.db"
for key, value in {
"sqlite_path": str(self.database), "sqlite_journal_mode": "DELETE",
"settings_encryption_key": Fernet.generate_key().decode(),
"jwt_secret": "source-installation-signing-secret-for-backup-tests",
"admin_username": "backup-admin", "admin_password": "a secure initial password",
"jellyfin_api_key": "environment-integration-secret", "setup_token": "local-setup-token",
"discord_webhook_url": "https://discord.example.invalid/api/webhooks/legacy-private-token",
}.items():
context = patch.object(settings, key, value)
context.start()
self.addCleanup(context.stop)
context = patch.object(backups, "_assets_root", return_value=self.root / "assets")
context.start()
self.addCleanup(context.stop)
db.init_db()
db.set_setting("sonarr_api_key", "database-integration-secret")
db.set_setting("site_login_message", "Restored configuration")
db.set_setting("installation_setup", "complete")
with closing(sqlite3.connect(self.database)) as conn, conn:
conn.execute("INSERT INTO requests_cache(request_id,title,payload_json) VALUES (3580,'Suits','{}')")
conn.execute(
"INSERT INTO signup_invites(code,enabled,created_at,updated_at) VALUES ('sha256:existing-invite',1,'now','now')"
)
self.assets = self.root / "assets"
(self.assets / "branding").mkdir(parents=True)
(self.assets / "branding" / "logo.png").write_bytes(b"branding fixture")
(self.assets / "artwork" / "tmdb" / "w342").mkdir(parents=True)
(self.assets / "artwork" / "tmdb" / "w342" / "poster.jpg").write_bytes(b"cached fixture")
def export(self, include_cache=True):
content, filename = backups.create_backup(PASSPHRASE, include_cache)
self.assertTrue(filename.endswith(".magent-backup"))
return content
def rewrite_archive(self, content, change):
decrypted = backups._decrypt(content, PASSPHRASE)
with zipfile.ZipFile(io.BytesIO(decrypted)) as archive:
files = {entry.filename: archive.read(entry) for entry in archive.infolist()}
change(files)
output = io.BytesIO()
with zipfile.ZipFile(output, "w") as archive:
for name, value in files.items():
archive.writestr(name, value)
return backups._encrypt(output.getvalue(), PASSPHRASE)
def test_round_trip_reencrypts_secrets_preserves_invites_and_restores_cache_on_restart(self):
content = self.export()
self.assertNotIn(b"database-integration-secret", content)
self.assertNotIn(b"environment-integration-secret", content)
original_auth_version = db.get_user_by_username("backup-admin")["auth_version"]
db.set_setting("site_login_message", "Live data before restart")
settings.settings_encryption_key = Fernet.generate_key().decode()
settings.jwt_secret = "destination-installation-signing-secret-for-backup-tests"
# Simulate a different host with different env-backed integration settings.
settings.jellyfin_api_key = "destination-env-value"
metadata = backups.stage_restore(io.BytesIO(content), PASSPHRASE)
self.assertTrue(metadata["include_cache"])
self.assertEqual(db.get_setting("site_login_message"), "Live data before restart")
self.assertIsNotNone(backups.backup_status()["pending_restore"])
staged_bytes = (self.database.parent / "backups" / "pending" / "database.sqlite3").read_bytes()
self.assertNotIn(b"database-integration-secret", staged_bytes)
self.assertNotIn(b"environment-integration-secret", staged_bytes)
self.assertNotIn(b"legacy-private-token", staged_bytes)
(self.assets / "branding" / "logo.png").write_bytes(b"changed logo")
(self.assets / "artwork" / "tmdb" / "w342" / "poster.jpg").unlink()
self.assertTrue(backups.apply_pending_restore())
self.assertEqual(db.get_setting("site_login_message"), "Restored configuration")
self.assertEqual(db.get_setting("sonarr_api_key"), "database-integration-secret")
self.assertEqual(db.get_setting("jellyfin_api_key"), "environment-integration-secret")
self.assertEqual(db.get_setting("discord_webhook_url"), "https://discord.example.invalid/api/webhooks/legacy-private-token")
self.assertEqual(db.get_setting("installation_setup"), "complete")
self.assertIsNone(db.get_setting("setup_token"))
self.assertEqual((self.assets / "branding" / "logo.png").read_bytes(), b"branding fixture")
self.assertEqual((self.assets / "artwork" / "tmdb" / "w342" / "poster.jpg").read_bytes(), b"cached fixture")
self.assertGreater(db.get_user_by_username("backup-admin")["auth_version"], original_auth_version)
with closing(sqlite3.connect(self.database)) as conn, conn:
self.assertEqual(conn.execute("SELECT title FROM requests_cache WHERE request_id=3580").fetchone(), ("Suits",))
self.assertEqual(conn.execute("SELECT code FROM signup_invites").fetchone(), ("sha256:existing-invite",))
self.assertTrue(conn.execute("SELECT value FROM settings WHERE key='sonarr_api_key'").fetchone()[0].startswith("enc:v1:"))
status = backups.backup_status()
self.assertIsNone(status["pending_restore"])
self.assertEqual(status["last_restore"]["status"], "restored")
self.assertTrue((self.database.parent / "backups" / status["last_restore"]["rollback_directory"] / "database.sqlite3").is_file())
self.assertFalse(backups.apply_pending_restore())
def test_wal_snapshot_contains_committed_uncheckpointed_rows(self):
with closing(sqlite3.connect(self.database)) as writer:
writer.execute("PRAGMA journal_mode=WAL")
writer.execute("PRAGMA wal_autocheckpoint=0")
writer.execute("UPDATE requests_cache SET title='Written in WAL' WHERE request_id=3580")
writer.commit()
self.assertTrue(Path(str(self.database) + "-wal").exists())
content = self.export()
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
self.assertTrue(backups.apply_pending_restore())
with closing(sqlite3.connect(self.database)) as restored:
self.assertEqual(restored.execute("SELECT title FROM requests_cache").fetchone()[0], "Written in WAL")
def test_managed_restore_preserves_destination_application_origin(self):
db.set_setting("magent_application_url", "https://source.example.test")
content = self.export()
db.set_setting("magent_application_url", "https://destination.example.test")
with patch.dict("os.environ", {"MAGENT_RUNTIME_MANAGED": "1"}):
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
self.assertTrue(backups.apply_pending_restore())
self.assertEqual(db.get_setting("magent_application_url"), "https://destination.example.test")
def test_manual_restore_retains_legacy_application_url_behavior(self):
db.set_setting("magent_application_url", "https://source.example.test")
content = self.export()
db.set_setting("magent_application_url", "https://destination.example.test")
with patch.dict("os.environ", {"MAGENT_RUNTIME_MANAGED": ""}):
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
self.assertTrue(backups.apply_pending_restore())
self.assertEqual(db.get_setting("magent_application_url"), "https://source.example.test")
def test_managed_restore_without_destination_origin_does_not_stage(self):
content = self.export()
with patch.dict("os.environ", {"MAGENT_RUNTIME_MANAGED": "1"}), \
patch("backend.app.services.public_urls.magent_public_url", return_value=""):
with self.assertRaisesRegex(backups.BackupError, "destination application address"):
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
self.assertIsNone(backups.backup_status()["pending_restore"])
def test_process_interruption_is_recovered_on_next_startup(self):
class ProcessStopped(BaseException):
pass
content = self.export()
db.set_setting("site_login_message", "Value before interrupted restart")
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
with patch.object(backups, "_replace_assets", side_effect=ProcessStopped):
with self.assertRaises(ProcessStopped):
backups.apply_pending_restore()
self.assertTrue((self.database.parent / "backups" / "restore-journal.json").exists())
self.assertEqual(db.get_setting("site_login_message"), "Restored configuration")
self.assertFalse(backups.apply_pending_restore())
self.assertEqual(db.get_setting("site_login_message"), "Value before interrupted restart")
self.assertEqual(backups.backup_status()["last_restore"]["status"], "rolled_back")
self.assertIsNone(backups.backup_status()["pending_restore"])
def test_crash_after_rollback_does_not_reapply_pending_restore(self):
class ProcessStopped(BaseException):
pass
content = self.export()
db.set_setting("site_login_message", "Value to retain")
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
replace_assets = backups._replace_assets
remove_tree = backups.shutil.rmtree
calls = 0
def fail_first_copy(source, target):
nonlocal calls
calls += 1
if calls == 1:
raise OSError("failed apply")
return replace_assets(source, target)
def interrupt_cleanup(path, *args, **kwargs):
if Path(path).name == "pending":
raise ProcessStopped()
return remove_tree(path, *args, **kwargs)
with patch.object(backups, "_replace_assets", side_effect=fail_first_copy), \
patch.object(backups.shutil, "rmtree", side_effect=interrupt_cleanup):
with self.assertRaises(ProcessStopped):
backups.apply_pending_restore()
journal = json.loads((self.root / "backups" / "restore-journal.json").read_text())
self.assertEqual(journal["phase"], "rolled_back")
self.assertFalse(backups.apply_pending_restore())
self.assertEqual(db.get_setting("site_login_message"), "Value to retain")
self.assertIsNone(backups.backup_status()["pending_restore"])
def test_missing_runtime_column_is_rejected_even_with_current_migration_version(self):
directory = self.root / "schema-test"
directory.mkdir()
backups._extract_archive(backups._decrypt(self.export(), PASSPHRASE), directory)
source = directory / "database.sqlite3"
with closing(sqlite3.connect(source)) as conn, conn:
conn.execute("ALTER TABLE users DROP COLUMN auto_search_enabled")
with self.assertRaisesRegex(backups.BackupError, "missing database columns"):
backups._validate_database(source)
def test_changed_encryption_key_since_staging_leaves_live_database_untouched(self):
content = self.export()
db.set_setting("site_login_message", "Current data")
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
settings.settings_encryption_key = Fernet.generate_key().decode()
with self.assertRaisesRegex(backups.BackupError, "configuration is invalid"):
backups.apply_pending_restore()
self.assertEqual(db.get_setting("site_login_message"), "Current data")
self.assertIsNotNone(backups.backup_status()["pending_restore"])
def test_excluding_disk_cache_keeps_database_cache_and_branding(self):
with zipfile.ZipFile(io.BytesIO(backups._decrypt(self.export(False), PASSPHRASE))) as archive:
self.assertIn("database.sqlite3", archive.namelist())
self.assertIn("files/branding/logo.png", archive.namelist())
self.assertFalse(any("artwork" in name for name in archive.namelist()))
def test_wrong_password_and_tampering_never_stage_or_touch_live_database(self):
content = self.export()
for bad_content, password in ((content, "incorrect password value"), (content[:-1] + bytes([content[-1] ^ 1]), PASSPHRASE)):
with self.subTest(password=password):
with self.assertRaisesRegex(backups.BackupError, "Incorrect passphrase or damaged"):
backups.stage_restore(io.BytesIO(bad_content), password)
self.assertIsNone(backups.backup_status()["pending_restore"])
self.assertEqual(db.get_setting("sonarr_api_key"), "database-integration-secret")
def test_path_traversal_unknown_files_and_checksum_failures_rejected(self):
content = self.export()
for name in ("../outside.txt", "/absolute.txt", "files/branding/../../../escape", "files/branding/script.py"):
with self.subTest(name=name):
malformed = self.rewrite_archive(content, lambda files: files.update({name: b"bad"}))
with self.assertRaises(backups.BackupError):
backups.stage_restore(io.BytesIO(malformed), PASSPHRASE)
malformed = self.rewrite_archive(content, lambda files: files.update({"files/branding/logo.png": b"tampered"}))
with self.assertRaises(backups.BackupError):
backups.stage_restore(io.BytesIO(malformed), PASSPHRASE)
self.assertFalse((self.root / "outside.txt").exists())
def test_size_limit_and_unsupported_schema_rejected(self):
content = self.export()
with patch.object(backups, "MAX_UPLOAD_BYTES", 16):
with self.assertRaisesRegex(backups.BackupError, "upload limit"):
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
with patch.object(backups, "MAX_EXPANDED_BYTES", 16):
with self.assertRaisesRegex(backups.BackupError, "Expanded backup"):
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
with closing(sqlite3.connect(self.database)) as conn, conn:
conn.execute("CREATE TRIGGER unsafe AFTER INSERT ON settings BEGIN DELETE FROM users; END")
# Validate the original fixture to avoid executing the malicious trigger in export.
with self.assertRaisesRegex(backups.BackupError, "unsupported database schema"):
backups._validate_database(self.database)
def test_unsupported_compression_is_rejected_before_expansion(self):
content = self.export()
rewritten = io.BytesIO()
with zipfile.ZipFile(io.BytesIO(backups._decrypt(content, PASSPHRASE))) as original:
with zipfile.ZipFile(rewritten, "w", compression=zipfile.ZIP_BZIP2) as target:
for entry in original.infolist():
target.writestr(entry.filename, original.read(entry))
with self.assertRaisesRegex(backups.BackupError, "unsafe archive entry"):
backups.stage_restore(io.BytesIO(backups._encrypt(rewritten.getvalue(), PASSPHRASE)), PASSPHRASE)
def test_cancel_is_idempotent_and_does_not_change_database(self):
content = self.export()
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
with self.assertRaisesRegex(backups.BackupError, "already staged"):
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
backups.cancel_restore()
backups.cancel_restore()
self.assertIsNone(backups.backup_status()["pending_restore"])
self.assertEqual(db.get_setting("sonarr_api_key"), "database-integration-secret")
def test_failure_after_database_replacement_rolls_back_both_database_and_files(self):
content = self.export()
db.set_setting("site_login_message", "Keep this current value")
(self.assets / "branding" / "logo.png").write_bytes(b"current logo")
backups.stage_restore(io.BytesIO(content), PASSPHRASE)
original = backups._replace_assets
calls = 0
def fail_once(source, target):
nonlocal calls
calls += 1
if calls == 1:
raise OSError("simulated interrupted copy")
return original(source, target)
with patch.object(backups, "_replace_assets", side_effect=fail_once):
with self.assertRaisesRegex(OSError, "interrupted copy"):
backups.apply_pending_restore()
self.assertEqual(db.get_setting("site_login_message"), "Keep this current value")
self.assertEqual((self.assets / "branding" / "logo.png").read_bytes(), b"current logo")
self.assertEqual(backups.backup_status()["last_restore"]["status"], "rolled_back")
self.assertFalse(backups.apply_pending_restore())
def test_api_requires_admin_and_restore_confirmation(self):
app = FastAPI()
app.include_router(backup_router.router)
with TestClient(app) as client:
self.assertEqual(client.get("/admin/backups").status_code, 401)
app.dependency_overrides[get_current_user] = lambda: {"username": "member", "role": "user"}
self.assertEqual(client.get("/admin/backups").status_code, 403)
self.assertEqual(client.post("/admin/backups/export", json={"passphrase": PASSPHRASE}).status_code, 403)
app.dependency_overrides[get_current_user] = lambda: {"username": "backup-admin", "role": "admin"}
status = client.get("/admin/backups")
self.assertEqual(status.status_code, 200)
self.assertEqual(status.headers["cache-control"], "no-store")
self.assertEqual(status.json()["max_expanded_bytes"], backups.MAX_EXPANDED_BYTES)
response = client.post("/admin/backups/export", json={"passphrase": PASSPHRASE})
self.assertEqual(response.status_code, 200)
self.assertEqual(response.headers["cache-control"], "no-store")
rejected = client.post("/admin/backups/restore", files={"file": ("test.magent-backup", response.content)},
data={"passphrase": PASSPHRASE, "confirmation": "wrong"})
self.assertEqual(rejected.status_code, 422)
restored = client.post("/admin/backups/restore", files={"file": ("test.magent-backup", response.content)},
data={"passphrase": PASSPHRASE, "confirmation": "RESTORE"})
self.assertEqual(restored.status_code, 202)
self.assertTrue(restored.json()["restart_required"])
self.assertEqual(client.delete("/admin/backups/restore").status_code, 200)
if __name__ == "__main__":
unittest.main()
-519
View File
@@ -1,519 +0,0 @@
"""Managed installation regression tests; use only disposable local files."""
import base64
from concurrent.futures import ThreadPoolExecutor
from contextlib import closing, redirect_stderr, redirect_stdout
import io
import json
import os
from pathlib import Path
import sqlite3
import stat
import tempfile
from threading import Barrier
import unittest
from unittest.mock import patch
from backend.app import container_bootstrap as bootstrap
class ContainerBootstrapTests(unittest.TestCase):
def setUp(self):
temporary = tempfile.TemporaryDirectory()
self.addCleanup(temporary.cleanup)
self.root = Path(temporary.name)
self.data = self.root / "data"
self.data.mkdir(mode=0o700)
self.state_path = self.data / bootstrap.STATE_FILENAME
self.database = self.data / "magent.db"
self.environment = {
"MAGENT_MANAGED_SECRETS": "true",
"MAGENT_APPLICATION_URL": "https://magent.example.test",
}
def prepare(self, **changes):
return bootstrap.prepare_environment({**self.environment, **changes}, self.data)
def state(self):
return json.loads(self.state_path.read_text(encoding="utf-8"))
def create_database(self, *, completed=0, admin=False):
with closing(sqlite3.connect(self.database)) as connection:
with connection:
connection.execute("CREATE TABLE installation_setup (id INTEGER PRIMARY KEY, completed INTEGER)")
connection.execute("INSERT INTO installation_setup VALUES (1, ?)", (completed,))
connection.execute("CREATE TABLE users (role TEXT)")
if admin:
connection.execute("INSERT INTO users VALUES ('ADMIN')")
def create_symlink(self, path, target, *, directory=False):
try:
path.symlink_to(target, target_is_directory=directory)
except (OSError, NotImplementedError) as exc:
self.skipTest(f"This platform cannot create test symlinks: {type(exc).__name__}")
def test_fresh_install_generates_independent_valid_random_secrets(self):
before = dict(self.environment)
prepared = self.prepare()
state = self.state()
self.assertEqual(self.environment, before)
self.assertEqual(set(state), {"version", *bootstrap.SECRET_NAMES})
self.assertEqual(state["version"], 1)
for name in ("JWT_SECRET", "SETUP_TOKEN"):
self.assertRegex(state[name], r"^[A-Za-z0-9_-]{64}$")
self.assertNotEqual(state["JWT_SECRET"], state["SETUP_TOKEN"])
self.assertEqual(len(base64.urlsafe_b64decode(state["SETTINGS_ENCRYPTION_KEY"])), 32)
for name in bootstrap.SECRET_NAMES:
self.assertEqual(prepared[name], state[name])
self.assertEqual(prepared["SQLITE_PATH"], str(self.database.absolute()))
self.assertFalse(self.database.exists())
self.assertEqual(list(self.data.glob(".magent-secrets-*")), [])
@unittest.skipUnless(os.name == "posix", "POSIX filesystem ownership/permissions")
def test_state_has_private_permissions_and_runtime_ownership(self):
self.prepare()
metadata = self.state_path.stat()
self.assertEqual(stat.S_IMODE(metadata.st_mode), 0o600)
self.assertEqual(metadata.st_uid, os.geteuid())
def test_separate_installations_get_different_secrets(self):
first = self.prepare()
other = self.root / "other"
other.mkdir(mode=0o700)
second = bootstrap.prepare_environment(self.environment, other)
for name in bootstrap.SECRET_NAMES:
self.assertNotEqual(first[name], second[name])
def test_restart_and_existing_database_reuse_exact_file_and_values(self):
first = self.prepare()
original = self.state_path.read_bytes()
original_modified = self.state_path.stat().st_mtime_ns
self.create_database(admin=True)
with patch.object(bootstrap.secrets, "token_bytes", side_effect=AssertionError("Must not regenerate")), \
patch.object(bootstrap.secrets, "token_urlsafe", side_effect=AssertionError("Must not regenerate")):
second = self.prepare()
self.assertEqual(first, second)
self.assertEqual(self.state_path.read_bytes(), original)
self.assertEqual(self.state_path.stat().st_mtime_ns, original_modified)
def test_disabled_mode_is_an_unchanged_copy_without_filesystem_access(self):
for value in (None, "false", "0", "no", "", " FALSE "):
with self.subTest(mode=value):
environment = {"JWT_SECRET": "legacy-key", "MAGENT_APPLICATION_URL": "invalid"}
if value is not None:
environment["MAGENT_MANAGED_SECRETS"] = value
result = bootstrap.prepare_environment(environment, self.root / "does-not-exist")
self.assertEqual(result, environment)
self.assertIsNot(result, environment)
self.assertFalse(self.state_path.exists())
def test_invalid_managed_mode_fails_before_writing(self):
with self.assertRaises(bootstrap.BootstrapError):
self.prepare(MAGENT_MANAGED_SECRETS="perhaps")
self.assertFalse(self.state_path.exists())
def test_auto_mode_generates_fresh_install_keys_without_explicit_jwt(self):
prepared = bootstrap.prepare_environment({"MAGENT_MANAGED_SECRETS": "auto"}, self.data)
self.assertTrue(self.state_path.exists())
self.assertEqual(prepared["MAGENT_MANAGED_SECRETS"], "true")
self.assertEqual(prepared["MAGENT_RUNTIME_MANAGED"], "1")
for name in bootstrap.SECRET_NAMES:
self.assertEqual(prepared[name], self.state()[name])
def test_auto_mode_preserves_explicit_jwt_manual_install_without_filesystem_access(self):
environment = {
"MAGENT_MANAGED_SECRETS": "auto",
"JWT_SECRET": "legacy-explicit-signing-key",
"SQLITE_PATH": "/existing/custom-database.db",
"API_DOCS_ENABLED": "true",
"MAGENT_APPLICATION_URL": "https://legacy.example.test",
"CORS_ALLOW_ORIGIN": "https://legacy.example.test",
}
prepared = bootstrap.prepare_environment(environment, self.root / "does-not-exist")
self.assertEqual(prepared, environment)
self.assertIsNot(prepared, environment)
self.assertNotIn("SETTINGS_ENCRYPTION_KEY", prepared)
self.assertNotIn("MAGENT_RUNTIME_MANAGED", prepared)
self.assertFalse(self.state_path.exists())
def test_auto_mode_whitespace_jwt_is_treated_as_unset(self):
prepared = bootstrap.prepare_environment({"MAGENT_MANAGED_SECRETS": "auto", "JWT_SECRET": " "}, self.data)
self.assertEqual(prepared["JWT_SECRET"], self.state()["JWT_SECRET"])
def test_absent_application_url_uses_fixed_defaults_without_claiming_an_origin(self):
prepared = bootstrap.prepare_environment({"MAGENT_MANAGED_SECRETS": "auto"}, self.data)
self.assertFalse(prepared.get("MAGENT_APPLICATION_URL"))
self.assertEqual(prepared["CORS_ALLOW_ORIGIN"], "http://localhost:3000")
self.assertEqual(prepared["AUTH_COOKIE_SECURE"], "false")
self.assertEqual(prepared["API_DOCS_ENABLED"], "false")
self.assertEqual(prepared["SQLITE_PATH"], str(self.database.absolute()))
def test_empty_application_url_is_deferred_to_setup(self):
prepared = self.prepare(MAGENT_APPLICATION_URL="")
self.assertEqual(prepared["MAGENT_APPLICATION_URL"], "")
self.assertEqual(prepared["CORS_ALLOW_ORIGIN"], "http://localhost:3000")
self.assertTrue(self.state_path.exists())
def test_managed_api_docs_cannot_be_enabled(self):
for value in ("true", "1", "yes", "on", "invalid"):
with self.subTest(value=value), self.assertRaisesRegex(bootstrap.BootstrapError, "API_DOCS_ENABLED"):
self.prepare(API_DOCS_ENABLED=value)
self.assertFalse(self.state_path.exists())
def test_saved_public_url_controls_restart_without_key_regeneration(self):
original = bootstrap.prepare_environment({"MAGENT_MANAGED_SECRETS": "auto"}, self.data)
state_bytes = self.state_path.read_bytes()
self.create_database(admin=True)
with closing(sqlite3.connect(self.database)) as connection:
with connection:
connection.execute("CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT)")
connection.execute("INSERT INTO settings VALUES ('magent_application_url', 'https://saved.example.test')")
restarted = bootstrap.prepare_environment({"MAGENT_MANAGED_SECRETS": "auto"}, self.data)
self.assertEqual(restarted["MAGENT_APPLICATION_URL"], "https://saved.example.test")
self.assertEqual(restarted["CORS_ALLOW_ORIGIN"], "https://saved.example.test")
self.assertEqual(restarted["AUTH_COOKIE_SECURE"], "true")
self.assertEqual(self.state_path.read_bytes(), state_bytes)
for name in bootstrap.SECRET_NAMES:
self.assertEqual(restarted[name], original[name])
def test_saved_public_url_wins_over_stale_deployment_url_on_restart(self):
self.prepare()
self.create_database(admin=True)
with closing(sqlite3.connect(self.database)) as connection:
with connection:
connection.execute("CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT)")
connection.execute("INSERT INTO settings VALUES ('magent_application_url', 'http://magent.lan:3000')")
restarted = self.prepare(CORS_ALLOW_ORIGIN="https://magent.example.test")
self.assertEqual(restarted["MAGENT_APPLICATION_URL"], "http://magent.lan:3000")
self.assertEqual(restarted["CORS_ALLOW_ORIGIN"], "http://magent.lan:3000")
self.assertEqual(restarted["AUTH_COOKIE_SECURE"], "false")
def test_invalid_saved_url_fails_closed_without_changing_keys(self):
self.prepare()
original = self.state_path.read_bytes()
self.create_database(admin=True)
with closing(sqlite3.connect(self.database)) as connection:
with connection:
connection.execute("CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT)")
connection.execute("INSERT INTO settings VALUES ('magent_application_url', 'https://user:secret@evil.test')")
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
self.assertEqual(self.state_path.read_bytes(), original)
def test_existing_database_or_recovery_sidecar_never_generates_replacement_keys(self):
for suffix in ("", "-wal", "-shm", "-journal"):
with self.subTest(suffix=suffix):
path = Path(str(self.database) + suffix)
path.write_bytes(b"existing installation data")
try:
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
self.assertEqual(path.read_bytes(), b"existing installation data")
self.assertFalse(self.state_path.exists())
finally:
path.unlink()
def test_lost_keys_after_initialization_are_not_recreated(self):
self.prepare()
self.create_database()
self.state_path.unlink()
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
self.assertFalse(self.state_path.exists())
def test_fresh_manual_secrets_conflict_without_writing_state(self):
for name in bootstrap.SECRET_NAMES:
with self.subTest(name=name), self.assertRaises(bootstrap.BootstrapError):
self.prepare(**{name: "synthetic-manual-secret"})
self.assertFalse(self.state_path.exists())
def test_matching_environment_values_are_accepted_but_conflicts_never_replace_file(self):
first = self.prepare()
original = self.state_path.read_bytes()
keys = {name: first[name] for name in bootstrap.SECRET_NAMES}
self.assertEqual(self.prepare(**keys), first)
for name in bootstrap.SECRET_NAMES:
with self.subTest(name=name), self.assertRaises(bootstrap.BootstrapError) as raised:
self.prepare(**{name: "conflicting-private-value"})
self.assertNotIn("conflicting-private-value", str(raised.exception))
self.assertEqual(self.state_path.read_bytes(), original)
def test_custom_database_location_is_rejected_without_touching_it(self):
custom = self.root / "other.db"
with self.assertRaises(bootstrap.BootstrapError):
self.prepare(SQLITE_PATH=str(custom))
self.assertFalse(custom.exists())
self.assertFalse(self.state_path.exists())
def test_missing_or_symlink_data_directory_is_rejected(self):
with self.assertRaises(bootstrap.BootstrapError):
bootstrap.prepare_environment(self.environment, self.root / "missing")
linked = self.root / "linked-data"
self.create_symlink(linked, self.data, directory=True)
with self.assertRaises(bootstrap.BootstrapError):
bootstrap.prepare_environment(self.environment, linked)
self.assertFalse(self.state_path.exists())
@unittest.skipUnless(os.name == "posix", "POSIX filesystem permissions")
def test_shared_writable_data_directory_is_rejected(self):
self.data.chmod(0o777)
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
self.assertFalse(self.state_path.exists())
def test_malformed_json_oversized_and_invalid_schema_never_get_replaced(self):
self.prepare()
valid = self.state()
invalid_states = [
b"not-json", b"\xff", b"x" * (bootstrap.MAX_STATE_BYTES + 1), b"[]", b"{}",
json.dumps({**valid, "version": True}).encode(),
json.dumps({**valid, "version": 2}).encode(),
json.dumps({**valid, "unexpected": "value"}).encode(),
json.dumps({**valid, "JWT_SECRET": None}).encode(),
json.dumps({**valid, "JWT_SECRET": "a" * 64}).encode(),
json.dumps({**valid, "JWT_SECRET": "short"}).encode(),
json.dumps({**valid, "SETUP_TOKEN": valid["JWT_SECRET"]}).encode(),
json.dumps({**valid, "SETTINGS_ENCRYPTION_KEY": "invalid-key"}).encode(),
json.dumps({**valid, "SETTINGS_ENCRYPTION_KEY": base64.urlsafe_b64encode(b"short").decode()}).encode(),
]
for index, payload in enumerate(invalid_states):
with self.subTest(case=index):
self.state_path.write_bytes(payload)
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
self.assertEqual(self.state_path.read_bytes(), payload)
def test_state_directory_is_not_replaced(self):
self.state_path.mkdir()
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
self.assertTrue(self.state_path.is_dir())
def test_state_symlink_is_not_followed_or_replaced(self):
self.prepare()
target = self.root / "original-secrets.json"
self.state_path.rename(target)
original = target.read_bytes()
self.create_symlink(self.state_path, target)
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
self.assertEqual(target.read_bytes(), original)
self.assertTrue(self.state_path.is_symlink())
@unittest.skipUnless(os.name == "posix", "POSIX filesystem permissions")
def test_publicly_readable_secrets_are_rejected_without_fixing_or_overwriting_them(self):
self.prepare()
original = self.state_path.read_bytes()
self.state_path.chmod(0o644)
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
self.assertEqual(stat.S_IMODE(self.state_path.stat().st_mode), 0o644)
self.assertEqual(self.state_path.read_bytes(), original)
@unittest.skipUnless(hasattr(os, "mkfifo"), "POSIX named pipes")
def test_named_pipe_state_is_rejected_without_blocking(self):
os.mkfifo(self.state_path, 0o600)
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
self.assertTrue(stat.S_ISFIFO(self.state_path.stat().st_mode))
def test_https_sets_matching_cors_and_secure_cookies(self):
prepared = self.prepare()
self.assertEqual(prepared["CORS_ALLOW_ORIGIN"], self.environment["MAGENT_APPLICATION_URL"])
self.assertEqual(prepared["AUTH_COOKIE_SECURE"], "true")
def test_explicit_http_lan_origin_disables_secure_cookie_flag_only(self):
prepared = self.prepare(MAGENT_APPLICATION_URL="http://192.0.2.10:3000")
self.assertEqual(prepared["CORS_ALLOW_ORIGIN"], "http://192.0.2.10:3000")
self.assertEqual(prepared["AUTH_COOKIE_SECURE"], "false")
def test_invalid_origin_fails_without_creating_keys(self):
origins = (
"not-a-url", "https://magent.example.test/", "https://magent.example.test/path",
"//magent.example.test", "ftp://magent.example.test", "http:/magent.example.test",
"https://user:password@magent.example.test", "https://@magent.example.test",
"https://magent.example.test?", "https://magent.example.test#",
"https://magent.example.test:0", "https://magent.example.test:65536",
"https://*.example.test", "https://magent.\ttest", "https://magent.example.test\\path",
" https://magent.example.test", "https://magent.example.test\x00",
)
for origin in origins:
with self.subTest(origin=repr(origin)), self.assertRaises(bootstrap.BootstrapError):
self.prepare(MAGENT_APPLICATION_URL=origin)
self.assertFalse(self.state_path.exists())
def test_cors_mismatch_or_cookie_scheme_conflict_fails_without_keys(self):
cases = (
{"CORS_ALLOW_ORIGIN": "https://elsewhere.example.test"},
{"AUTH_COOKIE_SECURE": "false"},
{"AUTH_COOKIE_SECURE": "0"},
{"AUTH_COOKIE_SECURE": "maybe"},
{"MAGENT_APPLICATION_URL": "http://magent.lan:3000", "AUTH_COOKIE_SECURE": "true"},
{"MAGENT_APPLICATION_URL": "http://magent.lan:3000", "AUTH_COOKIE_SECURE": "1"},
)
for changes in cases:
with self.subTest(changes=changes), self.assertRaises(bootstrap.BootstrapError):
self.prepare(**changes)
self.assertFalse(self.state_path.exists())
def test_racing_initializers_publish_and_return_one_complete_state(self):
barrier = Barrier(8)
def initialize(_):
barrier.wait(timeout=10)
return self.prepare()
with ThreadPoolExecutor(max_workers=8) as executor:
results = list(executor.map(initialize, range(8)))
for result in results:
self.assertEqual(result, results[0])
state = self.state()
for name in bootstrap.SECRET_NAMES:
self.assertEqual(state[name], results[0][name])
self.assertEqual(list(self.data.glob(".magent-secrets-*")), [])
def test_token_command_requires_managed_mode_and_does_not_create_state(self):
with self.assertRaises(bootstrap.BootstrapError):
bootstrap.setup_token({}, self.data)
self.assertFalse(self.state_path.exists())
with self.assertRaises((bootstrap.BootstrapError, FileNotFoundError)):
bootstrap.setup_token(self.environment, self.data)
self.assertFalse(self.state_path.exists())
self.assertFalse(self.database.exists())
def test_token_command_does_not_create_an_uninitialized_database(self):
self.prepare()
with self.assertRaises(bootstrap.BootstrapError):
bootstrap.setup_token(self.environment, self.data)
self.assertFalse(self.database.exists())
def test_token_command_returns_only_initial_token_using_readonly_closed_connection(self):
prepared = self.prepare()
self.create_database()
before = {path.name: path.read_bytes() for path in self.data.iterdir()}
connections = []
real_connect = sqlite3.connect
def connect(*args, **kwargs):
self.assertTrue(kwargs.get("uri"))
self.assertTrue(args[0].endswith("?mode=ro"))
connection = real_connect(*args, **kwargs)
with self.assertRaises(sqlite3.OperationalError):
connection.execute("INSERT INTO users VALUES ('admin')")
connections.append(connection)
return connection
with patch.object(bootstrap.sqlite3, "connect", side_effect=connect):
token = bootstrap.setup_token(self.environment, self.data)
self.assertEqual(token, prepared["SETUP_TOKEN"])
self.assertEqual({path.name: path.read_bytes() for path in self.data.iterdir()}, before)
for connection in connections:
with self.assertRaises(sqlite3.ProgrammingError):
connection.execute("SELECT 1")
def test_token_command_refuses_once_any_admin_exists_even_before_setup_completion(self):
self.prepare()
self.create_database(admin=True)
with self.assertRaises(bootstrap.BootstrapError):
bootstrap.setup_token(self.environment, self.data)
def test_token_command_refuses_completed_setup_even_without_admin(self):
self.prepare()
self.create_database(completed=1)
with self.assertRaises(bootstrap.BootstrapError):
bootstrap.setup_token(self.environment, self.data)
def test_token_command_refuses_unknown_or_invalid_database_state(self):
self.prepare()
for payload in (b"", b"not a SQLite database"):
with self.subTest(payload=payload):
self.database.write_bytes(payload)
with self.assertRaises(bootstrap.BootstrapError):
bootstrap.setup_token(self.environment, self.data)
self.assertEqual(self.database.read_bytes(), payload)
self.database.unlink()
self.create_database()
with closing(sqlite3.connect(self.database)) as connection:
with connection:
connection.execute("DELETE FROM installation_setup")
with self.assertRaises(bootstrap.BootstrapError):
bootstrap.setup_token(self.environment, self.data)
def test_existing_database_symlink_is_rejected_even_with_valid_state(self):
self.prepare()
self.create_database()
target = self.root / "other.db"
self.database.rename(target)
original = target.read_bytes()
self.create_symlink(self.database, target)
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
with self.assertRaises(bootstrap.BootstrapError):
bootstrap.setup_token(self.environment, self.data)
self.assertEqual(target.read_bytes(), original)
def test_existing_database_directory_is_rejected_even_with_valid_state(self):
self.prepare()
self.database.mkdir()
with self.assertRaises(bootstrap.BootstrapError):
self.prepare()
with self.assertRaises(bootstrap.BootstrapError):
bootstrap.setup_token(self.environment, self.data)
self.assertTrue(self.database.is_dir())
def test_startup_passes_keys_to_runtime_without_printing_them(self):
prepared = self.prepare()
stdout, stderr = io.StringIO(), io.StringIO()
with patch.dict(os.environ, self.environment, clear=True), \
patch.object(bootstrap.sys, "argv", ["bootstrap", "supervisord", "-c", "config"]), \
patch.object(bootstrap, "prepare_environment", return_value=prepared), \
patch.object(bootstrap.os, "execvpe") as execute, \
redirect_stdout(stdout), redirect_stderr(stderr):
self.assertEqual(bootstrap.main(), 0)
execute.assert_called_once_with("supervisord", ["supervisord", "-c", "config"], prepared)
self.assertIn("setup-token", stdout.getvalue())
self.assertEqual(stderr.getvalue(), "")
for name in bootstrap.SECRET_NAMES:
self.assertNotIn(prepared[name], stdout.getvalue() + stderr.getvalue())
def test_disabled_startup_does_not_print_managed_install_instructions(self):
stdout, stderr = io.StringIO(), io.StringIO()
environment = {"JWT_SECRET": "manual-test-value"}
with patch.dict(os.environ, environment, clear=True), \
patch.object(bootstrap.sys, "argv", ["bootstrap", "supervisord"]), \
patch.object(bootstrap.os, "execvpe") as execute, \
redirect_stdout(stdout), redirect_stderr(stderr):
self.assertEqual(bootstrap.main(), 0)
execute.assert_called_once_with("supervisord", ["supervisord"], environment)
self.assertEqual(stdout.getvalue() + stderr.getvalue(), "")
def test_cli_explicit_token_command_prints_only_token_not_other_keys(self):
prepared = self.prepare()
self.create_database()
retrieve = bootstrap.setup_token
stdout, stderr = io.StringIO(), io.StringIO()
with patch.dict(os.environ, self.environment, clear=True), \
patch.object(bootstrap.sys, "argv", ["bootstrap", "setup-token"]), \
patch.object(bootstrap, "setup_token", side_effect=lambda env: retrieve(env, self.data)), \
patch.object(bootstrap.os, "execvpe") as execute, \
redirect_stdout(stdout), redirect_stderr(stderr):
self.assertEqual(bootstrap.main(), 0)
execute.assert_not_called()
self.assertEqual(stdout.getvalue(), prepared["SETUP_TOKEN"] + "\n")
self.assertEqual(stderr.getvalue(), "")
self.assertNotIn(prepared["JWT_SECRET"], stdout.getvalue())
self.assertNotIn(prepared["SETTINGS_ENCRYPTION_KEY"], stdout.getvalue())
def test_cli_unexpected_io_failure_never_logs_sensitive_exception_details(self):
stdout, stderr = io.StringIO(), io.StringIO()
with patch.object(bootstrap.sys, "argv", ["bootstrap", "supervisord"]), \
patch.object(bootstrap, "prepare_environment", side_effect=OSError("private-secret-material")), \
redirect_stdout(stdout), redirect_stderr(stderr):
self.assertEqual(bootstrap.main(), 1)
self.assertEqual(stdout.getvalue(), "")
self.assertNotIn("private-secret-material", stderr.getvalue())
self.assertIn("Check volume permissions", stderr.getvalue())
if __name__ == "__main__":
unittest.main()
-122
View File
@@ -1,122 +0,0 @@
"""Unit checks for the release smoke harness; no Docker or network required."""
from email.message import Message
from email.parser import BytesParser
from email.policy import default
import importlib.util
from pathlib import Path
import unittest
from unittest.mock import patch
HELPER_PATH = Path(__file__).resolve().parents[2] / "scripts" / "container_smoke.py"
SPEC = importlib.util.spec_from_file_location("magent_container_smoke", HELPER_PATH)
smoke = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(smoke)
def response_headers(**changes):
headers = Message()
for key, value in {
"Content-Type": "text/html; charset=utf-8",
"Content-Security-Policy": "default-src 'self'; script-src 'self' 'nonce-test-nonce' 'strict-dynamic'",
"X-Content-Type-Options": "nosniff",
"X-Frame-Options": "DENY",
**changes,
}.items():
headers[key] = value
return headers
class ContainerPackagingHarnessTests(unittest.TestCase):
def test_backup_multipart_preserves_binary_content_and_required_fields(self):
content = b"MAGENT-BACKUP\x00\x01\xff\r\n\x00encrypted"
body, content_type = smoke.backup_restore_upload(content, "synthetic backup passphrase")
parsed = BytesParser(policy=default).parsebytes(
f"Content-Type: {content_type}\r\nMIME-Version: 1.0\r\n\r\n".encode() + body,
)
fields = {part.get_param("name", header="content-disposition"): part
for part in parsed.iter_parts()}
self.assertEqual(set(fields), {"passphrase", "confirmation", "file"})
self.assertEqual(fields["passphrase"].get_payload(decode=True), b"synthetic backup passphrase")
self.assertEqual(fields["confirmation"].get_payload(decode=True), b"RESTORE")
self.assertEqual(fields["file"].get_payload(decode=True), content)
self.assertEqual(fields["file"].get_filename(), "smoke.magent-backup")
def test_http_rejects_conflicting_body_encodings_without_network(self):
with patch.object(smoke.request, "urlopen") as urlopen:
with self.assertRaisesRegex(AssertionError, "only one encoding"):
smoke.http("/test", payload={}, raw=b"binary")
urlopen.assert_not_called()
def page(self, *, nonce="test-nonce", source="/_next/static/app.js", extra=""):
return (
f'<script nonce="{nonce}" src="{source}"></script>'
f'<script nonce="{nonce}">self.__next_f.push([])</script>'
'<link rel="stylesheet" href="/_next/static/app.css">'
f"{extra}"
).encode()
def test_static_assets_and_every_bootstrap_script_are_validated(self):
seen = []
def fake_http(path):
seen.append(path)
if path == "/login":
return self.page(), response_headers()
return b"static content", response_headers(**{"Content-Type": "application/javascript"})
with patch.object(smoke, "http", side_effect=fake_http):
assets = set()
self.assertEqual(smoke.check_page("/login", assets), "test-nonce")
self.assertEqual(assets, {"/_next/static/app.js", "/_next/static/app.css"})
self.assertEqual(seen, ["/login", "/_next/static/app.css", "/_next/static/app.js"])
smoke.check_page("/login", assets)
self.assertEqual(seen[-1], "/login")
self.assertEqual(len(seen), 4)
def test_nonce_mismatch_fails_before_fetching_assets(self):
with patch.object(smoke, "http", return_value=(self.page(nonce="wrong"), response_headers())):
with self.assertRaisesRegex(AssertionError, "script blocked by its CSP nonce"):
smoke.check_page("/login", set())
def test_missing_nonce_policy_is_rejected(self):
headers = response_headers(**{"Content-Security-Policy": "script-src 'self'"})
with patch.object(smoke, "http", return_value=(self.page(), headers)):
with self.assertRaisesRegex(AssertionError, "missing script nonce policy"):
smoke.check_page("/login", set())
def test_development_eval_policy_is_rejected(self):
headers = response_headers(**{
"Content-Security-Policy": "script-src 'nonce-test-nonce' 'strict-dynamic' 'unsafe-eval'",
})
with patch.object(smoke, "http", return_value=(self.page(), headers)):
with self.assertRaisesRegex(AssertionError, "development eval"):
smoke.check_page("/login", set())
def test_html_fallback_for_static_asset_is_rejected(self):
with patch.object(smoke, "http", return_value=(self.page(), response_headers())):
with self.assertRaisesRegex(AssertionError, "Asset returned HTML"):
smoke.check_page("/login", set())
def test_missing_executable_script_nonce_is_rejected(self):
page = self.page(extra='<script src="/_next/static/missing-nonce.js"></script>')
with patch.object(smoke, "http", return_value=(page, response_headers())):
with self.assertRaisesRegex(AssertionError, "script blocked by its CSP nonce"):
smoke.check_page("/login", set())
def test_inert_json_scripts_do_not_require_executable_nonce(self):
page = self.page(extra='<script type="application/ld+json">{"name":"Magent"}</script>')
with patch.object(smoke, "http", return_value=(page, response_headers())):
cache = {"/_next/static/app.js", "/_next/static/app.css"}
self.assertEqual(smoke.check_page("/login", cache), "test-nonce")
def test_external_scripts_are_not_followed_by_smoke_harness(self):
page = self.page(extra='<script nonce="test-nonce" src="https://external.invalid/app.js"></script>')
with patch.object(smoke, "http", return_value=(page, response_headers())):
with self.assertRaisesRegex(AssertionError, "Unexpected external executable asset"):
smoke.check_page("/login", {"/_next/static/app.js", "/_next/static/app.css"})
if __name__ == "__main__":
unittest.main()
-14
View File
@@ -99,7 +99,6 @@ class DuplicateAccountTests(TempDatabaseMixin, unittest.IsolatedAsyncioTestCase)
with db._connect() as conn:
conn.execute('UPDATE users SET jellyseerr_user_id=42 WHERE id=?', (self.extra,))
db.create_user('Other', 'Password-123456!', auth_provider='jellyfin', jellyseerr_user_id=42)
self.jf['users'].append({'id': 'd' * 32, 'name': 'Other'})
self.assertFalse((await duplicates.repair_duplicates(self.keep))['can_confirm'])
async def test_transaction_rolls_back_archive_and_history_on_failure(self):
@@ -167,16 +166,3 @@ class DuplicateAccountTests(TempDatabaseMixin, unittest.IsolatedAsyncioTestCase)
with TestClient(app) as client:
for path in ('check', 'confirm'):
self.assertEqual(client.post('/admin/identities/duplicates/' + path, json={'user_id': self.keep}).status_code, 403)
async def test_email_alias_consolidates_by_verified_id_and_preserves_activity(self):
with db._connect() as conn:
conn.execute("UPDATE users SET username='old@example.test',auth_provider='jellyseerr' WHERE id=?", (self.extra,))
db.upsert_user_activity('old@example.test', '127.0.0.1', 'browser')
preview = await duplicates.repair_duplicates(self.keep)
self.assertTrue(preview['can_confirm'], preview['issues'])
await duplicates.repair_duplicates(self.keep, self.keep, preview['revision'], {'username': 'admin'})
self.assertIsNone(db.get_user_by_id(self.extra))
with db._connect() as conn:
self.assertEqual(conn.execute('SELECT username FROM user_activity').fetchone()[0], 'Viewer')
self.assertFalse(db.create_user_if_missing('new-alias@example.test', 'unused', auth_provider='jellyseerr', jellyseerr_user_id=42))
-65
View File
@@ -1,65 +0,0 @@
import unittest
from unittest.mock import patch
from scripts.check_environment_docs import (
Setting,
check_documentation,
python_environment_names,
settings_inventory,
)
class EnvironmentDocumentationTests(unittest.TestCase):
def test_reference_covers_repository_variables_and_defaults(self):
errors, count = check_documentation()
self.assertGreater(count, 100)
self.assertEqual(errors, [], "\n".join(errors))
def test_settings_parser_preserves_implicit_names_alias_order_and_defaults(self):
source = '''
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_prefix="")
app_name: str = "Example"
service_url: str = Field(default=None, validation_alias=AliasChoices("SERVICE_URL", "OLD_URL"))
enabled: bool = Field(default=False, validation_alias="ENABLED")
interval: int = Field(default=60)
build_number: str = Field(default=BUILD_NUMBER)
'''
self.assertEqual(settings_inventory(source), [
Setting(("APP_NAME",), '"Example"'),
Setting(("SERVICE_URL", "OLD_URL"), "null"),
Setting(("ENABLED",), "false"),
Setting(("INTERVAL",), "60"),
Setting(("BUILD_NUMBER",), "@BUILD_NUMBER"),
])
def test_python_scanner_handles_reads_writes_and_bootstrap_mapping(self):
source = '''
os.getenv("METRICS_ENABLED", "false")
os.environ.get("WORKERS_ENABLED", "true")
environment.get("MANAGED_SECRETS", "auto")
prepared["GENERATED_KEY"] = "not-a-real-key"
other.get("NOT_AN_ENVIRONMENT_VARIABLE")
environment.get("lowercase-internal-key")
'''
self.assertEqual(python_environment_names(source), {
"METRICS_ENABLED", "WORKERS_ENABLED", "MANAGED_SECRETS", "GENERATED_KEY",
})
def test_scanning_never_executes_source_or_imports_settings(self):
source = '\ufeffraise RuntimeError("must not execute")\nos.getenv("SAFE_TO_SCAN")\n'
self.assertEqual(python_environment_names(source), {"SAFE_TO_SCAN"})
def test_reference_guard_reports_missing_variables_and_stale_defaults(self):
document = '| `RETRY_SECONDS` | `30` | Retry interval |'
source = 'class Settings(BaseSettings):\n retry_seconds: int = 60\n'
with patch("scripts.check_environment_docs.Path.read_text", side_effect=[document, source]), \
patch("scripts.check_environment_docs.runtime_environment_names", return_value={"NEW_FLAG"}):
errors, count = check_documentation()
self.assertEqual(count, 2)
self.assertIn("Undocumented environment variable: NEW_FLAG", errors)
self.assertTrue(any("Stale source default for RETRY_SECONDS" in error for error in errors))
if __name__ == "__main__":
unittest.main()
+2 -28
View File
@@ -1,5 +1,5 @@
import unittest
from unittest.mock import AsyncMock, patch
from unittest.mock import patch
from backend.app.config import settings
from fastapi import FastAPI
from fastapi.testclient import TestClient
@@ -16,13 +16,6 @@ class FeatureAccessTests(TempDatabaseMixin, unittest.TestCase):
secret = patch.object(settings, "jwt_secret", "feature-access-tests-only-secret-123456789")
secret.start()
self.addCleanup(secret.stop)
access = patch.object(
requests,
"_ensure_request_mutation_access",
new=AsyncMock(return_value=None),
)
access.start()
self.addCleanup(access.stop)
db.create_user('feature-viewer', 'Example-password123!', role='user')
db.create_user('feature-admin', 'Example-password123!', role='admin')
self.user = db.get_user_by_username('feature-viewer')
@@ -33,7 +26,7 @@ class FeatureAccessTests(TempDatabaseMixin, unittest.TestCase):
self.client.headers['Authorization'] = 'Bearer ' + create_access_token(self.user['username'], 'user')
def test_defaults_persist_and_invites_share_existing_setting(self):
self.assertEqual(permissions(self.user), dict(stats=True, requests=True, new_requests=True, issues=True, invites=False, ignore_profile_limits=False))
self.assertEqual(permissions(self.user), dict(stats=True, requests=True, new_requests=True, issues=True, invites=False))
update_permissions({'stats': False, 'invites': True}, self.user['username'])
db.init_db()
fresh = db.get_user_by_username(self.user['username'])
@@ -125,22 +118,3 @@ class FeatureAccessTests(TempDatabaseMixin, unittest.TestCase):
self.assertEqual(self.client.post('/portal/items', json={'kind': kind}).status_code, 403)
self.assertEqual(self.client.post('/portal/items', json={'kind': None}).status_code, 403)
self.assertEqual(self.client.post('/portal/items', json={}).status_code, 403)
def test_manual_override_permission_is_checked_again_at_download(self):
from types import SimpleNamespace
from unittest.mock import AsyncMock
from backend.app.models import Snapshot, RequestType
from backend.app.services import manual_releases
runtime=SimpleNamespace(jellyseerr_base_url=None,jellyseerr_api_key=None,sonarr_base_url='http://sonarr',sonarr_api_key='test')
snapshot=Snapshot(request_id='42',title='Example',request_type=RequestType.tv,raw={'arr':{'item':{'id':55}}})
release={'guid':'out','indexerId':1,'title':'Example','requiresOverride':True,'rejections':['Quality is not wanted in profile']}
payload={**release,'ignoreProfileLimits':True,'selectionToken':manual_releases.issue_selection(release,'42',self.user,'http://sonarr',55)}
collector=SimpleNamespace(configured=lambda:True,grab_release=AsyncMock(return_value={}))
with patch.object(requests,'get_runtime_settings',return_value=runtime),patch.object(requests,'build_snapshot',new=AsyncMock(return_value=snapshot)),patch.object(requests,'SonarrClient',return_value=collector),patch.object(requests,'save_action'):
self.assertEqual(self.client.post('/requests/42/actions/grab',json=payload).status_code,403)
collector.grab_release.assert_not_awaited()
update_permissions({'ignore_profile_limits':True},self.user['username'])
self.assertEqual(self.client.post('/requests/42/actions/grab',json=payload).status_code,200)
update_permissions({'ignore_profile_limits':False},self.user['username'])
self.assertEqual(self.client.post('/requests/42/actions/grab',json={**payload,'requiresOverride':False,'approved':True}).status_code,403)
collector.grab_release.assert_awaited_once()
-48
View File
@@ -1,48 +0,0 @@
import unittest
from types import SimpleNamespace
from unittest.mock import AsyncMock, patch
from backend.app import db
from backend.app.services import jellyfin_sync
from backend.app.services.jellyfin_identity import link_user, user_for_identity
from backend.app.routers import admin
from backend.tests.test_backend_quality import TempDatabaseMixin
class IdentitySyncTests(TempDatabaseMixin, unittest.IsolatedAsyncioTestCase):
async def test_sync_reuses_id_when_names_differ_and_preserves_settings(self):
db.create_user('old@example.test', 'Password-123456!', auth_provider='jellyseerr', jellyseerr_user_id=42,
auto_search_enabled=False, email='kept@example.test')
original = db.get_user_by_username('old@example.test')
runtime = SimpleNamespace(jellyfin_base_url='http://jf', jellyfin_api_key='test')
jf = SimpleNamespace(configured=lambda: True, get_users=AsyncMock(return_value=[{'Id': 'a' * 32, 'Name': 'NewName'}]))
with patch.object(jellyfin_sync, 'get_runtime_settings', return_value=runtime), \
patch.object(jellyfin_sync, 'JellyfinClient', return_value=jf), \
patch.object(jellyfin_sync, 'get_cached_jellyseerr_users', return_value=[{'id': 42, 'jellyfinUserId': 'a' * 32, 'email': 'upstream@example.test'}]), \
patch.object(jellyfin_sync, 'save_jellyfin_users_cache'):
self.assertEqual(await jellyfin_sync.sync_jellyfin_users(), 0)
self.assertEqual(await jellyfin_sync.sync_jellyfin_users(), 0)
kept = user_for_identity('a' * 32, 'http://jf')
self.assertEqual(kept['id'], original['id'])
self.assertFalse(kept['auto_search_enabled'])
self.assertEqual(kept['email'], 'kept@example.test')
self.assertIsNone(db.get_user_by_username('NewName'))
self.assertEqual(kept['auth_provider'], 'jellyfin')
async def test_resync_no_longer_deletes_accounts(self):
db.create_user('Keep', 'Password-123456!')
runtime = SimpleNamespace(jellyseerr_base_url='http://seer', jellyseerr_api_key='test')
with patch.object(admin, 'get_runtime_settings', return_value=runtime), \
patch.object(admin, '_fetch_all_jellyseerr_users', new=AsyncMock(return_value=[{'id': 42}])), \
patch.object(jellyfin_sync, 'sync_jellyfin_users', new=AsyncMock(return_value=0)), \
patch.object(admin, 'delete_non_admin_users') as delete:
result = await admin.jellyseerr_users_resync()
self.assertEqual(result['cleared'], 0)
delete.assert_not_called()
self.assertIsNotNone(db.get_user_by_username('Keep'))
def test_jellyfin_lookup_is_scoped_to_server(self):
db.create_user('Viewer', 'Password-123456!', auth_provider='jellyfin')
link_user('Viewer', 'a' * 32, 'http://jf')
self.assertIsNotNone(user_for_identity('a' * 32, 'http://jf'))
self.assertIsNone(user_for_identity('a' * 32, 'http://other-server'))
-175
View File
@@ -1,175 +0,0 @@
"""Real application HTTP checks for installation, cookies, and backup controls.
All persistence and artwork paths are isolated in temporary directories; workers,
logging file handlers, and the metrics listener are disabled for these tests.
"""
import io
from pathlib import Path
import tempfile
import unittest
from unittest.mock import patch
from fastapi.testclient import TestClient
from backend.app import db, main
from backend.app.config import settings
from backend.app.services import backups
OPERATOR_TOKEN = "installation-http-operator-token-test-123456789"
OWNER_PASSWORD = "installation-http-owner-password-123456789"
BACKUP_PASSPHRASE = "installation-http-backup-passphrase-123456789"
class InstallationHttpTests(unittest.TestCase):
def setUp(self):
self.temporary = tempfile.TemporaryDirectory(ignore_cleanup_errors=True)
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name)
for key, value in {
"sqlite_path": str(self.root / "magent.db"),
"sqlite_journal_mode": "DELETE",
"jwt_secret": "installation-http-test-jwt-secret-1234567890",
"settings_encryption_key": None,
"admin_username": "unused-environment-admin",
"admin_password": "",
"setup_token": OPERATOR_TOKEN,
"auth_cookie_secure": True,
"auth_cookie_domain": None,
"auth_cookie_samesite": "strict",
}.items():
context = patch.object(settings, key, value)
context.start()
self.addCleanup(context.stop)
for context in (
patch.object(main, "configure_logging"),
patch.object(main, "start_metrics"),
patch.object(main, "_background_tasks", []),
patch.object(main, "_background_started", False),
patch.object(backups, "_assets_root", return_value=self.root / "assets"),
patch.dict("os.environ", {"BACKGROUND_TASKS_ENABLED": "false"}),
):
context.start()
self.addCleanup(context.stop)
self.origin = str(settings.cors_allow_origin).rstrip("/")
self.client = self.enterContext(TestClient(main.app, base_url="https://magent.test"))
self.client.headers["Origin"] = self.origin
def create_owner(self):
response = self.client.post("/setup/bootstrap", json={
"setup_token": OPERATOR_TOKEN, "username": "owner", "password": OWNER_PASSWORD,
})
self.assertEqual(response.status_code, 201, response.text)
return response
def sign_in(self):
response = self.client.post("/auth/login", data={"username": "owner", "password": OWNER_PASSWORD})
self.assertEqual(response.status_code, 200, response.text)
self.assertIn(settings.auth_cookie_name, self.client.cookies)
auth_cookie = next(value for value in response.headers.get_list("set-cookie") if value.startswith(settings.auth_cookie_name + "="))
self.assertIn("HttpOnly", auth_cookie)
self.assertIn("Secure", auth_cookie)
self.assertIn("SameSite=strict", auth_cookie)
self.assertNotIn("Authorization", self.client.headers)
def test_fresh_setup_cookie_settings_completion_and_backup_round_trip(self):
status = self.client.get("/setup/status")
self.assertEqual(status.json(), {"setup_required": True, "needs_admin": True})
self.assertEqual(status.headers["cache-control"], "no-store")
self.assertIn("default-src 'none'", status.headers["content-security-policy"])
self.assertEqual(self.client.get("/setup/state").status_code, 401)
self.assertEqual(self.client.get("/admin/backups").status_code, 401)
self.create_owner()
self.sign_in()
self.assertEqual(self.client.get("/setup/state").json()["step"], "apps")
response = self.client.put("/admin/settings", json={
"jellyfin_base_url": "http://jellyfin.test:8096",
"jellyfin_api_key": "test-integration-key-for-setup",
"site_login_message": "Welcome to this installation",
})
self.assertEqual(response.status_code, 200, response.text)
self.assertEqual(response.json()["updated"], 3)
values = {row["key"]: row for row in self.client.get("/admin/settings").json()["settings"]}
self.assertEqual(values["jellyfin_base_url"]["value"], "http://jellyfin.test:8096")
self.assertIsNone(values["jellyfin_api_key"]["value"])
self.assertTrue(values["jellyfin_api_key"]["isSet"])
response = self.client.put("/setup/state", json={"step": "review"})
self.assertEqual(response.status_code, 200, response.text)
completed = self.client.post("/setup/complete")
self.assertEqual(completed.status_code, 200, completed.text)
self.assertTrue(completed.json()["completed"])
self.assertEqual(self.client.get("/setup/status").json(), {"setup_required": False, "needs_admin": False})
self.assertEqual(main._background_tasks, [])
exported = self.client.post("/admin/backups/export", json={
"passphrase": BACKUP_PASSPHRASE, "include_cache": False,
})
self.assertEqual(exported.status_code, 200, exported.text[:100])
self.assertTrue(exported.content.startswith(backups.MAGIC))
self.assertEqual(exported.headers["cache-control"], "no-store")
self.assertNotIn(b"test-integration-key-for-setup", exported.content)
restored = self.client.post("/admin/backups/restore", files={
"file": ("restore.magent-backup", io.BytesIO(exported.content), "application/octet-stream"),
}, data={"passphrase": BACKUP_PASSPHRASE, "confirmation": "RESTORE"})
self.assertEqual(restored.status_code, 202, restored.text)
self.assertTrue(restored.json()["restart_required"])
self.assertEqual(db.get_setting("site_login_message"), "Welcome to this installation")
self.assertIsNotNone(self.client.get("/admin/backups").json()["pending_restore"])
cancelled = self.client.delete("/admin/backups/restore")
self.assertEqual(cancelled.status_code, 200, cancelled.text)
self.assertIsNone(self.client.get("/admin/backups").json()["pending_restore"])
def test_cross_origin_bootstrap_and_authenticated_changes_are_rejected(self):
response = self.client.post("/setup/bootstrap", headers={"Origin": "https://unrelated.invalid"}, json={
"setup_token": OPERATOR_TOKEN, "username": "owner", "password": OWNER_PASSWORD,
})
self.assertEqual(response.status_code, 403)
self.assertFalse(db.has_admin_user())
self.create_owner()
self.sign_in()
response = self.client.put("/setup/state", headers={"Origin": "https://unrelated.invalid"}, json={"step": "review"})
self.assertEqual(response.status_code, 403)
response = self.client.post("/admin/backups/export", headers={"Origin": "https://unrelated.invalid"}, json={"passphrase": BACKUP_PASSPHRASE})
self.assertEqual(response.status_code, 403)
self.assertEqual(self.client.get("/setup/state").json()["step"], "apps")
def test_setup_validation_errors_do_not_echo_password_or_token(self):
secret_password = "private-password-marker-" + "p" * 1024
secret_token = "private-token-marker-" + "t" * 1024
for payload, secret in (
({"setup_token": OPERATOR_TOKEN, "username": "owner", "password": secret_password}, secret_password),
({"setup_token": secret_token, "username": "owner", "password": OWNER_PASSWORD}, secret_token),
({"setup_token": OPERATOR_TOKEN, "password": OWNER_PASSWORD}, OWNER_PASSWORD),
):
with self.subTest(secret=secret[:22]):
response = self.client.post("/setup/bootstrap", json=payload)
self.assertEqual(response.status_code, 422, response.text)
self.assertNotIn(secret, response.text)
self.assertNotIn(OPERATOR_TOKEN, response.text)
for error in response.json()["detail"]:
self.assertNotIn("input", error)
def test_backup_validation_errors_do_not_echo_passphrases(self):
self.create_owner()
self.sign_in()
passphrase = "private-backup-passphrase-marker-" + "p" * 1024
response = self.client.post("/admin/backups/export", json={"passphrase": passphrase})
self.assertEqual(response.status_code, 422)
self.assertNotIn(passphrase, response.text)
response = self.client.post("/admin/backups/restore", files={"file": ("archive", b"data")}, data={
"passphrase": passphrase, "confirmation": "RESTORE",
})
self.assertEqual(response.status_code, 422)
self.assertNotIn(passphrase, response.text)
self.assertIsNone(self.client.get("/admin/backups").json()["pending_restore"])
def test_real_middleware_rejects_oversized_bootstrap_before_creation(self):
response = self.client.post("/setup/bootstrap", content=b"x" * (17 * 1024), headers={"Content-Type": "application/json"})
self.assertEqual(response.status_code, 413, response.text)
self.assertFalse(db.has_admin_user())
if __name__ == "__main__":
unittest.main()
@@ -1,162 +0,0 @@
import asyncio
from pathlib import Path
import tempfile
import unittest
from unittest.mock import Mock, patch
import httpx
from fastapi import FastAPI, File, Request, UploadFile
from backend.app import db, main
from backend.app.config import settings
from backend.app.request_limits import InstallationBodyLimitMiddleware
from backend.app.services import setup
class InstallationLifecycleTests(unittest.IsolatedAsyncioTestCase):
def setUp(self):
temporary = tempfile.TemporaryDirectory(ignore_cleanup_errors=True)
self.addCleanup(temporary.cleanup)
patches = [
patch.object(settings, "sqlite_path", str(Path(temporary.name) / "magent.db")),
patch.object(settings, "jwt_secret", "installation-lifecycle-secret-1234567890"),
patch.object(settings, "settings_encryption_key", None),
patch.object(settings, "admin_password", ""),
patch.object(settings, "setup_token", "operator-setup-token-at-least-32-characters"),
patch.object(main, "_background_started", False),
patch.object(main, "_background_tasks", []),
patch.object(main, "start_metrics"),
patch.object(main, "configure_logging"),
patch.dict("os.environ", {"BACKGROUND_TASKS_ENABLED": "true"}),
]
for item in patches:
item.start()
self.addCleanup(item.stop)
async def test_fresh_start_waits_for_admin_and_completion_then_starts_workers_once(self):
with patch.object(main, "_launch_background_task") as launch:
await main.startup()
self.assertEqual(setup.get_public_setup_status(), {"setup_required": True, "needs_admin": True})
launch.assert_not_called()
setup.bootstrap_administrator(settings.setup_token, "owner", "new-password-12345")
await main._start_background_tasks()
launch.assert_not_called()
setup.complete_setup()
await main.app.state.on_setup_complete()
await main.app.state.on_setup_complete()
self.assertEqual(launch.call_count, 9)
async def test_upgraded_install_starts_normally_without_setup_token(self):
db.init_db()
db.create_user("owner", "existing-password-12345", role="admin")
settings.setup_token = ""
with patch.object(main, "_launch_background_task") as launch:
await main.startup()
self.assertFalse(setup.is_setup_required())
self.assertEqual(launch.call_count, 9)
async def test_disabled_workers_stay_disabled_after_setup(self):
setup.initialize_setup_state()
db.init_db()
setup.bootstrap_administrator(settings.setup_token, "owner", "new-password-12345")
setup.complete_setup()
with patch.dict("os.environ", {"BACKGROUND_TASKS_ENABLED": "false"}), patch.object(main, "_launch_background_task") as launch:
await main._start_background_tasks()
launch.assert_not_called()
async def test_bad_secret_stops_before_restore_or_database_initialization(self):
settings.jwt_secret = "short"
with patch.object(main, "apply_pending_restore") as restore, patch.object(main, "init_db") as initialize:
with self.assertRaisesRegex(RuntimeError, "JWT_SECRET"):
await main.startup()
restore.assert_not_called()
initialize.assert_not_called()
async def test_restore_failure_stops_before_initialization_and_workers(self):
with patch.object(main, "apply_pending_restore", side_effect=RuntimeError("restore failed")), patch.object(main, "init_db") as initialize, patch.object(main, "_launch_background_task") as launch:
with self.assertRaisesRegex(RuntimeError, "restore failed"):
await main.startup()
initialize.assert_not_called()
launch.assert_not_called()
async def test_startup_order_is_restore_then_setup_marker_then_schema(self):
calls = Mock()
calls.attach_mock(Mock(wraps=main.apply_pending_restore), "restore")
calls.attach_mock(Mock(wraps=main.initialize_setup_state), "setup")
calls.attach_mock(Mock(wraps=main.init_db), "schema")
with patch.object(main, "apply_pending_restore", calls.restore), patch.object(main, "initialize_setup_state", calls.setup), patch.object(main, "init_db", calls.schema):
await main.startup()
self.assertEqual([call[0] for call in calls.mock_calls], ["restore", "setup", "schema"])
def test_missing_token_does_not_allow_fresh_bootstrap(self):
setup.initialize_setup_state()
db.init_db()
settings.setup_token = ""
with self.assertRaisesRegex(RuntimeError, "SETUP_TOKEN"):
main._enforce_secure_startup_configuration()
def test_destination_environment_does_not_add_an_admin_to_restored_accounts(self):
db.init_db()
db.create_user("restored-owner", "existing-password-12345", role="admin")
with patch.object(settings, "admin_username", "host-bootstrap"), patch.object(settings, "admin_password", "new-host-password-12345"):
db.init_db()
self.assertIsNone(db.get_user_by_username("host-bootstrap"))
async def test_shutdown_cancels_workers_and_allows_next_start(self):
task = asyncio.create_task(asyncio.Event().wait())
main._background_tasks.append(task)
main._background_started = True
await main.shutdown()
self.assertTrue(task.cancelled())
self.assertEqual(main._background_tasks, [])
self.assertFalse(main._background_started)
class InstallationRequestLimitsTests(unittest.IsolatedAsyncioTestCase):
async def test_rejects_oversized_declared_body_before_parser(self):
app = FastAPI()
app.add_middleware(InstallationBodyLimitMiddleware)
@app.post("/setup/bootstrap")
async def bootstrap(request: Request):
self.fail("Body must be rejected before the endpoint")
async with httpx.AsyncClient(transport=httpx.ASGITransport(app=app), base_url="http://test") as client:
response = await client.post("/setup/bootstrap", content=b"{}", headers={"Content-Length": "999999"})
self.assertEqual(response.status_code, 413)
async def test_counts_chunks_with_missing_or_forged_content_length(self):
app = FastAPI()
app.add_middleware(InstallationBodyLimitMiddleware)
@app.post("/setup/bootstrap")
async def bootstrap(request: Request):
return await request.json()
async def chunks():
yield b'{"token":"'
yield b"a" * 17000
yield b'"}'
async with httpx.AsyncClient(transport=httpx.ASGITransport(app=app), base_url="http://test") as client:
for headers in ({}, {"Content-Length": "1"}):
response = await client.post("/setup/bootstrap", content=chunks(), headers=headers)
self.assertEqual(response.status_code, 413)
async def test_multipart_stream_limit_is_413_not_parser_500(self):
app = FastAPI()
app.add_middleware(InstallationBodyLimitMiddleware)
@app.post("/admin/backups/restore")
async def restore(file: UploadFile = File(...)):
return {"size": file.size}
async def chunks():
yield b'--boundary\r\nContent-Disposition: form-data; name="file"; filename="backup"\r\n\r\n'
yield b"a" * 2048
yield b"\r\n--boundary--\r\n"
with patch("backend.app.request_limits.RESTORE_BODY_LIMIT", 1024):
async with httpx.AsyncClient(transport=httpx.ASGITransport(app=app), base_url="http://test") as client:
response = await client.post("/admin/backups/restore", content=chunks(), headers={"Content-Type": "multipart/form-data; boundary=boundary"})
self.assertEqual(response.status_code, 413)
-3
View File
@@ -39,9 +39,6 @@ class IssueAcceptanceTests(TempDatabaseMixin, unittest.IsolatedAsyncioTestCase):
self.assertEqual(email.await_count, 1)
self.assertEqual(db.get_portal_item(item["id"])["status"], "awaiting_confirmation")
content = email.await_args.kwargs
self.assertEqual(content["subject"], f"Ready to try again? Magent issue #{item['id']}")
self.assertIn("affected content in Jellyfin", content["body_text"])
self.assertNotIn("grizzlyflix", content["body_html"].lower())
self.assertIn("YES — it works", content["body_html"])
self.assertIn("NO — still broken", content["body_html"])
self.assertIn(f"/issues/confirm/{item['id']}#yes", content["body_html"])
-30
View File
@@ -1,30 +0,0 @@
import unittest
from unittest.mock import patch
import httpx
from backend.app.clients.jellyfin import JellyfinClient
from backend.app.services.snapshot import jellyfin_item_matches_request
from backend.app.models import RequestType
class JellyfinMatchingTests(unittest.IsolatedAsyncioTestCase):
async def test_search_includes_punctuation_variant_and_provider_metadata(self):
calls=[]
def handle(request):
calls.append(request)
items=[{'Id':'animated','Name':'Avatar: The Last Airbender','ProductionYear':2005,'ProviderIds':{'Tmdb':'246'}}] if ':' in request.url.params['SearchTerm'] else [{'Id':'live','Name':'Avatar the Last Airbender','ProductionYear':2024,'ProviderIds':{'Tmdb':'82452'}}]
return httpx.Response(200,json={'Items':items})
original=httpx.AsyncClient
with patch('backend.app.clients.jellyfin.httpx.AsyncClient',side_effect=lambda **kw:original(transport=httpx.MockTransport(handle),**kw)):
result=await JellyfinClient('http://jellyfin','test').search_items('Avatar: The Last Airbender',['Series'])
self.assertEqual({i['Id'] for i in result['Items']},{'live','animated'})
self.assertTrue(all('ProviderIds' in r.url.params['Fields'] for r in calls))
matches=[i for i in result['Items'] if jellyfin_item_matches_request(i,title='Avatar: The Last Airbender',year=2024,request_type=RequestType.tv,request_payload={'tmdbId':82452})]
self.assertEqual([i['Id'] for i in matches],['live'])
def test_fallback_rejects_remakes_prefixes_and_conflicting_ids(self):
def match(item,payload=None):
return jellyfin_item_matches_request(item,title='Avatar: The Last Airbender',year=2024,request_type=RequestType.tv,request_payload=payload)
self.assertTrue(match({'Name':'Avatar the Last Airbender','ProductionYear':2024}))
self.assertFalse(match({'Name':'Avatar the Last Airbender','ProductionYear':2005}))
self.assertFalse(match({'Name':'Avatar','ProductionYear':2024}))
self.assertFalse(match({'Name':'Avatar the Last Airbender','ProductionYear':2024,'ProviderIds':{'Tmdb':'246'}},{'tmdbId':82452}))
self.assertTrue(match({'Name':'Localized title','ProviderIds':{'Tmdb':'82452'}},{'tmdbId':82452}))
-25
View File
@@ -1,25 +0,0 @@
import json
import logging
import unittest
from backend.app.logging_config import JsonLogFormatter, RequestContextFilter, bind_request_id, reset_request_id
class JsonLoggingTests(unittest.TestCase):
def test_json_formatter_includes_request_context(self) -> None:
token = bind_request_id("request-123")
try:
record = logging.LogRecord("magent.test", logging.INFO, __file__, 1, "hello %s", ("world",), None)
RequestContextFilter().filter(record)
payload = json.loads(JsonLogFormatter().format(record))
finally:
reset_request_id(token)
self.assertEqual(payload["level"], "INFO")
self.assertEqual(payload["logger"], "magent.test")
self.assertEqual(payload["request_id"], "request-123")
self.assertEqual(payload["message"], "hello world")
if __name__ == "__main__":
unittest.main()
-379
View File
@@ -1,379 +0,0 @@
"""Managed first-install origin claims use a token, not request routing headers."""
from concurrent.futures import ThreadPoolExecutor
import os
from pathlib import Path
import sqlite3
import tempfile
from threading import Barrier
import unittest
from unittest.mock import patch
from fastapi.testclient import TestClient
from backend.app import auth, db, main
from backend.app.config import settings
from backend.app.installation_origin import normalize_application_origin
from backend.app.security import create_access_token
from backend.app.services import setup
from backend.app.services.public_urls import magent_public_url
from backend.app.services.request_origins import can_claim_initial_origin, is_allowed_request_origin
PUBLIC_ORIGIN = "https://watch.example.test"
LOCAL_ORIGIN = "http://localhost:3000"
SETUP_TOKEN = "managed-origin-operator-token-for-tests-only-1234567890"
ADMIN_PASSWORD = "Managed-origin-password-for-tests!123"
class ApplicationOriginNormalizationTests(unittest.TestCase):
def test_exact_origins_are_canonicalized(self):
for value, expected in (
(PUBLIC_ORIGIN, PUBLIC_ORIGIN),
("HTTPS://WATCH.EXAMPLE.TEST:443/", PUBLIC_ORIGIN),
("http://magent.lan:80/", "http://magent.lan"),
("http://192.0.2.10:3000", "http://192.0.2.10:3000"),
("http://[fd00::10]:3000/", "http://[fd00::10]:3000"),
):
with self.subTest(value=value):
self.assertEqual(normalize_application_origin(value), expected)
def test_non_origins_and_ambiguous_values_are_rejected(self):
for value in (
"", "null", "*", "magent.lan", "//magent.lan", "https:/magent.lan",
"ftp://magent.lan", "https://user@magent.lan", "https://user:secret@magent.lan",
"https://magent.lan/path", "https://magent.lan/../", "https://magent.lan?",
"https://magent.lan#", "https://magent.lan?token=1", "https://magent.lan#fragment",
"https://*.magent.lan", "https://magent.lan:0", "https://magent.lan:65536",
"https://magent.lan:", "https://magent.lan\\path", " https://magent.lan",
"https://magent.\tlan", "https://magent.lan\x00",
):
with self.subTest(value=repr(value)), self.assertRaises(ValueError):
normalize_application_origin(value)
class ManagedSetupOriginTests(unittest.TestCase):
def setUp(self):
temporary = tempfile.TemporaryDirectory(ignore_cleanup_errors=True)
self.addCleanup(temporary.cleanup)
self.enterContext(patch.dict(os.environ, {"MAGENT_RUNTIME_MANAGED": "1"}))
for name, value in {
"sqlite_path": str(Path(temporary.name) / "managed-setup.db"),
"sqlite_journal_mode": "DELETE",
"jwt_secret": "managed-origin-test-signing-key-at-least-32-characters",
"settings_encryption_key": None,
"setup_token": SETUP_TOKEN,
"admin_username": "unused-environment-admin",
"admin_password": "",
"cors_allow_origin": LOCAL_ORIGIN,
"magent_application_url": None,
"magent_proxy_enabled": False,
"magent_proxy_base_url": None,
"auth_cookie_domain": None,
"auth_cookie_secure": False,
}.items():
self.enterContext(patch.object(settings, name, value))
setup.initialize_setup_state()
db.init_db()
# No production startup: no external workers, listeners or real data.
self.client = TestClient(main.app, base_url=PUBLIC_ORIGIN)
self.addCleanup(self.client.close)
def bootstrap(self, *, origin=PUBLIC_ORIGIN, headers=None, **changes):
request_headers = {} if origin is None else {"Origin": origin}
request_headers.update(headers or {})
return self.client.post("/setup/bootstrap", headers=request_headers, json={
"setup_token": SETUP_TOKEN,
"username": "first-admin",
"password": ADMIN_PASSWORD,
"application_url": PUBLIC_ORIGIN,
**changes,
})
def assert_unclaimed(self):
self.assertFalse(db.has_admin_user())
self.assertIsNone(db.get_setting("magent_application_url"))
self.assertEqual(setup.get_setup_state()["step"], "administrator")
def preflight(self, origin):
return self.client.options("/auth/login", headers={
"Origin": origin,
"Access-Control-Request-Method": "POST",
"Access-Control-Request-Headers": "Content-Type",
})
def admin_headers(self, origin=PUBLIC_ORIGIN):
return {
"Origin": origin,
"Authorization": f"Bearer {create_access_token('first-admin', 'admin')}",
}
def test_fresh_managed_install_does_not_trust_any_origin_before_claim(self):
self.assertTrue(can_claim_initial_origin())
for origin in (PUBLIC_ORIGIN, LOCAL_ORIGIN, "https://evil.example.test"):
with self.subTest(origin=origin):
self.assertFalse(is_allowed_request_origin(origin))
self.assert_unclaimed()
def test_valid_operator_claim_creates_admin_and_persists_url_atomically(self):
response = self.bootstrap(application_url="HTTPS://WATCH.EXAMPLE.TEST:443/")
self.assertEqual(response.status_code, 201, response.text)
self.assertEqual(db.get_setting("magent_application_url"), PUBLIC_ORIGIN)
self.assertIsNotNone(db.verify_user_password("first-admin", ADMIN_PASSWORD))
self.assertEqual(setup.get_setup_state()["step"], "apps")
self.assertFalse(can_claim_initial_origin())
self.assertTrue(is_allowed_request_origin(PUBLIC_ORIGIN))
self.assertFalse(is_allowed_request_origin(LOCAL_ORIGIN))
def test_missing_or_null_application_url_does_not_claim(self):
response = self.bootstrap(application_url=None)
self.assertEqual(response.status_code, 400, response.text)
response = self.client.post("/setup/bootstrap", headers={"Origin": PUBLIC_ORIGIN}, json={
"setup_token": SETUP_TOKEN, "username": "first-admin", "password": ADMIN_PASSWORD,
})
self.assertEqual(response.status_code, 400, response.text)
self.assert_unclaimed()
def test_missing_browser_origin_does_not_claim(self):
response = self.bootstrap(origin=None)
self.assertEqual(response.status_code, 403, response.text)
self.assert_unclaimed()
def test_wrong_token_cannot_claim_even_when_url_matches_evil_origin(self):
for origin in (PUBLIC_ORIGIN, "https://evil.example.test"):
with self.subTest(origin=origin):
response = self.bootstrap(origin=origin, application_url=origin, setup_token="wrong-token")
self.assertEqual(response.status_code, 403, response.text)
self.assertNotIn(SETUP_TOKEN, response.text)
self.assert_unclaimed()
def test_different_origin_and_spoofed_routing_headers_cannot_claim(self):
response = self.bootstrap(origin="https://evil.example.test", headers={
"Host": "watch.example.test", "X-Forwarded-Host": "watch.example.test",
"X-Forwarded-Proto": "https", "Sec-Fetch-Site": "same-origin",
})
self.assertEqual(response.status_code, 403, response.text)
self.assert_unclaimed()
def test_invalid_application_urls_never_claim_or_echo_secrets(self):
with patch.object(setup, "consume_bootstrap_attempt", return_value=None):
for value in ("", "https://user:secret@watch.example.test", PUBLIC_ORIGIN + "/path",
PUBLIC_ORIGIN + "?", "javascript:alert(1)", "//watch.example.test"):
with self.subTest(value=value):
response = self.bootstrap(application_url=value)
self.assertEqual(response.status_code, 400, response.text)
self.assertNotIn(SETUP_TOKEN, response.text)
self.assertNotIn(ADMIN_PASSWORD, response.text)
self.assert_unclaimed()
def test_other_state_changing_endpoints_do_not_inherit_bootstrap_origin_exception(self):
for path, method in (("/auth/login", "post"), ("/auth/jellyfin/login", "post"),
("/setup/complete", "post"), ("/setup/state", "put"),
("/setup/bootstrap/", "post"), ("/admin/settings", "put")):
for origin in (PUBLIC_ORIGIN, LOCAL_ORIGIN, "https://evil.example.test"):
with self.subTest(path=path, origin=origin):
response = getattr(self.client, method)(path, json={}, headers={"Origin": origin}, follow_redirects=False)
self.assertEqual(response.status_code, 403, response.text)
self.assertEqual(response.json()["detail"], "Cross-origin state change rejected")
self.assert_unclaimed()
def test_existing_admin_prevents_reclaim_and_url_replacement(self):
self.assertEqual(self.bootstrap().status_code, 201)
response = self.bootstrap(username="second-admin")
self.assertEqual(response.status_code, 409, response.text)
response = self.bootstrap(origin="https://evil.example.test", application_url="https://evil.example.test")
self.assertEqual(response.status_code, 403, response.text)
self.assertEqual(db.get_setting("magent_application_url"), PUBLIC_ORIGIN)
self.assertEqual(len(db.get_all_users()), 1)
def test_completed_install_cannot_reopen_origin_claim_after_admin_removal(self):
self.assertEqual(self.bootstrap().status_code, 201)
setup.complete_setup()
with db._connect() as connection:
connection.execute("DELETE FROM users")
connection.execute("DELETE FROM settings WHERE key='magent_application_url'")
self.assertFalse(can_claim_initial_origin())
self.assertEqual(self.bootstrap().status_code, 403)
self.assertFalse(db.has_admin_user())
def test_setting_write_failure_rolls_back_admin_and_setup_progress(self):
with db._connect() as connection:
connection.execute("""CREATE TRIGGER reject_origin BEFORE INSERT ON settings
WHEN NEW.key = 'magent_application_url'
BEGIN SELECT RAISE(ABORT, 'synthetic origin storage failure'); END""")
with self.assertRaises(sqlite3.IntegrityError):
setup.bootstrap_administrator(SETUP_TOKEN, "first-admin", ADMIN_PASSWORD, application_url=PUBLIC_ORIGIN)
self.assert_unclaimed()
def test_concurrent_claims_keep_the_winning_admin_and_origin_together(self):
barrier = Barrier(4)
def synchronized_hash(_):
barrier.wait(timeout=10)
return "test-only-precomputed-hash"
def claim(number):
try:
setup.bootstrap_administrator(
SETUP_TOKEN, f"owner-{number}", ADMIN_PASSWORD,
application_url=f"https://owner-{number}.example.test",
)
return number
except setup.SetupUnavailableError:
return None
with patch.object(setup, "hash_password", side_effect=synchronized_hash):
with ThreadPoolExecutor(max_workers=4) as executor:
winners = [number for number in executor.map(claim, range(4)) if number is not None]
self.assertEqual(len(winners), 1)
self.assertEqual([user["username"] for user in db.get_all_users()], [f"owner-{winners[0]}"])
self.assertEqual(db.get_setting("magent_application_url"), f"https://owner-{winners[0]}.example.test")
def test_first_https_login_uses_secure_cookie_without_restart(self):
self.assertEqual(self.bootstrap().status_code, 201)
self.assertFalse(settings.auth_cookie_secure)
response = self.client.post("/auth/login", headers={"Origin": PUBLIC_ORIGIN}, data={
"username": "first-admin", "password": ADMIN_PASSWORD,
})
self.assertEqual(response.status_code, 200, response.text)
cookie = next(value for value in response.headers.get_list("set-cookie")
if value.startswith(settings.auth_cookie_name + "="))
self.assertIn("Secure", cookie)
self.assertIn("HttpOnly", cookie)
self.assertEqual(self.client.get("/auth/me").status_code, 200)
def test_http_lan_claim_uses_non_secure_cookie_despite_static_secure_default(self):
origin = "http://magent.lan:3000"
self.assertEqual(self.bootstrap(origin=origin, application_url=origin).status_code, 201)
with patch.object(settings, "auth_cookie_secure", True):
self.assertFalse(auth._cookie_settings()["secure"])
self.assertTrue(auth._cookie_settings()["httponly"])
as_client = TestClient(main.app, base_url=origin)
try:
response = as_client.post("/auth/login", headers={"Origin": origin}, data={
"username": "first-admin", "password": ADMIN_PASSWORD,
})
self.assertEqual(as_client.get("/auth/me").status_code, 200)
finally:
as_client.close()
self.assertEqual(response.status_code, 200, response.text)
cookie = next(value for value in response.headers.get_list("set-cookie")
if value.startswith(settings.auth_cookie_name + "="))
self.assertNotIn("Secure", cookie)
self.assertIn("HttpOnly", cookie)
def test_login_origin_policy_and_saved_url_survive_setup_reinitialization(self):
self.assertEqual(self.bootstrap().status_code, 201)
setup.initialize_setup_state()
db.init_db()
self.assertFalse(can_claim_initial_origin())
self.assertEqual(db.get_setting("magent_application_url"), PUBLIC_ORIGIN)
for origin in (LOCAL_ORIGIN, "https://evil.example.test"):
response = self.client.post("/auth/login", headers={"Origin": origin}, data={
"username": "first-admin", "password": ADMIN_PASSWORD,
})
self.assertEqual(response.status_code, 403, response.text)
response = self.client.post("/auth/login", headers={"Origin": PUBLIC_ORIGIN}, data={
"username": "first-admin", "password": ADMIN_PASSWORD,
})
self.assertEqual(response.status_code, 200, response.text)
def test_managed_origin_policy_changes_with_saved_settings(self):
self.assertEqual(self.bootstrap().status_code, 201)
db.set_setting("magent_application_url", "http://magent.lan:3000")
self.assertFalse(is_allowed_request_origin(PUBLIC_ORIGIN))
self.assertFalse(is_allowed_request_origin(LOCAL_ORIGIN))
self.assertTrue(is_allowed_request_origin("http://magent.lan:3000"))
self.assertFalse(auth._cookie_settings()["secure"])
def test_unclaimed_install_does_not_grant_cors_to_any_browser_origin(self):
for origin in (PUBLIC_ORIGIN, LOCAL_ORIGIN, "https://evil.example.test"):
with self.subTest(origin=origin):
response = self.preflight(origin)
self.assertEqual(response.status_code, 400, response.text)
self.assertNotIn("access-control-allow-origin", response.headers)
response = self.client.get("/setup/status", headers={"Origin": origin})
self.assertEqual(response.status_code, 200, response.text)
self.assertNotIn("access-control-allow-origin", response.headers)
self.assert_unclaimed()
def test_claim_immediately_updates_cors_preflights_and_response_headers(self):
self.assertEqual(self.bootstrap().status_code, 201)
response = self.preflight(PUBLIC_ORIGIN)
self.assertEqual(response.status_code, 200, response.text)
self.assertEqual(response.headers["access-control-allow-origin"], PUBLIC_ORIGIN)
self.assertEqual(response.headers["access-control-allow-credentials"], "true")
self.assertIn("Origin", response.headers["vary"])
response = self.client.get("/setup/status", headers={"Origin": PUBLIC_ORIGIN})
self.assertEqual(response.status_code, 200, response.text)
self.assertEqual(response.headers["access-control-allow-origin"], PUBLIC_ORIGIN)
self.assertEqual(response.headers["access-control-allow-credentials"], "true")
def test_claimed_install_denies_localhost_and_foreign_cors_preflights_and_reads(self):
self.assertEqual(self.bootstrap().status_code, 201)
for origin in (LOCAL_ORIGIN, "https://evil.example.test", PUBLIC_ORIGIN + "/", "null"):
with self.subTest(origin=origin):
response = self.preflight(origin)
self.assertEqual(response.status_code, 400, response.text)
self.assertNotIn("access-control-allow-origin", response.headers)
response = self.client.get("/setup/status", headers={"Origin": origin})
self.assertEqual(response.status_code, 200, response.text)
self.assertNotIn("access-control-allow-origin", response.headers)
def test_admin_cannot_blank_or_malform_managed_url_or_partially_save_other_settings(self):
self.assertEqual(self.bootstrap().status_code, 201)
for value in (None, "", " ", False, 123, [], {}, "javascript:alert(1)", "//watch.example.test",
PUBLIC_ORIGIN + "/path", PUBLIC_ORIGIN + "?", "https://user:secret@watch.example.test"):
with self.subTest(value=value):
response = self.client.put("/admin/settings", headers=self.admin_headers(), json={
"site_login_message": "must-not-be-written",
"magent_application_url": value,
})
self.assertEqual(response.status_code, 400, response.text)
self.assertEqual(db.get_setting("magent_application_url"), PUBLIC_ORIGIN)
self.assertIsNone(db.get_setting("site_login_message"))
self.assertTrue(is_allowed_request_origin(PUBLIC_ORIGIN))
self.assertFalse(can_claim_initial_origin())
def test_admin_url_update_is_canonical_and_immediately_replaces_cors_origin(self):
self.assertEqual(self.bootstrap().status_code, 201)
next_origin = "https://new.example.test"
response = self.client.put("/admin/settings", headers=self.admin_headers(), json={
"magent_application_url": "HTTPS://NEW.EXAMPLE.TEST:443/",
})
self.assertEqual(response.status_code, 200, response.text)
self.assertEqual(db.get_setting("magent_application_url"), next_origin)
self.assertEqual(self.preflight(PUBLIC_ORIGIN).status_code, 400)
response = self.preflight(next_origin)
self.assertEqual(response.status_code, 200, response.text)
self.assertEqual(response.headers["access-control-allow-origin"], next_origin)
response = self.client.get("/setup/status", headers={"Origin": next_origin})
self.assertEqual(response.headers["access-control-allow-origin"], next_origin)
response = self.client.put("/admin/settings", headers=self.admin_headers(), json={"site_login_message": "stale"})
self.assertEqual(response.status_code, 403, response.text)
response = self.client.put("/admin/settings", headers=self.admin_headers(next_origin), json={"site_login_message": "new"})
self.assertEqual(response.status_code, 200, response.text)
self.assertEqual(db.get_setting("site_login_message"), "new")
def test_proxy_settings_cannot_replace_managed_origin_or_lower_cookie_security(self):
self.assertEqual(self.bootstrap().status_code, 201)
db.set_setting("magent_proxy_enabled", "true")
db.set_setting("magent_proxy_base_url", "http://proxy.example.test")
self.assertEqual(magent_public_url(), PUBLIC_ORIGIN)
self.assertTrue(is_allowed_request_origin(PUBLIC_ORIGIN))
self.assertFalse(is_allowed_request_origin("http://proxy.example.test"))
self.assertTrue(auth._cookie_settings()["secure"])
self.assertEqual(self.preflight(PUBLIC_ORIGIN).status_code, 200)
self.assertEqual(self.preflight("http://proxy.example.test").status_code, 400)
def test_unclaimed_managed_url_ignores_legacy_proxy_and_link_fallback(self):
db.set_setting("magent_proxy_enabled", "true")
db.set_setting("magent_proxy_base_url", "https://proxy.example.test")
self.assertEqual(magent_public_url("https://legacy.example.test"), "")
self.assertTrue(can_claim_initial_origin())
self.assertFalse(is_allowed_request_origin("https://proxy.example.test"))
self.assertEqual(self.preflight("https://proxy.example.test").status_code, 400)
if __name__ == "__main__":
unittest.main()
-102
View File
@@ -1,102 +0,0 @@
import asyncio
import unittest
from types import SimpleNamespace
from unittest.mock import AsyncMock, patch
from fastapi import HTTPException
from backend.app.config import settings
from backend.app.services import manual_releases as manual
from backend.app.routers import requests
from backend.app.models import Snapshot, RequestType
from backend.app.feature_access import permissions, update_permissions
from backend.app import db
from backend.tests.test_backend_quality import TempDatabaseMixin
class ManualSelectionTests(unittest.TestCase):
def setUp(self):
secret = patch.object(settings, 'jwt_secret', 'manual-selection-test-secret-123456789')
secret.start(); self.addCleanup(secret.stop)
self.user = {'username': 'viewer', 'role': 'user', 'features': {'ignore_profile_limits': True}}
self.release = {'guid': 'release', 'indexerId': 7, 'title': 'Example', 'requiresOverride': True,
'rejections': ['WEBDL-2160p is not wanted in profile']}
self.payload = {**self.release, 'ignoreProfileLimits': True,
'selectionToken': manual.issue_selection(self.release, '42', self.user, 'http://sonarr', 55)}
def test_profile_only_rejections_are_overridable(self):
for reason in ['WEBDL-2160p is not wanted in profile', 'Custom format score below minimum', 'File is larger than maximum size', 'Language is not wanted']:
self.assertTrue(manual.decision({'approved': False, 'rejections': [reason]})[1])
for reason in ['Unknown series', 'Release is blocklisted', 'No download client available', 'Already in queue']:
self.assertFalse(manual.decision({'rejections': [self.release['rejections'][0], reason]})[1])
self.assertFalse(manual.decision({'approved': True, 'downloadAllowed': False})[0])
def test_receipt_binds_request_user_source_item_and_release(self):
self.assertTrue(manual.verify_selection(self.payload, '42', self.user, 'http://sonarr', 55)['override'])
attempts = [({**self.payload, 'guid': 'other'}, '42', self.user, 'http://sonarr', 55),
(self.payload, '43', self.user, 'http://sonarr', 55),
(self.payload, '42', {**self.user, 'username': 'other'}, 'http://sonarr', 55),
(self.payload, '42', self.user, 'http://other', 55),
(self.payload, '42', self.user, 'http://sonarr', 56),
({**self.payload, 'selectionToken': 'forged'}, '42', self.user, 'http://sonarr', 55)]
for args in attempts:
with self.assertRaises(HTTPException): manual.verify_selection(*args)
def test_permission_revocation_and_literal_confirmation_enforced(self):
for payload, user, code in [(self.payload, {**self.user, 'features': {}}, 403),
({**self.payload, 'ignoreProfileLimits': 'true'}, self.user, 400)]:
with self.assertRaises(HTTPException) as error:
manual.verify_selection(payload, '42', user, 'http://sonarr', 55)
self.assertEqual(error.exception.status_code, code)
class ManualPermissionTests(TempDatabaseMixin, unittest.TestCase):
def test_default_off_individual_and_bulk(self):
for name in ('one', 'two'): db.create_user(name, 'Password123!', role='user')
one, two = [db.get_user_by_username(n) for n in ('one', 'two')]
self.assertFalse(permissions(one)['ignore_profile_limits'])
update_permissions({'ignore_profile_limits': True}, 'one')
self.assertTrue(permissions(one)['ignore_profile_limits'])
self.assertFalse(permissions(two)['ignore_profile_limits'])
update_permissions({'ignore_profile_limits': False})
self.assertFalse(permissions(one)['ignore_profile_limits'])
class ManualEpisodeSearchTests(unittest.IsolatedAsyncioTestCase):
def setUp(self):
access = patch.object(
requests,
'_ensure_request_mutation_access',
new=AsyncMock(return_value=None),
)
access.start()
self.addCleanup(access.stop)
async def test_episode_batch_is_bounded_and_exposes_next_page(self):
episodes = [{'id': i, 'seasonNumber': 1, 'monitored': True, 'hasFile': False} for i in range(1, 26)]
episodes += [{'id': 26, 'seasonNumber': 1, 'monitored': True, 'hasFile': True}]
active = peak = 0
async def search(identity):
nonlocal active, peak
active += 1; peak = max(peak, active)
await asyncio.sleep(0.001); active -= 1
return []
sonarr = SimpleNamespace(configured=lambda: True, get_episodes=AsyncMock(return_value=episodes), search_episode_releases=AsyncMock(side_effect=search))
runtime = SimpleNamespace(jellyseerr_base_url=None,jellyseerr_api_key=None,sonarr_base_url='http://sonarr',sonarr_api_key='test')
snapshot = Snapshot(request_id='42',request_type=RequestType.tv,title='Example',raw={'arr':{'item':{'id':55,'qualityProfileId':9}}})
with patch.object(requests, 'get_runtime_settings', return_value=runtime), patch.object(requests, 'build_snapshot', new=AsyncMock(return_value=snapshot)), patch.object(requests,'SonarrClient',return_value=sonarr), patch.object(requests,'save_action'):
first = await requests.action_search('42', {'username':'viewer','role':'user'})
second = await requests.action_search('42', {'username':'viewer','role':'user'}, offset=24)
self.assertEqual(first['nextOffset'],3); self.assertIsNone(second['nextOffset'])
self.assertEqual(sonarr.search_episode_releases.await_count,4)
self.assertLessEqual(peak,3)
self.assertEqual(first['totalMissingEpisodes'],25)
async def test_auto_search_preserves_current_profile(self):
for kind, service in [(RequestType.tv,'SonarrClient'),(RequestType.movie,'RadarrClient')]:
client=SimpleNamespace(configured=lambda:True, update_series=AsyncMock(), update_movie=AsyncMock(),
get_episodes=AsyncMock(return_value=[{'id':1,'seasonNumber':1,'monitored':True,'hasFile':False}]),
search_episodes=AsyncMock(return_value={'id':1}), search=AsyncMock(return_value={'id':1}))
runtime=SimpleNamespace(jellyseerr_base_url=None,jellyseerr_api_key=None,sonarr_base_url='http://sonarr',sonarr_api_key='test',radarr_base_url='http://radarr',radarr_api_key='test',sonarr_quality_profile_id=6,radarr_quality_profile_id=6)
snapshot=Snapshot(request_id='42',request_type=kind,title='Example',raw={'arr':{'item':{'id':55,'qualityProfileId':9}}})
with patch.object(requests,'get_runtime_settings',return_value=runtime),patch.object(requests,'build_snapshot',new=AsyncMock(return_value=snapshot)),patch.object(requests,service,return_value=client),patch.object(requests,'save_action'),patch.object(requests,'series_search_outcome',new=AsyncMock(return_value={'status':'attention','message':'Nothing queued'})),patch.object(requests,'movie_search_outcome',new=AsyncMock(return_value={'status':'attention','message':'Nothing queued'})):
await requests.action_search_auto('42',{'username':'admin','role':'admin'})
client.update_series.assert_not_awaited(); client.update_movie.assert_not_awaited()
-28
View File
@@ -91,9 +91,6 @@ class NewsletterConsentTests(NewsletterFixture, unittest.IsolatedAsyncioTestCase
result = await service.subscribe(self.user)
self.assertEqual(result['state'], 'pending')
rendered = sender.call_args.args[1]
self.assertEqual(rendered['subject'], 'Confirm your Magent newsletter subscription')
self.assertIn('NEW IN YOUR LIBRARY', rendered['body_html'])
self.assertNotIn('grizzlyflix', rendered['body_html'].lower())
self.assertNotIn('Arrival', rendered['body_html'])
url = re.search(r'https://[^\s]+', rendered['body_text']).group(0)
self.assertEqual(urlsplit(url).path, '/newsletter-subscription')
@@ -242,7 +239,6 @@ class NewsletterEditionTests(NewsletterFixture, unittest.TestCase):
store.complete_weekly(claimed, content(), END+timedelta(hours=1))
store.enqueue_due((END+timedelta(hours=1)).timestamp())
self.assertEqual(len(store.overview()['editions']), 1)
self.assertEqual(store.overview()['editions'][0]['subject'], 'Whats new in your library · 11 Sep 2026')
self.assertEqual(store.overview()['total'], 0)
self.assertEqual(store.settings()['next_send_at'], (END+timedelta(days=7)).timestamp())
self.assertTrue(config['enabled'])
@@ -384,30 +380,6 @@ class NewsletterCatalogTests(unittest.IsolatedAsyncioTestCase):
class NewsletterDeliveryTests(NewsletterFixture, unittest.IsolatedAsyncioTestCase):
async def test_new_draft_has_generic_subject_and_preserves_custom_intro(self):
custom_intro = 'News from our own media community.'
store.save_settings({**self.config, 'intro': custom_intro}, datetime.now(timezone.utc))
with patch.object(service, 'collect', new=AsyncMock(return_value=content())):
draft = await service.create_draft(self.user, 7)
self.assertTrue(draft['subject'].startswith('Whats new in your library · '))
self.assertEqual(draft['intro'], custom_intro)
def test_generic_email_template_preserves_custom_subject_and_intro(self):
custom_subject = 'Grizzlyflix weekend discoveries'
custom_intro = 'Welcome to our own <media> community.'
rendered = template.render(
{**content(), 'subject': custom_subject, 'intro': custom_intro}, {},
self.config['public_url'], self.runtime.jellyfin_public_url,
'https://beta.example.test/profile#newsletters',
)
self.assertEqual(rendered['subject'], custom_subject)
self.assertIn(custom_subject, rendered['body_text'])
self.assertIn(custom_intro, rendered['body_text'])
self.assertIn('Welcome to our own &lt;media&gt; community.', rendered['body_html'])
self.assertIn('Watch on Jellyfin', rendered['body_html'])
self.assertIn('Explore Jellyfin', rendered['body_html'])
self.assertNotIn('grizzlyflix', rendered['body_html'].lower())
async def test_weekly_worker_collects_once_and_delivers_to_confirmed_subscriber(self):
now = datetime.now(timezone.utc)
self.subscribe((now-timedelta(days=14)).timestamp())
-57
View File
@@ -1,57 +0,0 @@
import unittest
from datetime import datetime, timedelta, timezone
from types import SimpleNamespace
from unittest.mock import AsyncMock, patch
from backend.app.services import public_urls, newsletter_store, recap_store, newsletters, newsletter_catalog
from backend.tests.test_newsletters import NewsletterFixture
class PublicUrlTests(unittest.TestCase):
def resolve(self, application=None, proxy=None, enabled=False, legacy='https://legacy.test'):
with patch.object(public_urls,'get_runtime_settings',return_value=SimpleNamespace(
magent_application_url=application,magent_proxy_base_url=proxy,magent_proxy_enabled=enabled)):
return public_urls.magent_public_url(legacy)
def test_hosting_is_authoritative_with_proxy_and_path_support(self):
self.assertEqual(self.resolve('https://prod.test/'),'https://prod.test')
self.assertEqual(self.resolve('http://internal:3000','https://public.test/magent/',True),'https://public.test/magent')
self.assertEqual(self.resolve('https://prod.test','https://old-proxy.test',False),'https://prod.test')
self.assertEqual(self.resolve(),'https://legacy.test')
def test_invalid_configured_address_does_not_use_stale_legacy(self):
for value in ['javascript:alert(1)','https://user:password@host.test','https://host.test?key=secret','https://host.test/#fragment','https://host.test:99999','https://host.test/ bad']:
self.assertEqual(self.resolve(value),'')
class NewsletterHostingTests(NewsletterFixture, unittest.IsolatedAsyncioTestCase):
async def test_existing_draft_previews_using_hosting_without_duplicate_url(self):
draft=self.draft()
with newsletter_store.transaction() as c:
c.execute("UPDATE newsletter_settings SET public_url=''")
self.runtime.magent_application_url='https://prod.example.test'
with patch.object(public_urls,'get_runtime_settings',return_value=self.runtime),patch.object(newsletter_catalog,'posters',new=AsyncMock(return_value={})):
rendered=await newsletters.preview(draft['id'],draft['revision'])
self.assertIn('https://prod.example.test/profile#newsletters',rendered['body_html'])
self.assertIn('https://watch.example.test',rendered['body_html'])
self.assertNotIn('https://beta.example.test',rendered['body_html'])
self.assertEqual(recap_store.settings()['public_url'],'https://prod.example.test')
def test_scheduled_delivery_uses_current_hosting_address(self):
self.subscribe(when=100)
draft=self.draft()
newsletter_store.publish(draft['id'],draft['revision'],200,150)
self.runtime.magent_application_url='https://prod.example.test'
with patch.object(public_urls,'get_runtime_settings',return_value=self.runtime):
newsletter_store.enqueue_due(201)
delivery=newsletter_store.read_one('SELECT * FROM newsletter_deliveries WHERE edition_id=?',(draft['id'],))
self.assertEqual(delivery['public_url'],'https://prod.example.test')
self.runtime.magent_application_url='https://new.example.test'
self.assertEqual(newsletter_store.settings()['public_url'],'https://new.example.test')
def test_saving_schedule_uses_hosting_instead_of_client_address(self):
self.runtime.magent_application_url='https://prod.example.test'
with patch.object(public_urls,'get_runtime_settings',return_value=self.runtime):
result=newsletter_store.save_settings({**self.config,'public_url':'https://stale.example.test'},datetime.now(timezone.utc))
self.assertEqual(result['public_url'],'https://prod.example.test')
result=recap_store.save_settings({'enabled':False,'day':2,'hour':9,'public_url':''},datetime.now(timezone.utc))
self.assertEqual(result['public_url'],'https://prod.example.test')
-65
View File
@@ -1,65 +0,0 @@
import unittest
from types import SimpleNamespace
from unittest.mock import AsyncMock, AsyncMock, patch
from contextlib import ExitStack
from backend.app.routers import requests
class RecentStageTests(unittest.IsolatedAsyncioTestCase):
async def test_displayed_stage_controls_filter_and_pagination(self):
runtime = SimpleNamespace(jellyseerr_base_url='', jellyseerr_api_key='',
requests_data_source='prefer_cache', artwork_cache_mode='remote',
jellyfin_base_url='', jellyfin_api_key='')
rows = [dict(request_id=i, title=str(i), status=status, media_type='movie',
requested_by_id=10) for i, status in [(1,5),(2,5),(3,4),(4,6),(5,5),(6,2),(7,1),(8,3)]]
async def available(client, title, *args): return title in {'1','3','4','5'}
with ExitStack() as stack:
for name, value in [('get_runtime_settings', runtime), ('get_request_stage_cache', {1:{'ready':True},3:{'ready':True},4:{'ready':True},5:{'ready':True}}), ('_recent_cache_stale', False),
('active_repair_request_ids', {'5'}), ('get_request_cache_payload', None)]:
stack.enter_context(patch.object(requests, name, return_value=value))
stack.enter_context(patch.dict(requests._recent_cache, {'items':rows}))
stack.enter_context(patch.object(requests, '_request_is_available_in_jellyfin', new=AsyncMock(side_effect=AssertionError('Recent requests must not call Jellyfin'))))
user={'role':'user','username':'viewer','jellyseerr_user_id':10}
expected={'working':[2,5], 'ready':[1,3], 'partial':[4], 'approved':[6],
'pending':[7], 'declined':[8], 'in_progress':[2,4,5,6]}
for stage, ids in expected.items():
result=await requests.recent_requests(take=20,skip=0,days=0,stage=stage,user=user)
self.assertEqual([r['id'] for r in result['results']],ids,stage)
result=await requests.recent_requests(take=1,skip=1,days=0,stage='working',user=user)
self.assertEqual([r['id'] for r in result['results']],[5])
result=await requests.recent_requests(take=1,skip=0,days=0,stage='ready',user=user)
self.assertEqual(result['results'][0]['status'],4)
result=await requests.recent_requests(take=20,skip=0,days=0,stage='all',user={**user,'jellyseerr_user_id':99})
self.assertEqual(result['results'],[])
async def test_background_refresh_skips_fresh_rows_and_preserves_failures(self):
import time
rows=[{'request_id':i,'status':5,'title':str(i),'updated_at':'v1'} for i in [1,2,3]]
runtime=SimpleNamespace(jellyfin_base_url='http://jellyfin',jellyfin_api_key='test',requests_stage_refresh_minutes=15)
with ExitStack() as stack:
stack.enter_context(patch.object(requests,'get_runtime_settings',return_value=runtime))
stack.enter_context(patch.object(requests,'get_cached_requests_since',return_value=rows))
stack.enter_context(patch.object(requests,'get_request_stage_cache',return_value={1:{'source_updated':'v1','checked_at':time.time()}}))
stack.enter_context(patch.object(requests,'get_request_cache_payload',return_value={}))
check=stack.enter_context(patch.object(requests,'_request_is_available_in_jellyfin',new=AsyncMock(side_effect=[True,RuntimeError('offline')])) )
save=stack.enter_context(patch.object(requests,'save_request_stage_cache'))
await requests.refresh_local_request_stages()
self.assertEqual(check.await_count,2)
written=save.call_args.args[0]
self.assertEqual(len(written),1)
self.assertEqual(written[0][:3],(2,'v1',1))
from backend.tests.test_backend_quality import TempDatabaseMixin
from backend.app import db
class StagePersistenceTests(TempDatabaseMixin, unittest.TestCase):
def test_saved_stages_survive_initialization_and_actions_mark_due(self):
with db._connect() as conn:
conn.execute("INSERT INTO requests_cache (request_id, payload_json) VALUES (42, '{}')")
db.save_request_stage_cache([(42,'v1',1,12345)])
db.init_db()
self.assertTrue(db.get_request_stage_cache()[42]['ready'])
db.save_action('42','search_releases','Search','ok')
self.assertEqual(db.get_request_stage_cache()[42]['checked_at'],0)
self.assertTrue(db.get_request_stage_cache()[42]['ready'])
-27
View File
@@ -1,27 +0,0 @@
import unittest
from types import SimpleNamespace
from unittest.mock import AsyncMock, patch
from backend.app.routers import requests
from backend.app.models import Snapshot, RequestType
class RecheckMonitoringTests(unittest.IsolatedAsyncioTestCase):
async def test_series_restores_only_requested_seasons_and_episodes(self):
original={'id':10,'monitored':False,'qualityProfileId':7,'seasons':[{'seasonNumber':1,'monitored':False},{'seasonNumber':2,'monitored':False}]}
restored={**original,'monitored':True,'seasons':[{'seasonNumber':1,'monitored':True},{'seasonNumber':2,'monitored':False}]}
client=SimpleNamespace(get_series=AsyncMock(side_effect=[original,restored]),update_series=AsyncMock(),get_episodes=AsyncMock(side_effect=[[{'id':1,'seasonNumber':1,'monitored':False},{'id':2,'seasonNumber':2,'monitored':False}],[{'id':1,'seasonNumber':1,'monitored':True},{'id':2,'seasonNumber':2,'monitored':False}]]),monitor_episodes=AsyncMock())
snapshot=Snapshot(request_id='42',title='Test',request_type=RequestType.tv,raw={'arr':{'item':{'id':10}}})
runtime=SimpleNamespace(sonarr_base_url='http://sonarr',sonarr_api_key='test')
with patch.object(requests,'get_runtime_settings',return_value=runtime),patch.object(requests,'SonarrClient',return_value=client):
self.assertTrue(await requests._restore_request_monitoring(snapshot,{'status':2,'seasons':[{'seasonNumber':1}]}))
client.update_series.assert_awaited_once_with(restored)
client.monitor_episodes.assert_awaited_once_with([1],True)
async def test_movie_monitoring_preserves_profile_and_pending_is_noop(self):
movie={'id':10,'monitored':False,'qualityProfileId':7}
client=SimpleNamespace(get_movie=AsyncMock(side_effect=[movie,{**movie,'monitored':True}]),update_movie=AsyncMock())
snapshot=Snapshot(request_id='42',title='Test',request_type=RequestType.movie,raw={'arr':{'item':{'id':10}}})
runtime=SimpleNamespace(radarr_base_url='http://radarr',radarr_api_key='test')
with patch.object(requests,'get_runtime_settings',return_value=runtime),patch.object(requests,'RadarrClient',return_value=client):
self.assertFalse(await requests._restore_request_monitoring(snapshot,{'status':1}))
self.assertTrue(await requests._restore_request_monitoring(snapshot,{'status':2}))
client.update_movie.assert_awaited_once_with({**movie,'monitored':True})
-34
View File
@@ -1,34 +0,0 @@
import unittest
from datetime import timedelta
from unittest.mock import AsyncMock, patch
from backend.tests.test_insights import play, NOW, LIBRARIES
from backend.tests.test_email_recaps import fixture_report
from backend.app.services.insights import summarize
from backend.app.services.email_recaps import illustrated_recap
class ReportGraphicsTests(unittest.IsolatedAsyncioTestCase):
def test_patterns_deduplicate_and_handle_empty_history(self):
first = play()
second = play("second", PlaybackDuration=1800, EpisodeId="episode", ActivityDateInserted=(NOW-timedelta(days=1)).isoformat())
report = summarize([first, first, second], LIBRARIES, NOW-timedelta(days=7), NOW)
self.assertEqual(report["patterns"]["average_play_minutes"], 45)
self.assertEqual(report["patterns"]["longest_play_minutes"], 60)
self.assertEqual(report["patterns"]["weekend_percent"], 33.3)
self.assertEqual(sum(r["minutes"] for r in report["patterns"]["media"]), 90)
empty = summarize([], [], NOW-timedelta(days=7), NOW)
self.assertEqual(empty["patterns"]["average_play_minutes"], 0)
async def test_artwork_embedded_without_private_links_and_optional_on_failure(self):
report = fixture_report()
report["top_titles"][0]["artwork_url"] = "/insights/artwork/" + "a"*32 + "?token=123." + "b"*64
with patch("backend.app.services.email_recaps.get_runtime_settings"), patch("backend.app.services.insights_artwork.get_artwork", new=AsyncMock(return_value=(b"picture", "image/webp"))):
rendered = await illustrated_recap(report, {"username":"viewer"}, "https://example.test", "https://example.test/unsubscribe")
self.assertIn("cid:recap-title-0@magent", rendered["body_html"])
self.assertNotIn("?token=", rendered["body_html"])
self.assertEqual(rendered["inline_images"][0]["subtype"], "webp")
preview = await illustrated_recap(report, {"username":"viewer"}, "https://example.test", "https://example.test/unsubscribe", preview=True)
self.assertIn("data:image/webp;base64,", preview["body_html"])
self.assertNotIn("inline_images", preview)
with patch("backend.app.services.email_recaps.get_runtime_settings"), patch("backend.app.services.insights_artwork.get_artwork", new=AsyncMock(side_effect=RuntimeError())):
rendered = await illustrated_recap(report, {"username":"viewer"}, "https://example.test", "https://example.test/unsubscribe")
self.assertEqual(rendered["inline_images"], [])
+1 -70
View File
@@ -4,7 +4,7 @@ from types import SimpleNamespace
from unittest.mock import AsyncMock, patch
from fastapi import HTTPException
from backend.app.services.request_language import language_info, original_profile, is_original_profile, apply_original_to_movie, movie_search_outcome
from backend.app.services.request_language import language_info, original_profile, is_original_profile
from backend.app.routers import requests
@@ -54,7 +54,6 @@ class RequestLanguageTests(unittest.IsolatedAsyncioTestCase):
with patch.object(requests, 'get_runtime_settings', return_value=runtime), \
patch.object(requests, 'JellyseerrClient', return_value=seerr), \
patch.object(requests, '_resolve_request_destination', new=AsyncMock(return_value={'server_id': 1, 'profile_id': 6, 'root_folder': '/media'})), \
patch.object(requests, 'apply_original_to_movie', new=AsyncMock(return_value=None)), \
patch.object(requests, 'original_profile', new=AsyncMock(return_value=20)) as clone:
payload = {'mediaType': media_type, 'tmdbId': 1417, 'acceptOriginalLanguage': consent, 'seasons': [1]}
if expected is None:
@@ -66,71 +65,3 @@ class RequestLanguageTests(unittest.IsolatedAsyncioTestCase):
await requests.create_request(payload, {'username': 'viewer'})
self.assertEqual(seerr.create_request.await_args.kwargs['profile_id'], expected)
self.assertEqual(clone.await_count, int(expected == 20))
async def test_existing_radarr_movie_is_updated_and_read_back(self):
movie = {'id': 6940, 'tmdbId': 613, 'qualityProfileId': 9, 'monitored': True}
client = SimpleNamespace(get_movie_by_tmdb_id=AsyncMock(return_value=[movie]),
update_movie=AsyncMock(), get_movie=AsyncMock(return_value={**movie, 'qualityProfileId': 20}))
with patch('backend.app.services.request_language.original_profile', new=AsyncMock(return_value=20)):
self.assertEqual(await apply_original_to_movie(client, 613), 20)
self.assertEqual(client.update_movie.await_args.args[0]['qualityProfileId'], 20)
self.assertTrue(client.update_movie.await_args.args[0]['monitored'])
async def test_failed_profile_verification_does_not_claim_success(self):
client = SimpleNamespace(get_movie_by_tmdb_id=AsyncMock(return_value=[{'id': 6940, 'tmdbId': 613, 'qualityProfileId': 9}]),
update_movie=AsyncMock(), get_movie=AsyncMock(return_value={'qualityProfileId': 9}))
with patch('backend.app.services.request_language.original_profile', new=AsyncMock(return_value=20)):
with self.assertRaises(HTTPException):
await apply_original_to_movie(client, 613)
async def test_search_reports_real_outcomes(self):
for command_status, queue, expected in [('completed', [], 'pending'), ('failed', [], 'attention'),
('started', [], 'pending'), ('completed', [{'movieId': 6940}], 'downloading')]:
client = SimpleNamespace(get=AsyncMock(return_value={'status': command_status}),
get_queue=AsyncMock(return_value={'records': queue}), get_movie=AsyncMock(return_value={'hasFile': False}))
result = await movie_search_outcome(client, 6940, {'id': 1}, attempts=1, delay=0)
self.assertEqual(result['status'], expected)
async def test_language_endpoint_checks_consent_identity_and_access(self):
for payload in ({}, {'acceptOriginalLanguage': 'true'}, {'acceptOriginalLanguage': True, 'languageCode': 'es'}):
with patch.object(requests, '_request_language_context', new=AsyncMock(return_value=(SimpleNamespace(), 613, {'code': 'de'}))), \
patch.object(requests, 'apply_original_to_movie', new=AsyncMock()) as apply:
with self.assertRaises(HTTPException):
await requests.accept_request_language('3976', payload, {'role': 'admin'})
apply.assert_not_awaited()
with self.assertRaises(HTTPException):
await requests.accept_request_language('3976', {'acceptOriginalLanguage': True}, {'role': 'user', 'auto_search_enabled': False})
async def test_radarr_queue_filters_before_pagination(self):
from backend.app.clients.radarr import RadarrClient
client = RadarrClient('http://radarr.test', 'test')
with patch.object(client, 'get', new=AsyncMock(return_value={'records': []})) as get:
await client.get_queue(6940)
get.assert_awaited_once_with('/api/v3/queue', params={'movieIds': 6940, 'pageSize': 1000})
async def test_tv_search_distinguishes_no_download_and_queue(self):
from backend.app.services.request_language import series_search_outcome
client = SimpleNamespace(get=AsyncMock(return_value={'status': 'completed'}), get_queue=AsyncMock(return_value={'records': []}))
self.assertEqual((await series_search_outcome(client, 50, [{'id': 1}], attempts=1))['status'], 'pending')
client.get_queue.return_value = {'records': [{'seriesId': 50}]}
self.assertEqual((await series_search_outcome(client, 50, [{'id': 1}], attempts=1))['status'], 'downloading')
class SearchHandoffTests(unittest.IsolatedAsyncioTestCase):
async def test_radarr_completed_before_queue_refresh(self):
client = SimpleNamespace(get=AsyncMock(return_value={'status':'completed'}),
get_queue=AsyncMock(side_effect=[{'records':[]}, {'records':[]}, {'records':[{'movieId':2206}]}]),
get_movie=AsyncMock(return_value={'hasFile':False}))
result = await movie_search_outcome(client, 2206, {'id':1}, attempts=3, delay=0)
self.assertEqual(result['status'], 'downloading')
self.assertEqual(client.get_queue.await_count, 3)
async def test_sonarr_completed_before_queue_refresh(self):
from backend.app.services.request_language import series_search_outcome
client = SimpleNamespace(get=AsyncMock(return_value={'status':'completed'}),
get_queue=AsyncMock(side_effect=[{'records':[{'seriesId':999}]}, {'records':[{'seriesId':50}]}]))
result = await series_search_outcome(client, 50, [{'id':1}], attempts=2, delay=0)
self.assertEqual(result['status'], 'downloading')
-187
View File
@@ -1,187 +0,0 @@
"""Origin checks use operator configuration, never caller-controlled routing headers."""
from pathlib import Path
import tempfile
from types import SimpleNamespace
import unittest
from unittest.mock import AsyncMock, patch
from fastapi.testclient import TestClient
from backend.app import db, main
from backend.app.config import settings
from backend.app.routers import auth as auth_router
from backend.app.services import public_urls
from backend.app.services.request_origins import is_allowed_request_origin
PUBLIC_ORIGIN = "https://watch.example.test"
LOCAL_ORIGIN = "http://localhost:3000"
class RequestOriginTests(unittest.TestCase):
def setUp(self):
self.runtime = SimpleNamespace(
magent_proxy_enabled=False,
magent_proxy_base_url=None,
magent_application_url=PUBLIC_ORIGIN,
)
self.enterContext(patch.object(settings, "cors_allow_origin", LOCAL_ORIGIN))
self.enterContext(patch.object(public_urls, "get_runtime_settings", return_value=self.runtime))
def test_explicit_cors_and_configured_public_url_are_both_allowed(self):
self.assertTrue(is_allowed_request_origin(LOCAL_ORIGIN))
self.assertTrue(is_allowed_request_origin(PUBLIC_ORIGIN))
self.assertFalse(is_allowed_request_origin("https://unrelated.example.test"))
def test_scheme_hostname_case_and_default_ports_are_canonicalized(self):
for origin in (PUBLIC_ORIGIN, "HTTPS://WATCH.EXAMPLE.TEST", "https://watch.example.test:443"):
with self.subTest(origin=origin):
self.assertTrue(is_allowed_request_origin(origin))
self.runtime.magent_application_url = "http://watch.example.test:80"
self.assertTrue(is_allowed_request_origin("http://WATCH.example.test"))
self.assertFalse(is_allowed_request_origin("https://watch.example.test"))
self.assertFalse(is_allowed_request_origin("http://watch.example.test:8080"))
def test_nondefault_ports_must_match(self):
self.runtime.magent_application_url = "https://watch.example.test:8443/magent"
self.assertTrue(is_allowed_request_origin("https://watch.example.test:8443"))
self.assertFalse(is_allowed_request_origin("https://watch.example.test"))
self.assertFalse(is_allowed_request_origin("https://watch.example.test:443"))
def test_configured_subpath_does_not_become_part_of_origin(self):
self.runtime.magent_application_url = PUBLIC_ORIGIN + "/magent/"
self.assertTrue(is_allowed_request_origin(PUBLIC_ORIGIN))
self.assertFalse(is_allowed_request_origin(PUBLIC_ORIGIN + "/magent"))
def test_enabled_proxy_uses_configured_proxy_public_url(self):
self.runtime.magent_proxy_enabled = True
self.runtime.magent_proxy_base_url = "https://proxy.example.test/magent"
self.assertTrue(is_allowed_request_origin("https://proxy.example.test"))
self.assertTrue(is_allowed_request_origin(LOCAL_ORIGIN))
self.assertFalse(is_allowed_request_origin(PUBLIC_ORIGIN))
def test_unconfigured_public_url_only_allows_explicit_cors(self):
self.runtime.magent_application_url = None
self.assertTrue(is_allowed_request_origin(LOCAL_ORIGIN))
self.assertFalse(is_allowed_request_origin(PUBLIC_ORIGIN))
def test_invalid_or_non_origin_inputs_are_rejected(self):
for origin in (
"", "null", "*", "watch.example.test", "//watch.example.test",
"ftp://watch.example.test", "javascript:alert(1)",
PUBLIC_ORIGIN + "/", PUBLIC_ORIGIN + "/path",
PUBLIC_ORIGIN + "?query=true", PUBLIC_ORIGIN + "#fragment",
PUBLIC_ORIGIN + "?", PUBLIC_ORIGIN + "#",
"https://user@watch.example.test", "https://user:password@watch.example.test",
"https://watch.example.test@evil.example.test", "https://watch.example.test.evil.example.test",
"https://watch.example.test:0", "https://watch.example.test:65536",
"https://watch.example.test:invalid", "https://[invalid",
PUBLIC_ORIGIN + " https://evil.example.test", PUBLIC_ORIGIN + ",https://evil.example.test",
"https://watch.example.test\\@evil.example.test", PUBLIC_ORIGIN + "\n",
):
with self.subTest(origin=repr(origin)):
self.assertFalse(is_allowed_request_origin(origin))
def test_invalid_configured_public_url_does_not_authorize_an_origin(self):
for configured in (
"https://user:password@watch.example.test", PUBLIC_ORIGIN + "?token=private",
PUBLIC_ORIGIN + "#fragment", "javascript:alert(1)", "https://watch.example.test:65536",
):
with self.subTest(configured=configured):
self.runtime.magent_application_url = configured
self.assertFalse(is_allowed_request_origin(PUBLIC_ORIGIN))
self.assertTrue(is_allowed_request_origin(LOCAL_ORIGIN))
class RequestOriginHttpTests(unittest.TestCase):
def setUp(self):
temporary = tempfile.TemporaryDirectory(ignore_cleanup_errors=True)
self.addCleanup(temporary.cleanup)
self.runtime = SimpleNamespace(
magent_proxy_enabled=False,
magent_proxy_base_url=None,
magent_application_url=PUBLIC_ORIGIN,
)
for name, value in {
"sqlite_path": str(Path(temporary.name) / "origin-tests.db"),
"sqlite_journal_mode": "DELETE",
"jwt_secret": "request-origin-tests-jwt-secret-at-least-32-characters",
"settings_encryption_key": None,
"admin_username": "unused-environment-admin",
"admin_password": "",
"cors_allow_origin": LOCAL_ORIGIN,
"auth_cookie_domain": None,
"auth_cookie_secure": True,
}.items():
self.enterContext(patch.object(settings, name, value))
self.enterContext(patch.object(public_urls, "get_runtime_settings", return_value=self.runtime))
# Constructing without a context deliberately skips production startup:
# no migrations/workers/listeners/log files outside this temporary DB.
db.init_db()
self.client = TestClient(main.app, base_url=PUBLIC_ORIGIN)
self.addCleanup(self.client.close)
def test_public_origin_reaches_both_auth_handlers_with_localhost_cors_default(self):
for path in ("/auth/login", "/auth/jellyfin/login"):
with self.subTest(path=path):
response = self.client.post(path, data={}, headers={"Origin": PUBLIC_ORIGIN})
self.assertEqual(response.status_code, 422, response.text)
self.assertNotEqual(response.json().get("detail"), "Cross-origin state change rejected")
def test_explicit_cors_origin_remains_allowed(self):
response = self.client.post("/auth/login", data={}, headers={"Origin": LOCAL_ORIGIN})
self.assertEqual(response.status_code, 422, response.text)
def test_no_origin_keeps_existing_nonbrowser_behavior(self):
response = self.client.post("/auth/login", data={})
self.assertEqual(response.status_code, 422, response.text)
def test_caller_controlled_host_forwarding_and_fetch_headers_cannot_authorize_evil_origin(self):
for path in ("/auth/login", "/auth/jellyfin/login"):
for routing_headers in (
{},
{"Host": "evil.example.test"},
{"X-Forwarded-Host": "evil.example.test", "X-Forwarded-Proto": "https"},
{"Host": "evil.example.test", "X-Forwarded-Host": "evil.example.test", "Sec-Fetch-Site": "same-origin"},
{"Host": "watch.example.test", "X-Forwarded-Host": "watch.example.test", "Sec-Fetch-Site": "same-origin"},
):
with self.subTest(path=path, routing_headers=routing_headers):
response = self.client.post(path, data={}, headers={"Origin": "https://evil.example.test", **routing_headers})
self.assertEqual(response.status_code, 403, response.text)
self.assertEqual(response.json()["detail"], "Cross-origin state change rejected")
def test_null_path_query_and_userinfo_origins_are_rejected_before_login(self):
for origin in ("null", PUBLIC_ORIGIN + "/", PUBLIC_ORIGIN + "/path", PUBLIC_ORIGIN + "?query=1", "https://user@watch.example.test"):
with self.subTest(origin=origin):
response = self.client.post("/auth/login", data={}, headers={"Origin": origin})
self.assertEqual(response.status_code, 403, response.text)
def test_valid_local_login_works_from_configured_public_origin(self):
password = "origin-tests-valid-local-password"
db.create_user("origin-owner", password, role="admin")
response = self.client.post("/auth/login", data={"username": "origin-owner", "password": password}, headers={"Origin": PUBLIC_ORIGIN})
self.assertEqual(response.status_code, 200, response.text)
self.assertIn(settings.auth_cookie_name, self.client.cookies)
profile = self.client.get("/auth/profile")
self.assertEqual(profile.status_code, 200, profile.text)
self.assertEqual(profile.json()["user"]["username"], "origin-owner")
def test_valid_mocked_jellyfin_login_works_from_configured_public_origin(self):
jellyfin_runtime = SimpleNamespace(jellyfin_base_url="http://jellyfin.test:8096", jellyfin_api_key="test-api-key")
upstream = SimpleNamespace(
configured=lambda: True,
authenticate_by_name=AsyncMock(return_value={"User": {"Id": "test-jellyfin-id", "Name": "origin-viewer"}}),
get_users=AsyncMock(return_value=[]),
_extract_user_id=lambda _response: "test-jellyfin-id",
)
with patch.object(auth_router, "get_runtime_settings", return_value=jellyfin_runtime), patch.object(auth_router, "JellyfinClient", return_value=upstream), patch.object(auth_router, "get_cached_jellyseerr_users", return_value=[]):
response = self.client.post("/auth/jellyfin/login", data={"username": "origin-viewer", "password": "origin-tests-jellyfin-password"}, headers={"Origin": PUBLIC_ORIGIN})
self.assertEqual(response.status_code, 200, response.text)
upstream.authenticate_by_name.assert_awaited_once_with("origin-viewer", "origin-tests-jellyfin-password")
self.assertIn(settings.auth_cookie_name, self.client.cookies)
self.assertEqual(db.get_user_by_username("origin-viewer")["auth_provider"], "jellyfin")
if __name__ == "__main__":
unittest.main()
-36
View File
@@ -1,36 +0,0 @@
import sqlite3
import unittest
from backend.app.schema_migrations import run_schema_migrations
class SchemaMigrationTests(unittest.TestCase):
def setUp(self) -> None:
self.conn = sqlite3.connect(":memory:")
self.conn.execute(
"CREATE TABLE users (id INTEGER PRIMARY KEY, username TEXT NOT NULL UNIQUE, password_hash TEXT, role TEXT, created_at TEXT)"
)
self.conn.execute(
"CREATE TABLE signup_invites (id INTEGER PRIMARY KEY, code TEXT NOT NULL UNIQUE, created_at TEXT, updated_at TEXT)"
)
self.conn.execute("CREATE TABLE portal_items (id INTEGER PRIMARY KEY, kind TEXT, updated_at TEXT)")
self.conn.execute("CREATE TABLE requests_cache (request_id INTEGER PRIMARY KEY, created_at TEXT)")
def tearDown(self) -> None:
self.conn.close()
def test_migrations_are_versioned_and_idempotent(self) -> None:
self.assertEqual(run_schema_migrations(self.conn), [1])
self.assertEqual(run_schema_migrations(self.conn), [])
user_columns = {row[1] for row in self.conn.execute("PRAGMA table_info(users)")}
self.assertIn("auth_version", user_columns)
self.assertIn("email", user_columns)
request_columns = {row[1] for row in self.conn.execute("PRAGMA table_info(requests_cache)")}
self.assertIn("requested_by_id", request_columns)
applied = self.conn.execute("SELECT version, name FROM schema_migrations").fetchall()
self.assertEqual(applied, [(1, "legacy_columns_and_indexes")])
if __name__ == "__main__":
unittest.main()
-267
View File
@@ -1,267 +0,0 @@
from concurrent.futures import ThreadPoolExecutor
import os
import tempfile
from threading import Barrier
from types import SimpleNamespace
import unittest
from unittest.mock import AsyncMock, patch
from fastapi import FastAPI
from fastapi.testclient import TestClient
from backend.app import db
from backend.app.config import settings
from backend.app.routers import setup as setup_router
from backend.app.security import create_access_token
from backend.app.services import setup
SETUP_TOKEN = "operator-setup-token-for-tests-only-1234567890"
ADMIN_PASSWORD = "A-long-admin-password!123"
class SetupTests(unittest.TestCase):
def setUp(self) -> None:
self.temp = tempfile.TemporaryDirectory(ignore_cleanup_errors=True)
self.addCleanup(self.temp.cleanup)
for field, value in {
"sqlite_path": os.path.join(self.temp.name, "test.db"),
"sqlite_journal_mode": "DELETE",
"admin_username": "environment-admin",
"admin_password": "",
"jwt_secret": "setup-test-jwt-secret-only-1234567890",
"settings_encryption_key": "bWFnZW50LXNlY3VyaXR5LXRlc3Qta2V5LTMyLWJ5dGU=",
}.items():
context = patch.object(settings, field, value)
context.start()
self.addCleanup(context.stop)
context = patch.object(setup, "settings", SimpleNamespace(setup_token=SETUP_TOKEN))
context.start()
self.addCleanup(context.stop)
setup.initialize_setup_state()
db.init_db()
self.app = FastAPI()
self.app.include_router(setup_router.router)
self.client = TestClient(self.app)
self.addCleanup(self.client.close)
def bootstrap(self, **changes):
return self.client.post("/setup/bootstrap", json={
"setup_token": SETUP_TOKEN,
"username": "first-admin",
"password": ADMIN_PASSWORD,
**changes,
})
def admin_headers(self):
return {"Authorization": f"Bearer {create_access_token('first-admin', 'admin')}"}
def test_fresh_install_requires_setup_and_exposes_no_configuration(self):
response = self.client.get("/setup/status")
self.assertEqual(response.status_code, 200)
self.assertEqual(response.json(), {"setup_required": True, "needs_admin": True})
self.assertEqual(response.headers["cache-control"], "no-store")
self.assertEqual(self.client.get("/setup/state").status_code, 401)
def test_existing_install_migrates_as_completed_without_reopening_bootstrap(self):
with db._connect() as conn:
conn.execute("DROP TABLE installation_setup")
setup.initialize_setup_state()
self.assertEqual(setup.get_public_setup_status(), {"setup_required": False, "needs_admin": False})
self.assertIsNotNone(setup.get_setup_state()["completed_at"])
self.assertEqual(self.bootstrap().status_code, 409)
def test_missing_marker_fails_closed(self):
with db._connect() as conn:
conn.execute("DROP TABLE installation_setup")
self.assertFalse(setup.is_setup_required())
self.assertEqual(self.bootstrap().status_code, 409)
def test_marker_survives_restart_before_schema_initialization(self):
new_path = os.path.join(self.temp.name, "interrupted.db")
with patch.object(settings, "sqlite_path", new_path):
setup.initialize_setup_state()
setup.initialize_setup_state()
db.init_db()
self.assertTrue(setup.is_setup_required())
def test_empty_precreated_database_is_a_fresh_install(self):
new_path = os.path.join(self.temp.name, "empty.db")
with open(new_path, "wb"):
pass
with patch.object(settings, "sqlite_path", new_path):
setup.initialize_setup_state()
db.init_db()
self.assertTrue(setup.is_setup_required())
def test_environment_admin_uses_wizard_without_public_bootstrap(self):
with patch.object(settings, "admin_password", ADMIN_PASSWORD):
db.ensure_admin_user()
self.assertEqual(setup.get_public_setup_status(), {"setup_required": True, "needs_admin": False})
self.assertEqual(self.bootstrap().status_code, 409)
def test_valid_token_creates_local_admin_once_and_uses_password_hash(self):
response = self.bootstrap()
self.assertEqual(response.status_code, 201, response.text)
self.assertEqual(response.json(), {"status": "created", "username": "first-admin"})
user = db.verify_user_password("first-admin", ADMIN_PASSWORD)
self.assertIsNotNone(user)
self.assertEqual(user["role"], "admin")
self.assertEqual(user["auth_provider"], "local")
self.assertNotEqual(user["password_hash"], ADMIN_PASSWORD)
self.assertEqual(setup.get_setup_state()["step"], "apps")
self.assertEqual(self.bootstrap(username="second-admin").status_code, 409)
self.assertEqual(len(db.get_all_users()), 1)
def test_invalid_and_missing_operator_tokens_never_create_admin(self):
self.assertEqual(self.bootstrap(setup_token="incorrect").status_code, 403)
with patch.object(setup.settings, "setup_token", ""):
self.assertEqual(self.bootstrap().status_code, 403)
with patch.object(setup.settings, "setup_token", "too-short"):
self.assertEqual(self.bootstrap(setup_token="too-short").status_code, 403)
self.assertFalse(db.has_admin_user())
def test_non_ascii_token_fails_cleanly(self):
self.assertEqual(self.bootstrap(setup_token="invalid-\N{SNOWMAN}").status_code, 403)
self.assertFalse(db.has_admin_user())
def test_example_and_repeated_character_setup_tokens_are_rejected(self):
for token in (
"replace-with-a-separate-random-setup-token",
"CHANGE_ME_before_starting_this_installation",
"your-setup-token-goes-here-at-least-32-characters",
"a" * 64,
"0" * 64,
" " * 64,
):
with self.subTest(token=token), patch.object(setup.settings, "setup_token", token):
self.assertFalse(setup.setup_token_configured())
with self.assertRaises(setup.InvalidSetupTokenError):
setup.bootstrap_administrator(token, "owner", ADMIN_PASSWORD)
self.assertFalse(db.has_admin_user())
self.assertTrue(setup.setup_token_configured())
def test_password_policy_and_username_validation(self):
for username in (" ", "admin user", "admin\x7f", "admin\nname"):
with self.subTest(username=repr(username)):
self.assertEqual(self.bootstrap(username=username).status_code, 400)
self.assertEqual(self.bootstrap(password="short").status_code, 400)
self.assertFalse(db.has_admin_user())
def test_oversized_fields_and_unexpected_privileges_are_rejected(self):
self.assertEqual(self.bootstrap(password="x" * 1025).status_code, 422)
self.assertEqual(self.bootstrap(username="x" * 101).status_code, 422)
self.assertEqual(self.bootstrap(role="admin").status_code, 422)
self.assertFalse(db.has_admin_user())
def test_existing_normalized_username_is_not_replaced(self):
db.create_user("Taken", ADMIN_PASSWORD)
self.assertEqual(self.bootstrap(username="taken").status_code, 409)
self.assertFalse(db.has_admin_user())
self.assertEqual(len(db.get_all_users()), 1)
def test_bootstrap_attempts_are_persistently_limited(self):
for _ in range(setup.BOOTSTRAP_IP_ATTEMPTS):
self.assertEqual(self.bootstrap(setup_token="incorrect").status_code, 403)
setup.initialize_setup_state()
response = self.bootstrap()
self.assertEqual(response.status_code, 429)
self.assertGreater(int(response.headers["retry-after"]), 0)
self.assertFalse(db.has_admin_user())
with db._connect() as conn:
keys = [row[0] for row in conn.execute("SELECT key_hash FROM installation_setup_attempts")]
self.assertNotIn("testclient", keys)
def test_rate_limit_global_cap_and_expiry(self):
with patch.object(setup, "time", return_value=1000):
for number in range(setup.BOOTSTRAP_GLOBAL_ATTEMPTS):
self.assertIsNone(setup.consume_bootstrap_attempt(f"192.0.2.{number}"))
self.assertEqual(setup.consume_bootstrap_attempt("198.51.100.1"), 900)
with patch.object(setup, "time", return_value=1901):
self.assertIsNone(setup.consume_bootstrap_attempt("198.51.100.1"))
def test_concurrent_attempts_cannot_bypass_rate_limit(self):
with ThreadPoolExecutor(max_workers=8) as executor:
results = list(executor.map(lambda _: setup.consume_bootstrap_attempt("192.0.2.1"), range(12)))
self.assertEqual(results.count(None), setup.BOOTSTRAP_IP_ATTEMPTS)
def test_concurrent_bootstraps_create_only_one_admin(self):
barrier = Barrier(4)
def synchronized_hash(_):
barrier.wait(timeout=10)
return "test-only-precomputed-hash"
def create(number):
try:
setup.bootstrap_administrator(SETUP_TOKEN, f"admin-{number}", ADMIN_PASSWORD)
return True
except setup.SetupUnavailableError:
return False
with patch.object(setup, "hash_password", side_effect=synchronized_hash):
with ThreadPoolExecutor(max_workers=4) as executor:
results = list(executor.map(create, range(4)))
self.assertEqual(results.count(True), 1)
self.assertEqual(len(db.get_all_users()), 1)
def test_state_mutations_require_admin_and_progress_resumes(self):
self.assertEqual(self.bootstrap().status_code, 201)
db.create_user("viewer", ADMIN_PASSWORD)
user_headers = {"Authorization": f"Bearer {create_access_token('viewer', 'user')}"}
for path, method, kwargs in (
("/setup/state", "get", {}),
("/setup/state", "put", {"json": {"step": "review"}}),
("/setup/complete", "post", {}),
):
with self.subTest(path=path, method=method):
call = getattr(self.client, method)
self.assertEqual(call(path, **kwargs).status_code, 401)
self.assertEqual(call(path, headers=user_headers, **kwargs).status_code, 403)
response = self.client.put("/setup/state", json={"step": "preferences"}, headers=self.admin_headers())
self.assertEqual(response.status_code, 200)
setup.initialize_setup_state()
db.init_db()
self.assertEqual(setup.get_setup_state()["step"], "preferences")
self.assertTrue(setup.is_setup_required())
self.assertEqual(self.client.put(
"/setup/state", json={"step": "invalid"}, headers=self.admin_headers()
).status_code, 422)
def test_completion_invokes_worker_callback_and_cannot_reopen_bootstrap(self):
self.assertEqual(self.bootstrap().status_code, 201)
callback = AsyncMock()
self.app.state.on_setup_complete = callback
response = self.client.post("/setup/complete", headers=self.admin_headers())
self.assertEqual(response.status_code, 200, response.text)
self.assertTrue(response.json()["completed"])
self.assertIsNotNone(response.json()["completed_at"])
callback.assert_awaited_once()
self.assertFalse(setup.is_setup_required())
# A retry can restart an idempotent callback if the first response was
# interrupted, while keeping the original completion timestamp.
retry = self.client.post("/setup/complete", headers=self.admin_headers())
self.assertEqual(retry.json(), response.json())
self.assertEqual(callback.await_count, 2)
self.client.put("/setup/state", json={"step": "administrator"}, headers=self.admin_headers())
with db._connect() as conn:
conn.execute("DELETE FROM users")
self.assertEqual(self.bootstrap().status_code, 409)
self.assertEqual(setup.get_setup_state()["step"], "review")
def test_completion_requires_an_administrator(self):
with self.assertRaises(setup.SetupUnavailableError):
setup.complete_setup()
self.assertTrue(setup.is_setup_required())
def test_sync_callback_is_supported(self):
self.assertEqual(self.bootstrap().status_code, 201)
called = []
self.app.state.on_setup_complete = lambda: called.append(True)
response = self.client.post("/setup/complete", headers=self.admin_headers())
self.assertEqual(response.status_code, 200)
self.assertEqual(called, [True])
if __name__ == "__main__":
unittest.main()
-6
View File
@@ -1,6 +0,0 @@
# Optional source build; retain compose.yml's storage and security defaults.
# docker compose -f compose.yml -f compose.build.yml -p magent up -d --build
services:
magent:
image: magent:local
build: .
-29
View File
@@ -1,29 +0,0 @@
# Fresh installs: use Docker Compose CLI or a Portainer Docker Standalone stack.
# CLI: docker compose -f compose.yml -p magent up -d
# Instructions: docs/DOCKER.md or docs/PORTAINER.md.
# Configure the site address and connected apps in Magent's setup wizard.
# No Dockerfile, source checkout, .env file or shared default password is needed.
# Existing installations must keep their original data mount and keys.
services:
magent:
image: rephl3xnz/magent:latest
ports:
# LAN access by default. Restrict with a firewall; use HTTPS for public use.
# Only the frontend is published; it also serves /api.
- "3000:3000"
volumes:
# Contains the database, settings, cache and private generated keys.
- magent-data:/app/data
restart: unless-stopped
stop_grace_period: 30s
# For security, leave these settings unchanged unless you understand the risks.
read_only: true
cap_drop: ["ALL"]
security_opt: ["no-new-privileges:true"]
init: true
tmpfs:
- /tmp:rw,noexec,nosuid,size=64m,uid=1000,gid=1000
- /app/frontend/.next/cache:rw,noexec,nosuid,size=128m,uid=1000,gid=1000
volumes:
magent-data:
-13
View File
@@ -1,13 +0,0 @@
# Copy to /etc/magent/backend.env, owned by root with mode 0600.
# Generate independent secrets; these placeholders must never be deployed.
JWT_SECRET=REPLACE_WITH_INDEPENDENT_RANDOM_TOKEN
SETTINGS_ENCRYPTION_KEY=REPLACE_WITH_FERNET_KEY
SETUP_TOKEN=REPLACE_WITH_ANOTHER_RANDOM_TOKEN
MAGENT_MANAGED_SECRETS=false
MAGENT_APPLICATION_URL=https://magent.example.com
CORS_ALLOW_ORIGIN=https://magent.example.com
AUTH_COOKIE_SECURE=true
API_DOCS_ENABLED=false
SQLITE_PATH=/var/lib/magent/data/magent.db
LOG_FILE=/var/lib/magent/data/magent.log
BRANDING_SOURCE=data
-26
View File
@@ -1,26 +0,0 @@
[Unit]
Description=Magent API
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
User=magent
Group=magent
WorkingDirectory=/var/lib/magent
EnvironmentFile=/etc/magent/backend.env
Environment=PYTHONDONTWRITEBYTECODE=1
Environment=PYTHONUNBUFFERED=1
ExecStart=/opt/magent/.venv/bin/python -m uvicorn app.main:app --app-dir /opt/magent/backend --host 127.0.0.1 --port 8000 --workers 1
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
UMask=0077
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/magent/data
[Install]
WantedBy=multi-user.target
@@ -1,8 +0,0 @@
# Copy to /etc/magent/frontend.env. This file must not contain backend secrets.
NODE_ENV=production
NEXT_TELEMETRY_DISABLED=1
HOSTNAME=127.0.0.1
PORT=3000
NEXT_PUBLIC_API_BASE=/api
BACKEND_INTERNAL_URL=http://127.0.0.1:8000
MAGENT_APPLICATION_URL=https://magent.example.com
-25
View File
@@ -1,25 +0,0 @@
[Unit]
Description=Magent web frontend
Wants=network-online.target magent-backend.service
After=network-online.target magent-backend.service
[Service]
Type=simple
User=magent
Group=magent
WorkingDirectory=/opt/magent/frontend
EnvironmentFile=/etc/magent/frontend.env
# If command -v node reports another system-wide path, update this line.
ExecStart=/usr/bin/node /opt/magent/frontend/.next/standalone/server.js
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
UMask=0077
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/magent/frontend/.next/standalone/.next/cache
[Install]
WantedBy=multi-user.target
+28
View File
@@ -0,0 +1,28 @@
name: magent-beta
services:
magent:
build:
context: .
dockerfile: Dockerfile
env_file:
- ./.env
environment:
APP_NAME: Magent Beta
CORS_ALLOW_ORIGIN: https://beta.grizzlyflix.co.nz
MAGENT_APPLICATION_URL: https://beta.grizzlyflix.co.nz
MAGENT_API_URL: https://beta.grizzlyflix.co.nz/api
AUTH_COOKIE_NAME: magent_beta_auth
AUTH_STATE_COOKIE_NAME: magent_beta_logged_in
AUTH_COOKIE_DOMAIN: beta.grizzlyflix.co.nz
SQLITE_PATH: /app/data/magent.db
LOG_FILE: /app/data/magent.log
SITE_BANNER_ENABLED: "true"
SITE_BANNER_MESSAGE: "Beta environment"
SITE_BANNER_TONE: warning
ports:
- "${BETA_FRONTEND_BIND:-10.30.1.32}:3100:3000"
- "127.0.0.1:8100:8000"
volumes:
- ./data:/app/data
restart: unless-stopped
+4 -23
View File
@@ -1,29 +1,10 @@
services:
magent:
# Select a published immutable release tag or digest in .env.
image: ${MAGENT_IMAGE:?Set MAGENT_IMAGE to a published release tag or digest}
image: rephl3xnz/magent:latest
env_file:
- ./.env
environment:
JWT_SECRET: ${JWT_SECRET:?Generate an independent JWT_SECRET before starting}
SETTINGS_ENCRYPTION_KEY: ${SETTINGS_ENCRYPTION_KEY:?Set the original or newly generated Fernet key}
CORS_ALLOW_ORIGIN: ${CORS_ALLOW_ORIGIN:?Set the exact browser-facing origin}
MAGENT_APPLICATION_URL: ${MAGENT_APPLICATION_URL:?Set the browser-facing application URL}
ports:
# Keep the API internal; the frontend serves /api on this same port.
- "${MAGENT_BIND_ADDRESS:-127.0.0.1}:${MAGENT_HTTP_PORT:-3000}:3000"
- "3000:3000"
- "8000:8000"
volumes:
# Fresh installs only: existing installs must retain their original mount.
- magent-data:/app/data
restart: unless-stopped
stop_grace_period: 30s
read_only: true
cap_drop: ["ALL"]
security_opt: ["no-new-privileges:true"]
init: true
tmpfs:
- /tmp:rw,noexec,nosuid,size=64m,uid=1000,gid=1000
- /app/frontend/.next/cache:rw,noexec,nosuid,size=128m,uid=1000,gid=1000
volumes:
magent-data:
- ./data:/app/data
+13
View File
@@ -0,0 +1,13 @@
name: magent-production
services:
magent:
build: .
env_file:
- ./.env
ports:
- "10.30.1.32:3200:3000"
- "127.0.0.1:8200:8000"
volumes:
- ./data:/app/data
restart: unless-stopped

Some files were not shown because too many files have changed in this diff Show More