Skip to content

feat(simkl): comply with Simkl API conventions across Android, web, and proxies - #670

Merged
ProdigyV21 merged 4 commits into
ProdigyV21:mainfrom
Himanth-reddy:feat/simkl-api-convention-headers
Sep 8, 2026
Merged

feat(simkl): comply with Simkl API conventions across Android, web, and proxies#670
ProdigyV21 merged 4 commits into
ProdigyV21:mainfrom
Himanth-reddy:feat/simkl-api-convention-headers

Conversation

@Himanth-reddy

Copy link
Copy Markdown
Collaborator

Summary

This PR aligns all Simkl API calls across ARVIO with the official Simkl Headers and Parameters Conventions so requests are correctly identified and visible in Simkl's API Debug & Analytics dashboard.

Changes

  • Query Parameters: Ensure client_id, app-name=arvio, and app-version are appended to all Simkl requests (public catalog, scrobble, and sync) rather than only /oauth/pin.
  • Case Sensitivity: Enforce lowercase app-name=arvio per Simkl specification.
  • User-Agent: Add descriptive User-Agent headers across Android, Web, Netlify functions, and Supabase Edge functions.
  • Proxy Handlers: Update Netlify _backend.js and Supabase simkl-proxy to preserve/forward client_id, app-name, and app-version query parameters and User-Agent on all outgoing calls.
  • Tests: Added automated test to simkl-proxy.test.js validating that required query parameters and headers are attached to all proxied requests.

…nd proxies

- Append required query parameters (client_id, app-name, app-version) to all Simkl API requests per https://api.simkl.org/conventions/headers
- Ensure app-name is lowercase ('arvio') and app-version is properly formatted
- Pass descriptive User-Agent header across Android client, Web client, Netlify proxy, and Supabase Edge Function proxy
- Ensure client_id is preserved on all proxy requests (not just /oauth/pin)
- Add automated test verifying Netlify proxy attaches required query parameters and headers to outgoing requests
@github-actions github-actions Bot added area: android Changes to the Android app or Gradle build area: web Changes to web or Netlify sites area: backend Changes to backend services or infrastructure labels Sep 7, 2026
…oid, web, and proxies

- P0: Gate sync strictly on /sync/activities watermark even when force=true to avoid library sync storms
- P0: Implement continuous delta sync with GET /sync/all-items?date_from={watermark} and row merging; reserve sequential category sync (movies -> shows -> anime -> playback) for initial sync
- P0: Persist sync watermark and snapshot cache across process death and avoid aggressive window focus invalidation
- P1: Clamp scrobbler progress between 0-100 without multiplying small progress values (<1%)
- P1: Prevent movie unmarking from deleting library row and ratings by calling POST /sync/add-to-list with to: plantowatch
- P1: Add FIFO command queue to SimklScrobbler to prevent dropped STOP requests during auto-play advance
- P1: Trigger scrobbleStop when exiting playback at >=80% watched threshold
- P1: Standardize on Anime Path A (shows[] with use_tvdb_anime_seasons: true and extended=full_anime_seasons)
- P1: Batch season watched toggles into a single POST request
- P2: Update proxy allowlists for bare /sync/all-items, /sync/playback/episodes, and DELETE /sync/playback/{id} across Web, Netlify, and Supabase
- P2: Forward Retry-After header through all proxies and log warnings on HTTP 429 and 412
- P2: Support slug, simkl_id alias, unaired episode deduction, and immediate PIN polling termination on KO / device_code
@Himanth-reddy

Copy link
Copy Markdown
Collaborator Author

Simkl Official Integration Review Remediation

This commit addresses all findings and recommendations from the official Simkl integration review across Android, Web, Netlify functions, and Supabase Edge Functions:

P0 — Request Volume & Sync Storm Prevention

  • Strict /sync/activities Gating:
    • SimklSyncService.kt and simkl.ts now gate all library syncs behind /sync/activities. If activities.all matches the locally stored watermark, sync is completely skipped—even when force=true.
  • Continuous Delta Sync (date_from):
    • Reserved 3-part sequential category sync (moviesshowsanimeplayback) for initial sync only.
    • Subsequent syncs use GET /sync/all-items?date_from={watermark} to fetch deltas and merge changes into the existing snapshot (mergeMovieRows, mergeShowRows).
  • Watermark & Snapshot Disk Persistence:
    • Added simkl_sync_watermark persistence to profile-scoped DataStore (SyncProviderStore.kt) and cached snapshots to disk (simkl_snapshot_cache.json), ensuring state survives process death.
    • Eliminated aggressive window focus cache invalidation in Web; caches persist in localStorage and only invalidate upon account/token change.

P1 — Data Loss Prevention & Scrobbler Heuristics

  • Removed Progress Multiplier Bug:
    • Removed progress <= 1 ? progress * 100 logic across Android (SimklScrobbler.kt) and Web (simkl.ts). Small progress percentages (e.g. 0.5%–1.0%) are preserved accurately and clamped between 0 and 100.
  • Movie Unmark Without Rating/Library Erasure:
    • Unmarking a movie from watched now calls POST /sync/add-to-list with to: "plantowatch" instead of /sync/history/remove, preventing Simkl from purging the item row and user rating.
  • FIFO Scrobble Command Queue:
    • Added a FIFO queue in SimklScrobbler.kt to ensure rapid playback transitions (auto-advance / next episode) execute STOP before START without dropping commands.
  • Player Exit Watched Threshold (≥ 80%):
    • Exiting playback at ≥ 80% progress sends scrobbleStop instead of pause, enabling Simkl to mark the item as completed.
  • Anime Path A Standardized:
    • Scrobbles and history sync for anime now consistently send show with use_tvdb_anime_seasons: true and request extended=full_anime_seasons during full anime sync.
  • Batched Season Marking:
    • Marking a whole season watched in Web (DetailsDrawer.tsx) sends a single batched POST /sync/history payload with seasons: [{ number: seasonNumber }] instead of N individual episode calls.

P2 — Compliance, Proxy Allowlists & Resilience

  • Proxy Endpoint Expansion:
    • Added allowlist rules for bare GET /sync/all-items, GET /sync/playback/episodes, and DELETE /sync/playback/{id} across Web API route, Netlify proxy, and Supabase Edge Function.
  • Header & Error Resilience:
    • Proxies forward upstream Retry-After headers on HTTP 429.
    • Added interceptor logging for HTTP 429 and HTTP 412 (client_id_failed) responses.
    • Deducted not_aired_episodes_count from total episode counts when computing progress.
    • Added support for ids.slug and alternate ids.simkl_id keys, plus a Simkl badge on the Web details drawer linking to simkl.com.
    • Terminated polling loops immediately when KO or device_code is returned.

Verification

  • Android: 26/26 tests passed in SimklIntegrationTest.kt (./gradlew testPlayDebugUnitTest --tests "com.arflix.tv.data.repository.simkl.*").
  • Netlify: 58/58 tests passed (npm --prefix netlify-auth-site test).
  • Web: 51/51 tests passed (npm --prefix web test).
  • Web Build: Production build succeeded cleanly (npm --prefix web run build).

Arvin added 2 commits September 8, 2026 12:32
Keep account-scoped snapshots and their acknowledged watermark across process restarts, retain resolved anime IDs, and retry partial playback syncs. Reconcile deleted IDs only when removal markers change. Route web season writes through selected providers and surface failures before updating local watched state. Add Android and web regressions.
@ProdigyV21
ProdigyV21 merged commit 233be06 into ProdigyV21:main Sep 8, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: android Changes to the Android app or Gradle build area: backend Changes to backend services or infrastructure area: web Changes to web or Netlify sites

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants