Skip to content

Store IGDB data and refresh it after the response #254

Description

@engleek

Goal

Store each game's IGDB data in the database and refresh it in the background, instead of calling IGDB live on every game page. This picks up #112's design (background IGDB fetching), adapted to Cloudflare Workers (ADR 0002, #253).

Today

  • app/models/game.js calls getIGDBGame(game.igdb_slug) on every game page view, so each view makes 1–2 outside requests (the Twitch token, then IGDB).
  • app/utils/igdb.server.js caches the Twitch token with memory-cache, in process memory. On Workers that memory doesn't last between requests.
  • The slug is interpolated into the Apicalypse query (where slug = "${slug}") without escaping.
  • 213 games have an igdb_slug (production, 2026-10-04).

Design

  • Columns on game: igdb_data (JSON) and igdb_data_fetched_at.
  • Read path: the game page uses the stored igdb_data. If it's missing or older than a day, the refresh runs after the response (ctx.waitUntil on Workers; fire-and-forget on Node until then). The page never waits for IGDB. The only exception is a game whose slug was just set and has no data yet: show it without the IGDB section, then show it on the next view.
  • Avoid duplicate refreshes: set an igdb_refresh_started_at before fetching, and skip if it's under ~10 minutes old.
  • Cron: a scheduled job (a Cron Trigger on Workers) refreshes the stalest N games, so data stays fresh for games nobody visits. Keep N small enough for the free plan's limits.
  • Twitch token: store it with its expiry in a kv_store-style table (or KV), and refresh it shortly before it expires.
  • On slug change (create or edit): clear igdb_data, so the old game's data never shows.
  • No pgmq, no node-cron, no igdb-api-node: call the IGDB API with fetch and an escaped query.

Acceptance criteria

  • A game page view makes no request to IGDB or Twitch while its stored data is fresh.
  • Stale data is served immediately and refreshed after the response; two concurrent views start at most one refresh.
  • Changing a game's IGDB slug clears its data, and the next refresh fetches the new game.
  • A slug containing " can't change the IGDB query.
  • Unit tests cover staleness, the refresh guard, token expiry and query escaping. An e2e test runs against a local IGDB stand-in (like the Discord and Bluesky ones in playwright.config.js).
  • igdb-api-node and memory-cache are removed.

Notes

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions