Skip to content

ROADMAP.md

Latest commit

 

History

History
342 lines (299 loc) · 20.5 KB

ROADMAP.md

File metadata and controls

342 lines (299 loc) · 20.5 KB
Raw

LWT Roadmap

Direction for the next iteration. Strategic, not a release checklist — see CHANGELOG.md for shipped work.

North star

Make LWT reachable on a phone as a real app, starting with the FOSS audience (F-Droid) rather than the mass-market Play Store. The long-term prize is an app that does more in the browser and less on the server, so it can eventually run offline and sync — but each phase below is independently useful and we only pay for the hard parts when demand proves them worth it.

Strategic decisions (locked)

  • F-Droid first, not Play Store first. The FOSS audience tolerates the rough edges (entering a server URL, self-hosting) that mass-market users won't, so it needs less of the expensive zero-config work.
  • TWA is shelved. A Trusted Web Activity needs Chrome at runtime (fails on de-Googled phones) and reads to F-Droid as a NonFreeNetwork wrapper around a server we control. It was a shortcut for a different audience.
  • The mobile artifact is a configurable client, not a hardwired wrapper. A thin shell over the system WebView, pointed at a user-chosen server URL. This fits F-Droid's accepted "client for a self-hostable free service" pattern (Nextcloud-client style), and — unlike a TWA — it's reusable for Play Store later with a default instance baked in.
  • License is Unlicense (public domain) → zero F-Droid licensing friction.

Branding

  • lwt stays the umbrella for the project and the server. The original name is kept for everything technical.
  • The mobile app carries its own brand: Lukaisu. Locked 2026-06 (org.lukaisu.app, lukaisu.org registered), replacing the old language-reader codename. The consumer-facing client is where rebranding starts; the server stays LWT. App repo: ../lukaisu/.
  • Name deadline — met. The Android applicationId / package name is permanent once published to F-Droid/Play, so it had to be locked before Phase 2's first release. Done: org.lukaisu.app is committed in the app's Capacitor config and is final.
  • Smooth the seam in client copy: the Lukaisu app connects to "LWT servers" — the server-URL config screen should make that relationship clear.

Repository layout

  • App de-coupling (Phases 0, 1, 3, 4) lives in this repo (lwt/). It is the app evolving — not separable.
  • The mobile client (Phase 2) has its own repo: Lukaisu (../lukaisu/). Rationale: F-Droid builds reproducibly from a focused repo+tag (a thin Android project, not an Android subfolder buried in a 138k-LOC PHP monorepo); separate toolchain and release cadence; matches the workspace's one-repo-per-concern convention and the standard server/client split.
  • This roadmap stays in lwt/ — the published, version-controlled repo. The parent workspace folder is not a git repo, so a roadmap there would be an untracked, unshareable file. lukaisu/ROADMAP.md carries the build-focused detail; this file keeps the ecosystem strategy.

The keystone constraint

LWT renders HTML server-side (71 view templates). Until more logic moves to the frontend against /api/v1, any mobile wrapper is just a window to a server. So "more frontend, less server-rendered" is the enabling work — the thing that makes a good client today and offline/local-first possible tomorrow.

The raw count overstates the remaining work, and converting a view does not reduce it — a converted page keeps its .php file as a scaffold. Classified by what each template actually does (audited 2026-08-07): roughly 18 are already Alpine-rendered mount points, ~35 are static markup and form fields where conversion buys nothing, and only ~17 still render database rows into HTML. Most of those 17 are admin/config surfaces that are fine in an online WebView. Track the mobile-critical flows, not the file count.

Phase 0 — Foundations (now)

  • Confirm and document the strategy above.
  • API base URL decoupling: the frontend no longer assumes same-origin. @shared/api/client resolves an injectable server root (setApiServer, precedence runtime > localStorage lwt.apiServer > meta > same-origin), so the mobile client / self-host / future default instance can all point it at a chosen server. Unset = unchanged same-origin behavior.
  • Cross-origin auth plumbing (the seam's server + client halves): - [x] Opt-in CORS on /api/v1 via CORS_ALLOWED_ORIGINS (Cors helper + preflight handling in ApiV1::handleRequest). - [x] Client sends Authorization: Bearer (setAuthToken/getAuthToken, persisted as lwt.apiToken); server already validated Bearer tokens. - [x] First-run auth UX (GET /connect, clientAuth component): choose-server (probes /version) → log in or register against the auth API, storing the bearer token. Plus session longevity — proactive token refresh before the 30-day expiry, and lwt:auth-expired → /connect on a 401. Remaining for Phase 2: the Capacitor shell that hosts these screens and supplies a default server.
  • [~] Security hardening pass — continue the XSS phases. Gate for any shared/public exposure; non-negotiable before a public instance. Phase 8 shipped: four DOM sinks that built markup by string concatenation (bulk-translate inputs, multi-word markers, the language list, LibreTranslate errors) now set attributes and text through the DOM, and the json_encode-into-<script> convention is held by a codebase-wide invariant test instead of site-by-site vigilance. Ongoing — this bullet stays open until a full review, not just the XSS classes, has been done.

Phase 1 — Frontend de-coupling (the enabling work)

Reframe (post-audit): the mobile-critical pages are already client-rendered. read_desktop.php renders the word grid + state 100% client-side via textReader.renderTextContent() against /api/v1, with an offline prototype (offline-text-reader.ts + IndexedDB) started. review_desktop.php is "all UI rendered by Alpine.js." So Phase 1 is cutting the server-shell umbilical, not converting pages.

Status (re-audited 2026-08-07): complete as scoped. Every mobile-critical surface is shell-free, and the legacy-fragment cleanup — the last open item in this phase — has shipped.

This does not close #266. That issue asks for PHP to emit "only data for hydration", which is a wider bar than Phase 1's. The data half is met (config blobs carry boot parameters — ['textId' => …, 'langId' => …] — and everything else is fetched from /api/v1), but PHP still owns the markup half (audited 2026-08-08):

PHP still owns Where Scale
DOM scaffold x-data in views 53 views
Which JS modules load PageLayoutHelper::getRequiredModules() → <meta name="lwt-modules"> every page
i18n injection PageLayoutHelper::buildI18nScript() every page
Icons IconHelper::render() → server-rendered <i data-lucide> throughout

A bundled client used to work around all four with a build-time transpiler rather than removing them; that client and its transpiler are gone (see Phase 2), so nothing routes around this today. #266 stays open against that markup half; it is not a Phase 1 deliverable, and with the mobile client now independent it needs a fresh justification before it is worth paying for.

  • (Phase 0 gate) Injectable API base URL. Done in Phase 0 — same seam. @shared/api/client resolves an injectable absolute server root and everything client-rendered already routes through it. (Listed here too because it's the gate the rest of Phase 1 depends on.)

  • Reading — content + chrome now shell-free. The word grid + state render 100% client-side from /api/v1/texts/{id}/words (TextTermApiHandler → word_store.ts → text_reader.ts); offline prototype at shared/offline/offline-text-reader.ts. The reader's chrome is now client-rendered too: book-context nav from GET /texts/{id}/book-context (book_nav_renderer.ts) and the audio player from GET /texts/{id}/audio (markup-only audio_player.php + audioPlayer fetch), so read_desktop.php carries no per-text data and TextReadController dropped the server-side book/media plumbing. Verified live (book nav, audio reveal, no-regression plain read). Toolbar labels use __e() but resolve via the client i18n t(). Nothing outstanding — the global navbar was the last shared dependency and it shipped (next bullet).

  • Global navbar — shell-free. Shipped in 3.2.0 (d6e6a5203). PageLayoutHelper::buildNavbar() is gone; PHP now emits only the mount point <div id="navbar-root" data-current-page="…"> (buildNavbarPlaceholder()), and mountNavbar() fetches GET /api/v1/navbar (per-user language list, current language, theme state, admin/multi-user flags) and builds the markup in shared/components/navbar_renderer.ts. Labels resolve from the i18n bundle, so the payload stays locale-agnostic. Zero server-rendered navbar markup remains in src/**/*.php. This was the last chrome dependency spanning every page.

  • i18n → client (delivery mechanism). Shipped: GET /api/v1/i18n[/{locale}] (public; Translator::getAllTranslations()) returns the flat "namespace.key" => string bundle, merging English fallback — the same shape the page blob uses. shared/i18n/translator.ts gained loadI18nFromApi() (fetch + localStorage cache) and hydrateI18nFromCache() (sync first-paint), so a shell-free client gets strings without a server-rendered page. The server-injected blob stays as the default for SSR pages (additive, no breakage). Client boot wired: bootI18n() (called from main.ts) uses the blob when present, else hydrates from the localStorage cache and refreshes from the API, persisting the resolved locale (lwt.locale) for offline first-paint — so a bundled client now picks up strings on its own. Templates still calling __e() resolve client-side via t().

  • Review — verified shell-free. The review SPA (review_desktop.php → review_api.ts) renders entirely from /api/v1/review/* (next-word/status/config/table-words/tomorrow-count all exist). Removed the orphaned status_change_result.php (HTML fragment, no route/no caller; superseded by the JSON status_change_config.php). The legacy non-SPA review-AJAX page turned out not to exist — an earlier revision of this bullet called it "a separate, non-mobile entry — out of scope", but /review is the module's only route. Its five views and review_ajax.ts have been deleted: a "test" → "review" rename had moved the TypeScript to #term-review/#review-finished-area while the views still emitted #term-test/#test-finished-area, so the two halves could not have worked together even if something had wired them up.

  • Text list / library — shell-free. texts_grouped_app.ts client-renders the list from /texts/by-language/{id} + /texts/statistics, and the destructive bulk archive/delete now go through PUT /api/v1/texts/bulk-action (per-user scoped) instead of a same-origin form POST, so they work against a configurable API base. The remaining bulk actions (tag / review / reparse) intentionally stay on the form path — they need pickers/navigation and are desktop-admin, not mobile. __e() labels resolve via the i18n API once a page boots from it.

  • Vocabulary mgmt — shell-free; legacy fragments deleted. Re-audit (corrected): the modern reader's word actions already go through /api/v1/terms/* — word_store.ts/word_modal.ts call TermsApi.setStatus/createQuick/delete, and the unknown-word popup uses the API button family (createWellKnownButton/createIgnoreButton with a WordActionContext). The modern reader has no #frames-r, so the legacy target="ro" → *_result.php mechanism isn't even wired there. So the mobile-critical vocab flow needs no conversion — it's done. The remaining legacy/transitional code has since been removed. All nine dead-end *_result.php fragments are gone (save_result, edit_result, edit_term_result, edit_multi_update_result, hover_save_result, all_wellknown_result, bulk_save_result, and Book/import_result), along with the word_popup_interface.ts link-builders that fed them. The one file still named *_result.php is Vocabulary/Views/upload_result.php, kept on purpose: it is a live results table rendered by Alpine from GET /api/v1/terms/imported, not a server-rendered fragment. The term editor behind /word/edit, /word/edit-term and /words/{id}/edit now mounts the same API-driven component the reading view opens in a modal, and bulk save posts to POST /api/v1/terms/bulk.

  • Books — shell-free. /books and /book/{id} render from GET /api/v1/books, GET /api/v1/books/{id} (which nests the chapters) via bookList / bookDetail; the views carry a config blob and nothing else, and BookController no longer touches BookFacade to render either page. The blocker was the endpoint registry, not the client: DELETE /books/{id} and PUT /books/{id}/progress were rejected with 405 before dispatch even though BookApiHandler implemented both. A URL carrying an ID never matches a ROUTES key exactly, so lookup falls back to the first path segment — the books/chapters and books/progress keys were decorative, and the bare books entry allowed only GET and POST. EndpointMethodReachabilityTest now asserts real request shapes resolve, so the next drift fails a test instead of a feature.

  • Form POSTs — the write half, for the surfaces above (#262). Those surfaces read from /api/v1; several still wrote by posting a form to the page origin, which a client pointed at a different server cannot do. Converted: tags, user profile/password/preferences, standalone term creation, the text editor (POST /api/v1/texts, PUT /api/v1/texts/{id} — neither endpoint existed; TextsApi.create() had been calling a 404 since it was written), and the feed forms (POST/PUT /api/v1/feeds).

    Two mass-assignment holes closed on the way: `texts.TxLgID` and
`news_feeds.NfLgID` are client-supplied references into `languages`, and
a foreign key proves the row exists, not that the caller owns it. Both
form handlers passed the submitted value straight to the facade; the API
endpoints check ownership, and the form-POST routes that skipped it are
retired. Multi-user installs only.

Fourteen views still post, down from sixteen: admin (4), the imports
(`import_epub_form`, `upload_form`, `bulk_translate_form`), the feed
wizard's navigation steps (3), the archived-text and check forms, the
Microsoft link confirmation, and `edit_form` for its Check button alone.
Every one of those is a surface this phase deliberately leaves
server-rendered (below), so what remains of #262 is scoped by that list
rather than by view count.

Out of Phase 1 (leave server-rendered, fine in a WebView online): imports (file/web/youtube/whisper), admin/settings, language config, feeds.

Two exceptions inside that list, both because the work was a fix rather than a conversion:

  • The feed create/edit forms went to /api/v1 even though feeds are out of scope, because tracing them turned up that finishing the RSS wizard had saved nothing since 2026-08-08 (1e2216bb6). Step 4 posted to /feeds/edit, and that commit made the route a 302 to the manager SPA when the duplicated feeds list was retired; a redirect discards the body. Present in 3.4.0 and 3.4.1. Verified both ways against a live BBC RSS feed: on the parent commit the walk ends with news_feeds empty, on the fix it writes the row. The wizard's navigation steps (2 and 3) still post to /feeds/wizard — they drive a server-side session state machine that renders HTML previews of the fetched page, and moving that to the client is its own project.
  • The text editor's "Check" button still posts. It asks for a server-rendered parsing report rather than saving, so it names its own target with formaction while the form itself carries no action.

Definition of done per surface: renders entirely from /api/v1 JSON, no server-rendered partial carrying data, works against a configurable API base URL.

Keep the service worker (sw.ts) + manifest as the offline cache layer; expand coverage as each surface goes shell-free. Track progress as "% of mobile-critical flows that run without a server-rendered page."

Phase 2 — the mobile client (now independent)

Closed out of this repo, 2026-08. The client exists and shipped: Lukaisu (../lukaisu/) is a Capacitor 8 app on the system WebView, signed, fully FOSS, and live in its own F-Droid repo at https://fdroid.lukaisu.org/repo. Only the main-catalog submission is still ahead of it. See lukaisu/ROADMAP.md.

It no longer builds on anything here. Lukaisu owns its reading frontend outright under webapp/ (moved from lukaisu-server/src/frontend/ in its Phase M, 2026-07) and has gone local-first: an on-device Dexie database and TypeScript parsers run the read/save/review loop with no server at all, connecting one being optional. So the bundled-client ("Model B") build this phase used to describe — npm run build:app, vite.app.config.ts, build/php-view-prerender.mjs, the four standalone pages under src/frontend/app/ — had no consumer left, and has been deleted. It was never in CI or the Docker image; recover it from git history if a bundled build is ever wanted here again.

What that work bought this repo is kept and still earns its place, because it serves any remote or third-party client, not just a bundled one: the injectable API base URL, opt-in CORS, bearer-token auth, GET /connect, and the client i18n bundle — all listed under Phase 0, all independently useful.

Consequence for the north star: phone reach is no longer LWT's to deliver; Lukaisu delivers it, from its own code. The keystone-constraint argument above was written when a bundled client depended on shrinking the server-rendered surface — it now has to stand on what self-hosters reading in a browser get out of it, which is a weaker case and a smaller budget. Re-argue #266 on those terms before spending on it.

Phase 3 — Toward "5 minutes to reading" (mass-market prerequisites)

Only once the client exists and demand is real. This is where the hosting commitment and zero-config UX get paid for.

  • Guest / deferred-signup first-run ("read now, sign up later to sync") using existing Register + ClaimOrphanRows.
  • Stand up a hardened public instance — eyes open on the real cost: money is cheap (~$15/mo on the existing VPS), but backups, uptime, security patching, GDPR, and support are the actual burden.
  • Optional: Play Store build of the same client with the public instance as default. Apply to the main F-Droid catalog.

Phase 4 — Local-first + sync (conditional, far out)

The true offline F-Droid app, and the only thing that sheds per-user hosting. Do not start until offline demand or hosting cost justifies it.

  • Spike conflict resolution FIRST (the Anki problem) — it looks easy and isn't. Decide last-writer-wins+tombstones vs CRDT before committing.
  • Move reading + review data to client storage (SQLite-WASM / OPFS).
  • Port the corresponding logic from PHP to TypeScript.
  • Caveat: server-dependent parsers (MeCab/CJK, NLP lemmatizer, Whisper) stay server-side or need bundled WASM equivalents — plan per-language.

Watch-outs (carry forward)

  • Security before exposure — public instance multiplies the blast radius of any XSS.
  • Sync is the underestimated monster — quarantined to Phase 4 on purpose.
  • Two audiences pull opposite ways — FOSS/self-host vs zero-config mass-market. The configurable-server client is what lets one codebase serve both; protect that seam.