Skip to content

Drop Marshal from persistent storage in favour of Deriving_Json#859

Draft
balat wants to merge 83 commits into
masterfrom
deriving
Draft

Drop Marshal from persistent storage in favour of Deriving_Json#859
balat wants to merge 83 commits into
masterfrom
deriving

Conversation

@balat

@balat balat commented May 29, 2026

Copy link
Copy Markdown
Member

Summary

Replace Stdlib.Marshal with Deriving_Json for all persistent
storage in Eliom (Ocsipersist-backed cookies, session groups,
session data and Reference.eref). Adds a [@@deriving json]
annotation on the types involved and bumps the on-disk table names
where the format changes. Intended for the Eliom 13 release.

This is a breaking change. User code must now pass a JSON codec to
persistent references:

Reference.eref ~persistent:("name", [%json: t]) v
State.create_persistent_table ~scope ~json:[%json: t] "name"

Commits

  • Serialise persistent session groups with Deriving_Json — replaces
    Marshal.{to,from}_string for perssessgrp with a nominal payload
    record and adds [@@deriving json] on cookie_level.
  • Switch persistent-cookie table to Ocsipersist.Column.Json — bumps
    the cookie table suffix from _v5 to _v6 and annotates the cookie
    type + all its dependencies (scope_hierarchy, user_scope,
    full_state_name, timeout, perssessgrp) with [@@deriving json].
  • Switch session-group table to a typed Column.Json table — replaces
    Ocsipersist.Polymorphic with a typed functorial table; renames the
    table to __eliom_session_group_table_v2; annotates nbmax.
  • Drop Marshal from Reference and State persistent tables — the
    breaking API change. Reference.eref and State.create_persistent_table
    now take a Deriving_Json.t codec. The vestigial int64 column on
    every session value is dropped. Common.Ocsipersist.Polymorphic and
    Persistent_tables.create are removed.

Downstream

Companion PRs:

  • ocsigen-start: deriving branch (codec for Stringset in tips,
    Os.Date.local_calendar_json, demo template update, opam constraint
    relaxed, stale pin-depends removed)
  • ocsigen-toolkit: deriving branch (only opam constraint relaxed)
  • os_template: deriving branch (demo_ref.eliom codec update)

Test plan

  • dune build @check passes on the deriving branch
  • dune build passes on ocsigen-start and ocsigen-toolkit pinned
    against this branch
  • Manual smoke test of a persistent session round-trip in a real
    app (e.g. ocsigen-start)
  • Confirm that the Eliom manual + tutorial sections about
    Reference.eref have been updated (separate PRs)

balat added 25 commits June 22, 2026 14:37
Pure file renames, no content changes. For example:
  eliom_service.server.ml → service.server.ml
  eliom_content.eliom → content.eliom

Files NOT renamed (name conflicts with Stdlib/external libs):
  eliom_form (conflicts with Js_of_ocaml.Form)
  eliom_lazy (conflicts with Stdlib.Lazy)
  eliom_react (conflicts with react library)
  eliom_uri (conflicts with uri library)

Internal modules also kept unchanged:
  eliom_common, eliom_types, eliom_runtime, *_base, *_core, etc.

This commit does not compile: reference updates follow.
Update all internal references from Eliom_xxx to the new short names.
For example:
  Eliom_service.create → Service.create
  Eliom_parameter.int → Parameter.int

References updated in:
  - Renamed source files (74 files)
  - Internal modules (eliom_common, eliommod_*, etc.)
  - PPX code (ppx_eliom_server.ml, ppx_eliom_utils.ml)

References NOT changed (modules not renamed):
  Eliom_form, Eliom_lazy, Eliom_react, Eliom_uri
  Eliom_common, Eliom_runtime, Eliom_types, and all *_base/*_core modules
Create/update the Eliom module as the primary entry point:
  Eliom.Service, Eliom.Registration, Eliom.Content, etc.

Modules with name conflicts use the old names:
  Eliom.Form = Eliom_form  (Js_of_ocaml.Form conflict)
  Eliom.Lazy = Eliom_lazy  (Stdlib.Lazy conflict)
  Eliom.React = Eliom_react (react library conflict)
  Eliom.Uri = Eliom_uri    (uri library conflict)

Create client-side Eliom module (previously server-only).
Update dune: modules_without_implementation for renamed sig modules.
Regenerate dune.server and dune.client.
Provide backward-compatible module names (Eliom_service, etc.) in
separate libraries for existing code that hasn't migrated to the
new Eliom.Service style.

Usage: add eliom.compat.server (or .client) as a library dependency.

These libraries are intended to be deprecated and eventually removed
once downstream code has migrated.

Server: 22 compat modules (16 .ml + 6 sig-only .mli)
Client: 19 compat modules (12 .ml + 7 sig-only .mli)
- Migrate Eliom_X module references to Eliom.X (Eliom_service.create
  → Eliom.Service.create, Eliom_registration.App → Eliom.Registration.App,
  Eliom_content.Html.F → Eliom.Content.Html.F, etc.)
- Rename PROJECT_NAME_main.eliom → PROJECT_NAME_server.eliom
  (executables: testapp_server.bc, testapp_server.exe)
- Update Makefile.app and dune targets accordingly
- Eliom_xxx → Eliom.Xxx in source files
- api.odocl: use new short module names
- indexdoc: update documented module list
- filelist.ml: update module list for packaging
- _tags: disable warning 34 (OCaml 5.4), add logs.browser package
Define a named module type Content.Html.T (built from Tyxml's
Html_sigs.Make(Xml)(Svg.F.Raw').T plus Eliom's LINKS_AND_FORMS) and
use it as the parameter signature of Tools.Make instead of
[module type of Content.Html.F].

Avoids the OCaml strengthening bug that triggers when [module type of]
captures wrapped module paths through an alias.
Use dune's [(wrapped (module_name Eliom))] feature to expose the
public modules as [Eliom.Service], [Eliom.Content], [Eliom.Client],
... instead of the historical flat [Eliom_xxx] names.  Update PPX
client and dune configuration accordingly.
The module was incorrectly left with its old name despite being
visible to users via PPX-generated type annotations.

Also rename the local Client_value module in eliom_client_core.client.ml
to Client_value_registry to avoid shadowing.
Accessible as Eliom.Syntax via wrapping. Used by the PPX in
generated code (Eliom.Syntax.client_value, Eliom.Syntax.to_poly, etc.)

Also adds Syntax.to_poly (previously only in Lib.to_poly) since the
PPX generates Eliom.Syntax.to_poly in user code.
ppx_eliom_client.ml generates code referencing Eliom_client_core.
With wrapping, this must be Eliom.Eliom_client_core for user code.
Use eliom_expr to conditionally prefix with Eliom. when not -internal.
Visible to users via PPX-generated code (register_client_closure,
open_client_section, close_server_section, get_escaped_value).
…odules

The injection expression was hardcoded as Client_core.Syntax_helpers.get_injection
instead of going through eliom_expr which prepends Eliom. when not in -internal mode.
Breaking change: Eliom.run is now Eliom.App.run due to module wrapping.
The auto-generated wrapper module cannot contain val declarations,
so this cannot be covered by eliom-compat.
Define eliom-compat as its own (package ...) in dune-project (was
previously an [eliom.compat] sub-library) so that odoc no longer
collides between the modernized [Eliom.Xxx] names exported by [eliom]
and the legacy [Eliom_xxx] aliases exported by the compat lib.

The two libraries are renamed from eliom.compat.{server,client} to
eliom-compat.{server,client}.

Existing dune executables/installs that were implicit when there was
only one package now need an explicit (package eliom) stanza.
The old ocamlbuild-based doc system could not handle wrapped modules.
New build/gen_wikidoc.sh script:
- Uses dune-compiled .cmi files from _build/
- Creates short-name .cmi aliases (Bus.cmi -> include Eliom__Bus)
  so ocamldoc can resolve cross-references
- Gets include paths from dune's merlin config
- Generates wiki docs for both server and client
- Remove src/ocamlbuild/ (eliom.ocamlbuild plugin)
- Remove pkg/ (topkg packaging system)
- Remove build/build.ml, build/doc.ml, build/_tags (ocamlbuild config)
- Remove _tags, src/_tags, src/ppx/_tags, README.build
- Remove js_of_ocaml-ocamlbuild dependency from dune-project/opam
- Clean up ocamlbuild references in comments

The doc generation now uses build/gen_wikidoc.sh (previous commit).
The distillery templates in pkg/distillery/ are preserved.
When -internal-prefix Os is passed, the client PPX strips the Os.
wrapper prefix from type paths read from server .cmo files. This is
needed for compiling wrapped libraries like ocsigen-start that depend
on Eliom.
The eliom.opam.template is consumed by dune to append the
x-ci-accept-failures stanza into the generated eliom.opam.
Accessible as Eliom.Common from user code.
balat and others added 11 commits June 25, 2026 19:34
Run the Documentation workflow with a version input to freeze the dev docs
currently on gh-pages as /<version>/ and repoint `latest` at it (wodoc release,
which also refreshes versions.json). No rebuild — a release is exactly the dev
docs at that point. Push / version-less runs keep rebuilding dev only.
wodoc no longer reads a menu.wiki. The manual's left navigation is now a
(nav (section …)) stanza in doc/wodoc (with (group …) for the nested
chapters), and the CI step that overlaid doc/manual/menu.wiki from the
wikidoc branch is removed.
The converter turned the escaped wikicreole URL into broken markup
(`~http:~/~{e /your_server}`); restored it to a plain `http://` URL with
the host shown as an emphasised placeholder, in both the config and the
workflow-configuration manuals.
…n input into shells

- Add a 'concurrency' group so a master push (rebuilds dev/) and a release
  workflow_dispatch (freezes <version>/ and repoints latest) cannot push to
  gh-pages simultaneously (the second push would fail non-fast-forward).
- Pass the workflow_dispatch 'version' input through an env var and reference
  $VERSION in the run: shells instead of interpolating ${{ … }} directly, to
  avoid shell injection from the input.
Same cleanup as on master: drop the dead <<outline|<<header|...>> TOC markers,
convert the <<code language="ocaml"|...>> examples to odoc {[ ... ]} blocks, and
unwrap the {!exception ...} reference — all were wrapped in a bare {% ... %} (raw
markup with no target language, a wiki->mld leftover) that odoc renders verbatim
as <code>.
Keep the (hosted ...) table in sync with master: add the js_of_ocaml/tyxml
(subdir layout) and reactiveData (root) entries so cross-project links point
to ocsigen.org. Avoids reverting them when this doc branch merges.

Requires wodoc with the `subdir` layout (ocsigen/wodoc#4).
Keep the (hosted ...) table in sync with master: add (lwt lwt subdir) now
that Lwt's docs are deployed under the standard per-package layout, so this
modernization branch doesn't drop the Lwt redirect when it reaches master.
Eliom had two overlapping landing pages: overview.mld (the ocsigen.org
landing) and index.mld (the ocaml.org landing), both an "Eliom" introduction
with installation and getting-started examples. Keep the richer overview as
the single home page by renaming it to index.mld, and drop the redundant one.

On ocaml.org the documentation landing (/p/eliom/latest/doc/) is rendered
from index.mld; add @children_order so the manual pages appear in the sidebar
in the reading order of the reference manual instead of alphabetically. Point
the ocsigen.org landing at index.html.
The package home page (index.mld) is the ocsigen.org landing but was not
reachable from the left navigation. Add an "Overview" entry pointing at it,
before the Introduction.
Bring the deriving branch in line with master: convert hardcoded
ocsigen.org/<project> links (here version-less and root-relative) to odoc
cross-package references, quoting hyphenated page names, and repair the two
broken empty-page links (/tyxml/.html, /lwt/.html) to the project home. Keeps a
future merge from reintroducing the old URLs.
The API docs are now built with odoc/wodoc (see doc/wodoc and the
Documentation CI workflow); the gen_wikidoc.sh ocamldoc->wiki script,
the make wikidoc/doccp targets and the README section are no longer
used. The indexdoc menus are kept (still consumed by doc/wodoc).
@balat balat changed the base branch from modernize to master June 25, 2026 18:24
Eliom on this branch targets the Ocsigen Server 8.x API; the dependency
bound still allowed only 7.x (< 8.0), which made eliom uninstallable
against ocsigenserver 8.0. Bump the bound to (>= 8.0) (< 9.0).
balat and others added 13 commits June 26, 2026 16:44
The repository is already formatted with ocamlformat 0.29.0 in practice
(editor format-on-save), but `.ocamlformat` still pinned 0.28.1, so the
`lint-fmt` CI job — which installs the pinned version — failed on the
0.29.0-formatted code.

Bump the pinned version to 0.29.0 and reformat the few remaining files
that still carried 0.28.1 formatting (via `dune fmt`).
Ocsigen Server 8 removed the deprecated helpers id, leftright, map_filter
and the Option module from its baselib. Replace them with their stdlib
equivalents instead of relying on shims:

  - id            -> Fun.id
  - List.map_filter -> List.filter_map
  - Lib.Option    -> stdlib Option (map/iter, identical signatures)
  - leftright     -> Either.t, with Left/Right -> Either.Left/Either.Right

Also drop the now-unused `open Lib` in mod_sessiongroups.mli.
Eliom now targets the Ocsigen Server 8.x API. Bump the bound in
dune-project from (>= 7.0) (< 8.0) to (>= 8.0) (< 9.0) and regenerate
eliom.opam.
Released ocsipersist still targets the flat ocsigenserver module names; the
master pin tracks the Ocsigen Server 8 API.
CI: pin ocsipersist#master for Server 8
Require ocsigenserver >= 8.0 (and < 9.0)
# Conflicts:
#	.github/workflows/doc.yml
#	CHANGES
#	doc/client.indexdoc
#	doc/server-outputs.mld
#	dune-project
#	eliom.opam
#	src/lib/client.client.ml
#	src/lib/form_sigs.shared.mli
Eliom now targets the js_of_ocaml 6.4 API (merged from master). Bump the
js_of_ocaml/wasm_of_ocaml lower bounds to 6.4.0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant