nothing runs the gallery examples, so the docs can drift silently

[jooyoungseo]

Summary

Nothing in CI executes the code chunks in docs/examples.qmd, docs/examples-plotly.qmd or docs/examples-altair.qmd. The gallery can therefore drift out of step with the library — an example that no longer runs, or prose describing a reading the extractor no longer produces, will not fail anything.

tests/core/test_plot_type_stability.py checks that docs/stability.qmd lists every PlotType member, so the stability table is guarded. The worked examples are not.

Why it matters now

The gallery has recently taken on prose that states specific extraction behaviour rather than general description, because that behaviour is frequently non-obvious. From #690 alone:

• a gantt lane is named only when the axes carry both set_yticks() and set_yticklabels(), because the extractor trusts a lane name only against a FixedLocator;
• label= does not name a gantt lane;
• stackplot(labels=...) sets Z on a point only if label:, so unlabelled bands are distinguishable by stack order alone;
• levels=6 on a contour is six heights, not six curves — a height crossed twice draws one island per crossing.

Each is a claim about internal behaviour that a refactor could invalidate silently. The docs would still build; they would just be wrong, in the direction of telling a reader their chart is accessible in a way it no longer is.

The same class of drift already bit this repo the other way round: py-maidr #689 shipped a paragraph describing a default axis name while the chunk beneath it overrode that default, so the example never demonstrated the thing the paragraph was about. That was caught in review by a human reading, not by a check.

What would help

A CI job that extracts every {python} chunk from the three gallery pages and executes them, failing on any exception. That alone would catch a renamed argument, a moved import, or an example that stops producing a figure.

Stronger, if it is worth the maintenance: assert the emitted layer type per section, so a chart that silently starts reading as something else fails. A rough sketch of what the check does today when run by hand:

grid = ...  # extract chunk, exec, then read the schema
# Contour Plot   contour    n=6
# Lollipop Plot  lollipop   n=8
# Gantt Chart    gantt      lanes=['Design', 'Build', 'Launch']

That is the loop I have been running manually to verify gallery PRs; making it a job is mostly a matter of deciding where the expected values live.

Notes

Raised by the review on #690, which observed that the "measured against an installed build" claims in these pages are not re-verified by anything automated. Filing separately rather than expanding that PR.

Worth deciding whether the plotly and altair pages are in scope for the same job — plotly chunks build figures without a display and should run headless cleanly; the altair page delegates to the upstream Vega-Lite adapter, so there may be less to assert there beyond "the chunk runs".