Skip to content

51_release_checklist.md

Latest commit

 

History

History
128 lines (95 loc) · 5.21 KB

51_release_checklist.md

File metadata and controls

128 lines (95 loc) · 5.21 KB
Raw

Release Checklist

Use this checklist before publishing the repository or cutting a release snapshot.

Repository surface

  • data/, artifacts/, ckpt/, runs/, wandb/, and models/ are empty or absent.
  • No private paths, ad hoc local scripts, or machine-specific configs have been introduced.
  • No secrets, tokens, or cached credentials are present.

Required local gate

Run the unified local gate on a CPU machine with the CPU tier installed. It needs no GPU and no checkpoints:

bash scripts/90_audit/release_gate.sh

This gate currently runs:

  1. python -m pip check
  2. pytest -q tests, including tests/contract/test_release_surface.py
  3. fixture-based math smoke, with the CPU-tier import checks
  4. fixture-based code smoke
  5. dummy figure generation for the plotting path
  6. strict dataset manifest verification, when a prepared data/ is present
  7. model registry resolution (download_models.sh --dry_run 1)
  8. bash scripts/90_audit/pre_release.sh, which includes the prose scan
  9. a check that the gate itself left no artifacts in the working tree

Step 6 is skipped when data/ is absent, which is the state of a clean checkout, so a gate run that never prepared the datasets is a seven-step gate: prepare the datasets first if you want the full one.

Together, pre_release.sh and release_gate.sh verify:

  • Absence of hard-coded private paths
  • Absence of obvious leaked secrets
  • Absence of bundled datasets or generated release-surface artifacts
  • English prose with no em or en dashes, and no CJK outside the functional allowlist
  • Bash scripting syntax sanity
  • Python compilation and test suite sanity
  • That the gate itself leaves no artifacts in the working tree

Both scripts run on a CPU machine with the CPU tier installed. The same two stages run in CI on every push and pull request; see .github/workflows/ci-cpu.yml.

Run tests/contract/test_release_surface.py on its own when you only want the release surface contracts:

pytest -q tests/contract/test_release_surface.py

What is verified, and on what

CPU CI Release audit

Every push runs the CPU tier on Python 3.11: install from requirements/cpu.lock.txt, a dependency consistency check, the full unit suite, both fixture smoke tests, the prose scan, and the pre-release audit. That is what the badges above report.

Two things are worth naming because a reader cannot see them from a green badge.

  • The credit mechanism is checked without a GPU. tests/mechanism/ builds the paper's rollout tree by hand and compares the leave-one-out advantages against values computed on paper, so the arithmetic at the centre of the method can be confirmed on a laptop in seconds.
  • The GPU tier is installed and imported, not trained end to end, in CI. No hosted runner has the hardware. requirements/gpu.lock.txt records exactly which checks were run against it and which were not, and requirements/gpu-paper.lock.txt pins the environment that produced the published numbers.

Extended paper-facing checks

These are recommended before a public paper artifact release:

  1. Strict data verification:
bash scripts/10_data/prepare_all.sh --out_dir data --strict 1
  1. The command lines of the three training recipes, and of the evaluation of record, printed without starting anything:
for R in main long a3; do
  DRY_RUN=1 RECIPE=$R PRETRAIN=Qwen/Qwen3-4B-Instruct-2507 bash scripts/40_train/paper_train.sh
done
python scripts/70_rebuild/final_eval.py --policy Qwen/Qwen3-4B-Instruct-2507 --dry-run
  1. Optional release gate with a real HF base model:
HF_BASE='Qwen/Qwen3-4B-Instruct-2507' RUN_SFT_EVAL=1 bash scripts/90_audit/release_gate.sh

Packaging

  • pyproject.toml carries the release version, and CHANGELOG.md has an entry for it.
  • requirements/cpu.lock.txt was regenerated from a fresh virtual environment, and its header states when and how.
  • requirements/gpu.lock.txt was regenerated and its header lists what still needs a GPU to validate.
  • requirements/gpu-paper.lock.txt is untouched. It is a historical record.

Documentation

  • README.md matches the current release policy and entrypoints.
  • docs/00_getting_started.md matches actual installation and smoke commands.
  • docs/10_code_map.md and docs/20_implementation_audit.md still match the primary implementation path.
  • docs/30_data_sources.md and configs/data_manifest.yaml remain consistent.
  • Every relative documentation link resolves. tests/contract/test_release_surface.py checks this, so a moved file cannot slip through.

Governance and metadata

  • LICENSE, CITATION.cff, and THIRD_PARTY_NOTICES.md are up to date.
  • .github/workflows/ci-cpu.yml still matches the intended release gate, and its two jobs are green on the release commit.
  • Community files (CONTRIBUTING.md, CODE_OF_CONDUCT.md, SECURITY.md) still describe the current workflow.