[feature/capgen-v1] Add missing check for (metadata-)protected variables, reorganize how Claude keeps track of future work

CLAUDE.md

@@ -0,0 +1,82 @@
+# CCPP framework — capgen v1
+
+## Follow-up work: one list, tracked in git
+
+`doc/followups.md` is the **single source of truth** for deferred items, open
+questions, and transient shims awaiting removal.
+
+- Add new follow-ups there.  Do not start a second list in another document.
+- Cite items by ID (`FU-014`).  Other documents reference IDs; they must not
+  restate the items.
+- Closed items keep their row, with the date and the reason.
+- `doc/constituents_overhaul.md` is the register of record for the
+  constituents area and keeps its own status taxonomy; `doc/followups.md` §2
+  indexes into it rather than duplicating it.
+
+## Memory reconciliation (this project spans several machines)
+
+Work on this repository happens on **more than one machine**, and each has
+its own auto-memory, task list, and scratch notes.  None of that travels.
+`doc/followups.md` is what travels.
+
+**Run a reconciliation sweep when any of these is true:**
+
+1. The user asks for one.
+2. You are about to add a new item to `doc/followups.md`.
+3. This machine is missing from the reconciliation log in `doc/followups.md`
+   §7, or its entry is older than the newest commit touching that file.
+
+**The sweep:**
+
+1. Read every local, machine-scoped store you have access to:
+   - the auto-memory directory for this project (`MEMORY.md` and the
+     individual memory files it indexes),
+   - any task/todo list held by the session or the harness,
+   - scratch notes and `RESUME.md`-style files in working directories
+     outside the repository (e.g. sibling repro directories).
+2. For each item found, decide what it is:
+   - **Durable project work** — deferred work, an open question, a decision
+     and its rationale, a shim-removal trigger, a known test failure, a
+     cross-repo dependency.  These belong in `doc/followups.md`.
+   - **Machine-local fact** — where clones live on *this* machine, local
+     paths, shell/toolchain setup, personal working preferences.  These stay
+     in auto-memory and must not be copied into the repository.
+   - **Session-scoped noise** — intermediate reasoning, superseded plans.
+     Discard.
+3. For each durable item, check whether `doc/followups.md` already covers it
+   (match on substance, not wording).  If not, add a row with a new ID, the
+   date it was raised, and a `file:line` or document-section pointer.  If it
+   is covered but the local store has extra detail — a rationale, a
+   reproduction, a decision that was made — fold that detail in.
+4. If a local memory contradicts `doc/followups.md`, the *newer* evidence
+   wins; correct the stale one and say which you changed.
+5. Update the reconciliation log (`doc/followups.md` §7) with this machine's
+   hostname and the date.
+6. Report what you added, folded in, or corrected.  Do not silently rewrite
+   existing rows.
+
+**Do not delete auto-memory entries just because they were copied into
+`doc/followups.md`** — replace the durable content with a one-line pointer to
+the ID so the local store stays useful without becoming a rival list.
+
+## Verifying claims about original capgen
+
+When the question is "what did original capgen actually emit?", read
+`origin/develop:test/*/*_host_integration.F90` — those files record the real
+expected call lists and are the ground truth.  Reasoning from an early
+return in `scripts/suite_objects.py:match_variable` (or any other single
+code path) is **not** proof and has produced a wrong, pushed commit
+(`501d1c0`, since reverted: it claimed original capgen never put
+register-phase `ccpp_constituent_properties_t` args on a call list;
+`test_advection_host_integration.F90` lists them in both `test_outvars1`
+and `test_reqvars1`).
+
+## Documentation cross-references
+
+Committed documents must not cite auto-memory files.  Memory is per-machine,
+so such a reference is dangling for every other machine and for every
+reviewer.  Four such references accumulated before 2026-07-28
+(`project_implementation_status.md`, `design_constituent_api.md`,
+`design_constituents_mutability.md`, `design_constituent_host_wins.md`) and
+none of the targets existed in this clone.  Cite a committed document, a
+`file:line`, or a `doc/followups.md` ID instead.

capgen/generator/suite_resolver.py

@@ -1709,6 +1709,19 @@ def _resolve_one_arg(
         # Both found — host takes precedence (suite data shouldn't duplicate host).
         source = 'control' if host_entry.is_control else 'host'
 
+    # A protected host variable is read-only to physics.  Passing it to an
+    # intent(out)/intent(inout) dummy is also invalid Fortran, so without
+    # this the generated cap fails to compile instead.
+    if (host_entry is not None and host_entry.protected
+            and intent in ('out', 'inout')):
+        raise CCPPError(
+            "Variable '{}' (standard_name='{}') is declared intent({}) by "
+            "scheme '{}' phase '{}', but the host marks it protected; only "
+            "intent(in) is allowed for a protected variable".format(
+                local, std_name, intent, scheme_name, phase
+            )
+        )
+
     # ---- build access expression -----------------------------------------
     if host_entry is not None:
         # ``host_entry.access_path`` is the verbatim form from

capgen/metadata/metadata_table.py

@@ -772,7 +772,7 @@ def _prop_snapshot(self) -> Dict[str, str]:
 
     # ------------------------------------------------------------------
     def validate(self, require_intent: bool, context: ParseContext) -> None:
-        """Check that all required attributes are present.
+        """Check that all required attributes are present and consistent.
 
         Parameters
         ----------
@@ -784,7 +784,8 @@ def validate(self, require_intent: bool, context: ParseContext) -> None:
         Raises
         ------
         CCPPError
-            If any required attribute is missing.
+            If a required attribute is missing, or the variable is
+            ``protected`` with an intent other than ``in``.
         """
         required = {'standard_name', 'dimensions', 'type'}
         if require_intent:
@@ -796,6 +797,11 @@ def validate(self, require_intent: bool, context: ParseContext) -> None:
                     self.local_name, sorted(missing), context
                 )
             )
+        if self.protected and self.intent not in (None, 'in'):
+            raise CCPPError(
+                "Variable '{}' is marked protected but is intent {}, "
+                "at {}".format(self.local_name, self.intent, context)
+            )
 
     # ------------------------------------------------------------------
     def __repr__(self) -> str:

doc/briefing.md

@@ -253,32 +253,31 @@ control-variable arguments to the public entry points.
 
 ### 7.1 Deferred — to be resolved in upcoming work
 
-- **Constituents overhaul.**  Three reform proposals on the table
-  (`doc/constituents_overhaul.md`); decision pending an upcoming
-  meeting.  Pieces involved: framework setter additions
-  (`set_advected`, `set_diagnostic_name`, `set_default_value`),
-  `is_match` relaxation, Class A vs Class B property classification.
-- ~~**Validator host-metadata check.**~~  **Landed 2026-06-01**:
-  `ccpp_validator.py --host-files` validates `type = host` and
-  `type = ddt` tables against the Fortran (`doc/migration.md` §7.4).
-- **Codegen-time scheme-registration cross-check.**  Today's
-  registration check is at runtime
-  (`ccpp_initialize_constituents`).  Stronger options: new metadata
-  attribute `registers_std_names = a, b, c` on register-phase
-  tables; cross-check at codegen.
-- **Nested-subcycle `ccpp_loop_counter` semantics.**  When a scheme
-  inside a deeply nested subcycle asks for `ccpp_loop_counter`, it
-  currently resolves to the **outermost** loop's counter.  None of
-  the in-tree physics catalogs uses the inner-counter case.
-- **`ccpp_datafile.py --host-files` repurpose.**  The current
-  `--host-files` returns the generated host-API file; should be a
-  filtered list of *input* host metadata files (parallel to the new
-  `--scheme-files`).  Deferred.
-- **`ccpp_host_constituents.F90` suppression** when no suite touches
-  constituents (file is correct-but-empty under host-wins; should
-  not be emitted at all).
-- **Python linter / formatter pass.**  Pick `ruff`, apply across
-  `capgen/`.
+Tracked in **`doc/followups.md`**, the single source of truth.  This
+section used to carry its own bullet list; it drifted out of step with the
+parallel lists in `migration.md` §8 and `redesign_prompt.md`, and all three
+were merged there on 2026-07-28.
+
+The headline items for a reader of this brief:
+
+- **Constituents overhaul** — three proposals on the table
+  (`doc/constituents_overhaul.md` §8); decision pending a meeting and
+  gating the framework setter additions.  `followups.md` FU-020, FU-003.
+- **Enforce `protected`** — a scheme can currently write a host variable
+  the host marked read-only; original capgen errored, capgen v1 does not.
+  FU-014.
+- **Codegen-time scheme-registration cross-check** — today's check is at
+  runtime.  FU-002.
+- **Nested-subcycle `ccpp_loop_counter` semantics** — resolves to the
+  outermost counter.  FU-001.
+- **Transient shims** — `--legacy-mode`, `--gfs-dim-aliases`,
+  `--legacy-auto-clone-constituents`, CAM-SIMA's `capgen_compat/`, each
+  with an explicit removal trigger.  FU-010 … FU-013.
+
+Landed since this section was first written: the validator host-metadata
+check (FU-008, 2026-06-01).  Closed as *decided against*: suppressing
+`ccpp_host_constituents.F90` when unused (FU-009) — see that row for why,
+and do not re-propose it.
 
 ### 7.2 Intentionally NOT supported
 

doc/code_walkthrough_DRAFT.md

@@ -392,7 +392,7 @@ Three questions prebuild developers always ask:
 - constituent-ness is ultimately the **host’s** decision. A scheme that only *reads* a name
   need not re-flag it — capgen infers it from the set of names *some* scheme flags (“rule
   b”). If the host declares the name as an ordinary variable, that wins
-  (`design_constituent_host_wins`).
+  (the **host-wins** rule, §8.3).
 
 **2. Where/how are constituents registered?** Exactly one way to declare a *new* one (Rule 1):
 a **register-phase** scheme returns an `intent=out, allocatable` array of
@@ -495,7 +495,8 @@ a reliable source of the standard name. At init, the framework fills each index
 
 > **Host-wins:** if the host itself declares the `index_of_*` / framework names, the resolver
 > short-circuits to ordinary host-arg resolution (the constituent path is skipped). That’s the
-> `design_constituent_host_wins` rule.
+> **host-wins** rule, implemented by the `host_dict` short-circuit in
+> `_resolve_constituent_arg` (`capgen/generator/suite_resolver.py`).
 
 ### 8.4 The whole constituent axis — `number_of_ccpp_constituents`
 

doc/constituents.md

@@ -666,21 +666,22 @@ framework files (listed under `<utilities>` in `datatable.xml`):
 The host's CMake should query `ccpp_datafile.py --utility-files` to
 get the absolute paths to these files at the right output location.
 
-> **These four are listed only when some suite touches constituent
-> state.** `<utilities>` answers "what do the generated caps need",
-> which is all capgen can determine from metadata. If the *host's own
-> Fortran* uses the constituent API — `use ccpp_constituent_prop_mod`
-> in host code rather than only through the generated caps — then the
-> host needs these modules compiled even for a suite with no
-> constituents, and capgen cannot see that. Such a host must add them
-> to its build itself; querying `--utility-files` alone will silently
-> produce a build that fails with `Cannot open module file
-> 'ccpp_constituent_prop_mod.mod'` the first time someone configures a
-> constituent-free suite.
+> **These four are listed unconditionally** (since 2026-07-27, FU-009).
+> `ccpp_host_constituents.F90` is generated for every run — with a
+> zero-size table when no suite touches constituent state — and it
+> `use`s `ccpp_constituent_prop_mod`, so the framework sources are
+> always a dependency of the generated caps and always appear in
+> `<utilities>`.
 >
-> CAM-SIMA is exactly this case and declares them in
-> `cime_config/host_framework_deps.py`. See `constituents_overhaul.md`
-> §4.17 for the failure and the reasoning.
+> Before that change they were scoped to suites that actually used
+> constituents, on the reasoning that `<utilities>` answers "what do the
+> generated caps need".  That broke hosts whose *own* Fortran does `use
+> ccpp_constituent_prop_mod` outside the generated caps — something
+> capgen cannot see from metadata — with `Cannot open module file
+> 'ccpp_constituent_prop_mod.mod'` the first time someone configured a
+> constituent-free suite.  See `constituents_overhaul.md` §4.17, and
+> §4.18 for the unresolved question of whether a host may `use`
+> framework modules directly at all (FU-021).
 
 ---
 
@@ -869,16 +870,18 @@ message naming the offending token.
 
 ### Open work items
 
-- **Unconditional `ccpp_host_constituents.F90` emission.** The
-  generator currently emits `ccpp_host_constituents.F90` for every
-  build, even when no scheme or host actually uses the constituent
-  system (no `ccpp_constituent_properties_t(:)` register-phase arg,
-  no `is_constituent`-flagged scheme arg, no framework-named
-  `index_of_<X>` / `ccpp_constituents` / etc. claimed by capgen).
-  When the host owns its own indices (SCM/GFS) and no scheme exercises
-  the constituent path, the generated file is dead code that should be
-  suppressed.  Tracked as a deferred item; the `host_dict` precedence
-  rule above already keeps the file *correct* (empty) in that case.
+Tracked in `doc/followups.md`; constituent-specific items are indexed in
+its §2, which points into `doc/constituents_overhaul.md`.
+
+One correction to what this section used to say: **`ccpp_host_constituents.F90`
+is emitted unconditionally, and that is deliberate** (FU-009, decided
+2026-07-27).  It was previously listed here as dead code to be suppressed.
+The host cap re-exports this module's public API, so gating emission on
+suite content would make `<host>_ccpp_cap`'s interface expand and contract
+with the suite — not something host code can compile against.  With no
+constituent state the module is still valid: a zero-size table,
+`ccpp_number_constituents` answers 0, `ccpp_constituents_array` returns a
+zero-size array, and host loops over it are no-ops.
 
 ---
 

doc/constituents_overhaul.md

@@ -964,7 +964,9 @@ construction.
 
 ## 5. Property classification (Class A vs Class B)
 
-Proposed in `design_constituents_mutability.md` 2026-05-12. Each
+Proposed 2026-05-12.  (The original write-up lived in an auto-memory
+design note that does not travel between machines and is no longer
+available; the classification below is now the record.)  Each
 constituent property is conceptually owned by either the scheme
 (physics-portable, immutable once instantiated) or the host
 (host-configuration, mutable post-instantiation).
@@ -1284,8 +1286,10 @@ keeping.**
 
 ## 9. Appendix: framework setter inventory
 
-(For reference during the meeting. Reproduced from
-`design_constituents_mutability.md`.)
+(For reference during the meeting. Originally reproduced from an
+auto-memory design note that does not travel between machines; this table
+is now the record. Verify against
+`src/ccpp_constituent_prop_mod.F90` before relying on it.)
 
 `ccpp_constituent_properties_t` methods (`src/ccpp_constituent_prop_mod.F90`):
 
@@ -1341,9 +1345,8 @@ setters that delegate to the underlying `ccpp_constituent_properties_t`.
 ## Cross-references
 
 - `doc/constituents.md` — capgen's user-facing constituents reference.
-- `design_constituent_api.md` (memory) — capgen's per-instance option-A design.
-- `design_constituents_mutability.md` (memory) — extended design notes incl. class A/B classification.
-- `project_implementation_status.md` (memory) — current implementation state and deferred items.
+- `doc/followups.md` — deferred items and open questions across the project;
+  §2 indexes this document's own §4 / §7 / §8.
 - `scripts/constituents.py` — original capgen's host-cap generator.
 - `src/ccpp_constituent_prop_mod.F90` — framework.
 - `capgen/generator/host_constituents.py` — capgen's host-side module emitter.