Skip to content

Allow excluding fields from ncdata_check - #517

Open
jimmielin wants to merge 1 commit into
ESCOMP:developmentfrom
jimmielin:hplin/exclude_check_data
Open

Allow excluding fields from ncdata_check#517
jimmielin wants to merge 1 commit into
ESCOMP:developmentfrom
jimmielin:hplin/exclude_check_data

Conversation

@jimmielin

Copy link
Copy Markdown
Collaborator

Tag name (required for release branches):
Originator(s): @jimmielin
AI tools used (if applicable; please also add the "AI-generated code" label to the PR):
What: claude-fable:5
How: Discussed the implementation, wrote most of the logic and tests.

Description (include the issue title, and the keyword ['closes', 'fixes', 'resolves'] followed by the issue number):

  • Allows excluding certain fields based on a .gitignore-like glob pattern from physics_check_data.
  • This feature is expected to be used so ncdata_check_err=.true. can be turned on in tests that read multi-stage snapshots where some variables are explicitly intended to be ignored as they would not match.

Describe any changes made to build system: N/A

Describe any changes made to the namelist: host-side ncdata_check_exclude option - default empty

List any changes to the defaults for the input datasets (e.g. boundary datasets): N/A

List all files eliminated and why: N/A

List all files added and what they do: N/A

List all existing files that have been modified, and describe the changes:
(Helpful git command: git diff --name-status development...<your_branch_name>)

M       cime_config/namelist_definition_cam.xml
  - add ncdata_check_exclude

M       src/core_utils/string_core_utils.F90
  - pure core_glob_match and core_glob_list_excluded

M       src/physics/utils/phys_comp.F90
  - add &physics_nl ncdata_check_exclude

M       src/physics/utils/physics_data.F90
  - check_field_2d/3d/4d skip rows whose report label matches the exclusion list
  - write out the excluded list always so it is not a silent drop.

M       test/unit/fortran/src/core_utils/test_string_core_utils.pf
  - comprehensive tests for the added glob matching string utils.

If there are new failures (compared to the test/existing-test-failures.txt file),
have them OK'd by the gatekeeper, note them here, and add them to the file.
If there are baseline differences, include the test and the reason for the
diff. What is the nature of the change? Roundoff?

derecho/intel/aux_sima:

derecho/gnu/aux_sima:

derecho/nvhpc/aux_sima (test is run via Github workflow. Only run the test manually if we need to save new baselines):

If this changes climate describe any run(s) done to evaluate the new
climate in enough detail that it(they) could be reproduced:

CAM-SIMA date used for the baseline comparison tests if different than latest:

…check

Adds an ordered glob-pattern exclusion list for physics_check_data rows,
so test configurations can exclude variables that hold a different
pipeline stage than the check snapshot by design (with the reason
recorded next to the patterns, e.g. in a testmod). With all expected
differences excluded, ncdata_check_err=.true. turns the check into a
hard pass/fail test.

- string_core_utils: core_glob_match ('*'-only wildcard matcher) and
  core_glob_list_excluded (gitignore-style ordered first-match-wins
  decision; a leading '!' keeps a matching name). Both pure; pfUnit
  tests added (14 cases, passing).
- physics_data: check_field_2d/3d/4d skip rows whose report label
  matches the exclusion list; patterns are stored module-side via new
  set_check_field_exclusions, so the generated physics_check_data is
  unchanged. Excluded rows are buffered and listed in the check output
  by flush_check_field_verbose -- exclusion is never silent. Matching
  is on the label a row is reported under, which for
  constituent-dimensioned fields is the per-constituent file variable
  name (e.g. aerochem_vmr_bc_a1).
- phys_comp: new physics_nl entry ncdata_check_exclude (char*256(200)),
  broadcast and handed to physics_data; active patterns echoed at init.
- namelist_definition_cam.xml: entry with semantics and example.
@jimmielin jimmielin self-assigned this Jul 2, 2026
@jimmielin jimmielin added enhancement New feature or request AI-generated code labels Jul 2, 2026
@nusbaume
nusbaume requested a review from peverwhee July 8, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI-generated code enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants