Skip to content

fix(archive): anchor component name when matching history files - #5021

Draft
chengzhuzhang wants to merge 1 commit into
ESMCI:masterfrom
chengzhuzhang:fix/anchor-component-name-in-hist-match
Draft

fix(archive): anchor component name when matching history files#5021
chengzhuzhang wants to merge 1 commit into
ESMCI:masterfrom
chengzhuzhang:fix/anchor-component-name-in-hist-match

Conversation

@chengzhuzhang

Copy link
Copy Markdown

Problem

ArchiveBase.get_all_hist_files() builds its match regex by concatenating the component name directly onto the extension pattern:

string = model + r"\d?_?(\d{4})?(_d\d{2})?\." + ext

and then uses re.search, so the component name matches anywhere in the filename — including inside another component's name. The concrete case is E3SM's atmosphere components: eam is a substring of scream, so the eam spec matches EAMxx history files:

>>> archiver.get_all_hist_files("casename", "eam", rundir)
['casename.scream.h.AVERAGE.nmonths_x1.0001-01.nc']   # not an eam file

casename.scream.h... contains eam (in scr-eam) followed by ., and eam's extension h\d*.*\.nc$ then matches the rest.

In a real case only one atmosphere spec is loaded, so this does not corrupt production archiving. Where it does bite is anywhere several specs are in play at once — test_st_archive, which loads every comp_archive_spec in the model — so a component's spec can silently "cover" for another component's broken or missing pattern, and the test passes when it should not. That is exactly what happened in E3SM: a wrong hist_file_extension for scream went unnoticed because eam's pattern matched the files instead (E3SM-Project/E3SM#6190, E3SM-Project/E3SM#8589).

Fix

Anchor the component name at a filename boundary — start of name, or preceded by ., -, or _:

string = r"(?:^|[\.\-_])" + model + r"\d?_?(\d{4})?(_d\d{2})?\." + ext

The leading ^ alternative keeps the existing f.startswith(model) case working, where a file is named after the component rather than the case. The added group is non-capturing, so the existing (\d{4})? / (_d\d{2})? group numbering is unchanged (nothing reads those groups here, but it keeps the diff behavior-neutral).

Every separator that appears before a component name in the naming conventions I could find is covered: casename.eam.h0…, casename.eam_0002.e…, casename.clm2_0002.h0…, ga0xnw.mom6.frc…, casename.mpaso.hist.am…, and the long test-name form …C.fake_testing_only_20160816_164150-20160816_164240.cpl.h.nc.

Tests

Two new tests in CIME/tests/test_unit_xml_archive_base.py, written as pytest functions per AGENTS.md:

  • test_component_name_is_not_matched_as_substring — with both an eam and a scream spec loaded, each component returns only its own history file. Fails on master, passes with this change.
  • test_component_name_at_start_of_filename — covers the ^ alternative, i.e. a file named after the component instead of the case.

Verified locally:

  • pytest CIME/tests/test_unit_xml_archive_base.py CIME/tests/test_unit_case_st_archive.py — 14 passed.
  • pytest CIME/tests/ -k unit — 486 passed, 15 skipped, 3 failed; the same 3 (test_unit_doctest::test_lib_docs and two test_unit_system_tests::test_generate_baseline*) fail identically on unmodified master in this environment, so they are pre-existing and unrelated.
  • black --check clean on both files.

Note

_get_extension() in the same module builds its regex the same way and has the same substring exposure. I left it alone to keep this change small, since it is only reached with a single component's model name at a time; happy to extend it here if you would prefer both fixed together.

Analysis and patch produced by Claude (Claude Code).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 28.65%. Comparing base (f8e59ff) to head (31474a7).
⚠️ Report is 104 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5021      +/-   ##
==========================================
+ Coverage   28.48%   28.65%   +0.16%     
==========================================
  Files         262      266       +4     
  Lines       38490    39024     +534     
  Branches     8146     8278     +132     
==========================================
+ Hits        10964    11181     +217     
- Misses      26268    26568     +300     
- Partials     1258     1275      +17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant