Allow use of wildcards in definition of seimisc observations/responses - #14118
Open
achaikou wants to merge 4 commits into
Open
Allow use of wildcards in definition of seimisc observations/responses#14118achaikou wants to merge 4 commits into
achaikou wants to merge 4 commits into
Conversation
As it has been pointed out, we start having too much duplication between observations and config, yet we do not have a good place to put it into. So creating a new file to keep common logic. ReservoirData was chosen as a general term to describe RFT/Seismic/Summary response or observation data.
As there was no constrain on realizations, all realizations were joined with all responses, resulting in too many dots on the plot.
achaikou
added
the
release-notes:unreleased-feature-changes
Contributor
There was a problem hiding this comment.
Pull request overview
Adds support for glob-style wildcards when defining seismic observations and seismic response input files, reducing configuration duplication for families of sim2seis-generated CSVs. The change centralizes shared seismic utilities (tolerance checks, response/observation alignment, and pattern-to-filepath resolution) and updates config + GUI/storage call sites accordingly.
Changes:
- Introduce
SeismicDatautilities, including filename-only wildcard expansion and shared tolerance/location alignment helpers. - Update seismic response/observation loading paths to support wildcard patterns and reuse shared utilities.
- Extend/adjust unit tests to cover wildcard expansion and multi-realization visualization behavior.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/ert/unit_tests/gui/ertwidgets/test_ensemble_widget.py | Refactors response-plot assertions and updates test to cover multiple seismic responses. |
| tests/ert/unit_tests/config/test_seismic_config.py | Updates tests to use real temp files and adds coverage for wildcard patterns in SEISMIC response inputs. |
| tests/ert/unit_tests/config/test_observation_declaration.py | Adds coverage for wildcard patterns (and literal metacharacters) in seismic observation CSV paths. |
| src/ert/storage/local_ensemble.py | Switches seismic response/observation alignment helper to SeismicData. |
| src/ert/gui/tools/manage_experiments/ensemble_widget.py | Switches seismic response/observation alignment helper to SeismicData. |
| src/ert/config/seismic_config.py | Uses wildcard expansion for seismic response file discovery and shared tolerance logic. |
| src/ert/config/_reservoir_data_utils.py | Adds SeismicData utilities (pattern resolution, KDTree proximity checks, and alignment helper). |
| src/ert/config/_observations.py | Enables wildcard expansion for seismic observation CSV loading via SeismicData.resolve_pattern_filepaths. |
Suppressed comments (1)
src/ert/config/_reservoir_data_utils.py:128
resolve_pattern_filepaths()returns matches in filesystem iteration order (Path.iterdir()), which is not guaranteed to be deterministic. Sorting the matched paths (e.g., by filename) would make response/observation key resolution stable across platforms and runs.
compiled_pattern = re.compile(fnmatch.translate(filename_pattern))
matching_paths = [
path
for path in search_dir.iterdir()
if path.is_file() and compiled_pattern.fullmatch(path.name)
]
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #14118 +/- ##
==========================================
- Coverage 91.94% 91.93% -0.01%
==========================================
Files 482 483 +1
Lines 33494 33533 +39
==========================================
+ Hits 30795 30830 +35
- Misses 2699 2703 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Changes to other tests are mostly caused by additional check for directory existence that doesn't play nicely with mocked_files.
achaikou
force-pushed
the
seismic_wildcard
branch
from
August 6, 2026 14:51
980f466 to
7bbe069
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
Suppressed comments (2)
src/ert/config/_reservoir_data_utils.py:122
resolve_pattern_filepaths()only checkssearch_dir.exists(). Ifbasedir/pattern_path.parentexists but is a file (not a directory), the lateriterdir()call will raiseNotADirectoryErrorinstead of the intendedon_error(...). Similarly, the non-glob branch should ensure the resolved path is a file (not justexists()) so directory paths don’t get treated as response/observation files.
search_dir = Path(basedir) / pattern_path.parent
if not search_dir.exists():
raise on_error(
f"Directory '{search_dir.absolute()}' does not exist "
"or is not accessible."
)
src/ert/config/_reservoir_data_utils.py:129
- The wildcard branch returns
matching_pathsin the filesystem’siterdir()order, which is not guaranteed to be stable across platforms/filesystems. Sorting the matches (e.g., by filename) makes response/observation ordering deterministic and avoids potential order-dependent behavior/flaky tests.
matching_paths = [
path
for path in search_dir.iterdir()
if path.is_file() and compiled_pattern.fullmatch(path.name)
]
achaikou
marked this pull request as ready for review
August 6, 2026 15:39
ajaust
reviewed
Aug 10, 2026
ajaust
left a comment
ajaust
left a comment
Contributor
There was a problem hiding this comment.
The changes look good. 🙂
I only have some minor comments and questions.
Comment on lines
+114
to
+129
| filename_pattern = pattern_path.name | ||
| has_glob = any(char in filename_pattern for char in "*?[") | ||
| if not has_glob: | ||
| path = search_dir / filename_pattern | ||
| if not path.exists(): | ||
| raise on_error( | ||
| f"File '{path.absolute()}' does not exist or is not accessible." | ||
| ) | ||
| return [path] | ||
|
|
||
| compiled_pattern = re.compile(fnmatch.translate(filename_pattern)) | ||
| matching_paths = [ | ||
| path | ||
| for path in search_dir.iterdir() | ||
| if path.is_file() and compiled_pattern.fullmatch(path.name) | ||
| ] |
Contributor
There was a problem hiding this comment.
We have two explicit paths for the case that we get a full filename or a filename pattern with wildcards. Why can't the regex path also handle the case of filenames without wildcards?
| write_default_seismic_file_content(path_with_literal_wildcard) | ||
| write_default_seismic_file_content(path_fitting_to_wildcard) | ||
|
|
||
| wildcard_pattern = "test**" |
Contributor
There was a problem hiding this comment.
Do we need the double wildcard, i.e., **, or would this also work with a single wildcard, i.e., test* to match both files?
|
|
||
|
|
||
| @pytest.mark.usefixtures("use_tmpdir") | ||
| def test_that_seismic_config_supports_blob_pattern(): |
Contributor
There was a problem hiding this comment.
Suggested change
| def test_that_seismic_config_supports_blob_pattern(): | |
| def test_that_seismic_config_supports_glob_pattern(): |
|
|
||
|
|
||
| @pytest.mark.usefixtures("use_tmpdir") | ||
| def test_that_seismic_observation_filenames_can_be_blob_pattern(file_context_token): |
Contributor
There was a problem hiding this comment.
Suggested change
| def test_that_seismic_observation_filenames_can_be_blob_pattern(file_context_token): | |
| def test_that_seismic_observation_filenames_can_be_glob_pattern(file_context_token): |
| TOLERANCE: ClassVar[float] = 0.1 | ||
|
|
||
| @classmethod | ||
| def get_too_close_coordinate_pairs(cls, coordinates: Any) -> set[tuple[int, int]]: |
Contributor
There was a problem hiding this comment.
Copilot claims that we restrict the typing of coordinates to Sequence[tuple[float, float]] | np.ndarray. Do you think we can restrict it this much?
Contributor
|
Should we also add the wildcard support somewhere to the documentation? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue
Resolves #13913
Approach
It has been confirmed that we want wildcards, not regex.
git rebase -i main --exec 'just rapid-tests')When applicable
merge screenshot-PR in ert-testdata before merging this PR.