Improve error message for invalid design matrix cells - #14123
Conversation
The list of disallowed contents was duplicated and could drift.
The row number in the Excel file typically differs from the internal row number of the Polars dataframe since we remove empty lines, remove headers for the design sheet, and 1-based counting within Excel.
There was a problem hiding this comment.
Pull request overview
This pull request improves DESIGN_MATRIX validation feedback by reporting spreadsheet row numbers (rather than indices after internal filtering) and by clarifying that failures can be due to either empty cells or explicitly rejected cell values. It also documents the rejected values in the user-facing configuration reference and updates/extends unit tests to pin the new behavior (including handling of blank rows).
Changes:
- Track original Excel row numbers while reading both design and default sheets, and use them in validation error messages.
- Treat disallowed textual values (
NONE/NULL/NAN, case-insensitive, whitespace-trimmed) as invalid and reflect that in the error wording. - Add/update documentation and unit tests to cover the new error messaging and row-number stability (including blank/leading blank rows).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tests/ert/unit_tests/sensitivity_analysis/test_design_matrix.py | Updates expected error messages/row numbers and adds regression tests ensuring blank rows and leading blanks don’t shift reported rows. |
| src/ert/config/design_matrix.py | Preserves spreadsheet row provenance via _drop_empty_rows, rewords validation errors, and centralizes disallowed cell values. |
| docs/ert/reference/configuration/keywords.rst | Documents which design matrix cell values are considered invalid. |
| docs/ert/getting_started/howto/design_matrix.rst | Updates the DESIGN_MATRIX validation description and links to the documented invalid cell values note. |
84b59f2 to
a971127
Compare
Cells with certain values are rejected by the DesignMatrix parser. The old error message would always report an empty cell, even if it had an disallowed value, which was confusing for users.
The rejected values were undocumented. A cell holding a word such as NONE is visibly non-empty, so an error calling it empty or invalid is hard to act on without a list of what is accepted.
a971127 to
33d9cd0
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #14123 +/- ##
==========================================
+ Coverage 91.85% 91.93% +0.07%
==========================================
Files 484 482 -2
Lines 33581 33503 -78
==========================================
- Hits 30847 30800 -47
+ Misses 2734 2703 -31
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Issue
Resolves #14108
Approach
Errors for empty or invalid design matrix cells reported a row index into
the already-filtered dataframe, so "Row 1" rarely matched the row the user
had to fix. Rows are now read without dropping blanks, filtered in-process,
and the originating spreadsheet row is carried into the message.
Both sheets anchor their read with
skip_rows, which stops the reader fromtrimming blank rows above the data; without it the numbers drift by the
number of leading blanks. A parametrised test pins this.
The message also no longer claims a cell is empty when it holds a rejected
word such as NONE, and the accepted values are documented.
(Screenshot of new behavior in GUI if applicable)
git rebase -i main --exec 'just rapid-tests')When applicable
merge screenshot-PR in ert-testdata before merging this PR.