Skip to content

feat: Add usePlainNumberFormatForAllCells option to read numeric cells at full precision regardless of cell format - #1061

Open
vladislav-ishchenko wants to merge 1 commit into
nightscape:mainfrom
vladislav-ishchenko:plain-number-all-cells
Open

feat: Add usePlainNumberFormatForAllCells option to read numeric cells at full precision regardless of cell format#1061
vladislav-ishchenko wants to merge 1 commit into
nightscape:mainfrom
vladislav-ishchenko:plain-number-all-cells

Conversation

@vladislav-ishchenko

Copy link
Copy Markdown

Closes #1053.

What

New boolean read option usePlainNumberFormatForAllCells (default false): every non-date numeric cell read into a string column — including cached numeric formula results — is rendered through the existing PlainNumberFormat (full precision, no scientific notation), ignoring the cell's number format. Date-formatted cells keep their formatted rendering, text cells stay verbatim, non-finite values keep POI's display rendering. V2 column naming honors the option too, so a numeric header cell is named consistently with its own data cells.

usePlainNumberFormat only registers PlainNumberFormat for the General/@ format strings, so cells with an explicit number format still render their rounded/scientific display value: 84.789 under 0.00 reads as "84.79", and large formatted numbers read as scientific notation (#126, #771). Widening the registration isn't viable — the custom-format map is keyed by exact format string and shared with date rendering, and 2+-part ; conditional formats bypass the map entirely — so the format-independent path branches on the cell instead.

Also in this PR

PlainNumberFormat appended the unstripped BigDecimal, so single-significant-digit values below 1e-3 gained a spurious trailing zero from Double.toString's d.0E-x mantissa: 0.0005 read as "0.00050". It now appends the stripped value. Only that value class is affected (a sweep over 400k random doubles found no other differences), which means usePlainNumberFormat=true on a General-format cell holding 0.0005 now reads "0.0005" instead of "0.00050".

Tests

Both engines (V1 + V2), including the maxRowsInMemory streaming path: explicit-format rounding vs. plain rendering, General-format scientific notation, text cells verbatim, date cells, a cached numeric formula result, the 0.0005 trailing-zero case, numeric header naming, and a usePlainNumberFormat-only read pinning the boundary between the two options. The V1 tests go through the spark.read.excel(...) DSL so the option-key plumbing is exercised as well.

…s at full precision regardless of cell format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Option to read numeric cells at full precision regardless of the cell's number format

1 participant