Core: Keep NETFLIX_UNSAFE_PARQUET_ID_FALLBACK_ENABLED until 2.0 release - #17444
Conversation
|
FYI @gaborkaszab if you agree with my assessment. |
|
Fine for me. Thx, @dramaticlly ! |
kevinjqliu
left a comment
There was a problem hiding this comment.
LGTM
this was originally marked for 2.0.0 deprecation (https://github.com/apache/iceberg/pull/9324/changes#diff-0ffc14da79d0ef5ce2e3e7da1f9241439061059ae5828c50c38ebf875281e2feR75)
And changed to 1.12.0 in this PR: dramaticlly@831b4ea#diff-0ffc14da79d0ef5ce2e3e7da1f9241439061059ae5828c50c38ebf875281e2feR84
It seems like there are other 2.0.0 -> 1.12.0 changes in that PR. Should we also look at the other changes to see if we should push them back to 2.0.0?
|
@nssalian fyi for 1.12.0 release |
Thank @kevinjqliu. I think it's ok for now, we can have a separate PR if we need to delay more deprecations to 2.0 on case by case. This |
|
thanks @dramaticlly for the fix and everyone for the review, especially @kevinjqliu for digging out the history. |
partially revert the deprecation cycle change of NETFLIX_UNSAFE_PARQUET_ID_FALLBACK_ENABLED in 831b4ea. The reason is this introduce a significant behavior change where parquet id is no longer position based and require explicit name mapping.
This can impact any imported parquet files which is not written by iceberg, particular in addFiles procedure and migrated hive tables.
The effort required to remove this deprecated field can be found in https://github.com/dramaticlly/iceberg/pull/12/changes. I don't think it worth the significant effort before 2.0