Skip to content

Core: Keep NETFLIX_UNSAFE_PARQUET_ID_FALLBACK_ENABLED until 2.0 release - #17444

Merged
stevenzwu merged 2 commits into
apache:mainfrom
dramaticlly:NETFLIX_UNSAFE_PARQUET_ID_FALLBACK_ENABLED
Aug 1, 2026
Merged

Core: Keep NETFLIX_UNSAFE_PARQUET_ID_FALLBACK_ENABLED until 2.0 release#17444
stevenzwu merged 2 commits into
apache:mainfrom
dramaticlly:NETFLIX_UNSAFE_PARQUET_ID_FALLBACK_ENABLED

Conversation

@dramaticlly

@dramaticlly dramaticlly commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

partially revert the deprecation cycle change of NETFLIX_UNSAFE_PARQUET_ID_FALLBACK_ENABLED in 831b4ea. The reason is this introduce a significant behavior change where parquet id is no longer position based and require explicit name mapping.

This can impact any imported parquet files which is not written by iceberg, particular in addFiles procedure and migrated hive tables.

The effort required to remove this deprecated field can be found in https://github.com/dramaticlly/iceberg/pull/12/changes. I don't think it worth the significant effort before 2.0

@github-actions github-actions Bot added the core label Jul 31, 2026
@dramaticlly

Copy link
Copy Markdown
Contributor Author

FYI @gaborkaszab if you agree with my assessment.

@gaborkaszab

Copy link
Copy Markdown
Contributor

Fine for me. Thx, @dramaticlly !

@kevinjqliu kevinjqliu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

this was originally marked for 2.0.0 deprecation (https://github.com/apache/iceberg/pull/9324/changes#diff-0ffc14da79d0ef5ce2e3e7da1f9241439061059ae5828c50c38ebf875281e2feR75)

And changed to 1.12.0 in this PR: dramaticlly@831b4ea#diff-0ffc14da79d0ef5ce2e3e7da1f9241439061059ae5828c50c38ebf875281e2feR84
It seems like there are other 2.0.0 -> 1.12.0 changes in that PR. Should we also look at the other changes to see if we should push them back to 2.0.0?

@kevinjqliu

Copy link
Copy Markdown
Contributor

@nssalian fyi for 1.12.0 release

@dramaticlly

Copy link
Copy Markdown
Contributor Author

LGTM

this was originally marked for 2.0.0 deprecation (https://github.com/apache/iceberg/pull/9324/changes#diff-0ffc14da79d0ef5ce2e3e7da1f9241439061059ae5828c50c38ebf875281e2feR75)

And changed to 1.12.0 in this PR: dramaticlly@831b4ea#diff-0ffc14da79d0ef5ce2e3e7da1f9241439061059ae5828c50c38ebf875281e2feR84 It seems like there are other 2.0.0 -> 1.12.0 changes in that PR. Should we also look at the other changes to see if we should push them back to 2.0.0?

Thank @kevinjqliu. I think it's ok for now, we can have a separate PR if we need to delay more deprecations to 2.0 on case by case.

This NETFLIX_UNSAFE_PARQUET_ID_FALLBACK_ENABLED in particular requires significant change and also shift the default behavior for externally provided parquet files, I think keep until 2.0 for major version change is more appropriate.

@nssalian nssalian left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dramaticlly

Comment thread core/src/main/java/org/apache/iceberg/SystemConfigs.java Outdated
@stevenzwu
stevenzwu merged commit c0ecc6d into apache:main Aug 1, 2026
36 checks passed
@stevenzwu

Copy link
Copy Markdown
Contributor

thanks @dramaticlly for the fix and everyone for the review, especially @kevinjqliu for digging out the history.

@dramaticlly
dramaticlly deleted the NETFLIX_UNSAFE_PARQUET_ID_FALLBACK_ENABLED branch August 3, 2026 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants