Core: Support explicit row ID assignment in manifest rewrites

[timothyw553]

Problem

Iceberg format v3 requires row IDs for data files. A large-table upgrade may calculate those assignments in parallel, but RewriteManifests currently cannot reserve a caller-defined row-ID range while replacing manifests directly.

This matters when the table must keep its existing snapshot ancestry. An OSS Spark streaming checkpoint stores an Iceberg snapshot ID. If an upgrade removes that snapshot from the current lineage, the stream cannot resume.

Change

This adds a low-level API for distributed manifest replacement:

• ManifestFiles.rewriteDataManifestWithFirstRowIds writes replacement data manifests containing EXISTING entries with explicit first_row_id values.
• Each replacement preserves the original entry's snapshot ID, data sequence number, and file sequence number.
• RewriteManifests.reserveRowIdRange advances the table's next-row-id while committing those manifests.
• The commit rejects non-v3 tables, partial replacement of legacy data manifests, concurrent snapshot changes, concurrent next-row-id changes, invalid ranges, and overlaps within a manifest.
• The caller remains responsible for validating that row-ID ranges are globally disjoint across manifests. This is documented because the core commit intentionally does not collect every manifest entry onto one process.

Normal manifest rewrites are unchanged unless the new API is used.

Correctness requirements

A caller using this API must preserve the live data-file set, replace every current data manifest that lacks row IDs, preserve entry sequence metadata, and assign one non-overlapping row-ID range per live file.

Validation

• Full TestRowLineageAssignment suite
• Spark 4 structured-streaming test through both Hive and REST catalogs
• The streaming test checkpoints a v2 table, performs the v3 manifest replacement, appends data, and resumes twice from the same checkpoint without loss or replay
• API/core Revapi
• API, core, and Spark formatting checks
• git diff --check