How to test an Iceberg V3 migration on Amazon S3 Tables

Start with the format boundary

An Athena dependency blocks an in-place Iceberg V3 upgrade on S3 Tables unless that access path changes: AWS says Athena cannot read V3 tables. On September 30, 2026, AWS completed S3 Tables’ V3 type support with geometry, geography, unknown, nanosecond timestamps, and column defaults. Variant was already available, and AWS had announced deletion vectors and row lineage on November 26, 2025. The migration question is whether every production access path supports the required schema and operations. Sources: AWS

AWS’s launch blog says existing V2 readers can keep working after an upgrade until V3 features are used. The Iceberg specification, however, requires implementations to reject table versions beyond their supported version. S3 documentation calls the upgrade one-way, although existing Parquet files are reused. Do not treat the blog’s assurance as migration guidance: unchanged files do not establish that a reader accepts the table metadata. The following plan proposes tests; the research included no executed workloads. Sources: AWS, Apache Iceberg, AWS News Blog

Inventory operations and deployed engines

Inventory readers, writers, and maintenance tools, including scheduled jobs, notebooks, dashboards, streaming applications, and recovery utilities. Record engine versions, connector and Java archive versions, catalog endpoints, authorization paths, and required operations. AWS lists basic V3 support from Amazon EMR 7.12 and AWS Glue 5.1. The launch blog specifies Spark 4.0+, Glue 6.0+, or EMR 8.1+ for the complete new type set, with Parquet-only support. The table below separates these thresholds from operation-specific restrictions. Sources: AWS, AWS News Blog

Qualify the existing schema before adding new types. Redshift excludes structs, lists, maps, binary, universally unique identifiers, time, variant, spatial types, nanosecond timestamps, and unknown in V3. It also excludes equality-delete tables and materialized views over V3 tables, with availability limited to documented deployment configurations. Upgrading changes its Iceberg timestamptz mapping from timestamp to timestamptz. Trino 483 labels V3 experimental and excludes row-level updates, deletes, the optimize operation, column defaults, and encryption. Shared engine ancestry does not establish identical service support. Sources: Trino, AWS

Preserve a baseline and isolate each change

Build independent test tables with representative distributions, partitions, file sizes, schema history, and accumulated deletes. Include a table with repeated updates and compactions; a fresh load does not reproduce that history. Preserve an independent recovery copy in the original format and rehearse redirecting applications to it. AWS describes the upgrade as one-way, so the recovery plan should also account for writes accepted after cutover. Sources: AWS

Capture row counts, business-key uniqueness, partition aggregates, selected complete rows, snapshot references, schema field identifiers, properties, and file and delete metadata. Run the same application queries through every production access path. Record plans, scan bytes, latency, compute consumption, and maintenance activity. Keep engine, partitioning, compression, and schema changes separate from the initial format comparison. Use the following proposed sequence to isolate differences.

1. Measure the original format with current production settings.
2. Test the upgraded format with otherwise unchanged settings.
3. Compare both formats with merge-on-read enabled.
4. Add selected new types and defaults in a separate case.
5. Repeat correctness and performance checks after managed compaction.

Test schema behavior before and after compaction

Use values that expose precision loss and coercion. AWS warns that engines limited to microseconds can truncate nanosecond timestamps. Test events one nanosecond apart, ordering, equality joins, range predicates, and application serialization. Include equivalent instants with different offsets and daylight-saving boundaries. For variant, exercise missing keys, explicit nulls, arrays, mixed numeric and string values, and deeply nested objects. Sources: AWS

The specification assigns initial-default to pre-existing rows, while changes to write-default affect future writes. Test these behaviors separately by adding a currency field with a U.S. dollar default, inserting rows, and changing the write default to euros. Insert another cohort, then check historical rows, both cohorts, and explicit-null rows independently. Unknown reads as null; unknown, variant, geometry, and geography require null defaults. Include null-only evolution tests and spatial fixtures covering empty values, polygons with holes, and antimeridian cases. Sources: Apache Iceberg

Repeat every fixture through each writer and reader after maintenance. AWS says compaction writes shredded variant Parquet by default, potentially breaking older readers, and documents write.variant.shredding.enabled=false as a control. The launch blog excludes variant, geometry, geography, and nanosecond columns from compaction sort keys. Other columns remain available for sort and Z-order strategies. Verify the intended maintenance configuration and treat silent coercion as a correctness failure. Sources: AWS, AWS News Blog

Exercise deletion vectors and lineage across commits

AWS’s launch example sets format-version to 3, then configures delete, update, and merge modes to merge-on-read. Inspect metadata to confirm that the tested write path produces deletion vectors. Iceberg permits at most one vector per data file per snapshot and requires new deletes to incorporate applicable previous vectors or position deletes. Existing position deletes can remain after upgrade, but V3 prohibits writing new position-delete files. Use successive commits to test these rules. Sources: Apache Iceberg, AWS News Blog

1. Compare sparse and clustered deletes with equal affected-row counts.
2. Delete additional rows from the same files, then repeat the delete.
3. Update non-key and partition fields, then run mixed insert, update, and delete batches.
4. Reinsert a deleted business key and exercise overlapping writers with retries.
5. Compare expected key sets before and after compaction, including retained snapshots across the upgrade.

At each stage, capture business keys, row identifiers, and last-updated sequences. V3 requires lineage for newly created rows; equality-delete updates represent removal followed by a new row. Rewrites must preserve applicable existing non-null lineage values. Redshift documents null lineage for pre-upgrade data until the first post-upgrade write initializes it, and requires explicit selection of lineage pseudo-columns. Require stable identity for unchanged rows and investigate maintenance that makes unchanged records appear modified. Sources: Apache Iceberg, AWS

Test incremental consumers separately from lineage visibility. Filtering the current table by last-updated sequence cannot itself return deleted rows or reconstruct every intermediate update between checkpoints. Pin each batch to a defined snapshot and record its sequence boundary. Advance the checkpoint only after the downstream transaction succeeds. Induce crashes around checkpoint persistence, verify deletion propagation, and measure scan bytes independently of result size.

Set workload gates before approving the upgrade

AWS published a deletion-vector demonstration on September 17, 2025 using 10,000 rows, EMR 7.10.0, Spark 3.5.5, Iceberg 1.9.2, Glue Catalog, and ordinary S3 storage. With both versions using merge-on-read, reported delete times were 3.126 seconds for V2 and 1.407 seconds for V3. Delete-file sizes were 1,801 bytes and 475 bytes, respectively. The displayed methodology runs V2 first, provides no repeated-trial distribution, and measures row-count queries. It does not establish a general production speedup or S3 Tables maintenance savings. Sources: AWS Big Data Blog

Benchmark representative operations, including repeated small batches and sustained mutation. Vary the fraction of rows changed, separating random from clustered changes and whole-partition deletes from sparse deletes. Randomize execution order, distinguish cold and warm runs, and collect enough repetitions for median and tail latency. Measure planning time, retries, bytes read and written, file counts, maintenance duration, and total compute, storage, and maintenance cost. Record exact library builds: Apache’s release notes list fixes involving lineage inheritance, nanosecond predicate pushdown, deletion-vector references, and commit validation. Agree on the following acceptance criteria before testing. Sources: Apache Iceberg

• Require every production access path to pass its actual schema and operations.
• Resolve all unexplained row, precision, default, and lineage differences.
• Verify correctness through retries, repeated mutations, compaction, and consumer restarts.
• Meet latency and total-cost thresholds agreed before testing.
• Rehearse recovery to the original format, including writes accepted after cutover.

Resolve documentation conflicts against the exact deployment before approval. AWS’s September announcement states availability wherever S3 Tables is available, while its compatibility matrix retains a partial-Region footnote for variant. Verify the intended Region and seek clarification where required access paths have ambiguous support. Record the decision with named builds, tested schemas, mutation patterns, maintenance behavior, measured outcomes, and the rehearsed recovery procedure. Sources: AWS