Iceberg

What is Apache Iceberg?

Iceberg is the open table format that turned cloud object storage into a real analytical database. Snapshots, ACID, schema evolution, time travel — on plain Parquet files, readable by every serious engine.
Start the Iceberg course See the lakehouse guide

The problem Iceberg solves

Before Iceberg (and Delta and Hudi), analytical data on S3 lived in Hive-style directory partitions. It worked, badly. No atomic writes. Schema changes rewrote everything. Concurrent readers could see half-written data. Renaming a column was a data migration.

Iceberg fixes this at the metadata layer. Data files stay as Parquet; a stack of JSON/Avro manifests tracks which files belong to which snapshot. Every write is atomic, every read sees a consistent snapshot.

The architecture in one picture

  • Data files — Parquet (usually), stored anywhere object storage is cheap.
  • Manifest files — Avro lists of data files with column stats for pruning.
  • Manifest list — an Avro list of manifests that make up one snapshot.
  • Table metadata (JSON) — schema, partitioning spec, snapshot history.
  • Catalog — points at the current metadata JSON. REST, Glue, Nessie, Polaris, Unity all speak Iceberg in 2026.

Killer features

  • Hidden partitioning. You partition by day(event_ts) once. Queries filter on event_ts directly — Iceberg prunes correctly without users knowing the partition scheme.
  • Schema evolution by ID. Rename, add, drop, reorder columns — no rewrite, no broken downstream jobs.
  • Time travel. Query any past snapshot. Roll back a bad load in one command.
  • Row-level updates. MERGE, UPDATE, DELETE on data lake tables — CDC and GDPR delete without rebuilding partitions.
  • Engine-agnostic. Same table, read by Spark, Trino, Snowflake, DuckDB, Flink. No vendor lock-in.

Iceberg vs Delta vs Hudi

In 2026 the industry has effectively converged on Iceberg as the interchange format. Snowflake, BigQuery, Databricks, and Redshift all read Iceberg natively. Delta Lake still leads inside Databricks. Hudi remains strong for high-frequency streaming upserts. Learn Iceberg first — it's the biggest job market and the most portable skill.

Where to go next

DataForge's Iceberg course walks through table creation, partition evolution, MERGE, time travel and catalog choices with real datasets. Pair with the data lakehouse guide for the architectural picture and the Kafka tutorial for streaming into Iceberg.

FAQ

What is Apache Iceberg?
Apache Iceberg is an open table format for huge analytic datasets stored in cloud object storage (S3, GCS, ADLS). It sits between raw Parquet files and a query engine (Spark, Trino, Snowflake, DuckDB, Flink), giving you database-like features — ACID transactions, schema evolution, hidden partitioning, time travel — on top of a plain file layout.
Iceberg vs Delta Lake vs Hudi — what's different?
All three are open table formats solving the same problem. Iceberg has the widest engine support (Snowflake, BigQuery, Databricks, Trino, Spark, Flink, DuckDB all read/write it natively in 2026) and is the format the industry is standardizing on. Delta is stronger inside Databricks; Hudi shines for streaming upserts. If you're starting today, pick Iceberg.
Is Apache Iceberg a database?
No — it's a table format. It defines how metadata and data files are laid out in object storage. You still need a query engine (Spark, Trino, DuckDB) to run SQL against Iceberg tables, and a catalog (REST, Glue, Nessie, Polaris, Unity) to track which tables exist.
How does Iceberg handle schema evolution?
Iceberg tracks columns by ID, not by name or position. That means you can rename, reorder, add and drop columns without rewriting data files — something Hive-style partitioned tables famously can't do. Old snapshots keep the old schema; new snapshots reflect the change.
What is time travel in Iceberg?
Every write to an Iceberg table produces a new snapshot. You can query any past snapshot with SELECT ... FOR VERSION AS OF <snapshot_id> or FOR TIMESTAMP AS OF <ts>. Perfect for auditing, debugging bad loads, and reproducing a downstream model exactly.

Ready to start?

7 days free. Then less than a coffee per month.

Start the Iceberg course