Blog
Deep dives into building Streambed — the challenges, design decisions, and lessons learned building a Postgres-to-Iceberg CDC engine.
The small-file problem gets worse with CDC
Why Streambed is adopting DuckLake for Postgres-to-DuckDB analytics. The small-file problem is well known in the lakehouse world, but it gets worse for CDC workloads. Every flush to a lakehouse table has a cost. It creates more files, more metadata, and more commit work. Updates and deletes make it worse, especially with copy-on-write. Streambed started with Iceberg as the target format. Iceberg is mature, widely supported, and a good choice when you want an open table format across many engines. But Streambed has a narrower path: Postgres WAL → Streambed → DuckDB query server So the question became simpler: If DuckDB is already the query engine, is Iceberg still the best default target for small, repeated CDC flushes? We started testing DuckLake to answer that.
August 30, 2026
Compaction in Streambed
Streambed is getting more mature. Big data has a small problem or more precisely small file problem.
August 9, 2026
Why I Built Streambed
Every Postgres database eventually hits the same wall: analytical queries that are too slow for production but too important to ignore.
April 12, 2026