Amazon Aurora PostgreSQL now supports direct querying of Apache Iceberg and Parquet data in your data lake
~2 min read
devopsawsaurorapostgresqlduckdbicebergdata-lakeai-agents
Aurora PostgreSQL can now directly query operational data together with data stored in your data lake in Apache Iceberg and Apache Parquet formats, using your existing PostgreSQL applications and tools. By eliminating the need to extract, transform, and load (ETL) structured data from data lakes into your operational database, you reduce operational complexity and simplify application development.
Aurora can also query data lakes managed in Iceberg REST Catalog (IRC)-compatible catalogs — access across a breadth of analytics systems without moving or duplicating data. Whether you're powering real-time dashboards, enriching transactions with historical context, or building AI agents that reason over both live and archived data, it all happens through a single, familiar interface.
Previously, combining recent transactional data in Aurora with historical records in S3 meant building reverse ETL pipelines that duplicated data, increased infrastructure costs, and required ongoing engineering to keep everything synchronized. That burden grows as you embed AI agents into applications — you can't predict and pre-replicate every dataset an agent might need.
The team behind DuckDB (DuckLabs) recently joined Amazon, and this capability is the first visible integration: DuckDB is now embedded directly within Aurora PostgreSQL. You can query live operational data — including uncommitted writes — alongside your data lake in a single query. Query processing stays within Aurora: no additional network hops, no ETL duplication.
You query Iceberg tables managed via the AWS Glue Data Catalog, plus Parquet and Iceberg data in Amazon S3 and S3 Tables — all with familiar PostgreSQL syntax and existing applications and tools. Because the capability is built on DuckDB, future improvements to the open-source engine keep flowing into Aurora and other AWS services.
This kills the reverse-ETL pattern for a whole class of workloads. If your applications — or your AI agents — need to mix fresh transactional data with historical lake data, they can now do it in one PostgreSQL query with zero duplication. The DuckDB-inside-Aurora approach is a genuinely interesting architectural direction worth watching.