alt Hacker NewsCurious to learn more about how people are using it?
Are they downloading parquet files and running analyses locally, or are they connecting to Iceberg-like data lake and leveraging DuckDBs query engine capabilities or have you exposed an interface (REST, UI) to query your data?
Replies
We use DuckDB WASM with parquet to build dashboards in-browser. It's cool to be able to write SQL directly in a browser and not have to rely on REST/Graphql/etc to access the data layer.
I've got a couple of different use cases:
- ETL pipelines running on K8s nodes. Using their streaming processing engine means I can run smaller pods/nodes if needed, for datasets that may have required large dataframe-like transformations that may have buffered a big dataset into memory previously.
- A CLI distributed to an internal team to do a postprocessing step on a large modeling dataset - to get it into a consumable format and upload it to a bucket as a .db file.
- A SvelteKit app that used the node duckdb bindings to attach to the .db on the bucket and explore the results through a suite of BI tools. These tables have millions of rows, and would be pretty heavy to store in PG. The DuckDB version works really, really well.
We use WASM DuckDB as the target for an in-browser agentic feature. Generated SQL runs against the user's individual tables that then feed in-browser dashboards.
Excellent performance.
Realtime full MSSQL database mirroring into DuckDb to do a complex reporting. Everything is in-process. DuckDb database mapped to temp storage and recreated on app restart. Still order of magnitude faster then doing a direct query over MSSQL Server (2ms vs 40+ seconds on same query).
Some devs in team still cannot believe that there is no cheating, that it's possibe, that some 60Mb DB can do queries faster then MSSQL Server with just around 250Mb+ of memory overhead.
(.Net 10 + DuckDB.NET package)
I'm using duckdb/duckdb-go as query engine for my Go services: moving hot data from Postgres to Parquet files on S3 or to Iceberg; querying cold data on Iceberg, ... instead of using different Go libraries.
Yes.
I have used it with WASM for some web applications for web use. I have also used with locally for querying 100 gigs of data. And I have used it in the cloud as the serverless gold layer for Apache superset.
maybe a niche use case but i've found it's perfect to store & query random trivia/gameshow questions based on filters for my personal clones of things like Family Feud and Jeopardy
I use it locally with parquet files
ETL from DynamoDB into Ducklake
I actually used it in an interview. I downloaded the csv/parquet file and load it into DuckDB.
In the future, I also plan to use it for testing production data pipelines: imagine you have a streaming cdc pipeline running in development env, and at the end you can dump every parquet into DuckDB as a verification — the end result should be the same. I could also use the same Database for testing, but I like DuckDB somehow.