alt Hacker NewsA Preview of DuckDB v2.0
Comments
DuckDB is one of the things I've been most excited about in a long time. Introduced it to projects at 3 companies since 2023, greatly lowering resource requirements and running it in a variety of environments. Just having the ability to do out of core bigger than memory data processing on lower end consumer grade hardware is remarkable.
Thanks to the team for everything!
I'm looking for a lightweight client-server database where I can connect 3 or 4 GUI clients to a single database and concurrently edit the database. Low transaction volumes (probably a few edits per minute). Would DuckDb + Quack be suitable?
Hate to bring it up, but 10,000 commits in less than 6 months is a lot. Is AI a major contribute here?
Is AI use for accelerated development of a beloved tool like DuckDB enough to quiet lingering doubters?
We have bet early on DuckDB and Ducklake for Windmill and couldn't be happier. The focus on server/client mode is interesting, it opens the way for orchestrators like ours to have "lite" workers/jobs for duckdb that connect to one central bigger beefy nodes and improve the overall efficieny. I'm very curious if benchmark shows that there are performance benefits to do so thanks to co-location and overall less cpu cycles wastes.
I <3 DuckDB. It has become one of my go to tools for storing, data processing , integrations and now even graph. More importantly it's fun to use because it is so portable. Looking forward to v2.
<3 duckdb run realtime analytics pipeline using a (moderately popular) stream processing engine I built on top of DuckDB. Looking forward to what duckdb provides in terms of perf out of the box!
https://github.com/turbolytics/sql-flow
DuckDB has been a fantastic engine to build on (in python), and processes thousands of events per second, day in an day out, without issue
If you like DuckDB, please consider funding DB research [1]!
I’m mostly using Exasol these days (the concurrency and smooth scaling to multi-node is just too seductive), but with the introduction of Quack I might take another look at DuckDB. I’ll have to see how well it handles many agents reading and writing to it concurrently.
I am a crew member of Joy Of Coding (https://joyofcoding.org) where we invited Hannes to do a talk. He is a great speaker. Seeing this we will need to invite him another time!
> The VARIANT type shipped in DuckDB v1.5, and the way to think about it is JSON on steroids. Basically, imagine if JSON were fast. [...] DuckDB automatically detects the common structure hidden in your semi-structured data and “shreds” it, so it compresses well in storage
I am really looking forward to this hitting v2.0. I can't stand uncompressed JSON - so space-inefficient. But heterogenous JSON in parquet files is such a pain because of schema differences causing fields to be silently dropped. Having DuckDB solve this is exactly what I've been looking for.
Excited about a stable C++ API for extensions!
I made a dry run extension a few months ago (https://github.com/aleda145/duckdb-dryrun), will be so nice to build it just once and know that it will always work.
Also urge anyone to make an extension, the template makes it quite smooth: https://github.com/duckdb/extension-template
Love DuckDB. It’s so fast and portable! I mainly use the query engine as part of my ETL process for creating SQLite database. I’d love to pitch it at work, but we’re heavily invested in BigQuery, which makes it a bit difficult.
The last year of DuckDB enhancements feel like the shift from in-process execution engine (which it is phenomenal at) to an engine that can serve as the foundation of a cloud data warehouse. I know the founders were reticent about not wanting to build that, but I have a feeling it is in the works.
If I could have a pet feature added to DuckDB, it would be some form of native ordered table. In a database like Clickhouse or any of the dedicated time series DBMSes or log stores, there’s a built-in concept that a table might have an order, and the database will optimize based on the order. But, for databases that are logically just bags of rows (traditional DBMSes and also DuckDB [0]), you either need an index or you need to rely on full table scans or at least scans of big blocks. DuckDB does the latter really well, but I think it would be quite nice for some workflows to have explicit ordering. Also, I bet compression could work a lot better with ordering hints.
All that being said, I’m quite excited about DuckDB 2.0. I want to give the improved VARIANT support a try.
[0] Documentation on DuckDB’s native format is rather sparse AFAICT. But the DDL has nothing resembling an ordered table.
I look forward to DuckDB being the engine that underpins the next gen of analytical data tooling. DuckLake already looks amazing and with the Quack protocol seems like it will be a great natural fit for lots more types of tooling, such as sensor data etc.
It's funny to me that we still don't have incremental materialized views. All of the parts are there (export state, agg_state (forget fn name), finalize). I wonder if they're avoiding an explicit war with clickhouse or something. I do recall they mentioned they want to add this to ducklake.
Incremental MVs are ClickHouse's best feature. If DDB adds this, the last moat is distributed query execution.
I love DuckDB genuinely more than sqlite even though they do completely different things but DuckDB has like for me the perfect mix between simplicity, embedded capabilities and expressiveness. (and actual Time and Date Types).
We've built our whole platform around DuckDB at Hex. Our product truly could not exist without it.
It's sad that almost no migration framework supports DuckDB, and overall support is highly limited, but it's a great product I've been using daily for 6 months without encountering a single issue. I hope v2 brings DuckDB more attention and increases third-party support!
Sometimes I prefer DuckDB query language to MySQL or Postgres. It is not even about performance, it just feels right. I just connect to my DBS from DuckDB just to use its query language. And besides it gives you a single language frontend.
I was curious to see they are advertising OLTP-like transactional processing speed. It would be super convenient to have one DB for OLTP and OLAP purposes!
Has anybody here tried using it that way?
(though I don't see any way to deal with write skew and other transactional guarantees - no SERIALIZABLE optimisitic concurrency, no SELECT FOR UPDATE pessimistic concurrency, etc)
Please document the new "extensible PEG-based parser" for extension authors
Really looking forward to that new Async system, especially when reading/querying against thousands of parquet files. This is going to monumentally affect me and my work - I have to query against millions of massive parquet files and the speed has already been rather wonderful, but if those metrics are to be even 100% in range, this is going to make life so much better.
DuckDB is seriously an incredible utility.
> A repository is a name, a URL prefix, and one or more RSA public keys that are trusted to sign the extensions served from it.
Is it too late to beg Hannes and Mark to let us have something like minisign instead of RSA? :)
Very excited for 2.0, congrats folks!
Funny to think one of my favorite software projects this decade is basically "lets make it easy to host your own OLAP database".
How stable is DuckDB these days? I remember it was very buggy compared to Clickhouse.
This is cool
What about the runtime size? I care this because I intend to run a stripped WASM version of DuckDB in browser.
Are there improvements in how memory_limit works? I often had DuckDB get OOM killed because it went beyond its limit. It's definitely one of the reasons why I usually have an AI tune the environment for my datasets.
Great work!
I built a browser tool for querying local Parquet, CSV, JSON, Excel, Arrow, Avro, DBF, and SQLite files with DuckDB-WASM.
Most probably after DuckDB v2.0 release I will revamp my tool as well.
I've been working on a demo database project, and have been really impressed by the UI. So glad they decided to put more effort into it, it has made building a "follow along" tutorial really nice.
DuckDB is so cool, game changer when it comes to local data processing.
With some of these changes, it appears to be encroaching on clickhouse territory. Or are they still very different products?
DuckDB keeps getting better and better. I wonder when something like Apache Gluten will pick it up as a backend.
well done to the duckDB team - one of the features I'm waiting for is real time materialized views.
Was hoping to see procedural functionality like PL/pgSQL... regardless, an astonishing project overall.
How does DuckDB compares with PostgreSQL / MariaDB ?
In terms of project trajectory this is also an interesting contrast. DuckDB is "SQLite, but for OLAP".
DuckDB 1.0 was in 2024. DuckDB 2.0 (new API, new storage format, new ...) is in 2026.
SQLite has been 3.x since 2004.
Would be really cool if they were to add statistical functions too. I'd jump at the chance of getting to use this over pandas
[flagged]
[flagged]
[flagged]
[dead]
Disappointed, since I was expecting they would rewrite the implementation from C++ to Zig. I bet that would increase the number of positive pull requests they get, since most developers prefer to stay away from C++ nowadays.
Same problem, different day
Looks like an awesome release, but the smell of AI from that post is horrid.
Here is a wild idea: is it really so hard to edit out sentences structured and punctuated like this - it's so painfully obvious and distracts from the content. The effect is real.
Super excited about Quack (partially due to the name). I use duckdb for both analytics and runtime, but I do have to serve/handle/manage a giant, multi-GiB duckdb file as effectively a runtime artifact[1]. I'm aware that this isn't the _perfect_ database for this, but the mix of it being fast, having spatial support, sane coding interfaces, great dbt integration, and me being able to do everything between "run a giant several hundred step dbt pipeline" to "query the output of said pipeline" to "read/query a csv on disk" with the exact same tool is just so nice. If I could centrally manage said asset more akin to a traditional database, I'd be very happy.
I've partially solved this with separate databases for different steps in the data pipeline(s) and have even experimented with Clickhouse as a complete alternative, but I really like way too many things about duckdb to replace it.
[1]: If you care: https://skaldmaps.com/blog/2026/07/zip-codes-are-a-bad-spati...