The Emergence of DuckDB: A Game-Changer in the Database World
The emergence of DuckDB has been one of the most surprising developments in the database world over the past couple of years. The “unapologetically single node” vectorized column store, which was developed by Hannes Mühleisen, eschews the complexity that comes with its bigger distributed cousins, while still delivering lots of usable SQL performance.
A Humble Beginning
Mühleisen developed DuckDB while working as a professor of data engineering at Radboud University Nijmegen in The Netherlands. He has since gone on to become the co-founder and CEO of DuckDB Labs, which provides support for the database, which has been called SQLite for analytics.
A Conversation with Hannes Mühleisen
We recently caught up with Mühleisen, who we named a BigDATAwire Person to Watch for 2024. Here’s our conversation:
BigDATAwire: Were you surprised at the success of DuckDB? What do you see for the future of the software?
Hannes Mühleisen: Most new Open Source projects never gain any traction, so that was also our expectation in the beginning. However, before starting DuckDB we had experimented with a hacky prototype and that did get some people excited so we had the confidence to start from scratch and dedicate the next couple of years of our life to it. Of course, we are absolutely humbled by the now widespread adoption of DuckDB, and we were certainly surprised by the magnitude.
The Future of DuckDB
For the future, we hope that choosing DuckDB for data processing will become a no-brainer, something nobody will have to argue for in their team. We are quite looking forward to releasing a 1.0 of DuckDB this year.
An Interesting Use Case
BDW: What’s the most interesting use case for DuckDB that you have heard of?
HM: When Andre Kohn first showed DuckDB-WASM to us, I was pretty blown away. DuckDB-WASM is a version of DuckDB that runs entirely within a Web Browser using the WebAssembly framework. I would not have thought that compiling a fairly complex C++ project like DuckDB to WASM would be feasible or efficient. I have been proven wrong and the result is powering a new generation of analytics apps. Besides WASM, people often show use cases in surprising environments to us like someone running DuckDB on “big iron” IBM mainframes or running data processing entirely within Lambda functions.
The Right Call
BDW: How do you think DuckDB would have turned out if you had taken venture capital money and were based in Silicon Valley instead of The Netherlands?
HM: I think we made the right call not taking venture capital. Because DuckDB’s development started in academia, we had already been able to quietly work on the software for a couple of years and get some traction already. So by the time we founded the spin-off company DuckDB Labs, we already had customers lined up that wanted to pay for consulting and support for DuckDB. So we did not really need investment to get going, and we still don’t need investment to push the envelope in data engines, even with a small team. Being based in Amsterdam certainly helped as well, operating costs are much lower than in Silicon Valley and a certain level of oddness is somehow acceptable from Europeans. For example, I feel that our bet on single-node operations would have been more difficult elsewhere.
Outside the Professional Sphere
BDW: Outside of the professional sphere, what can you share about yourself that your colleagues might be surprised to learn – any unique hobbies or stories?
HM: Well, as some people know DuckDB is named after my pet duck “Wilbur.” He has since flown away to start a duckie family, or so I hope. I had chosen a duck as a pet because I live on a historic sailing ship in the Amsterdam city center with my family. Sometimes, we even take the old rust bucket out sailing and as hobbies go, it’s one of the more intense ones.
Conclusion
DuckDB has emerged as a game-changer in the database world, offering a unique approach to data processing that eschews complexity in favor of simplicity and usability. With its vectorized column store and single-node architecture, DuckDB is poised to revolutionize the way we process data.
Frequently Asked Questions
Q: What is DuckDB?
A: DuckDB is a vectorized column store database that offers a unique approach to data processing, eschewing complexity in favor of simplicity and usability.
Q: How did DuckDB come to be?
A: DuckDB was developed by Hannes Mühleisen, a professor of data engineering at Radboud University Nijmegen, and is now the foundation of DuckDB Labs, a company that provides support for the database.
Q: What is the future of DuckDB?
A: The future of DuckDB looks bright, with the company planning to release a 1.0 version of the database in the near future.
