alt Hacker NewsWait, do you insert the data from S3 into duckdb? I was just doing select from file.
Replies
fastasucan • 05/05/2025
Maybe its your terminal that chockes because it tries to display to much data? 500GB should be no problem.
Wait, do you insert the data from S3 into duckdb? I was just doing select from file.
Maybe its your terminal that chockes because it tries to display to much data? 500GB should be no problem.
Nope, just reading from S3. Check this out: https://duckdb.org/2024/07/09/memory-management.html