Works fine for me on TB+ datasets. Maybe you were doing in-memory rather than persistent database and running out of RAM? https://duckdb.org/docs/stable/clients/cli/overview.html#in-...
Wait, do you insert the data from S3 into duckdb? I was just doing select from file.
Wait, do you insert the data from S3 into duckdb? I was just doing select from file.