alt Hacker NewsIf the data is in Parquet they are already indexed in a sense. No further indexing necessary.
If they are stored in DuckDB’s native format (which I don’t use), it supports some state of the art indices.
https://duckdb.org/docs/stable/sql/indexes.html
I find Parquet plenty fast though.
Ah thanks, of course. I was thinking of dealing with millions of (Geo)JSON files adding up to terabytes, without copying/duplicating them though, mostly indexing. I used to do that with postgres foreign data wrappers and had hopes for duckdb :-). But that's a question for SO or other forum.