Whoa. Is that first query building an index of random filesystem json files on the fly?
It's not an index, it's just (probably parallel) file reads
That being said, it would be trivial to tweak the above script into two steps, one reading data into a DuckDB database table, and the second one reading from that table.
alt Hacker News
It's not an index, it's just (probably parallel) file reads
That being said, it would be trivial to tweak the above script into two steps, one reading data into a DuckDB database table, and the second one reading from that table.