CSV are a poor format to access from S3.
Should convert them to parquet then access and analytics becomes cheap and fast.
I agree. That's how our data is produced. We constantly generate real time data into CSV. As far as I can tell, I can't append to parquet file.
alt Hacker News
I agree. That's how our data is produced. We constantly generate real time data into CSV. As far as I can tell, I can't append to parquet file.