alt Hacker NewsThis is a extremely common issue that happens in growing firms.
You start off with everything in Postgres, it makes the most sense. Soon you realize some tables are growing really huge - usually some sort of time-series or log data reaching 10TB+. You can no longer fit it in one node. You can try you luck with some sharding extensions, but they add complexity to upgrades.
In that case it makes total sense to move these large tables off Postgres, and I think Clickhouse is a straight up replacement here. You can still keep your relational heavy tables in Postgres.
Yes it affects you ability to cleanly join data, and guarantee 100% consistency. With some smart application code, and schema design, you can replace parts of Postgres with Clickhouse for the big data problem.
I did this exact thing this year. I moved about >5 TB of event and log data into Clickhouse. I went from having to increase Postgres storage capacity every few months ($$$) to never thinking about it. Migrating to Clickhouse took me from constant timeouts on analytics queries to all queries returning in well under 100ms, and automated data retention using TTLs going from nightly pruning jobs that failed constantly (Postgres hates DELETEs on large tables) to again, never thinking about it. Because of this, I was able to increase retention per-account from a mere 30 days like I had -- just to keep performance and storage costs under control -- to much, much longer. Huge fan.