logoalt Hacker News

bijowo1676yesterday at 11:00 PM1 replyview on HN

problem is table design and write amplification. Every row insert triggers update into every index, so you get classic amplification problem.

Separate your table into Cold (with all indexes and bells and whistles) and Hot (heap table with no indexes except PK).

Insert as many rows as you want into Hot heap, and then move them in the background into cold in batches, so that index recalculation is amortized across many rows, instead of per-row.

Another poster suggested partitioning, thats the same idea: separate Hot and Cold data into partitions and keep hot partition as heap


Replies

appplicationtoday at 4:48 AM

That’s a really cool idea I had not heard before, thank you for sharing this. It also feels like the type of thing a db ought to be able to do under the hood. I wonder why this is not a config (though there’s a pg extension for everything so maybe it does exist)

show 2 replies