Dynamic height of virtual scrolling elements is a thing. You just need to recalculate the scrollable height on the fly. tanstack's does it, as do some of the nicer grid libraries.
To be fair I haven't looked at any solutions in about a decade lol
To be fair I haven't looked at any solutions in about a decade lol