How are you going to keep your place if the order changes anyway? What is your place if the order has changed?
If the order changes, all bets are off regardless of pagination or infinite scroll.
When asked to load more items, simply take the entire list (which may have changed) and remove the items already shown on the screen.
When asked to load more items, simply take the entire list (which may have changed) and remove the items already shown on the screen.