The fact that, in general, you don’t know what was on the previous pages, if anything. The user may just randomly decide to open page 3 without having visited the previous two pages. Or they clicked “next” after having page 2 open on their computer for several days and meanwhile they used your website on a different device.
If implementing this idea, it wouldn't just be page 3 any more, it would be page numbered 3 excluding IDs 75833,6857362,2737,...
Reddit page links include both the number of the first item on the page (only cosmetic) and the last item on the previous page (which actually determines what is displayed).