> In fact a decent % of people stops shopping on your site if there's a few ms lag.

While it's still true, I have read that the accepted lag today is higher than 10-15 years ago, because they have lower expectations due to a general decline in page load speed. (React pages with spinners/placeholders, newsletter popups, higher page weights etc.)