I have no idea about gaming but in the music world you start noticing latency when it is above 5ms and the difference between 5 and 8ms is tangible. For the most part anything under 10ms is workable and you will quickly adapt as long as the latency is constant. Under 10ms of latency is a level of latency we are used to dealing with, if you are playing guitar and your amp is 10 feet from you, you have ~9ms of latency. Our brains are pretty good at adapting to these short latencies as long as they are predictable.

The visual latency on gaming could be different but I suspect not, I think it is more an issue that some people fixate on the latency, others just accept and adapt to it. Games do have more possible sources of latency, visual, audio and io, and if these can all be different that can be difficult; years ago I had an issue with this and midi, that one really threw me off. Games may also not model the physics of sound? does sound travel slower than light in games? That could worsen the problem since sound does travel slower in real life and we are used to that, we expect it.