I think you're conflating lossy encoding degrading fidelity with the main problem that plagues most audience recordings: the crowd is in the foreground and the band is in the background. One is nearly imperceptible to most people that haven't spent decades in studios like Neil Young, and the other is immediately obvious to everyone.