The issue there is that everyone has an inflated sense of their ability to drive at speed, and it's "other drivers" that are the problem.
Growing up in Australia, my dad pointed out two things to me, learning to drive: 1) that in my state at that time, you were required to drive at the maximum speed that was: within the limit, appropriate to the conditions and your ability to control the vehicle, and 2) that if it was to directly and imminently avoid an accident, a driver is permitted, for the minimum time possible, to ignore any road law (i.e. you cannot be cited for crossing over into oncoming traffic if it was to avoid traffic about to hit you. It was also commonly used for people cited for speeding while passing another vehicle - you'd appeal, and say "While I realize that I am still required to obey the speed limit when passing, as I was doing so, the vehicle I was passing adjusted its speed putting me in a situation where I felt the safest course to avoid an accident was to accelerate past them and then return to the speed limit").
But yes, as someone who was both a paramedic for 14 years, I've seen more than my fair share of drivers who think that they are safe (which, to be clear, is general commentary - I obviously don't know you).
If all the many many other drivers on the road are passing you, going 20 mph (30km/h) faster, you are driving unsafely, regardless of the speed limit.