alt Hacker NewsI think I read somewhere (and in any case it matches my experience) that the first user will catch 50% of the problems and that with 5 users you can catch close to 90%. So human testing is not just invaluable, it's pretty cheap for the benefits it brings.
Replies
Probably you learned that from them! The first user is 31%, from their testing.
https://www.nngroup.com/articles/why-you-only-need-to-test-w...
Its not just QA and bug hunting.
You want to hire your target audience, which may be vastly different from yourself, and suddenly you discover that the language is throwing them off, the color scheme brings different meanings, and they just don't understand the flow which felt completely natural for you.
It's more like the first user catches all of the critical bugs and a handful of users catch all of the high impact bugs. Cause that's kinda the definition of the impact of critical and high impact bugs.
At every UX job I've had we generally ran usability tests with 6 participants. This was before the online usability testing tools proliferated and we had a lab in our office (one way mirror with observation room and everything). Every UX researcher I worked with said based on the available research and personal experience it is as you said: more than 5-6 participants would hit diminishing returns pretty hard and wasn't really worth it.