Most people have experienced what 無為 speaks to without even knowing about the concept or trying to follow it.

That's the whole point. In fact, if you're trying too hard to achieve 無為, you defeat the purpose.