Yes, people mix up the concepts of authentication and authorization (access control). Authentication can be really simple if you rely on a standard like JWT.
Authorization is what's difficult and dangerous.