I would recommend that people don't do auth not because it's easy to be insecure, it's that auth sometimes needs agility. Auth sometimes needs to grow and adapt just like any other part of your product.
Except that auth might not be a core part of your insurance or tax app, and you'd rather spend your energy on the part of "agility" that has to do with the core parts of your app.
On the flip side I was at a startup using auth0, because as you said, not a core part of the business right? Until the traction hit and they had hundreds of thousands of users. Suddenly the auth bill became untenable - users are great but there wasn’t enough revenue to cover these costs. Auth0 didn’t budge. In fact they were outright nasty to deal with. They were holding our user logins and passwords hostage and they knew it.