That's true, but in another sense I think it's just that trying to "do it all" encryption-wise is significantly harder than some people realize. Having encryption that's really "safe" raises barriers to casual use that most casual users aren't really willing to accept.

Like suppose I use Matrix only on my phone, so I just have the one device. Then I lose my phone and have to get a new one. How do I regain access to my account, including all of my old messages? Or suppose I (still using Matrix only on my phone) decide I need to log out because I want to let someone else (a friend, my kid) use my phone for a while and don't want them snooping in my messages. How do I retain access to all the messages I receive while logged out?

I'm not saying these are problems with Matrix; they would be problems with any service that attempts to cover the same bases (in particular, e2ee with forward secrecy and multiple independent devices). The average user's conception of a messaging service is "I can log in with my password and then have total access to all of my messages, past, present and future." There are too many ways to break that assumption if you try to have perfect forward secrecy and all these other desiderata that encryption wonks care about but normal people don't. I think this is one reason there's still a big gap between comments on HN saying "matrix still works fine for me" and the tales of "I tried to get my grandma to use this and it was a disaster". I don't think it makes sense to try to roll Matrix out for general use, or say it's "the best messaging app" until it can smoothly handle all of those use cases.