If the chat is truly E2E there is no way a data breach can happen on the server side. The same applies if the app is only saving chat logs locally. [1]
Now, if the threat scenario is someone implanting a compromised version of the IM app on every device out there, and siphoning data from the device itself, then it's a completely different scenario.
[1] although this could be intercepted by an attacker compromising the IM servers, if the app is not distributed/P2P
Logs are stored on local devices and many people back them up in whatever cloud (majority not encrypted).
You or the other person could lose the device and someone could use your PIN/password (something as simple as shoulder surfing while you use it). There could also be a leak in whatever cloud service you're using, or the data could get subpoenaed because of some dumb law that gets passed, some rogue employee, etc. It's a huge liability no matter how you look at it.