I mean, if you're worried about Signal being a bad actor you also should probably be worried about Intel being a bad actor, and they hold the keys to SGX (especially because the biggest threat, if you're worried about this at all, is going to be governments compelling the involved companies to hand over data or attempt to intercept messages). And Signal is also a third party to your communications, that's how it works. But nothing about SGX makes me think Signal is more trustworthy, it doesn't meaningfully remove actions that they could take to compromise my communications.