I agree the FOSS parts are fine. The problem is that there are reasons you have to carry a cell phone, and there are not FOSS alternatives for those things.

Patching binaries of such mandatory garbage is a nice idea though!

This could solve the main problem I had with graphene in the past, which is that some applications I had to use (like for parking or charging a car or whatever) crashed on startup with a Null Pointer Exception, and the set of broken apps changed from week to week.

I imagine binary patching could be used to bypass hardware attestation checks too (you'd need to extract the key from the enclave, but that's usually pretty trivial, depending on the phone model). None of the apps I used required that.