It's "on" enough to detect the activity needed to wake it back up. But will a powered-off phone still be pinging cell towers or making WiFi requests?

Use one of the few phones with hardware kill switches or removable batteries.

If you cover your phone with an antielectrostatic bag it can't communicate; that is a Faraday cage.

Since people around you will think you are also wearing a tinfoil hard, you had better stick to the phones with hardware switches as sibling comment mentions