I mean that's what you'd expect turning off the whole of siri. They provide granular options to turn off individual aspects like the listening for "hey siri" and such. If you wholesale disable the entirety of siri you are going to find it annoying if you use features provided by siri.