That's not a good argument. First, music is not speech. It doesn't require linguistic processing. Second, you can hear chords instead of individual voices, so polyphony can be perceived as shifting harmony, and it usually is.
It's incorrect to reduce polyphony to just shifting harmonies. Harmony is of course the constraint but the whole point is about intertwined voices - various melodies that your brain can process simultaneously. Music is not speech, but it's not that far from the language as it seems in terms of processing. When you hear 2 melodic lines at the same time, you're brain needs to encode them separately, in a similar manner to parallel speech streams.
It's incorrect to reduce polyphony to just shifting harmonies. Harmony is of course the constraint but the whole point is about intertwined voices - various melodies that your brain can process simultaneously. Music is not speech, but it's not that far from the language as it seems in terms of processing. When you hear 2 melodic lines at the same time, you're brain needs to encode them separately, in a similar manner to parallel speech streams.