I read the paper's abstract. They're not actually claiming what commenters here think they are. They're saying that there's some sort of pipeline processing for the task of "listening to speech" and when the subject switches focus from listening to one speaker to another, that pipeline has to drain. And the new stream pipeline has to get loaded up. Not actually surprising when you think about how it would have to work. The same thing likely happens with vision.