The averages a "standard" fMRI analysis produces highlights brain areas which may not even have been involved in the majority of subjects, because the pattern is so wide spread. That is in contrast with your usual average or median over e.g. height. It's a bit like averaging squares on a chess board and concluding that the opening is played in the middle two columns.