I remember my earlier days of Linux of having to compile a kernel module to read from cdrom. Seems like Linux has gone too far in the other direction of having modules that you will probably never need.
That's the same thing that people say about MS Office: nobody uses more than 15% of its feature set, but everyone uses a different 15%. "Linux" having these modules is what keeps it relevant and prevalent in different fields and niches. Whether distro's should ship this many modules by default is a different question, but then we're no longer talking about Linux the kernel.
Not only are there a vast amount of modules you’ll never need, the distros build and distribute most of them.