That's Google as the hardware OEM, not Google as the OS/platform vendor. They should be standing on Qualcomm's neck until they upstream their drivers and whatever else is necessary to make it practical for anyone to run updated kernels on their hardware, the same as it has worked for PCs for decades.
FWIW, when Windows NT was ported to mobile it also was compiled against binary blobs for specific Qualcomm SoCs. It's not an Android deficiency; what works on PCs just doesn't really work in mobile-land.