Microsoft (MSAI) showed that quality >> quantity with their Phi model a couple of years ago. It was a small model, but trained on very high quality data, and did better than much larger models.