I got a degree in statistics, and what I remember from those times is that we spent maybe 10% of our time learning how to build the model and 90% of the time correctly interpreting the model output. Of course, in the time I learned statistics it was almost exclusively applied in hard science, finance, and actuary. The wild west of machine learning hadn't gotten started. I wonder if they teach it differently now.

When we start using big datasets where n went into the tens of millions we kinda stopped bothering because margins were so small it was effectively certain you found the mean of some range. However I think in that time we forget what confidence scoring actually is and how quickly multiplied alpha can balloon on you.