There's a researcher at Harvard, Arthur C. Brooks, who wrote a book on this, the importance of the default mode network of the brain that is engaged when you do nothing or are bored.

People who drown out boredom or silence tend to eliminate wandering thoughts, introspection, autobiographical thinking, pondering of relations, effectively the kind of internal reflection that's necessary from time to time. It's pretty common now to hear people say that they're uncomfortable with silence exactly for that reason.