At least some cognition researchers think yes it is better to wait than to context switch.

Learning to just sit and "be bored" can train you to have stronger attention on one thing.