Of course the infinite loop should run as expected.

It breaks the most fundamental debugging expectations (such as "delete code until problem disappears") if the fundamental, minimal building blocks of a language, when on their own, do random rubbish.

To understand a program that does something, better first understand a program that does nothing.

As a fan of sensible analogies:

You put a salad bowl with vinegar into the fridge and notice that when you do that, the fridge stinks afterwards. You try again without the vinegar, then without the salad. In C++ world, upon receiving the empty bowl, the fridge detonates ("it is not useful"), blowing up your house. That is not OK.