I do see your point on that one, but phrases have an origin.

Of course is like an abbreviation of something like ‘in the natural course of things’. Which has become more like just ‘yes’ over time. In the usage of ‘yes’ it’s easier to argue it could be one word.

... which is in fact in both the OED and MW dictionaries.