A compiler that can't magically know how to find headers that don't exist in the expected directory?

Yes, that is the case for pretty much every compiler. I suppose you could build the headers into the binary, but nobody does that.