I think it feels off because an HTML file can include scripts, fonts, images, videos, styles, and probably a few other things. But not HTML. It can probably be coded with a custom element (<include src=.../>). I would be surprised if there wasn't a github repo with something similar.
I created something like this relatively recently. The downside is of course that it requires JavaScript.
https://github.com/benstigsen/include.js