Linux has been dealing with this kind of thing for over a decade now. Specialized SoCs love their memory carveouts.