I would look at osxcross[0] if you haven't looked into it already. It can at least do the "build stuff targeting macos" part (well, relatively speaking). Swift support kinda breaks down, unfortunately, and actually getting it set up is a bit of a hassle. but it does work.

[0]: https://github.com/tpoechtrager/osxcross