Every claim I've seen about unity builds being fast just never rings true to me. I just downloaded the rad debugger and ran the build script on a 7950x (about as fast as you can get). A debug build took 5s, a release build 34s with either gcc or clang.
Maybe it's a MSVC thing - it does seem to have some multi-threading stuff. In any case raddbg non-clean builds take longer than any of my rust projects.
I use unity builds day in day out. The speed up is an order of magnitude on a 2m+ LOC project.
If you want to see the difference download unreal engine and compile the editor with and without unity builds enabled.
My experience has been the polar opposite of yours - similar size rust projects are an order of magnitude slower than C++ ones. Could you share an example of a project to compare with?