Optimizations is one reason for modern languages to compile slowly, but it's certainly not the only reason. Even at -O0, C++ and Rust are much slower to compile. The culprits are more likely much more advanced type-systems and monomorphization passes for generics. Slow compile times are much more inherent to the languages (compared to Turbo Pascal, certainly).
Not really. In addition to Ada and D mentioned by a sibling comment, Ocaml or F# have even richer type systems than Rust, and compile times are better.
Yes, compiling Rust in the release mode could take long time and it would be worth all this baggage - monomorphization, trait resolution etc. But there is no reason, except not giving shit, to no have a fast and simple pipeline for debug builds.
Not even, regarding type systems, see D or Ada compilers.
Or if you prefer to travel back to 1993, here is a developer experience using C++ as if it was Lisp Machine/Smalltalk environment with incremental compilation and hot code reloading.
"Lucid Energize Demo"
https://www.youtube.com/watch?v=pQQTScuApWk
With one of the very first ideas of a LSP could be like,
https://dreamsongs.com/Files/Energize.pdf
We got into a culture that accepted slow compilers is normal, it doesn't have to be.