> but fortunately it's trivial to just not write unsafe Rust

The point of unsafe Rust is to "leave no room for a lower-level language". When writing lower-level facilities it's not uncommon at all to have to resort to unsafe. In practice, relying on the idiomatic standard library support within an unsafe block that might be breaking Safe Rust invariants is almost a sure way of shooting oneself in the foot as soon as the library function is doing anything more complicated than taking a pure value as input and returning a value as output.