Try turning off LTO. Their default codex-rs/Cargo.toml uses `lto = "fat"`, which is... expensive and slow and... you really really don't need it for a local build that you're not distributing.

In my experience, although the build is a little slow, it's that LTO step that takes a million years.