Everyone’s eventually going to run a binary they downloaded from the same place, if you’ve already decided to do that, why is a curled install script worse?
The issue does not have to do with whether the download is a binary or source code. It has to deal with verifying the integrity of the download before installation.
Curl piped into a shell command provides no means to verify that the download is uncorrupted and unmodified before running it. For example, whenever I download software manually I check the downloaded file against the verified checksums to ensure that I have an unmodified version. Ideally I check this with gpg --verify on the signed checksum file (against the source's public key). This is a standard procedure for many organizations [1]. If you just download something and immediately run it without this step, you could potentially run a hacked version of the installation script.
Because it normalizes a practice that, while acceptable in context of a well known project with numerous dedicated eyeballs such as Rust language, is not a generally acceptable method of installing software.