There has never been a real-world OpenSSH exploit that allows a server to RCE a client that connected to it without a bunch of dubious qualifiers. Connecting to a random SSH server is much, much less dangerous than running a random binary or executing a random curl install script, both of which people do all the time, and is probably about on par with the likelihood of a random website escaping your browser's sandbox and RCEing you.
Agreed, bugs in the terminal emulator are probably more concerning. The attack surface of those is much larger (there are some pretty wild ANSI escape sequences, and terminal emulators are often granted pretty wide disk access permissions on systems that have them if they're also used for local development).
I believe the recent cve-2026-55200 in libssh2 (client-side library) was allowing exactly this. https://nvd.nist.gov/vuln/detail/cve-2026-55200 ("Remote attackers can send crafted SSH packets with excessively large packet_length values to corrupt heap memory and achieve remote code execution.")
Of course the other abouts that you whatted (such as random curl install scripts, binaries, etc.) are still more dangerous.
If you properly set up your ssh client (No agent forwarding or X11 forwarding)
Malicious servers can send malicious terminal escape codes. For example https://www.sentinelone.com/vulnerability-database/cve-2026-...
Web browsers are generally built with security in mind. Terminal emulators surely much less so. The OpenSSH client probably sits somewhat in between, generally developed with security in mind, but not necessarily consistently expecting malicious servers.