The original U2F tokens didn't suffer from this problem that much. It contained one secret, it relied on the website to store and return an opaque website-and-account-specific blob, which was decrypted by the token and used to sign the login request. It would be reasonably easy to expand this in a way where the server also stores a backup credential for use by your second/third/whatever hardware token, encrypted with an asymmetric key during the enrollment of the primary token.

But this went out of the window when some genius decided that usernames were too complicated, so Passkeys had to be discoverable, which means they have to be fully stored token-side. Which of course has the nice side benefit of essentially killing hardware tokens and forcing people into using their Android/iOS/Windows device for it.