Think of it like SSH authorized keys but automated for the web. Instead of storing the keys in a file; it stores them in a hardware security module (yubikey, or TPM).
Registration generates an asymmetric key pair between your passkey, and the website. Login is the usual challenge/response process. The biggest step forward is phishing resistance. A fake login page can relay a TOTP code, but not the passkey challenge/response.