X.509 needs to be replaced with a simple key you get for free during domain registration. Then the blast radius of key compromise is a single domain name and not the whole internet.
You mean like DNSSEC DANE?
The blast radius of that is every subdomain of whoever got compromised. Having only ICANN being capable of breaking the entire internet is a world of difference from what we have today, and may be the best we can do... and we should be using that, but it's not perfect either.
How does a client know what public key to expect for a given domain, then?
If they key is distributed by DNS, then you need to secure DNS, e.g. with DNSSEC. Well, with DNSSEC the TLDs and root basically have the same status as CAs (although it does cut down the huge number of CAs that can issue certs).