> What the internet would need is something like IPv6 with automatic encryption via IPSEC, with IKE provided by DNSSEC.
I understand the appeal of this vision, but I think history has shown that it's not consistent with the realities of incremental deployment. One of the most important factors in successful deployment is the number of different independent actors who need to change in order to get some value; the lower this number the easier it is to get deployment. By very rough analogy to the effectiveness of medical treatments, we might call it the Number To Treat (NTT).
By comparison to the technologies which occupy the same ecological niches on the current Internet, all of the technologies you list have comparatively higher NTT values. First, they require changing the operating system[0], which has proven to be a major barrier. The vast majority of new protocols deployed in the past 20 years have been implementable at the application layer (compare TLS and QUIC to IPsec). The reason for this is obviously that the application can unilaterally implement and get value right away without waiting for the OS.
IPv6 requires you not only to update your OS but basically everyone else on the Internet to upgrade to IPv6. By contrast, you can just throw a NAT on your network and presto, you have new IP addresses. It's not perfect, but it's fast and easy. Even the WebPKI has somewhat better NTT properties than DNSSEC: you can get a certificate for any domain you own without waiting for your TLD to start signing (admittedly less of an issue now, but we're well into path dependency).
Even if we stipulate that the specific technologies you mention would by better than the alternatives if we had them -- which I don't -- being incrementally deployable is a huge part of good design.
[0] DNSSEC doesn't strictly require this, but if you want it to integrate with IKE, it does.
> First, they require changing the operating system
This was done very quickly with IPv6; most major vendors had adequate support very early. This shows that it can be done when the companies involved actually want to do it.
> IPv6 requires you not only to update your OS
Blatantly false. AFAIK, all mainstream OSs today has enough IPv6 support to work adequately in a theoretical IPv6-only environment.
> Even the WebPKI has somewhat better NTT properties than DNSSEC: you can get a certificate for any domain you own without waiting for your TLD to start signing (admittedly less of an issue now, but we're well into path dependency).
Wait for CDS and CDNSKEY record support to be more widespread among TLDs (some support it today, and from what I can see, the number is increasing). Then you don’t need even your registrar to be involved in you DNSSEC deployment, you can just enable DNSSEC in your DNS server and let it deploy automatically.
> being incrementally deployable is a huge part of good design.
Oh, agreed.
> [0] DNSSEC doesn't strictly require this, but if you want it to integrate with IKE, it does.
Yes, this kind of new feature would have to be implemented in a backwards compatible way, with fallback to normal connections if the other end does not support it. One idea would be to put KEY records in the reverse lookup zones; only if such a record exists will you get automatic IPsec.