Not at all, it doesn't even need to be PKI. But if it was, your routers would be the CA. Or more practically, whatever device is responsible for addressing, also responsible as the authority over those addresses. Your DHCP server would also be the CA for your LAN. Even a simple ND/ARP would require a claim (something like a short byte value end-devices can lookup/cache) that allows it to make that "the address x.x.x.x is at <mac>" statement. Smarter schemes might allow the network forwarder (router) to translate claims to avoid end devices looking up and caching lots of claims locally (and it would need to be authorized to do so).

You wouldn't need TLS. this scheme i just thought would actually decentralize/federate PKI a lot more. If you have a public address assigned, your ISP is the IP-CA. I don't want to get into the details of my DNS replacement idea, but similar to network operators being authorities over the addresses they're responsible for, whoever issued you a network name is also the identity authority over that name (so DNS registrars would be CA's). Ideally though, every device would be named, and the people that have logical control over the address will also be responsible for the name and all identity authentication and claims over those addresses and names. You won't have freaking google and browsers dictating which CA root to trust, it will instead be the network you're joining that does that (be it your DHCP server, or your ISP is up for debate, but I prefer the former). Ideally, your public key hash is your address. How others reach you would be by resolving your public key from your identity, the traffic will be sent to your public key (or see my sibling comment for the concept of cryptographic identity). All names would of course be free, but what we call "DNS" today will survive as an alias to those proper names. so your device might be guelo.lan123.yourisp.country but a registrar might sell you a guelo.com alias that points to the former name.

The implications of this scheme are wild, think about it!

Rogue trust providers will be a problem, but only to their domain. right now random CA roots can issue domains for anything. with the scheme I proposed, your country can mess with its own traffic, as can your isp, as can you over your lan. You won't be able to spoof traffic for a different lan, or isp using their name.

Solve all the problems at their foundations!