alt Hacker NewsIPv6 zones in URLs are a mistake
Comments
And it gets even more fun when browsers such as firefox implemented this, then decided no we won't do it and removed the feature -- now there's no way to access your router web interface over link-local address...
(rationale being that whatwg said no: https://github.com/whatwg/url/issues/392 ; firefox bug https://bugzilla.mozilla.org/show_bug.cgi?id=700999 )
The "solution" is to use a proxy such as https://github.com/twisteroidambassador/prettysocks/tree/ipv... which incidentally encode the % as a `s` and handle special URLs like this http://fe80--1ff-fe23-4567-890as3.ipv6-literal.net for you through the socks dns resolution feature... I've never found anything else that works recently -_-
You complain about URL encoding ? Enter UNC encoding ...
https://devblogs.microsoft.com/oldnewthing/20100915-00/?p=12...
> \\fe80--1ff-fe23-4567-890as3.ipv6-literal.net\share
Hasn’t everyone moved on to ULAs now?
To explain, IPv6 link local addresses are like using a MAC address to send packets. You wouldn’t ever host services on a LL address and things that do are doing it wrong. Every v6 router should advertise a ULA prefix to all downstream clients. If you want to connect to your router’s web UI you’d use its universal local address—not its link local—and avoid all of these problems. This is exactly why zones were deemed mistake and replaced by ULAs and this was 10 years ago… at least!
"IPv6 is weird. One of the more strange parts of the standard is that every interface's link local addresses are in fe80::whatever`."
How is IPv6 weird here, it's the exact same thing in IPv4, no? If you have two different network interfaces, you have to identify which is which somehow, either by assigning a specific IP range to it or by adding some kind of identifier.
Making zones part of addresses in the first place was probably a mistake, I agree, but the problem of address conflicts when users can choose arbitrary addresses certainly isn't a design flaw of IPv6.
IPv6 zones are great. IPv6 neighbor discover is great. Link local doesn’t need SLAAC or DHCPv6 server.
It seems clear that browsers would need to special case this and implement OS specific rules for what a valid Zone ID is.
Browsers have overloaded the URL/search/AI magic bar for years. I don’t quite see how IPv6 Zone ID are that different. They already auto append or auto hide the www subdomain.
That's a bit of a stretch. First, IPv4 can't handle this scenario at all. It's an IPv6 feature. So, let's just be thankful that this exists. Amen.
Second, if you don't want to use interface IDs, you can just enable ULAs on your networks, and routing will take you to the correct interface.
> In order to disambiguate what's the host and what's the port, you typically format the IPv6 address in square brackets, so fe80::4 on port 80 would look like this:
> [fe80::4]:80
I really do wish they'd just stuck with dots. Or if we must upend things, commit to the bit and change the character to separate ports.
I ran into some of these issues when working on IPv6 validation in a library. I found that if you just call system functions like inet_pton, you would also get OS-dependent restrictions on what zone identifiers are valid! This isn't ideal so I wound up just making an IPv4/IPv6 parser with a very liberal zone ID production. Said library also supported URLs, and I did not implement it to parse the IPv6 literal as percent encoded in this edge case, but it winds up working both ways anyways. Is this good? Maybe not: maybe it would've been better to pick a strict subset instead. However, whether or not that would be better depends on specific use cases. Unfortunately, there is just no perfect answer sometimes.
> And with the right scope it looks like this:
[fe80::4%eth0]:80
About here my I felt my heart start to beat really fast and I started to hyperventilate.
I'll just accept that this is as much of a nightmare as it seems.
"so if you have a packet destined to fe80::4, how do you disambiguate it?"
Routing tables get you to the destination but I think the question is about which source address to use ie which network card/interface to use as source - after all, they are all in fe80::.
For a destination in fe80:: the OS will pick the one on the right interface (in effect the IPv6 version of ARP).
You never use fe80:: as a source for a network beyond fe80:: because it and they are link local addresses. You'll send to the default gateway/GoLR/etc unless you have more explicit routes and set your source address as your IPv6 "identity" which might be one of many.
Anyway, here's your problem:
"But if you try to parse this as a URL in Go, you get an error:"
Go needs fixing!
No they are not. There is frankly no other way to handle the cases that IPv6 zones do.
I've never really got why this is so complicated. My interpretation of [] syntax in URLs is "[ enters into a raw address mode", "] exits the raw address mode" and "the characters between the brackets are opaque address characters to be passed to getaddrinfo()".
(It basically has to be this way, or the URL syntax would need to be updated to support future address families with their own address formats. New address families can be loaded at runtime, including ones that didn't exist at the time your current software was compiled -- and this is handled properly by the BSD socket API -- so hardcoding possible address formats is incorrect.)
The _only_ character that needs special handling is ], and if you're willing to declare that you can't be bothered to support link-local addresses at all then declaring that you'll support anything except addresses containing a "]" should be far easier.
Are URLs of link local addresses a common thing with IPv6? I don’t think I’ve ever encountered one myself (but my home network supports ULAs and more importantly DNS).
I've just been implementing a bunch of URL-related utility functions in Go. Decided the most pragmatic solution of handling IPv6 addresses in URL hosts is to outright reject zone identifiers because of the ambiguity in how to parse / serialize them, and the inconsistent ways others have done it (or most of the time, not done it).
RFC 3986 says "This syntax does not support IPv6 scoped addressing zone identifiers." Makes sense because '%' is a reserved character for percent encoding (hence the %25 that Go's net/url expects).
The URL Standard explicitly states "Support for <zone_id> is intentionally omitted."
> In theory, there is guidance for how to properly handle IPv6 zones in user interfaces in RFC 9884, but there's no such guidance for URLs.
RFC 6874: Representing IPv6 Zone Identifiers in Address Literals and Uniform Resource Identifiers (https://www.rfc-editor.org/rfc/rfc6874.html)
Which says that, yes, you need to %-encode the %, so a URL containing a host of fe80::4%eth0 becomes http://[fe80::4%25eth0]/. Yes, that's ugly. Sorry.
> TL;DR: computers were a mistake.
I agree entirely.
(For what it's worth, I am a maintainer of Go's net/url package, and I believe net/url correctly handles zone ids in URLs. It's always possible there's something wrong I'm not aware of. Please let me know if there is!)
In Rust there is the same problem. The `url::Url` library does not support `%<zone_id>`.
`http::Uri` does, and it accepts both `%` and `%25`.
https://play.rust-lang.org/?version=stable&mode=debug&editio...
This just proves that Go URL formatting was a mistake. IPv6 addresses existing long before Go decided on how they should be formatting strings. Python has a similar problem with parts of its standard library.
This is what happens when language and standard library designers ignore a spec like IPv6 for a couple of decades.
Nothing is more idiomatic Go than ignoring inconvenient edge cases.
I use IPV6 ULA to deal with this. (IOT link local only connections). Mind, I also ensured scope routing/etc worked, too, as part of the unit test family.
Couldn't the need for Zones have been solved with ARP-like probing? I.e. if you don't know on which interface to route a link local address, try pinging the address from each interface, and see which one responds.
This response to the post announcing the article is very telling:
I thought fe80::whatever was only for link local, and link local was only for 1-1 communication with router for SLAAC.
After you'd get a unique local than thebn would be used for normal routing needs.
Did I get the wrong?
Also, thank you windows for not having consistent interface ids after reboot. I had to rewrite a configuration file every startup with powershell in order to tackle this case.
I would go further and argue IPv6 zones is a mistake.
I know it solve some real problem. but the cost for solving those is just too high. A careful planning and manual configuration can avoid those.
IPv6 promised a 128bit address space, but we got 128bit + arbitrary length of string instead. This force lots of complication on to application developer.
[dead]
More strange. Stranger. This is strange. Stranger? Who are you?
I don't even understand what's being complained about here. If you want a % in a Uri you need to encode it. It's not rocket science
It gets worse than that.
The Python `ipaddress` library has an `ip_address` address that returns either an IPv4Address or IPv6Address if the passed string is a valid IPv4 or IPv6 address, or throws a ValueError if the address is invalid.
I've seen code that uses that function to determine if a user-supplied string is a valid IP before passing it to a command line. At first glance, that seems fine, but some shell metacharacters are valid in the IPv6 zone ID.
`fe80::1%a;whoami>${PATH:0:1}tmp${PATH:0:1}pwned` is a valid IPv6 IP, and if you did `ping fe80::1%a;whoami>${PATH:0:1}tmp${PATH:0:1}pwned`, you'd have the output of `whoami` written to /tmp/pwned.
Obviously, people shouldn't writing code that puts user input into a shell call without the proper method of execution (ie, shell=False when using subprocess.Popen), but people often think "I validated it, it's fine" and then get popped because their validation wasn't as good as they thought it was.
EDIT: In case it isn't clear, `${PATH:0:1}` is necessary in the attack payload because a `/` is invalid in a zone ID. `${PATH:0:1}` is a tricky way to get a `/` character by just grabbing the first character of your PATH environment variable.