logoalt Hacker News

gnabgib • today at 1:22 AM • 1 reply • view on HN

No, it's not. UTF-16 supports the same characters as UTF-32 and UTF-8 (all three support all of Unicode). UTF-16 does that by having... variable width characters, in the form of 2-UTF16 code units for some chars.

UTF-16 wastes a bunch of space (when you don't need 16 bits to mark a character), and doesn't avoid variable-width encoding. UTF-8 tries to optimize space (but is ASCII skewed) with more variance (but once you can't infer length from bytes, what's the difference?) UTF-32 is a space hog at the benefit of predictable length encoding (a full Int32 to represent the null char, or a carriage return)


Replies

octoberfranklin • today at 1:34 AM

Congratulations, you've discovered why UTF-16 is stupid. You pay the space penalty of two-byte characters and the complexity penalty of variable-length characters.

It's the worst of both worlds!

Seriously nobody advocates UTF-16. The smart people advocate UCS-2.

➕ show 1 reply