In Java, unsigned arithmetic is available through an API and, as you said, it is pretty much only needed when marshalling to certain wire protocols or for FFI. Built-in unsigned types are useful primarily for bitfields or similar tiny types with up to 6 bits or so.
I miss them for doing bit juggling like file headers or networking packets.
However I do concede writing a few helper methods isn't that much of a burden.