I agree that mixing signed and unsigned types creates friction. After trying to use unsigned for sizes and indices for a while, I changed my mind and now use signed types almost everywhere, except for modulo arithmetic and bit masks, and I found this removes all problems I had before.

I don't think the usual arithmetic conversions are very problematic, IMHO there is a lot of FUD about this. The rule is also very simple: "Operands are converted to the higher ranked type but at least to the rank of int, and to the signed type if it covers all values of both types and otherwise to the unsigned version." This is the complete rule.

I like the your final version, but it is a bit too clever for my taste.