There is a really convincing set of arguments against this idea by Robert Seacord[1]. I used to be in the signed size camp, but I've come around to preferring unsigned as much as possible because it's much easier to reason about. I think there are far more footguns than people realize when it comes to signed integers.

[1] https://www.youtube.com/watch?v=82jVpEmAEV4