> or is it just Rust just being confusing?

It doesn't mean "byte length", so much as "byte" happens to be the element type. Unicode is conventionally represented as UTF-8, so the container can't be directly indexed to yield a character.