There a point on the spectrum where someone cannot function without supervised housing and another one where you have non verbal autistic folks.
But I agree broken is not the right word for those poor folks because they were never functional to begin with!
We need some word and "differently abled" is just silly.
Being blind is a problem. It's not a moral judgment, you aren't less of a person, but you aren't going to be driving any time soon.