>“technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate, and use it,”
If critical decisions affecting human life—such as hiring, lending, crime prediction, and welfare—are processed in an opaque black box, people will lose their fundamental right to explain their context or appeal against the machine's algorithmic verdicts
I feel much much smaller version of it already happened. Many many customer facing processes in companies (from customer support to lost & found departments) almost all automated to a point if you are outside of the norm you are screwed. Time you are going to spent is higher than the value you are going to get most of the time.
For the very same reason these companies are very anti-diversity. Because everything designed for a majority of you are minority in terms of your style of living (which many minorities are) then you are going to struggle, happens to me all the time, and one of the reasons I actually moved out of the US to country that I feel more included).
What is old made new. Franz Kafka passed away June 2nd 1924.
Substitute technology for religion and he's probably one of the least authorized persons to question this.
This is already happening. Most companies are using AI screeners for hiring now. They reject qualified people all the time.