logoalt Hacker News

elevationtoday at 5:15 PM1 replyview on HN

Older systems were imperfect and were understood to be. I've meet veterans who joined the Navy at 14 or 16. I've met many College students who can pass as old enough to buy alcohol, especially with a fake id. Dead people are sometimes registered to vote. We know this and have systems to try to catch these exceptions.

But cellphone access is different; it's assumed to be perfect, but it's increasingly being moderated by machine learning heuristics that serve as judge, jury, and executioner, severing your services if a couple of your actions trigger a fuzzy approximation to some of the training data.

AI moderation helps suppress spammers, but it's also punishing false positives, and there is just no recourse. Any ID system that piggybacks on "Apple | Google" is effectively shunning some non trivial portion of society. Governments of the people need to provision their own tech systems that are accessible to all citizens, even those who have run afoul of an AI moderation system.


Replies

elevationtoday at 5:28 PM

This year, an octogenarian friend got locked out of his android phone permanently. He had never had a PIN on his Samsung phone.

It started when he signed up at a new bank, giving them his phone number. Somehow the bank enrolled his in their online banking system, which notified Samsung, who remotely initiated the "let's give your phone a pin" flow, presumably to protect him during online banking. (This happened without his knowledge -- he had not installed the bank's phone app.)

Later that day, when his phone went into a modal "let's setup a pin" screen, he panicked, assuming an attacker had gained control of his phone, since this was not something he initiated. No button would let him exit the screen, so he powered it down. Now, when he powers it up, it demands a pin, but he doesn't know what pin that would be. The only way to get the phone back would be to factory reset it, meaning he'd be wiping his data. He had the money to replace his phone, but that may not be true of every citizen, especially at his age.

People assume digital auth systems are perfect. But you don't hear from consumers who can't get online to tell you "I've lost access."

I've shared some other similar stories: a widow who got banned for life from facebook within minutes of making an account from an apple device on a consumer ISP with her real cell phone number.

A coworker attempted to sell his son's sporting goods on facebook marketplace and was banned for life with no appeal because AI thought it was "weapons."

Some high school students each made a gmail address from the same laptop one afternoon, only to be banned the next day. Each supplied their own cellphone number, but the accounts got shut down, presumably because multiple accounts were being created from the same device too rapidly.

AI moderation means there are a ton of unwritten rules, and private companies will keep you out of their platforms if you break them. That's fine, but it means governments have no business serving their citizens from these exclusive platforms.

> e signed up at a new bank, giving them his phone number. Somehow

show 2 replies