alt Hacker NewsIt's illustrative that there are even errors in how the humans have formulated the task and documented the errors.
They propose:
Valentino → Female Child
Yet this is obviously a boy name, else it would be Valentina. Is this model going to return confidence values? No, they want decisions. This is an inherent problem in AI with reductive logic for these tasks.
Likewise, dates. Is 17/08/22:
- 20170822 (Chinese write the year first, and in speech and chat they will say '17' not '2017'.)
- 20220817 (Everywhere But North America.)
Is 04/06/2009:
- 20090604 (Non-North American)
- 20090406 (American)
Without knowing the cultural context, the location, the document type, the document author, etc, you could be getting it wrong.
Replies
It’s a surname. (Edit you can see this in the first row they use it as an example surname)
>Is this model going to return confidence values? No, they want decisions.
The system doesn't return confidence values because that's not what they are designing. The models, however, do and to me it's interesting that Jev's measure of confidence is pretty well calibrated to its correctness.
>Without knowing the cultural context, the location, the document type, the document author, etc, you could be getting it wrong.
And? There's no perfect process, just ones that asymptotically approach correctness at exponentially increasing cost. They own the system and get to decide what works for them.
Germans write dates like 17.02.2026
It’s an interesting problem. And i have following thoughts to it?
- Are all cases even decidable?
- Shouldn’t one false positive rank higher than a hundred rank negative.
-Shouldnt’t the AI just abstain from decisions to avoid false positives.
- Is it smarter to seperate this in a lot of small sub problems and to define this as a rule set of small yes no decicions, why does this need to be solved agentically?
- You should definitely clean up deterministically first.
I feel like if you really want to solve this problem, you would approach it differently. This seems like a wrong approach and a bad article on top?