logoalt Hacker News

runakotoday at 4:34 AM0 repliesview on HN

Disclaimer: I do not like to read LLM-generated text any more than anyone else.

IMHO a big problem with Pangram in particular is that they market it as a reliable tool that can be used to catch students cheating. This can obviously have disastrous effects on young lives, because it is not as reliable as they suggest.

Per their own benchmarks, they do not achieve 100% accuracy even on text that is published on the Internet, and which is likely encoded into the models themselves.

There is validity to their goals, but that is overshadowed by the irresponsible way in which it is marketed.

(All of this, swirling in a context where students are being told that they absolutely must become proficient at using LLMs to do exactly this kind of work by the highest levels of state and federal governments, faculty leadership, as well as the leaders of the workforce into which they hope to graduate. The message to youth is extremely muddled at best.)