alt Hacker NewsMy experience is using Pangram quite often with lots of writing of all flavors (including a bunch of known origin).
As for my own writing, I didn't do this experiment, but one of my co-workers did -- and over 176 posts spanning 22 years, all 176 (well, 177 now with my latest) are 100% human. This is not hugely surprising in that (in addition to me having actually written them!) my voice is very... distinctive. What would be more entertaining would be to try to get an LLM to write like me and fool Pangram that way. I still think that this would be difficult based on the experiences that I've heard, but it wouldn't surprise me if you could pull it off (and I would assuredly find the result entertaining!).
In the dimensions that we use Pangram in the most actionable sense (namely, to audit our own public writing), I am unconcerned about false positives, and leave it to Oxide authors to rework/recast as needed. (Though it sounds like Freddie didn't even need to do that -- he just needed to provide a longer sample.)
Now that you’ve seen it can be brittle (e.g. if a small sample is provided, per this single case), would it be sensible to add a disclaimer to the post? It’s a great ad for the tool (& I’d love for a perfect tool to exist!), so it’ll sell subscriptions & we wanna make sure that some teacher out there doesn’t falsely accuse a kid, or engineer doesn’t think worse of their colleague unfairly, etc.
False negatives are mentioned, but the false positive is what could hurt people.
To human writing. Thank you!