logoalt Hacker News

colesantiagoyesterday at 11:22 PM3 repliesview on HN

(This is the part where you provide extensive extraordinary evidence to your claim)


Replies

rcxdudetoday at 8:34 AM

Scam might be too strong a word but it certainly has far higher false positive rates than they are claiming, and their output is at best misleadingly presented: https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...

show 1 reply
pixl97yesterday at 11:50 PM

No, they are the ones making claims, especially their CEO saying things like a 1/10000 false positive rate. Their own testing showed a 2% rate, which is insanely high when you talk about the number of papers students turn in. Worse their testing methodology compared it with pre-llm documents and not post llm documents that were human written (much harder and more expensive to verify), by treating language as static.

show 1 reply
platinumradtoday at 9:00 AM

Pangram is subjectively very useful and I personally subscribe, but the burden of proof is on them. The product is very much "trust me bro" and I fear that if they ever try to improve recall both their precision and reputation will tank.

show 1 reply