alt Hacker NewsAI outperforms law professors in Stanford Law study
Comments
This contradicts my anecdata.
Recently, I tasked Opus 4.6 to study a new Czech building permit law in conjunction with some waste disposal regulations and the result was disappointing. The model could not stop drawing conclusions from obsolete regulations in its training dataset, even when given the fulltext of the new law. The usual "you are totally right" also applied and its conclusions were most of the time obviously wrong even to a human with cursory knowledge of the subject.
I ended with studying the relevant regulations myself over the weekend.
I skimmed portions of the study but didn't manage to figure out whether this actually measures a preference for confident mediocrity.
He is basically an AI professor for law. This study just confirms his existence:
Stanford and its donors of course want to replace anyone but its administrators, so they cheer on such anti-intellectual nonsense.
Yes yes, the IPO is near.
[flagged]
[dead]
[dead]
[flagged]
More great news from the prestigious university where 40% of students claim they are disabled
https://fortune.com/article/rise-in-elite-students-seeking-a...
and where they wanted to ban words such as "chief", "stupid", "karen" and "American"
https://reason.com/2022/12/21/stanford-elimination-harmful-l...
Marc Andreessen argued that we've already reached AGI. He says that the top AI models give better answers than 99% of people he has access to, and he has access to some of the best people in their field.
I'm getting more convinced. I mean, sure it makes dumb mistakes sometimes but its a particular set of self serving mistakes, commenting out tests in order to pass. We obv don't want this behavior but I wouldn't say it's dumb.
It'll be like the Turing test, which we just blew past years ago and no one cared. After all the hand-wringing about sentience and rights of the AI if it passes the Turing test, and now we just have AI bots running 24/7 writing slop.
How does everyone else feel?
interesting