When I see Dario say all this I wonder what he's seeing that seems so differentiated from my POV

I wonder how much of Anthropic's success is based on a horde of prompt to code examples that they post-train on. We shouldn't assume Opus' web dev skills are the same as general intelligence

Here's something Sonnet 4 did (Jul 2025) that I don't think a human intern would do when I made a wikisource tool (transcript follows)

#-#-#

F:

let’s see how the wikisource tool wrks.. let’s try to match “Mary hath chosen that good part” in one of the bible sources they should have

Claude: Great idea! Let me first search for some biblical content on WikiSource, then use the content match tool.

wiki_source_search

Perfect! Let me try the Douay-Rheims version of Luke, which should have the Mary and Martha story:

wiki_source_page_content_match

No matches! Let me try a slightly different phrasing – maybe it’s “hath chosen the good part”:

wiki_source_page_content_match

Still nothing. Let me try searching for just “Martha” to see if the story is there:

F:

dummy .. why would use kjv wording in a different edition lol

Claude: Haha you’re absolutely right! I was being completely silly – of course the KJV wording “Mary hath chosen that good part” wouldn’t be in the Douay-Rheims translation. Different translations use different language entirely.

What I Did Wrong: • Used King James Version phrasing in a Catholic translation • The Douay-Rheims says “Mary hath chosen the best part” • I was looking for “good part” (KJV wording)

Should Have Started With: Just searching for “Martha” or “Mary” to see what the actual text says in that translation, then used the exact wording I found.

#-#-#

It's almost comedic.. "Perfect! Let me try the Douay-Rheims version of Luke" is where if you were watching a horror movie you'd say "Don't go down to the basement fridge!"

And I think a human when they first see the text match fail would go "hmm--ohh" but Claude starts doing permutations. This I guess is the side effect of Reinforcement Learning and system prompts that amount to saying: "Just do it. Don't ask questions. Just do it."