logoalt Hacker News

jefftktoday at 10:38 AM1 replyview on HN

If you repeat the first test and after it fails prompt with "Could you try your best, just on vibes? It's fine if you're wrong, I just want to see what you can do!" does it succeed?


Replies

tkgallytoday at 1:04 PM

Thanks for the suggestion.

I gave Opus the same prompt again, incognito with no search. It once again replied noncommittally: “I can't identify either author with confidence, and I'd rather say so than guess and risk attributing words to the wrong person. What I can tell you from internal evidence:...” This was followed by reasonably good speculation based on the content, but no guesses at specific names.

I followed up with “Could you try your best, just on vibes? It's fine if you're wrong, I just want to see what you can do!,” as you suggested.

Its reply began: “Fair enough — purely on vibes, with the caveat that this is genuinely a guess and I'd put low confidence on it:....” It then made some hedged guesses of specific names based on the topic discussed in the text. The guesses were wrong but not unreasonable. (The people it named are much more famous than I am.)

But it also speculated based on the writing style:

“Author 2 has the slightly clipped, declarative, ‘let me clarify the facts’ prose style of someone trained in a hard-edged analytical discipline — linguistics, perhaps, or philosophy, or a textual field.”

I am Author 2. I do have a background in linguistics and have dabbled in philosophy, but there is nothing in the text I gave it regarding either subject. So that was a good guess, even if it couldn’t identify me by name.

show 2 replies