alt Hacker NewsHonest question, how do you know if it's pulling from context vs from memory?
If I use Opus 4.6 with Extended Thinking (Web Search disabled, no books attached), it answers with 130 spells.
Replies
ozim • yesterday at 10:24 PM
Exactly there was this study where they were trying to make LLM reproduce HP book word for word like giving first sentences and letting it cook.
Basically they managed with some tricks make 99% word for word - tricks were needed to bypass security measures that are there in place for exactly reason to stop people to retrieve training material.
➕ show 2 replies
ck_one • yesterday at 10:34 PM
When I tried it without web search so only internal knowledge it missed ~15 spells.
clanker_fluffer • yesterday at 10:16 PM
What was your prompt?
One possible trick could be to search and replace them all with nonsense alternatives then see if it extracts those.