logoalt Hacker News

SwellJoetoday at 5:30 PM0 repliesview on HN

Asking them what they think is happening is actually not reliable, though? They don't always know how they came to a conclusion after the fact. It is probable that it's roughly similar to the path they took to get there, since it's the same weights, but it's not certain. And, if you make it standard practice to always collect that data (e.g. if you have an automated tool to ask the model to explain itself after every action to log it), it seems like you might find yourself being blocked for violating terms of service. It looks like "distilling".

In short, there are workarounds, but they're not guaranteed to work forever and they're likely to bump into terms of service.