What would a honeypot mean in this scenario? Like they are hoping you'll accidentally send keys? Or that you'll divulge private company information? It seems about the same as any other AI model from OpenAI/OpenRouter etc right?
If you accidentally send private keys to someone logging that data, you need to rotate keys.
But please let me know if you were thinking about something else, I'm interested.
1.) predators asking how-to 2.) wanna-be serial killers or other types like that asking how-to 3.) asking for info related to state secrets
I've seen Experiential Labs insert advertising including the tracked redirect links for them at the end of responses. So who knows what they do with the Chain Of Thought.
And we tend to forget, but we implicitly give any llm provider we use full access to all the tools available in the harness, often including shell, so no changes to the model are required to do malicious things.