logoalt Hacker News

alexaholic • today at 7:19 AM • 0 replies • view on HN

Glancing over the methodology[1] made me curious about three things:

1. Is there a benchmark measuring how much better an LLM performs when you tell it "you are an X"?

2. Does telling an AI agent "you are an AI agent" actually help? Is it the LLM equivalent of reminding a senior developer they're a senior developer, making them feel guilty about being lazy and finally getting them to do their job?

3. What happens when the persona conflicts with the task? If you tell it "you are a PM" and then ask it to write code, does it struggle? Produce worse code? Spawn a sub-agent and tell it they're a senior developer?

[1] https://artificialanalysis.ai/methodology/intelligence-bench...