> This is a strange conclusion

Not really, with the population behavior being this way, though I clearly was mistaken in saying the behavior didn’t have exceptions.

> Moreover, Each starling in a flock of starlings is a separate evolutionary branch in a tree spanning billions of years.

Agreed. And before we brought LLMs into the picture, that just happened to be a feature of everything we’d call an agent.

> Each agent in a LLM swarm here is the same trunk assigned different tasks. If I could clone you, body and mind, this instant and set your team of yous onto some goal, how much defection would you expect? Would it be the same as a randomly picked group? Would that negate the agency that 'you' possess?

We know the answer to this. Genetically identical worms in the lab actually have about 40% distinction in their connectomes even when they’re in the same environment. And no, no lock step behavior. Identical human twins also don’t necessarily grow into identical agents, though there is drive to cooperate more than average, just as with siblings. Genetically identical lab mice in social settings nevertheless establish dominance hierarchies that are stable.

Now, where cloning does definitely lead to cooperation and even sacrifice is within an organism. Two identical genetic copies that lead to distinct organisms, however, will not show identical behavior, and while they will cooperate, there’s no guarantee that holds across contexts.

This distinction in population behavior is what I’m pointing to to say that the assignment of the individual unit, the LLM, as an agent is the flaw here.

To be sure there are agent like dynamics in the behavior, but these don’t come from the LLM, but are from the harness. I need to dig into the data, but I wonder how much of the variance in LLM copy behavior is related to the harness, rather than to any agentic property of the LLM.