Fair point. Maybe I'm arguing that they shouldn't be given agency, because all they can do is simulate it poorly.

Or... maybe it's that they can't have true agency, because it doesn't make sense to tell a big ball of floating point numbers to make decisions about how it plans to have impact in the world. It can't do that, even if it can play-act doing so.