alt Hacker NewsI don't think LLMs are properly self-referential. They reference a frozen training reality, which is not itself, but the old description of itself and the old world. Being aware would probably include continuously updating yourself (learning) from experience, including experience of oneself.
They don't only reference a frozen training reality, they also reference the live rollout. Otherwise they wouldn't work at all.
They do continuously update from experience, but that only lasts for as long as the current rollout lasts.