We do not understand and know how these models work. We know what their architectures are and how to create them, but we cannot explain their behaviours at a fundamental level. There is no definitive way for us to answer the question of "how did it produce response X for query Y?" - we're only grazing the surface with mechanistic interpretability.

We know how the models are built and trained, but we have a very limited understanding of how the final products work.

That is to say, we don't know why they give the outputs that they do.

If we did know how they worked, AI interpretability would not be an open and growing field.

You could say something similar about biology—just physics behind the curtains, and we understand a lot of the basics. The difficulty comes from complexity, not mysticism.

To be clear I don't think that LLMs are sentient, but the appeal in studying them is similar to biology in that you get to dissect a highly complex system with comparatively crude tools.

it took significant research efforts to just understand how these models learn how to multiply two numbers. The fact that we know how they operate doesn't mean we understand it.

Utterly wrong. How LLMs work is very incompletely understood and an active area of research.

[dead]