The same as the difference between a thing and a statistical model of a thing.

They don't always match up.

One has a human behind it and one has a matrix latent space. Usually human understanding is more "real"

Stallman vehemently states llms are bullshit generators with no true understanding.

But sometimes you can approximate your problem and unfold an llm interlocator to producing the strings you need for your task