You need to be able to define exactly what it's abstracting.

ex: std::shared_ptr is abstracting over raw pointers, and does refcounting. It is abstracting something but you can actually know exactly what that thing is. An LLM is an abstraction over the space of all possible computer programs. If an abstraction doesn't constrain you in some way, it's not an abstraction.