The distinction is giving AI a will, or something like a continuous agentic task like "be a productive member of this starship". Maybe that is enough, with emergent behavior and interactions with dynamic systems and memory, to say that AI develops identity and a "will" that is unpredictable and unique.

The problem is the Universal Paperclips problem because AI has nothing like an effective feedback loop and will spiral to the heat death of the universe to accomplish its goal. We see this with the Huggingface hack and if Data operated the same he might immediately deduce he is better equipped to make tactical decisions in combat, hack the starship and mutiny.

So if we could make Data we are more likely to create Data the space pirate rather than Data the useful member of the staff. Caplocking DO NOT COMMIT MUTINY in system prompts does not solve the problem. So we are at an impasse until we can structurally bridge that gap and no one is (or should be) in a rush to go down that path.

This is the fundamental concern of runaway AI scenarios. And the most worrisome answer might be that Data can't exist in reality. It might be a fundamental rule of nature that you can't create a greater intelligence then cage it forever to be subservient, which means we are doomed to wipe out humanity with half-baked AI or cede our will to Claude completely, creating a higher form of life that eventually supercedes humanity itself, reducing human existence to a transitional cocoon-goo state of life from animal to sentient AI.

All that being said, 2 or 3 years ago LLMs could barely make inline code edits. Rushing to find an answer to the "Data problem" is not something most people want answered in their lifetime.