That’s an interesting thing to think about. Still, it’s important for us to remind ourselves that “looks very slightly like” is not the same as the real thing. The A in AI stands for artificial.
The summary of this paper describes my sentiment in better words than I have:
https://www.nature.com/articles/s41599-025-05868-8
It’s very easy for the average person to mistake linguistic ability and simulated problem solving for intelligence and sentience.
Without knowing what makes consciousness possible, the paper cannot justify biology as necessary - it mistakes a lack of evidence for conscious AI for proof that conscious AI is impossible.
I do not believe current AI or LLMs are conscious, but there is no proof one way or another that they can or cannot be. The paper authors are making up their own definitions and building an argument from them