>Equivalent statements could be made about how human brains are not magic, just biology - yet I think we still think.
They're not equivalent at all because the AI is by no means biological. "It's just maths" could maybe be applied to humans but this is backed entirely by supposition and would ultimately just be an assumption of its own conclusion - that human brains work on the same underlying principles as AI because it is assumed that they're based on the same underlying principles as AI.
Well, a better retort would be "Human brains are not magic, just physics. Protons, neutrons and electrons don't think".
But I think most people get what GP means.
But parent didn't try to apply "it's just maths" to humans. He said one could just as easily say, as some do: "Humans are just biology, hence they're not magic". Our understanding of mathematics, including the maths of transformer models is limited, just as our understanding of biology. Some behaviors of these models have taken researches by surprise, and future surprises are not at all excluded. We don't know exactly how far they will evolve.
As for applying the word thinking to AI systems, it's already in common usage and this won't change. We don't have any other candidate words, and this one is the closest existing word for referencing a computational process which, one must admit, is in many ways (but definitely not in all ways) analogous to human thought.
Human brains might not be explained by the same type of math AI is explained with, but it will be some kind of math...
> that human brains work on the same underlying principles as AI
That wasn't the assumption though, it was only that human brains work by some "non-magical" electro-chemical process which could be described as a mechanism, whether that mechanism followed the same principles of AI or not.
Straw man. The person who you're responding to talked about "equivalent statements" (emphasis added), whereas you appear to be talking about equivalent objects (AIs vs. brains), and pointing out the obvious flaw in this argument, that AIs aren't biology. The obvious flaw in the wrong argument, that is.
Unless you're supposing something mystical or supernatural about how brains work, then yes, it is "just" math, there is nothing else it could be. All of the evidence we have shows it's an electrochemical network of neurons processing information. There's no evidence that suggests anything different, or even the need for anything different. There's no missing piece or deep mystery to it.
It's on those who want alternative explanations to demonstrate even the slightest need for them exists - there is no scientific evidence that exists which suggests the operation of brains as computers, as information processors, as substrate independent equivalents to Turing machines, are insufficient to any of the cognitive phenomena known across the entire domain of human knowledge.
We are brains in bone vats, connected to a wonderful and sophisticated sensorimotor platform, and our brains create the reality we experience by processing sensor data and constructing a simulation which we perceive as subjective experience.
The explanation we have is sufficient to the phenomenon. There's no need or benefit for searching for unnecessarily complicated alternative interpretations.
If you aren't satisfied with the explanation, it doesn't really matter - to quote one of Neil DeGrasse Tyson's best turns of phrase: "the universe is under no obligation to make sense to you"
If you can find evidence, any evidence whatsoever, and that evidence withstands scientific scrutiny, and it demands more than the explanation we currently have, then by all means, chase it down and find out more about how cognition works and expand our understanding of the universe. It simply doesn't look like we need anything more, in principle, to fully explain the nature of biological intelligence, and consciousness, and how brains work.
Mind as interdimensional radios, mystical souls and spirits, quantum tubules, none of that stuff has any basis in a ruthlessly rational and scientific review of the science of cognition.
That doesn't preclude souls and supernatural appearing phenomena or all manner of "other" things happening. There's simply no need to tie it in with cognition - neurotransmitters, biological networks, electrical activity, that's all you need.