I think what is needed, and what will happen, is to pull apart concepts like intelligence, consciousness, and language use, and create narrower and more rigorous definitions of what those are.

My go-to analogy is the concept of heat and warmth, which was the subject of philosophical speculation and inquiry for centuries. But ultimately it was not useful or predictive until it was redefined very narrowly by scientists as a specific measurable attribute of a collection of molecules. Arguably this redefinition itself was an important component of the establishment of “science” as a discipline distinct from philosophy.

This let scientists work with heat rigorously, even though non-scientists still have a largely colloquial vocabulary about it, e.g. “this coat is warmer than my last one.” On matters like consciousness, we’re all still smashed together into colloquial land, arguing about definitions when we think we are arguing about concepts.

I do think the current advancements in AI may help us develop the new vocabulary we need to quantitatively reason about intelligence and language, and in doing so, help better define / constrain what the we mean by consciousness. We think of LLMs now as a tool we can use to do work, but another view is that they are a sensor that directly correlates physical inputs to intelligent outputs. Sort of like how burning can be used to measure the energy contained in various materials, in addition to keeping us warm.