alt Hacker NewsNeal Stephenson touches on this topic in several of his novels. Probably the most concrete of this is "Fall; or Dodge in Hell" which involves a simulation of people's scanned brains rediscovering qualia and constructing their own simulated world from scratch. In the book, two of the deceased digital "souls" eventually mate and produce digital offspring and the whole simulation starts consuming more and more resources.
His Baroque Cycle series also touches on this in several places. One funny side plot involves a freed African slave (Dappa) who speaks dozens of different languages and is highly intelligent and an aristocratic person who maintains that of course this former slave (who is obviously a lot smarter than this aristocrat) is just a trained monkey that naturally is not conscious even though he is quite clever with language. The same books also have a lot of side plots involving Leibniz and various attempts to build thinking/computing machines.
The Dappa plot is probably the closest to a lot of debates there will be around AGI with people likely to insist for all sorts of reasons rooted in philosophy, religion, etc. that even though the AGI walks, talks, etc. like a duck, it can't be a duck. At some point, we'll have AGI that pass any test we can think of and we'll still have people arguing that these cannot be conscious.
Replies
> At some point, we'll have AGI that pass any test we can think of and we'll still have people arguing that these cannot be conscious.
You've already drawn your line in the sand (i.e. they are conscious). In that case, you can't also claim that we should continue producing them by the millions at the flick of a switch.
The AI-is-conscious crowd will have to choose - either they are conscious, in which case they should not be birthed, or they are not conscious in which case we can use them as tools. You can't have both and still be logically consistent.
The clinching thing for me is how the AGI is supposed to work. If it's the same as our theory of how we work, then fine, it's one of us. It wouldn't even have to work very well.
> Probably the most concrete of this is "Fall; or Dodge in Hell" which involves a simulation of people's scanned brains rediscovering qualia and constructing their own simulated world from scratch.
This is a great example for a discussion about The Hard Problem. Here is the description from the book of the inner experience of this scanned brain as it gets booted up:
> What came next could not, of course, be described without using words. But that was deceptive in a way since he no longer had words. Nor did he have memories, or coherent thoughts, or any other way to describe or think about the qualia he was experiencing. And those qualia were of miserably low quality. To the extent he was seeing, he was seeing incoherent patterns of fluctuating light. For people of a certain age, the closest descriptor for this was “static”: the sheets, waves, and bands of noise that had covered the screens of malfunctioning television sets. Static, in a sense, wasn’t real. It was simply what you got out of a system when it was unable to lock on to any strong signal—“Strong” meaning actually conveying useful, or at least understandable, information. Modern computer screens were smart enough to just shut down, or put up an error message, when the signal was lost. Old analog sets had no choice but to display something. The electron beam was forever scanning, a mindless beacon, and if you fed it nothing else it would produce a visual map of whatever was contingently banging around in its circuitry: some garbled mix of electrical noise from Mom’s vacuum cleaner, Dad’s shaver, solar flares, stray transmissions caroming off the ionosphere, and whatever happened when little feedback loops on the circuit board got out of hand. Likewise, to the extent he was hearing anything, it was just an inchoate hiss.
The Hard Problem asks: who is experiencing this qualia and why is there an experiencer at all? Stephenson writes how this simulated brain is experiencing static as it condenses into meaningful patterns, but he implicitly starts with someone experiencing this static qualia. If this is the very beginning of the simulated brain booting up, where did this experiencer come from?
> At some point, we'll have AGI that pass any test we can think of and we'll still have people arguing that these cannot be conscious.
Because you're conflating intelligence with consciousness. There is no test for consciousness. In fact, you can't even prove that other human beings are conscious, you only know that you yourself are because it's self evident to you (cogito ergo sum, I think therefore I am). The whole point of the hard problem is that you can imagine something exactly like a human being that passes every test of being a human being (e.g. an AI) but still not be sure that it has any inner experience.
> At some point, we'll have AGI that pass any test we can think of and we'll still have people arguing that these cannot be conscious.
Is there a distinction in your mind between consciousness and intelligence? Is it possible, for example, for a machine to solve complex problems but not be conscious? Or vice versa, can an animal or a person be very unintelligent yet still conscious?