A Multi-Level view of LLM Intentionality

Prompted by Keith Frankish's recent streamed discussion of LLM intentionality on YouTube, there's a particular idea I wanted to share which I'm not sure is widely enough appreciated but which I think gives a valuable perspective from which to think about LLMs and what kinds of intentions they may have. This is not an original idea of my own -- at least some of my thinking on this was sparked by reading about the Waluigi problem on LessWrong.

In this post, I'm going to be making the case that LLMs might have intentionality much like ours, but please understand that I'm making a point in principle and not so much arguing for the capabilities of current LLMs, which are probably not there yet. I'm going to be talking about what scope there is to give the benefit of the doubt to arbitrarily competent future LLMs, albeit ones that follow more or less the same paradigms as those of today. I'm going to try to undermine some proposed reasons for skepticism about the intentions or understanding of LLMs, not because I think the conclusions are wrong but because I think the arguments are too weak to support them.

I should also note that I will follow Keith (and Daniel Dennett, and others) in assuming an interpretivist account of intentions. That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it *really* has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case).

The basic idea I want to introduce is that LLMs might have intentions and agency on multiple levels, so we may be missing something if we restrict our analysis to one level alone.

Psychophysical Harmony

I wanted to write about the argument from psychophysical harmony against naturalism, even though it has no force for those like me who think that talk of qualia and phenomenality in general is confused. For the argument to make any sense at all, we will first have to assume that such considerations as the Knowledge Argument (i.e. the Mary's Room thought experiment from Frank Jackson) have persuaded us that physicalism is inadequate for the task of accounting for phenomenal experience. The argument from psychophysical harmony then goes farther, suggesting that there cannot be any sort of neutral natural explanation at all for the mysterious appropriateness of our experience in representing the physical world. Instead we might need to invoke God, or if not God then perhaps some other benevolent purposeful force or principle such as John Leslie's axiarchism.

Strong Emergence and Free Will

In this post I will discuss whether the idea of strong emergence can help us to make sense of free will and a few related issues. These ideas have been on my mind lately largely due to the above discussion, but I've been thinking about this for a while.

Applying Illusionism to Physical Reality

Many of you would be aware already of illusionism, if in fact there were many readers of this blog. But in case it's needed, illusionism, exemplified by Keith Frankish and Daniel Dennett, is an approach to the philosophy of consciousness which claims that qualia (the ineffable irreducible feelings associated with conscious experience, e.g. the redness of red), are entirely illusory and do not actually exist. As such, on this view the Hard Problem of Consciousness as defined by David Chalmers simply dissolves -- we need only explain why we believe we experience qualia, we do not need to explain how it is that qualia can be produced by physical stuff. As a bonus, illusionism may claim that even the idea of qualia are incoherent.

Illusionism is, I would say, a species of functionalism, and so is compatible with and largely overlaps with other glosses on functionalism such as computationalism. There may be corner cases where some illusionists may disagree with some computationalists, especially on how best to describe things, but the stories they tell about human consciousness are mostly compatible. As such, I count myself as an illusionist, a functionalist and a computationalist.

What I wanted to write about today is a strong rhyming I've noticed between the illusionist view of consciousness and how I think of the stuff of physical reality.

The Computer Metaphor

Is the brain actually a computer? Or is this a metaphor or analogy which is often taken too literally? This issue has come up a few times in the past couple of years, and I wanted to give my take because some common responses do not seem right to me.

I'm going to argue the brain really is literally a computer and that this is not a metaphor or analogy at all. Many of the experts who disagree with this are neuroscientists, biologists and philosophers. We have much to learn from them about how the brain works, but that's not the only issue. To answer the question of whether a brain is a computer we also need to consider what a computer is, and I feel this side of the equation is too often neglected.

Rescuing Computationalism with Platonism

In my last post I discussed some issues with identifying objectively which computations a physical system could legitimately be interpreted as instantiating. Computationalism is usually taken to be the view that all it takes to create a conscious mind is to implement the right computation, so the idea that we can't tell objectively when a computation is implemented implies either that there is no objective fact of the matter regarding when minds exist, that all minds exist (everywhere) or that no minds exist. None of these conclusions is particularly appealing!

I find the arguments discussed on the last post to be somewhat persuasive. Indeed, I had had similar concerns before becoming aware of these. What's more, I think the problem may be worse than even Putnam, Searle and Bishop have suggested.

Putnam, Searle and Bishop: The Failure of Physicalist Computationalism

I wanted to come out of blog dormancy to write up my thoughts on what I feel is a very important argument against computationalism. The argument advances the view that there is no objective fact of the matter about which computations a physical system is computing, and if this is the case it would certainly seem to problematise computationalism (the view that what it is to be a conscious mind is just to perform the right kind of computation).

In this post I will explain the argument and some of the common responses to it. I'll reserve my own response (which is quite different from that of most computationalists) for a future post.

Dualism is Not a Dirty Word

There are two major flavours of naturalism - methodological naturalism and metaphysical or philosophical naturalism. The former essentially means adopting the attitude that the latter is true as a pragmatic step without actually committing to the view that it is literally so. As such, only metaphysical naturalism is really in need of clarification.

I have already explained on this blog that I am a metaphysical naturalist, and what I think this entails, in particular the distinction between the supernatural and the natural. All too often these are unhelpfully defined simply as antonyms of each other.

To recap briefly on that earlier article, and I think to clarify and expand it in the light of recent thinking on the subject, I propose the following definition:

Naturalism is the view that the physical universe is fundamentally intelligible.

Let's extend this to make a number of other statements which I regard as equivalent or derivable from that basic definition.

(For the purposes of this discussion, a physical phenomenon or effect is an object, event or series of events which could in principle be objectively detected or verified by independent observers.)

In the naturalist view:

• All physical effects have physical causes
• All high-level physical phenomena can in principle be described in terms of lower-level physical phenomena
• The most fundamental, basic level of physical phenomena corresponds to the fundamental laws of physics, which are expressible in terms of mathematics

In my view, this definition rules out libertarian free will, for example, because the decisions of people have physical effects yet libertarians reject any physical, deterministic account of human decision-making.

Most naturalists seem to be physicalists, believing only physical stuff such as matter and energy to exist, and many even think that physicalism is identical to naturalism. It isn't though, and this distrust of the non-physical is partly responsible for an unwarranted contempt for the alternative view - dualism.

Attacking the Evolutionary Argument Against Naturalism

My last post presented what I feel is the strongest form of the Evolutionary Argument Against Naturalism (EAAN) promoted by Alvin Plantinga.

Today I will attempt to refute it.

Rather than argue as some have that we can rely on our reason because of tools developed over centuries by a process of cultural evolution, I want to show that even the primitive faculties provided to us by evolution have given us abilities in this regard.

Unlike Plantinga, I certainly do not accept the premise that evolution and naturalism imply a low probability that we can reliably form beliefs.

Strong AI: The Gödel Problem

In my previous post, I alluded to an argument made by John Lucas and popularised by Roger Penrose against the possibility of AI, and this is that Gödel's Incompleteness Theorems suggest that even weak AI is impossible. We could never make a computer system that behaved intelligently.

Loosely put, Gödel's Incompleteness Theorems put fundamental limits on what can be achieved by working with formal systems.

In particular, some argue that they show that formal systems can never be as intelligent as human beings. In this post, I argue against this view and propose that humans are subject to the same limitations.

Strong AI: Naturalism and AI

In this article, I want to examine the issues presented by the problem of consciousness when examined from a naturalistic or physicalist point of view. What I present in this article is probably not a knock-down argument proving the computational theory of consciousness (Strong AI) within this framework. Rather it shows that the alternatives are rather problematic.

Strong AI: The Illusion of Real Consciousness

Perhaps we are no more conscious that the automatons we imagine in our thought experiments when considering questions about artificial intelligence and philosophy of mind.

In this view, all we have to explain is why we believe we are conscious, without explaining why we actually are. "Real consciousness" is an illusion, and consciousness is simply comprised of the functions it affords, namely the ability to be self-reflective, to deliberate, to perceive the world, etc., in such a way that the entity believes itself to have a subjective experience or "real consciousness".

If we allow the assumptions of Weak AI, namely that it should be possible in principle to create a computational artificial intelligence which has all the functionality of a human brain, then all of these abilities can be achieved. If we deny Strong AI, then we must suppose that the AI will falsely believe itself to be conscious.

If we understand how artificial intelligences might believe themselves to be conscious, then we can understand by analogy why biological intelligences might share those beliefs. In other words, if we have explained our belief that we are conscious (by analogy to our artificial intelligences), then what, exactly, remains to be explained?

Strong AI: Evolutionary Parsimony

In this post, I will attempt to make an argument for Strong AI from an evolutionary perspective. Please keep the assumptions stated in my previous post in mind.

In Defence of Strong AI: Meta-meaning

Yesterday's post was quite long and wide-ranging, with a lot of examples etc. It focused specifically on how it might be possible to get semantics from syntax, particularly with reference to The Chinese Room thought experiment.

However I feel that I could sum up my position more succinctly by looking at the meaning of the word "meaning" itself.

In Defence of Strong AI: Semantics from Syntax

Is it possible to get semantics from syntax?

If it were impossible to get semantics (meaning) from syntax (symbols and formal rules for manipulating them) then we probably wouldn't have bothered broadcasting the Arecibo image to the universe in the hope of communicating with aliens.

Without any way to explain the meaning of this image, it seems to consist of syntax alone. Whether it could be deciphered by aliens in practice or not, it at least seems possible in principle, which seems to suggest that it may be possible in some cases to derive semantics from syntax.

The assertion that you can't, that there is no way to get from syntax to semantics, is perhaps the strongest response from John Searle to the systems/virtual mind refutation of his Chinese Room thought experiment (as outlined in my post on the topic).

Searle illustrates his point by saying that if he is in The Chinese Room and has a formal system of rules (syntax) for manipulating Chinese words, he has no way to get from this syntax to the semantics of what is being said. If he has no way to do this, then how can the system do it?

In Searle's view, the systems/virtual mind reply to the Chinese Room is just hand-waving until this is explained.

Let's see if I can attempt to do just that.

In Defence of Strong AI: The Modal Argument Refuted

In this video, philosopher and Christain theologian Alvin Plantinga outlines his modal argument, which in my view successfully establishes that he is not identical with his brain or body.

He interprets this to mean that he is something more than physical, and presumably that he has a soul. Physicalism refuted? Nope!

In Defence of Strong AI: Qualia and Mary's Room

Photo: brokenview

The Mary's Room thought experiment considers a brilliant neuroscientist who has been raised and educated in an entirely colourless environment. It was originally proposed by Frank Jackson as follows:

Mary is a brilliant scientist who is, for whatever reason, forced to investigate the world from a black and white room via a black and white television monitor. She specializes in the neurophysiology of vision and acquires, let us suppose, all the physical information there is to obtain about what goes on when we see ripe tomatoes, or the sky, and use terms like ‘red’, ‘blue’, and so on. She discovers, for example, just which wavelength combinations from the sky stimulate the retina, and exactly how this produces via the central nervous system the contraction of the vocal cords and expulsion of air from the lungs that results in the uttering of the sentence ‘The sky is blue’. [...] What will happen when Mary is released from her black and white room or is given a color television monitor? Will she learn anything or not?

On the day she is allowed to emerge from her confinement, she sees the colourful world for the first time. "Wow!" she says, as she takes it all in. Though she knew everything there was to know about the human brain and everything there was to know about the physics of colour, she had never before known what it was like to experience it first-hand.

But if she now knows something she didn't know before, then evidently she could not have known everything there was to know about colour before her escape. This surely proves that conscious experience transcends physics and neuroscience. Physicalism (the belief that everything is the result of physical interactions) must simply be wrong. Right?

Well, not really...

In Defence of Strong AI: Philosophical Zombies Refuted

One concept that often comes up in discussions about consciousness and strong AI is that of the philosophical zombie.

Generally, a philosophical zombie is something which looks like and behaves like a human being, but has no conscious experience of the world. It may deceive us by protesting that it is conscious and claiming that it does feel pain and experience sensations just as we do, but it is just an automaton mimicking a real sentient being. It is no more aware than a character in a novel or a two-line computer program that prints to the screen the message "Hello World! (I'm conscious, by the way)".

Opponents of strong AI will often bring up the idea of a philosophical zombie to illustrate that it is conceivable that something can behave as though it were conscious while actually having no inner experiences whatsoever. David Chalmers, in The Conscious Mind, discusses the idea of an entire world, perhaps indistinguishable from our own, which is populated entirely by philosophical zombies.

While those who bring up the concept may not claim that philosophical zombies are possible in this universe, they do maintain that the concept is at least logically possible or coherent. I disagree.

Essential identity: Meet your future selves

While discussing my ideas about identity with a friend, it became apparent that I hadn't delved sufficiently deeply into the existential consequences of cloning yourself body and mind. This friend appeared to agree with me on most of what I had written, but when I proposed a new thought experiment to him, he came to very different conclusions to me.

The thought experiment is simple enough, and asks whether you have any interest in the welfare of your clone should you clone yourself.

Essential identity: Cogito ergo sum

If there's one part of us that we consider to be the seat of our identities, it is our brains.