A Multi-Level view of LLM Intentionality

Prompted by Keith Frankish's recent streamed discussion of LLM intentionality on YouTube, there's a particular idea I wanted to share which I'm not sure is widely enough appreciated but which I think gives a valuable perspective from which to think about LLMs and what kinds of intentions they may have. This is not an original idea of my own -- at least some of my thinking on this was sparked by reading about the Waluigi problem on LessWrong.

In this post, I'm going to be making the case that LLMs might have intentionality much like ours, but please understand that I'm making a point in principle and not so much arguing for the capabilities of current LLMs, which are probably not there yet. I'm going to be talking about what scope there is to give the benefit of the doubt to arbitrarily competent future LLMs, albeit ones that follow more or less the same paradigms as those of today. I'm going to try to undermine some proposed reasons for skepticism about the intentions or understanding of LLMs, not because I think the conclusions are wrong but because I think the arguments are too weak to support them.

I should also note that I will follow Keith (and Daniel Dennett, and others) in assuming an interpretivist account of intentions. That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it *really* has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case).

The basic idea I want to introduce is that LLMs might have intentions and agency on multiple levels, so we may be missing something if we restrict our analysis to one level alone.

Rescuing Computationalism with Platonism

In my last post I discussed some issues with identifying objectively which computations a physical system could legitimately be interpreted as instantiating. Computationalism is usually taken to be the view that all it takes to create a conscious mind is to implement the right computation, so the idea that we can't tell objectively when a computation is implemented implies either that there is no objective fact of the matter regarding when minds exist, that all minds exist (everywhere) or that no minds exist. None of these conclusions is particularly appealing!

I find the arguments discussed on the last post to be somewhat persuasive. Indeed, I had had similar concerns before becoming aware of these. What's more, I think the problem may be worse than even Putnam, Searle and Bishop have suggested.

Putnam, Searle and Bishop: The Failure of Physicalist Computationalism

I wanted to come out of blog dormancy to write up my thoughts on what I feel is a very important argument against computationalism. The argument advances the view that there is no objective fact of the matter about which computations a physical system is computing, and if this is the case it would certainly seem to problematise computationalism (the view that what it is to be a conscious mind is just to perform the right kind of computation).

In this post I will explain the argument and some of the common responses to it. I'll reserve my own response (which is quite different from that of most computationalists) for a future post.

Dualism is Not a Dirty Word

There are two major flavours of naturalism - methodological naturalism and metaphysical or philosophical naturalism. The former essentially means adopting the attitude that the latter is true as a pragmatic step without actually committing to the view that it is literally so. As such, only metaphysical naturalism is really in need of clarification.

I have already explained on this blog that I am a metaphysical naturalist, and what I think this entails, in particular the distinction between the supernatural and the natural. All too often these are unhelpfully defined simply as antonyms of each other.

To recap briefly on that earlier article, and I think to clarify and expand it in the light of recent thinking on the subject, I propose the following definition:

Naturalism is the view that the physical universe is fundamentally intelligible.

Let's extend this to make a number of other statements which I regard as equivalent or derivable from that basic definition.

(For the purposes of this discussion, a physical phenomenon or effect is an object, event or series of events which could in principle be objectively detected or verified by independent observers.)

In the naturalist view:

• All physical effects have physical causes
• All high-level physical phenomena can in principle be described in terms of lower-level physical phenomena
• The most fundamental, basic level of physical phenomena corresponds to the fundamental laws of physics, which are expressible in terms of mathematics

In my view, this definition rules out libertarian free will, for example, because the decisions of people have physical effects yet libertarians reject any physical, deterministic account of human decision-making.

Most naturalists seem to be physicalists, believing only physical stuff such as matter and energy to exist, and many even think that physicalism is identical to naturalism. It isn't though, and this distrust of the non-physical is partly responsible for an unwarranted contempt for the alternative view - dualism.

Consciousness is not like Photosynthesis

I've been posting some comments on Massimo Pigliucci's blog recently in defense of Strong AI, after a couple of posts attacking the computational theory of mind.

In the first, he uses John Searle's Chinese Room argument to demonstrate that a computer cannot be conscious. If you've been paying attention to this blog, you should know what I think of that argument!

In the second post, he uses an article by Jack Copeland which undermines the position of philosophers who take the Church-Turing thesis to imply that a Turing machine can do any computation performable by any machine. Unfortunately, Pigliucci seems to fail to realise that while Copeland is technically correct, the philosophers he criticises are no less so unless some very unlikely propositions turn out to be true (i.e. that the laws of physics are not computable and the brain is a hypercomputer).

But what motivated me to write this post is the same tired old argument that keeps coming up again and again in these discussions, the analogy to photosynthesis.

Strong AI: Conclusion

I've taken a long hiatus while I tried to think if there was anything substantive I had left to say about Strong AI. I didn't come up with much, so I'll wrap it up with this post.

Strong AI: The Gödel Problem

In my previous post, I alluded to an argument made by John Lucas and popularised by Roger Penrose against the possibility of AI, and this is that Gödel's Incompleteness Theorems suggest that even weak AI is impossible. We could never make a computer system that behaved intelligently.

Loosely put, Gödel's Incompleteness Theorems put fundamental limits on what can be achieved by working with formal systems.

In particular, some argue that they show that formal systems can never be as intelligent as human beings. In this post, I argue against this view and propose that humans are subject to the same limitations.

Strong AI: Naturalism and AI

In this article, I want to examine the issues presented by the problem of consciousness when examined from a naturalistic or physicalist point of view. What I present in this article is probably not a knock-down argument proving the computational theory of consciousness (Strong AI) within this framework. Rather it shows that the alternatives are rather problematic.

Strong AI: The Illusion of Real Consciousness

Perhaps we are no more conscious that the automatons we imagine in our thought experiments when considering questions about artificial intelligence and philosophy of mind.

In this view, all we have to explain is why we believe we are conscious, without explaining why we actually are. "Real consciousness" is an illusion, and consciousness is simply comprised of the functions it affords, namely the ability to be self-reflective, to deliberate, to perceive the world, etc., in such a way that the entity believes itself to have a subjective experience or "real consciousness".

If we allow the assumptions of Weak AI, namely that it should be possible in principle to create a computational artificial intelligence which has all the functionality of a human brain, then all of these abilities can be achieved. If we deny Strong AI, then we must suppose that the AI will falsely believe itself to be conscious.

If we understand how artificial intelligences might believe themselves to be conscious, then we can understand by analogy why biological intelligences might share those beliefs. In other words, if we have explained our belief that we are conscious (by analogy to our artificial intelligences), then what, exactly, remains to be explained?

Strong AI: Evolutionary Parsimony

In this post, I will attempt to make an argument for Strong AI from an evolutionary perspective. Please keep the assumptions stated in my previous post in mind.

Strong AI: The Story So Far

Let's take Strong AI as the viewpoint that the essential capabilities of the brain are computational, meaning that were we to reproduce these same computational functions in an artificial machine such as a silicon computer, then that machine would have a mind just as real, just as conscious as a human mind.

In my recent posts, I have argued against some common criticisms of this view.

I have shown how The Chinese Room may establish that a piece of computational hardware cannot become conscious, but it does not show that the software running on that hardware is unconscious.

I have shown that the concept of a philosophical zombie does nothing to cast doubt on Strong AI, for if Strong AI is true then philosophical zombies are probably logically incoherent. Philosophical zombies are nothing more than a restatement of the viewpoint that Strong AI is not true.

I have shown that qualia, with specific reference to Mary's Room, do not in themselves disprove physicalism, but are probably simply brain states which act as labels for certain percepts and concepts.

I have agreed with Alvin Plantinga that the mind and body are distinct entities (like hardware and software), but argue that this is a conclusion of no consequence which no thoughtful person should be surprised by and which does not imply that the soul can survive the destruction of the body.

I have also explained why I disagree with the assertion that semantics cannot be derived from syntax, and sketched how I think semantics can be viewed as a special case of syntax.

What I have not done yet is given some positive reasons for the suspicion that Strong AI is true. In my following posts I will attempt to do so.

In Defence of Strong AI: Meta-meaning

Yesterday's post was quite long and wide-ranging, with a lot of examples etc. It focused specifically on how it might be possible to get semantics from syntax, particularly with reference to The Chinese Room thought experiment.

However I feel that I could sum up my position more succinctly by looking at the meaning of the word "meaning" itself.

In Defence of Strong AI: Semantics from Syntax

Is it possible to get semantics from syntax?

If it were impossible to get semantics (meaning) from syntax (symbols and formal rules for manipulating them) then we probably wouldn't have bothered broadcasting the Arecibo image to the universe in the hope of communicating with aliens.

Without any way to explain the meaning of this image, it seems to consist of syntax alone. Whether it could be deciphered by aliens in practice or not, it at least seems possible in principle, which seems to suggest that it may be possible in some cases to derive semantics from syntax.

The assertion that you can't, that there is no way to get from syntax to semantics, is perhaps the strongest response from John Searle to the systems/virtual mind refutation of his Chinese Room thought experiment (as outlined in my post on the topic).

Searle illustrates his point by saying that if he is in The Chinese Room and has a formal system of rules (syntax) for manipulating Chinese words, he has no way to get from this syntax to the semantics of what is being said. If he has no way to do this, then how can the system do it?

In Searle's view, the systems/virtual mind reply to the Chinese Room is just hand-waving until this is explained.

Let's see if I can attempt to do just that.

In Defence of Strong AI: The Modal Argument Refuted

In this video, philosopher and Christain theologian Alvin Plantinga outlines his modal argument, which in my view successfully establishes that he is not identical with his brain or body.

He interprets this to mean that he is something more than physical, and presumably that he has a soul. Physicalism refuted? Nope!

In Defence of Strong AI: Qualia and Mary's Room

Photo: brokenview

The Mary's Room thought experiment considers a brilliant neuroscientist who has been raised and educated in an entirely colourless environment. It was originally proposed by Frank Jackson as follows:

Mary is a brilliant scientist who is, for whatever reason, forced to investigate the world from a black and white room via a black and white television monitor. She specializes in the neurophysiology of vision and acquires, let us suppose, all the physical information there is to obtain about what goes on when we see ripe tomatoes, or the sky, and use terms like ‘red’, ‘blue’, and so on. She discovers, for example, just which wavelength combinations from the sky stimulate the retina, and exactly how this produces via the central nervous system the contraction of the vocal cords and expulsion of air from the lungs that results in the uttering of the sentence ‘The sky is blue’. [...] What will happen when Mary is released from her black and white room or is given a color television monitor? Will she learn anything or not?

On the day she is allowed to emerge from her confinement, she sees the colourful world for the first time. "Wow!" she says, as she takes it all in. Though she knew everything there was to know about the human brain and everything there was to know about the physics of colour, she had never before known what it was like to experience it first-hand.

But if she now knows something she didn't know before, then evidently she could not have known everything there was to know about colour before her escape. This surely proves that conscious experience transcends physics and neuroscience. Physicalism (the belief that everything is the result of physical interactions) must simply be wrong. Right?

Well, not really...

In Defence of Strong AI: Philosophical Zombies Refuted

One concept that often comes up in discussions about consciousness and strong AI is that of the philosophical zombie.

Generally, a philosophical zombie is something which looks like and behaves like a human being, but has no conscious experience of the world. It may deceive us by protesting that it is conscious and claiming that it does feel pain and experience sensations just as we do, but it is just an automaton mimicking a real sentient being. It is no more aware than a character in a novel or a two-line computer program that prints to the screen the message "Hello World! (I'm conscious, by the way)".

Opponents of strong AI will often bring up the idea of a philosophical zombie to illustrate that it is conceivable that something can behave as though it were conscious while actually having no inner experiences whatsoever. David Chalmers, in The Conscious Mind, discusses the idea of an entire world, perhaps indistinguishable from our own, which is populated entirely by philosophical zombies.

While those who bring up the concept may not claim that philosophical zombies are possible in this universe, they do maintain that the concept is at least logically possible or coherent. I disagree.

In Defence of Strong AI: The Chinese Room Refuted

It has become obvious to me that in order to fully explain my world view, I must first defend strong AI.

Strong AI is the position that it is possible in principle to have an artificial intelligence, implemented as a computing machine much like the machines we have today, which would be capable of consciousness, feeling and sensation. This is in contrast to the position of weak AI, which only claims that we could in principle make a computing machine which would merely behave as if it were conscious.

This is important not just for the ethics of how we might treat sentient computers but because it cuts to the heart of what it is that our minds actually are. If our minds are dependent on something other than simple computation, this puts limits on certain questions we might ask about our universe. For example, if computation cannot produce consciousness, then we immediately know that we cannot all be living in a computer simulation.

Firstly, I would like to point out the problems I see with some of the most popular criticisms of strong AI, starting with perhaps the most famous: John Searle's "The Chinese Room". Later I will attempt to build a positive case of my own.