A Multi-Level view of LLM Intentionality

Prompted by Keith Frankish's recent streamed discussion of LLM intentionality on YouTube, there's a particular idea I wanted to share which I'm not sure is widely enough appreciated but which I think gives a valuable perspective from which to think about LLMs and what kinds of intentions they may have. This is not an original idea of my own -- at least some of my thinking on this was sparked by reading about the Waluigi problem on LessWrong.

In this post, I'm going to be making the case that LLMs might have intentionality much like ours, but please understand that I'm making a point in principle and not so much arguing for the capabilities of current LLMs, which are probably not there yet. I'm going to be talking about what scope there is to give the benefit of the doubt to arbitrarily competent future LLMs, albeit ones that follow more or less the same paradigms as those of today. I'm going to try to undermine some proposed reasons for skepticism about the intentions or understanding of LLMs, not because I think the conclusions are wrong but because I think the arguments are too weak to support them.

I should also note that I will follow Keith (and Daniel Dennett, and others) in assuming an interpretivist account of intentions. That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it *really* has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case).

The basic idea I want to introduce is that LLMs might have intentions and agency on multiple levels, so we may be missing something if we restrict our analysis to one level alone.

Putnam, Searle and Bishop: The Failure of Physicalist Computationalism

I wanted to come out of blog dormancy to write up my thoughts on what I feel is a very important argument against computationalism. The argument advances the view that there is no objective fact of the matter about which computations a physical system is computing, and if this is the case it would certainly seem to problematise computationalism (the view that what it is to be a conscious mind is just to perform the right kind of computation).

In this post I will explain the argument and some of the common responses to it. I'll reserve my own response (which is quite different from that of most computationalists) for a future post.

Strong AI: Conclusion

I've taken a long hiatus while I tried to think if there was anything substantive I had left to say about Strong AI. I didn't come up with much, so I'll wrap it up with this post.

Strong AI: The Gödel Problem

In my previous post, I alluded to an argument made by John Lucas and popularised by Roger Penrose against the possibility of AI, and this is that Gödel's Incompleteness Theorems suggest that even weak AI is impossible. We could never make a computer system that behaved intelligently.

Loosely put, Gödel's Incompleteness Theorems put fundamental limits on what can be achieved by working with formal systems.

In particular, some argue that they show that formal systems can never be as intelligent as human beings. In this post, I argue against this view and propose that humans are subject to the same limitations.

Strong AI: Naturalism and AI

In this article, I want to examine the issues presented by the problem of consciousness when examined from a naturalistic or physicalist point of view. What I present in this article is probably not a knock-down argument proving the computational theory of consciousness (Strong AI) within this framework. Rather it shows that the alternatives are rather problematic.

Strong AI: The Illusion of Real Consciousness

Perhaps we are no more conscious that the automatons we imagine in our thought experiments when considering questions about artificial intelligence and philosophy of mind.

In this view, all we have to explain is why we believe we are conscious, without explaining why we actually are. "Real consciousness" is an illusion, and consciousness is simply comprised of the functions it affords, namely the ability to be self-reflective, to deliberate, to perceive the world, etc., in such a way that the entity believes itself to have a subjective experience or "real consciousness".

If we allow the assumptions of Weak AI, namely that it should be possible in principle to create a computational artificial intelligence which has all the functionality of a human brain, then all of these abilities can be achieved. If we deny Strong AI, then we must suppose that the AI will falsely believe itself to be conscious.

If we understand how artificial intelligences might believe themselves to be conscious, then we can understand by analogy why biological intelligences might share those beliefs. In other words, if we have explained our belief that we are conscious (by analogy to our artificial intelligences), then what, exactly, remains to be explained?

Strong AI: Evolutionary Parsimony

In this post, I will attempt to make an argument for Strong AI from an evolutionary perspective. Please keep the assumptions stated in my previous post in mind.

Strong AI: The Story So Far

Let's take Strong AI as the viewpoint that the essential capabilities of the brain are computational, meaning that were we to reproduce these same computational functions in an artificial machine such as a silicon computer, then that machine would have a mind just as real, just as conscious as a human mind.

In my recent posts, I have argued against some common criticisms of this view.

I have shown how The Chinese Room may establish that a piece of computational hardware cannot become conscious, but it does not show that the software running on that hardware is unconscious.

I have shown that the concept of a philosophical zombie does nothing to cast doubt on Strong AI, for if Strong AI is true then philosophical zombies are probably logically incoherent. Philosophical zombies are nothing more than a restatement of the viewpoint that Strong AI is not true.

I have shown that qualia, with specific reference to Mary's Room, do not in themselves disprove physicalism, but are probably simply brain states which act as labels for certain percepts and concepts.

I have agreed with Alvin Plantinga that the mind and body are distinct entities (like hardware and software), but argue that this is a conclusion of no consequence which no thoughtful person should be surprised by and which does not imply that the soul can survive the destruction of the body.

I have also explained why I disagree with the assertion that semantics cannot be derived from syntax, and sketched how I think semantics can be viewed as a special case of syntax.

What I have not done yet is given some positive reasons for the suspicion that Strong AI is true. In my following posts I will attempt to do so.

In Defence of Strong AI: Meta-meaning

Yesterday's post was quite long and wide-ranging, with a lot of examples etc. It focused specifically on how it might be possible to get semantics from syntax, particularly with reference to The Chinese Room thought experiment.

However I feel that I could sum up my position more succinctly by looking at the meaning of the word "meaning" itself.

In Defence of Strong AI: The Chinese Room Refuted

It has become obvious to me that in order to fully explain my world view, I must first defend strong AI.

Strong AI is the position that it is possible in principle to have an artificial intelligence, implemented as a computing machine much like the machines we have today, which would be capable of consciousness, feeling and sensation. This is in contrast to the position of weak AI, which only claims that we could in principle make a computing machine which would merely behave as if it were conscious.

This is important not just for the ethics of how we might treat sentient computers but because it cuts to the heart of what it is that our minds actually are. If our minds are dependent on something other than simple computation, this puts limits on certain questions we might ask about our universe. For example, if computation cannot produce consciousness, then we immediately know that we cannot all be living in a computer simulation.

Firstly, I would like to point out the problems I see with some of the most popular criticisms of strong AI, starting with perhaps the most famous: John Searle's "The Chinese Room". Later I will attempt to build a positive case of my own.

The conscious beneficiaries of morality

To whom do we have a moral duty? Sam Harris identifies "conscious beings" as the beneficiaries of our morality.

This is a very broad category, and perhaps Harris is correct. The momentum of history appears to be with him, in that our moral circle is ever-expanding. We first felt this duty only to our kin, then later to our nation, our race, our species.

In the present day, many are concerned with the well-being of members of other species. The future may bring yet more expansion of this moral circle, perhaps even bringing artificially intelligent constructs into the fold. 

Harris seems to include certain animals in his circle of morality, which will please animal rights advocates. On the other hand, pro-life campaigners will be disappointed that Harris seems to exclude human embryos. That some people may reasonably disagree with him calls the objectivity of his axiom into question, however this is the topic of an earlier post.

Let's assume Harris has correctly identified the correct description for moral beneficiaries. In this post, I discuss the problem of deciding objectively who fits it.