A Multi-Level view of LLM Intentionality

Prompted by Keith Frankish's recent streamed discussion of LLM intentionality on YouTube, there's a particular idea I wanted to share which I'm not sure is widely enough appreciated but which I think gives a valuable perspective from which to think about LLMs and what kinds of intentions they may have. This is not an original idea of my own -- at least some of my thinking on this was sparked by reading about the Waluigi problem on LessWrong.

In this post, I'm going to be making the case that LLMs might have intentionality much like ours, but please understand that I'm making a point in principle and not so much arguing for the capabilities of current LLMs, which are probably not there yet. I'm going to be talking about what scope there is to give the benefit of the doubt to arbitrarily competent future LLMs, albeit ones that follow more or less the same paradigms as those of today. I'm going to try to undermine some proposed reasons for skepticism about the intentions or understanding of LLMs, not because I think the conclusions are wrong but because I think the arguments are too weak to support them.

I should also note that I will follow Keith (and Daniel Dennett, and others) in assuming an interpretivist account of intentions. That is, we should ascribe intentions to a system if and only if it helps to predict and explain the behaviour of the system. Whether it *really* has intentions beyond this is not a question I am attempting to answer (and I think that it is probably not determinate in any case).

The basic idea I want to introduce is that LLMs might have intentions and agency on multiple levels, so we may be missing something if we restrict our analysis to one level alone.

Psychophysical Harmony

I wanted to write about the argument from psychophysical harmony against naturalism, even though it has no force for those like me who think that talk of qualia and phenomenality in general is confused. For the argument to make any sense at all, we will first have to assume that such considerations as the Knowledge Argument (i.e. the Mary's Room thought experiment from Frank Jackson) have persuaded us that physicalism is inadequate for the task of accounting for phenomenal experience. The argument from psychophysical harmony then goes farther, suggesting that there cannot be any sort of neutral natural explanation at all for the mysterious appropriateness of our experience in representing the physical world. Instead we might need to invoke God, or if not God then perhaps some other benevolent purposeful force or principle such as John Leslie's axiarchism.