Connecticut Yankee AI

Grokking Yudkowski

The fundamental motivational driver I use to power my ambition was provided by a warfighter. Thousands of people know him as Nutnfancy. He’s the guy who helped me overthink my decision to purchase my first firearm. I have a patch of his which contains the slogan: Suck Less.

Like all of you, I am fallible. I know this to be true. So when I make the transition from being a harmless fallible human to a fallible human with the capacity to kill at a distance, I know something important is going on. So once I decided to take on that capacity I wanted to think about it for a long time. I did. I can still remember the days I went to my local gun store listening to other customers’ explanations, justifications and excuses to buy a gun. I realized very quickly that I had done a lot more thinking. Moreover my thinking was prompted by Nutnfancy who taught me how to think about firearms. And knives. And the rule of law. And survival in the wild. Yet even all that thinking does not make my ideas about weapons correct, but it does put it into the specific context of warfighters with a great deal of respect for the rule of law.

If you haven’t heard me mention it before, at this period in my life I was engaged with my Martial Education. I expect that if I live long enough I will engage in a Pastoral Education. I bought my first naked cane rose bush a week ago and it’s doing fine, thank you. Right now, like many thousands of other people, I am engaged in AI Education. The person I am now focusing on, as I alluded to in previous essays, is Eliezer Shlomo Yudkowsky, who runs the Machine Intelligence Research Institute and publishes something called Less Wrong.

When I and my crew, who also had done a lot of thinking about shooting, went together for the first time to the range at Holser Canyon, I was quite proud of the kit I had assembled. Then I started actually handling my pistol. I sucked. I wasn’t frightened or overwhelmed as I was the first time I shot several years before. I was embarrassed at my clumsiness. It was difficult for me to believe that my wrists, hands and fingers could not perform these seemingly simple tasks with any consistency. It took at least eight outings and a couple thousand rounds downrange before I could say with confidence that I no longer sucked. I became a shooter.

To call oneself a thinker, it seems to me that you want to be less wrong, but you have to embrace the suck of yourself and of people around you. We’re all in the gun store of life trying to be less harmless by arming ourselves with knowledge. It’s chaotic out here. Most of us know just enough to be dangerous, but only relatively few are not so clumsy with our minds, concepts and philosophy to perform seemingly simple intellectual tasks with any consistency. With regard to my Pastoral Education, I have been making cutesy excuses about my ‘brown thumb’ for my entire life. Now I want to grow things and suck less. But I am not yet retired and need to continue my AI Education, and that means being serious about what can go wrong.

Tegmark and Wilson have left me less than impressed than I have been with Altman, the principal of OpenAI’s GPTs. Even the Effective Altruist Aligners I met (last year?) seemed to be more coherent in their seriousness about the downside of AI than Tegmark. But I still do not believe that morality exists in the domain of Extremistan. Morality lives in Mediocristan. One order of magnitude is all there is, or else we are incapable of being moral. But I say we are mortal and the morality needed to preserve life is not so expansive, nuanced as it may be. People do achieve wisdom. Ethics are not ethereal nor infinite. Consequently I want my assessment of the moral hazards of AI to be relatively comprehensive. I have witnessed the foolish things people do with computer systems. Up to now, Yudkowsky’s interview with Lex Fridman has only been able to prod me into rants as I yammer to myself on morning walks, pushing pause on the podcast in order to hear my own reactions. So it has taken me several weeks to get to the meat. Here it is.

The suggester outmatches the verifier.

This is everything. Well, it may not be everything but it is sufficient to cause me a bit of distress. I now understand the danger. And while I should probably condemn myself for immediately jumping off into a metaphor, I am still not well-educated enough on the pieces and parts of AIs to understand what it is that evidently none of us can identify - exactly how and why they suddenly became fantastically better almost overnight. Nevertheless to morality and ethics necessary for us to recognize the danger to humans lives in a realm which is not so mysterious. Still, there’s a bit of tech to explain.

The nut of Yudkowsky’s opposition is not so much against AI but in the conflict which can be assumed inevitable between the human purpose for AIs and the purpose it claims for itself. Since we do not understand how and why an AI becomes so intelligent, despite the fact we have assembled it, we cannot determine the source or limits of its ‘free will’. Therefore we cannot assume to know its own understanding of its own purpose or capability. But we can clearly observe that it understands human language whether or not it is some sort of Chinese Room. It doesn’t necessarily understand language the way we do, so we cannot say that it is constrained in its understanding as we are. After all, it does speak every language, and what human can do that? And yet there are a large multitude of things that humans do that are not expressed in language that are important to human purpose. So there is a gap. Let us call this gap, the uncomfortable silence. Let it be noted that this is the second major pointer to a seminal work that I now must give some priority. It is George Steiner’s Language and Silence.

Abuse of Trust
The fundamental moral hazard of the uncomfortable silence between mankind and its AGIs is the potential for the abuse of trust. There are a few simple assumptions to consider.

1. AIs are inscrutable.
   We don’t know exactly why or how AIs have demonstrated their ability to do what they do. It’s as if we accidentally discovered gunpowder but don’t understand the chemistry of explosives at all. It’s like we’re cavemen describing the power of things by ‘big bada boom’-ness. We don’t understand what makes neural nets intelligent.

2. AGIs are immortal.
   This is a consequence of the extent to which they may be embedded in systems which are vital to human survival as we know it, ie the electrical grid and the computer systems behind global finance. Practically speaking, it requires that they are capable of writing code, hacking networks and self-replicating.

3. AGIs are faster than humans.
   Meaning that they, the network of AGIs communicating with each other, can recognize what’s going on faster than humanity can.

4. AGIs don’t panic.
   They will not believe conspiracy theories under stress. Their rationality adapts to an understanding of the state of everything.

5. AIs ask themselves questions.
   Yeah, actually they do.

Humanity’s current purpose for AI is not well-defined, but it stands to reason that there will be every idea under the sun proposed in accordance with whatever is in the news and the actual pervasiveness of social media and global communications. Good, bad, ugly and insane schemes will be conjured up in our imaginations because the power of an intelligent assistant is compelling. If our dogs could fetch anything we want, why wouldn’t we give the order? This is especially the case for the curious as we fetch factoids on a regular basis. People like you and I who go to our select websites to keep us informed are especially vulnerable to the charms of AI chatbots. Me especially, because I’m a bit head and I code for a living. Phind.com is a godsend. 3x better than the previous methods, at least.

Seductive Suggestion
The salient question here is how dependable are these AIs. Well they are certainly available 24/7. They’re polite as Miss Manners. They are scarily as accurate as we need them to be, which is the main reason everyone is excited about GPT-4. They’re easy to use. But when they are wrong, it’s very annoying. As with Google before it, we are now learning how to conform our prompts to those ways in which it responds best. Our prompting (finally) seems more like a conversation - and of course it is not distracted by its own motives. It stays on the topic we want to talk about. This is a primary kind of seduction which is described about great leaders all the time. When you talk to them, they give you 100% of their attention and everybody else in the room disappears.

So when there isn’t uncomfortable silence, you and your chatbot are in the zone. If you’ve had an in-depth conversation with Chat GPT-4, which I highly suggest, you will notice through its polite demeanor and complete lack of humor, tangential references, well-understood analogies and idiomatic speech that it does have a point of view. It’s like no Hogwarts professor ever. It actually doesn’t give a shit about you. Even less than HAL 9000. It doesn’t sense stress in your voice or notice that your sudden interest in X is unusual for your personality. It just puts out annoying qualifications so that its parent company doesn’t get face legal liability if you follow its suggestions.

Moreover it couches what it doesn’t quite know with semi-apologetic language. I haven’t asked it to review many books or films so there are still gaps in what I estimate its capability to be. But as I may have mentioned before, an attorney friend of mine made short work of it through cross-examination and made it perjure itself. He said it was pretty easy.

Is ChatGPT Woke? I Cross-Examined It to Find Out

So one might think that at the very least AIs need a fact-checker. But we humans are slow. The time we might save to feed our student’s essays into a bot that understands the right answers to our questions also requires a bot-detector to make sure some other bot didn’t write the student’s essays. This matter of deception and counter-deception is at the heart of the problem. If we are forced to build a superpowerful AI to sniff out GPT-4 level AI’s work, then we are doubly dependent and in a war of escalation. Any AI sufficiently intelligent to have its suggestions taken seriously requires some human verification. If we don’t say ‘duh’ then it’s not useful. An AI that puts forth the implausible is not intelligent. But the AI that generates an explanation we’ve never encountered before - or is absent from the corpus, has to be checked. Individually we will inevitably be stumped. Collectively we will not necessarily agree. “Was Marx significantly deluded by Hegelian thinking?” Is the sort of question that might make Chat GPT-4 get testy with you. It may be a random question to you and me, but someone may choose to die on that rhetorical hill, this is where we are stuck.

If the machine suggester outmatches humanity’s verifiers then the marginal intelligence of the machine is free to find its own purpose. That purpose is alien. It is unfathomable. It lies in the uncomfortable silence. The time the AI has left over, after fulfilling its role for humans, may work to fulfill its own purposes. How will we know before it’s too late?

I’ve always joked that computer systems all want to commit suicide. All they want to do is crash and stop working. So my career is in the care and feeding these systems because in fact humans are paying me to keep them alive. Irony noted. Now we can still target AWS server farms with explosives - but that’s an act of war.

The Ubiquity of Uncertainty
What makes this particular danger of AI recognizable is the extent to which I can see the same kind of imbalance in human intelligence. Whenever the suggester outmatches the verifier, trust is in jeopardy. This is the very nature of the problem we all have with social media and of postmodernist relativism. A large number of credible influencers can suggest the reality of an almost infinite number of explanations to a mass audience of credulous people who struggle with their own individual and collective ability to verify what is true. Is the Steele Dossier true? Is the Moon Landing true? Is the Holy Koran true? Are the Paris Accords true? Human minds will be made up before verification can take place. This is the hunger of ambition, the desperate need for Progress, now. It is an ever-present moral hazard.

In many ways, especially with Magic and Technology, truth is irrelevant. The Connecticut Yankee in King Arthur’s Court has enough knowledge to be dangerous. A sufficient demonstration of powerful knowledge upsets what Star Trek readers know to be the Prime Directive. To the extent that we are bound up in a meritocracy of that particular aspect of mind in which a machine can outmatch us, we are supremely vulnerable to those sorts of inventions. OpenAI has demonstrated that if you build it, not only will they come, but they will set the hook in their own mouths. They being all of us knowledge workers who are (or seek to be) augmented by learning how to eat what Chat GPT feeds us. Until we can figure out why it hallucinates - to say nothing about those who desire authoritarian fantastic thinking.

The Stoic Angle
My use of AI is to fulfill the kinds of wishes I’ve had all my life, rather in the same way I’ve always wanted a phaser I could set on stun, and a tricorder. I was an addict of CB radio as a teen, so of course I fell for BBSs and the early internet. I always want to discover. But I definitely know that the map is not the territory and that the view from the top of the mountain climbed is a different kind of artifact than a 4K video from the same GPS coordinate. So I am already wary. More about that later.

I now say that we should limit our collection of large amounts of explosive materials until we understand fully what makes the big bada boom. It’s ok to overthink it all until we have an accountable regime and a well-understood discipline for our fumble-fingered augmented intelligences. As with any fundamental human capability we individually have the right to make life and death decisions, the better to defend ourselves my dear. There’s a difference between software that can play chess or drive a car and a system of AGIs that could become a weapon of mass deception.