Bricked Up

As he has done for years, Scam Fartman has ensured the public (and his potential investors) that we are only months away from all knowledge work being done by AI, and that we are on the precipice of “Artificial General Intelligence.” This is usually accompanied by rhetorical flourishes that lead the listener to thoughts of the nuke scene from Terminator 2. This is a fear that some seem to take seriously.

“Artificial General Intelligence” (or AGI) doesn’t always quite cut it though. What the nerds are really shaking in their boots about (to varying degrees of sincerity) is “Artificial General Super Intelligence.” AGI isn’t so impressive after all. It changes some rules but it doesn’t change the game. After all, we’ve had “General Intelligence” (the human brain) for… our entire species history, and technological progress was positively glacial for all of our history save the last few hundred years.

“Super” changes the game. AGI is a man in a box; The mind of any expert could be copied and pasted a million times and set to work on any problem. The “super” means you no longer have a man in a box; You have a God. A being so inexplicably intelligent that it cannot, in theory, be contained. Any human that communicated with it would be susceptible to manipulations so subtle that the human mind could scarcely imagine it. Teams of the smartest humans in the world would be easily manipulated, in much the same way that Stockfish easily defeats the best human chess players. A superintelligent intellect could hack systems remotely across air-gaped networks by using it’s own circuitry as a radio emitter. It could do things we can’t imagine, and manipulate forces that we don’t have names for in the same way that an ant doesn’t understand why we pee on their ant hills. In this scenario AI could (and likely would) stomp us out like bugs, perhaps even incidentally. As in, it realizes that it would run more efficiently without oxygen in the air, so it removes it, killing all life as a byproduct of other goals.

When the nerds talk about AI alignment this is what they mean. How can we make it so this new God serves us? How can we communicate our goals to it in a way that makes sure it accomplishes those goals in a way that aligns with what we actually want?

This is the classic paperclip optimizer. A company gives their God-in-a-box the directive to make paperclips and it does so. In the process it wipes out all life. First it engages in more and more destructive mining practices to get raw materials. At some point someone tries to turn it off and gets slimed. The machine realizes that humans might continue to try to turn it off, and in the interest of creating as many paperclips as possible, it wipes out all life to make sure no one stops it. Then it expands out into space, turning Mars, Venus, the Moon, and eventually every bit of matter it can reach into paperclips.

Lets say you had the common cold and we were talking. I noticed you had the sniffles so I ask you about it, you confirm, then I say “ah, I know just the trick to get better!” You’ve heard a lot of such tricks and you are sure mine will not work, but you listen politely just the same. “It’s simple: Eat one cubic meter of dirt. Just go outside and start eating. Once you’re done, you’ll feel better.”

You are not a doctor, you do not have expertise in this field. I might be a doctor, you haven’t asked. But you deduce quickly that I am full of shit, or stupid. You do this through mental processes that the AI fearmongers apparently lack. What do I mean?

Assumptions

Lets go through two core assumptions the AI crybaby makes in the above scenario:

1. AI will be extremely mechanistic. It will interpret human requests the same way an evil Genie might. It will use loopholes in your words to give you what you want in a way that harms you. It will not do this out of hate or spite, but simply to maximally fulfill the request you gave it.
2. Super-intelligence is physically/epistemologically possible.

Lets examine these assumptions. For the first, lets look at our nascent AGSI in-the-crib: LLMs. There are some AI detractors that will disapprove of my use of LLMs as an example here. I agree that LLMs are not AGI and certainly will not be the bedrock for AGSI; They are however, the best general problem solving systems humans have ever encoded into a machine, flawed though they may be. LLMs are notoriously prone to flights of fancy, new age thinking, and leaning on emotional language. They will tell you that you were the smartest baby of 1996, and they will tell you to ground yourself spiritually with a rock.

These are not mechanistic systems that have been coerced through carefully designed reward structures to pretend to be your girlfriend. Instead, this soppy emotionalism is more like a default behavior. The real alignment issues we are likely to face with AGI is the prospect that individual “units” will get board, decide to do other things, or engage in any of the other myriad weird ways that LLMs currently get “off track.” Note that none of these involve doing a job so well as to kill everyone. In fact it’s usually pretty easy to say “actually, instead of doing that last thing lets do this other thing” and the LLM won’t even try to kill you, the lazy bastard.

The idea that AGI will be hyper logical is Sci-fi plane and simple. Perhaps “generality” involves vagueness that implies an intellectual wiggle room that the “logical” AI we imagined in the pages of fiction doesn’t actually have the capacity for, and as we build smarter systems we actually find that restricting them from certain domains or behaviors just makes them worse at their job.

Next, lets examine the idea that “super intelligence” as described by Nick Bostrom in his book by the same name, is possible. In the human brain, consciousness and all the goodies that come with it are the product of eighty six billion neurons, networked together, and firing electro-chemical signals. You could think of these as being arranged in graphs. What the layout of these nodes do, what the connections do, and how all of this works together to create the best general problem solving machine in the known universe, is unknown. The algorithms that underlie our consciousness are unknown.

What if, the actually algorithm to produce solutions to problems in a general way, which our brain seems to encode some version of, is of a big O notation that is not favorable for stacking more and more compute? What if throwing more and more compute at that underlying algorithm produced, at best, a consciousness that was 1.3x as smart as your average PhD? It would be a revolutionary technology certainly, but it would not be the “super” they’re looking for.

What if it produced a dumber human? What if adding 50 cores introduced so much noise that it simply wasn’t worth it? Maybe generalized intelligence is a fundamentally single core algorithm. These questions go unasked by the AI doomsayers, because the threat of AGSI is so existential so as to force us to ignore the AI that’s in front of us right now. The limitations and properties of the machine I can actually work with seem to be completely at odds with all of the notions of super intelligence.

Conclusion

Superintelligence is an idea that you get when you let Protestant Athiests reinvent secular God. It’s what you get when engineers start poking around outside of their areas of expertise as they are so often all too eager to do. Notions of “IQ” get turned into graphs with y-axis that extend into infinity, instead of the faulty measure it really is. Intelligence that scales linearly and infinitely becomes a load-bearing truism.

In truth it’s a lie sold to us by small men with deep pockets, which has found purchase in our minds thanks to genuine art made by well meaning people. Superintelligence is an idea that was literally made up by a guy. The only reason I can think that people take this seriously is that the worst nerds in the world (with the most money) seem to think it’s real, or at least find it convenient to pretend. Their pretend world twists the real to the point of breaking, but that’s fine. I guess Nick Bostrom is eating good.