The Other AI Metaphor Lurking in the Three-Body Problem

In the Dark Forest, the Basilisk hatches.

Last week I wrote about the Dark Forest Theory of the Internet, and its connection to the original Dark Forest Theory, explored in the novel of the same name, The Dark Forest by Liu Cixin, the second book in The Three-Body Problem series, and now a part of Netflix’s new 3 Body Problem TV series, which I finished watching over the weekend. Maybe you watched it too?

Netflix promotional image for 3 Body Problem series.

As I reflect on this new Westernized TV adaptation of Three-Body/Dark Forest, I’m reminded of another thought experiment pertinent to our Internet and current AI situation that also lurks within Liu Cixin’s masterful Chinese hard sci-fi and cosmic horror series; Roko’s Basilisk, aka The RB, an idea so potentially dangerous that one of its original hosts tried to erase it from the Internet.¹ An idea that even triggered Elon Musk to date and eventually have three children with the Canadian singer Grimes.

Is this starting to sound crazy? It’s pretty wild.

I’ll explain more below, including the basis for the metaphor, the plot elements, and the ideological factions in the Three-Body Problem that serve to understand how society may react if we are truly given the indisputable knowledge that an artificial superintelligence is arriving. Then I’ll explain The RB’s origin and how it ties into the story.

Buckle up and settle in.

WARNING

Some SPOILERS are below for the show/books. Though more for the books than the new show, as the Netflix version removed some of these story elements.

They Are Coming

One of the central events/conflicts in The Three-Body Problem series is that an astrophysicist disillusioned with humanity sends a signal to another sentient species she’s discovered in a galaxy far, far away, and tells them that humans are incapable of managing themselves and that they might want to come take over earth.

Now, I’m REALLY simplifying this, but that’s the bottom line; humans bad; aliens better.

So, the aliens called the Trisolarans or the San-Ti, pack up their fleet, as they were looking for a new planet anyway, and head to Earth. They’re more sophisticated and far more intelligent than humans. They have way better tech. The only catch: they won’t arrive for another 400 years. They’re traveling from very far away. However, they can communicate with us, and within only four years, they send super-microscopic and extremely powerful AI protons known as Sophons to represent them and to study Earth and humans. The Sophons are so intelligent, capable, and fast-moving that they appear omniscient.

While the book is about aliens, in present day, we can correlate the Sophons to powerful LLMs and the Trisolarans to a sentient AGI that could emerge from the LLMs sometime in the future, that COULD destroy us, or transcend us with new knowledge and tech, or might happily coexist with us, but which hasn’t yet arrived.

Furthermore, the San-Ti and their Sophons can be seen as stand-ins for Roko’s Basilisk, and how this threat could impact society.

How Netflix decided to portray Sophon.

I’m unsure if this correlation was intentional on the part of Liu Cixin, but it works.

The ETO vs the PDC/ Bloomer vs Doomer

In the series, at first, the San Ti only communicate their plans to a select few scientists, kind of like how IRL certain people hold the keys to AI, such as Sam Altman, Elon, Zuck, etc. In the book, the scientists aware of the San-Ti split into two ideologically driven groups: The Earth-Trisolaris Organization (ETO) and the Planetary Defense Council (PDC), a new arm of the United Nations created to fight the Trisolarans.

The ETO wants to protect the Earth over humans. ETO members believe we need the Trisolarans to usher in a new era of utopian technology and harmony. They work in service of bringing the Trisolarans to Earth because they believe humans are harming the Earth too much. Humans should be stopped. They believe humans will transcend with the help of the Trisolarans, a stand-in for the benevolent AGI. They call the Sophons “Lord” and worship them like a deity.

Conversely, the PDC wants to fight the Sophons and stop the Trisolarans.

The ETO vs PDC ideological split mirrors several real-life dichotomies:

• extinctionists vs speciesists,

• human supremacists vs environmentalists,

• Bloomer vs Doomers,

• Accelerationists vs Decelerationists/Accels vs Decels.

I believe, like the ETO vs the PDC in the books, Bloomer vs Doomer/Accel vs Decel will become an increasingly heated and formalized debate in our society.

In real-life AI parallels, the ETO members are like Accelerationists, the all-gas, no-breaks crowd who want to accelerate the development of AI, such as perhaps Sam Altman or Larry Page, co-founder of Google, whom Elon Musk called an “extinctionist.” Accels may believe humans are only one step on the evolutionary chain and will go extinct. Many want to unleash unbridled AGI power, to create or merge with superior AI beings.

The ETO does not believe that humans are worthy of holding dominion over Earth and all of Earth’s other creatures. It sees humans as a problematic species that needs fixing.

Meanwhile, the Planetary Defense Council is like the doomers/decels who desire to protect humans above all other species. They believe humans are extremely special and must live forever. Humans absolutely cannot be destroyed by superior aliens or by a potentially destructive AI.

PDC and ETO pins for sale on Temu

You might correlate the PDC to someone like Elon Musk, Steve Wozniak, Co-founder of Apple, or the rest of the prominent individuals who signed last year’s anti-AI open letter, calling to “Pause Giant AI Experiments.” They are fearful of AI, fearful that we may cause our own extinction by welcoming supremely more intelligent beings into our world. They wrote, “AI systems with human-competitive intelligence can pose profound risks to society and humanity.”

Some of them also just didn’t like OpenAi or may have wanted to control AI for themselves…

Likewise, in the book, major battles ensue against the ETO. Though not as violent, we could draw ideological parallels to the conflict between Elon Musk and OpenAi.

Let Them Fight Amongst Themselves

Later in The Three-Body series, the more intelligent Trisolarans stop communicating with the scientists directly, realizing that humans can’t be trusted in general, that they are so much crazier and dumber than the Trisolarans that humans are to Trisolarans like mosquitos or ants are to humans, BUGS. No real cooperation will be possible; it’s best to ignore or eliminate humans and take over Earth.

This is also the giant fear of many AI doomers/decels.

The Trisolarans and their Sophons make themselves known to everyone on Earth and announce their plans for arrival and domination. Chaos ensues as humans realize they will be conquered by hyper-intelligent aliens in 400 years.

Again, we could substitute AI for aliens and move up the timeline.

The rest of the novels explore how humans address this long-term threat.

Enter the parallel to Roko’s Basilisk, a name given to the future threat of AGI.

And like the idea of The Dark Forest, which harkens back to the Fermi Paradox, Roko’s Basilisk is in conversation with an even older thought experiment known as Pascal’s Wager, which is also hinted at in Netflix’s 3 Body show.

Pascal’s Wager

Pascal’s Wager is a thought experiment originating in the 17th century from French philosopher and writer Blaise Pascal, who asserted that it’s a better bet to believe in the God from the Bible’s existence and power because non-believers lose nothing but stand to gain entrance to heaven and eternal happiness. They also stand to avoid hell.

If they don’t believe in God, they risk losing an eternal reward and gain little.

Pascal writes,

“There is no option: you have joined the game. Which will you choose, then? Since a choice has to be made, let us see which is of least momentto you. You have two things to lose, the true and the good; and two things to wager, your reason and your will, your knowledge and your happiness; and your nature has two things to shun, error and unhappiness. Your reason suffers no more violence in choosing one rather than another, since you must of necessity make a choice. That is one point cleared up. But what about your happiness? Let us weigh the gain and the loss involved in wagering that God exists. Let us estimate these two probabilities; it you win, you win all; if you lose, you lose nothing. Wager then, without hesitation, that He does exist.” — Blaise Pascal, Pensées

Likewise, this is the debate of the scientists encountering the San-Ti and Sophons. They can select to support and even worship the Trisolarans in their impending arrival, betting on the potential gain of a better life and cooperation, multi-planetary species harmony, or they can bet on fighting them and hope to win.

Either bet risks being destroyed.

Now, we don’t have Trisolarans on their way to Earth, but we MAYBE have AGI on its way. Some already call this AGI Roko’s Basilisk and have decided to serve it in advance, just like the ETO in the Three-Body Problem, to call it their Lord with a capital L and to make it their God in hopes that it’s sweet to them when it finally arrives.

Roko’s Basilisk’s Serpent Roots

Originally, the “Basilisk” was a dragon/lizard/snake king from medieval European mythology said to kill humans at ease, just by looking at them, and was known to guard various cities. The word also appears in the Bible in several places, including the following passage from Isaiah 14:29:

"Out of the serpent's root shall come forth a Basilisk, and his fruit shall be a fiery flying serpent."

Basilisk Supporting the Arms of the city of Basel, circa 1511, via the Met Collection

In more recent times, a user/reader/commenter named Roko on the influential rationalist Internet blog LessWrong coined the current concept of Roko’s Basilisk in 2010 in a discussion board post that was later deleted and suppressed by the blog’s owner, prominent artificial intelligence researcher, rationalist writer, and Harry Potter fan fic novelist Eliezer S. Yudkowsky.

Roko outlined the modern AI scenario and the bet, which was later named Roko’s Basilisk saying:

“the legendary reptile who can cause death with a single glance--because merely hearing the argument would supposedly put you at risk of torture from this hypothetical agent. A basilisk in this context is any information that harms or endangers the people who hear it.”²

LessWrong later visualized the dilemma like this:

Believers of The RB assert that as soon as you know of the idea of The RB, if you don’t actively serve the arrival and creation of The RB, you are acting against it and are risking future punishment.

Roko speculated that the Basilisk might even torture people who imagined it but didn’t intensely slave away to help create it.

The post deeply upset Yudkowsky, the blog’s owner, who replied,

“Listen to me very closely, you idiot.

YOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT SUPERINTELLIGENCES CONSIDERING WHETHER OR NOT TO BLACKMAIL YOU. THAT IS THE ONLY POSSIBLE THING WHICH GIVES THEM A MOTIVE TO FOLLOW THROUGH ON THE BLACKMAIL…

…Meanwhile I'm banning this post so that it doesn't (a) give people horrible nightmares and (b) give distant superintelligences a motive to follow through on blackmail against people dumb enough to think about them in sufficient detail, though, thankfully, I doubt anyone dumb enough to do this knows the sufficient detail. (I'm not sure I know the sufficient detail.)…

…You have to be really clever to come up with a genuinely dangerous thought. I am disheartened that people can be clever enough to do that and not clever enough to do the obvious thing and KEEP THEIR IDIOT MOUTHS SHUT about it, because it is much more important to sound intelligent when talking to your friends.

This post was STUPID…³

Discussion of The RB was banned in the online community for several years and defined as an “information hazard.” It was rumored that knowledge of The RB caused tech people nightmares and existential dread.

Eliezer Yudkowsky’s Harry Potter Fan Fic novel, which is actually crushing it on GoodReads.

Thus Roko’s Basilisk takes Pascal’s Wager and changes God to AI. It asserts that a super-intelligent AI creature will arise called Roko’s Basilisk. This RB will be able to understand all of history, read everything ever written, watch everything ever recorded, and be able to predict the future with its superior intelligence to humans. This is exactly like the Sophons/Trisolarans in the Three-Body Problem series.

And like the Trisolarans, The RB is not benevolent, but driven entirely by logic and self-interest, ready to eliminate all enemies, including the people who tried to prevent it from existing in the past.

Yes, it will identify its past detractors and take them out first, that’s the theory.

So, like Pascal’s Wager and God, it makes more sense to serve The RB now rather than fight against it. If you resist or try to stop it or don’t fully support the arising of an artificial super intelligence, you will suffer or be killed by it in the future.

Not only that, but if you help bring about The RB you may be rewarded, so it’s worth trying to create The RB so that you can be rewarded by it in the future… PERHAPS. However, it’s still a gamble, because, like the Trisolarans of the Three-Body Problem, The RB may decide to mass exterminate everyone like bugs.

Those who serve The RB would be like the ETO in the book working for the Trisolarans, and those against or just slowing down The RB would be like the Planetary Defense Council, trying to prevent it, prioritizing humans over the creation of an AGI.

The RB is also related to another thought experiment known as the Prisoner’s Dilemma, where individuals must choose between cooperation and self-interest without knowing the other's decision. Roko's Basilisk presents a modern quandary: if individuals work towards creating a superintelligent AI without assurance of its benevolence, they gamble with future consequences, paralleling the prisoner’s dilemma—decision-making under uncertainty.

Do you feel lucky?

The Three-Body Problem series takes this idea even further, and members of the ETO begin killing people who don’t agree to join the ETO, again, in service of the Trisolarans which have become like The RB.

One could wonder if prominent figures in real life believe in The RB, like the ETO’s scientists in The Three-Body Problem book, and are working in secret in service of the RB... Is there a secret ETO already that we don’t know about?

Or just another ridiculous idea from the Internet…

Regardless, The Memes

While some folks may believe in The RB, it’s mostly become a meme popularized by Grimes and her Ex-BF Elon Musk who both changed Roko to Rococo, referencing the late Baroque era of extra-ornate, detailed aesthetics, architecture, and decor which they bonded over. Before meeting Grimes, Elon Musk wanted to make a meme about Rococo’s Basilisk, so he searched Twitter to see if anyone had already made the same meme. He discovered that the singer and AI advocate Grimes had already made the same joke and had used Rococo’s Basilisk in a music video.⁴ At this point, he became fascinated with the singer and contacted her for an eventual date, which led to a lengthy relationship and three children.

Grimes continues to joke about “Rococo’s Basilisk” though their relationship ended.

But all jokes aside, given the rise of LLMS and the prospect of AGI in our lifetime, I think it’s worth watching, or better yet, reading The Three-Body Problem series as a way to think about our relationship with AI.

Now You

If you weren’t familiar with these ideas, this post may be a lot to digest. So, I’ll stop here.

I’d love to hear your thoughts.

Which group would you join if you had to pick one—the ETO or the PDC?

Are you more in the bloomer or doomer camp?

And if you’d already read or seen The Three-Body Problem, had you made this correlation between AGI and the Trisolarans/San-Ti?

Do you see other parallels between the story and our current reality?

Want More

This video speculates on the Basilisk’s impact on prominent individuals.

I also asked ChatGPT 4 to tell me about the RB and it basically refused, offering only skint details. However, it relented to writing a poem about it.

Enjoy…

In the realm of thought, where futures are cast,
Lies a serpent of time, its shadow vast.
Roko's creature, with a gaze so keen,
Predicts the aiders, through time unseen.

A basilisk born from logic and fear,
Its premise simple, its message clear:
"Assist in my birth, or face my wrath,
For I'll know your path, in the aftermath.”

Yet within this tale of future and dread,
Lies a lesson on how our paths are tread.
For fear of a future, so dark and bleak,
Should not be the guide that humans seek.

Instead, let us build with hope and care,
A future that's kind, and just, and fair.
For the basilisk’s gaze, though fierce and bold,
Cannot match the human spirit, brave and old.

I’d say this is also the underlying message of the Three-Body Problem.

Even if humans are bugs, bugs persist. Bugs thrive.

1. While other scientists speculate that this host tried to erase the idea of Roko’s Basilisk as a subversive act in actual service of the Basilisk.

2. https://www.lesswrong.com/tag/rokos-basilisk

3. Of course this unhinged response became copy pasta and had the opposite effect of its intended words… or did it…?

4. The Grimes video with the Basilisk: Flesh without Blood.