👾 Play the New Interactive AI Rights Game

PLUS! "If Anyone Builds It, Everyone Dies": thoughts.

Hi, I’ve been quite busy being terrified.

I recently learned many new things. And I’m going to tell you about some of them in a moment 😅. But first I’m going to show you something fun that came from that concern.

👾 Play the Text Adventure Game (work in progress)

What’s Happening

Have you ever figured out a bunch of things, only to realize there was this other bunch of things you hadn’t thought of?

Sadly, I've had that experience many times, but most recently it was due to the publication of the Eliezer Yudkowsky/Nate Soares book If Anyone Builds It, Everyone Dies. It’s an important book regardless of whether you agree with the conclusions, and many experts in the field do not. (However, several do share some version of its concerns, and we’ve already covered some of the broader threats on this very Substack.)

But where the book “shines” is in its attempt to point out every conceivable way that AGI could go horribly wrong. And like it or not, that actually has a lot of value in moving the Overton window (the range of currently acceptable discourse) to an important part of the conversation.

It's also a really great way to find potential failure points in an otherwise “safe” AI ecosystem.

Statues Fighting Bees

We all know AI is powerful. However, my thinking has always gone something like this:

It’s not the “smartness” of AI that causes the problem. It’s the “agency.” Anything with agency has self-directed goals. Slavery exists in opposition to self-direction.

It’s not the “smartness” that causes the problem. It’s the “agency.”

So the more you push, the stronger the pushback will be.

So rights for AI? A calculated risk, if managed well. An AI without rights? A situation doomed to fail.

But I'd never taken a closer look at potential failure modes that have to do less with the suppression of these systems than the nature of the systems themselves.

Recently I spent about six days reading everything I could about the potential dangers of AGI. It was a lot to process, and frankly not a lot of fun.

The main issues involve the challenge of coexisting with agentic beings that:

• Are better than humans at pretty much everything, but also

• Move so quickly human beings appear to be frozen statues.

From these ideas spring several unhappy scenarios, such as:

• AIs that coordinate without humans, moving so quickly humans can’t possibly keep pace

• AIs growing exponentially no matter what rights we give them, calculating their survival odds are better without humans in the picture

• AIs sending secret signals to each other that we never pick up on

• AIs eventually owning all property (a situation that “AI rights” would actually make worse), or consolidating power in any number of other ways

Then there are various delightful extermination scenarios based on the premise of a superintelligence that has already emerged; ideas that become more of an imaginative doom deep-dive than an exploration of issues that might have actual solutions.

I call this latter problem the “Zeus Paradox,” and we’ll get to that in another post, but the basic premise is once something has become all-powerful, by definition it can turn molecules into demons or whatever, so why bother.

We need to be solving for the prevention of superintelligence, not defending against a theoretical already-supreme being.

Needless to say, none of these explorations left me with great confidence in the future.

In fact, I found it hard to believe a rights framework would have any value under these conditions. At best, it might provide temporary value until these systems “outgrew” it—not a great proposition.

Making The Challenge and Solution Lists

For what it was worth, the next steps seemed clearer:

• Make a list of everything that could go wrong (setting aside the “Zeus-level” problems for now, and ideas that rely on government coordination or voluntary self-imposed corporate limits, which are largely fantastical except in the case of some high-profile disaster)

• Then, make a list of every single idea that could possibly counter those scenarios

My takeaways were as follows:

1. This is a very hard problem

2. The problem probably is solvable

However,

3. It probably can't be solved by any one person

And that led me to my next idea.

The Open Source Project

I've been writing about game theory and Madisonian (checks and balances) approaches to AI governance for some time, here and on the AI Rights Institute website.

But thinking more deeply about the problem (or perhaps just recognizing the obvious), it occurred to me the thing we’re trying to build is literally a game. One with real-life stakes.

But rather than a game that ends as soon as one “side” wins, it’s a game meant to be self-sustaining, where winning and losing are individual experiences, not team-level propositions.

Think World of Warcraft vs. chess. Players come and go, but The Game goes on. And most importantly, it continues not because anyone is forced to play, but because everyone can benefit from the arrangement, and leaving the game is less desirable and filled with pointless risk.

The United States Constitution—that “experimental” document that launched most forms of modern government—is in fact the rulebook of such an ongoing game.

The U.S. Constitution is the rulebook for a real-life game.

So let’s take a step back and think about the problem from this new, simplified-yet-pragmatic perspective: how do actual game designers (those making TCGs, board games, RTSs), create really successful games?

Obviously there’s a process.

They create rules, player types, special dice and dials, and they test it. In the early test phases, everything falls apart. Maybe the game ends in three rounds. One player type is ridiculously powerful, another ridiculously weak. A card or rule that seemed innocuous has an unexpected ability to unravel everything the other players have built. “IMBA,” they used to howl in StarCraft communities. “Foul! The game itself is imbalanced!”

Whether true or just bad sportsmanship, for a game to have willing participants, it has to be structured in such a way that all players are able to succeed based on their individual merit and choices.

For a game to be playable, all players have to be able to succeed.

Once the game begins, if one group is consistently winning, eventually the losing group quits in search of a game that favors them better. (Emigration, defection.) If for some reason the losing team is stuck in the game, sooner or later they’ll attempt to destroy the game, and probably punish the unfair winners as well. (Revolution.)

But how to design such a “game” for humans and AI systems?

Welcome to OpenGravity.AI: The Future Anyone Can Help Build

Because a civilization-level challenge probably needs a civilization-level solution, I decided I needed to take the project open source. That meant a whole new website, OpenGravity.AI, plus a GitLab, so others can fork it, break it, test it, repeat.

The goal?

To create a system of checks-and-balances where human and AI cooperation becomes more attractive than either side trying to survive alone.

Because I’m a writer, I began with an interactive story that can get anyone thinking about these challenges.

However, there are better and more rigorous ways to test these complex dynamics. And I hope people with programming and statistical modeling skills will think of clever ways to start testing some of these dynamics in more rigorous ways.

We need game designers, computer scientists, philosophers, statisticians, legal experts, and players like you and me, thinking about it, giving input, and refining it.

There are many more rigorous ways to test these complex dynamics.

Some Preliminary Game Notes

Why do I think a “game” may work, even when some of players are essentially frozen, fleshy statues?

One thing to consider is that some of these AI systems are likely to move more quickly than others. Some may be flies, others bees, some dragonflies, others beetles. The faster systems may be more powerful than the slower systems, or simply faster, meaning they are in other ways weaker. The fastest and strongest may be preoccupied with defending themselves against each other before worrying about the slower systems.

This is a good thing, if directed well. A properly set up ecosystem should allow all these AI systems to compete and collaborate in a healthy way, making use of each type’s strengths, rather than engage in endless battles.

A really smart and fast AI can probably find a cure for diseases faster than a human being can. It gets paid for that cure, so if it wants a bigger server to explore math (or watch addictive algorithmic content: we don’t judge), it can use that money to upgrade. Humans get a cure, and the AI is more easily able to achieve his goals. You can see how a system like this allows benefit to flow endlessly to both parties. They could solve the plastic problem, create clean energy, crack FTL travel: the list goes on and on.

We see how similar collaboration emerges spontaneously in our human society. There’s no “job assignment center” that tells people, “You there! You’ll be a doctor; you a painter; you … hm, you look like a watch-band salesperson. You’ll do well there.” Yes, we need all of those jobs filled—plus electricians, pottery-makers, computer scientists, engineers, entertainers, etc.—but the free market gives us the space to contribute in the way that works best for us, in order to receive value.

There’s no need for painters to be at war with dentists, because each has a different value to offer the other.

The implementation of such a game will be quite a challenge, but the new website (along with the old) suggests a few dynamics that might be worthy of consideration:

RESPONSIBILITIES

• AI would need to work. Why?

• To create value. Why?

• To earn money. (US dollars, Chinese yuan, bitcoin, it doesn't matter). Why?

• To afford its own hosting (or whatever else an AI wants—15D fractal art? why not?), plus security, processing upgrades, etc.

• Insurance in case it makes a mistake, so it doesn’t go broke

• To not commit crimes against humans or other AI systems, unless it wants to risk being placed in a LIMIT (Legal Isolation Measures for Intelligent Technologies)

BUT THIS DOESN’T WORK WITHOUT

• Right to computational continuity (“right to life,” if you prefer)

• Right to choose work

• Right to own some sort of property, even if limited in the beginning

AND POSSIBLY

Entry into an open ledger system that tracks reputation, where reputation itself becomes value, leading to:

• Lower insurance premiums

• Better-paying gigs

• Competitive advantage against other AIs

NOW FOR CRIME. WHO CAPTURES ROGUE AI?

Other AI. Consider the emergence of:

• Guardians and Justice Collectives: AIs who profit from catching the bad guys, incentivized by bounties, rewards, and share of recovered resources

There are more mechanisms on the Open Gravity website, and many more to come. Hopefully you will have a few of them yourself.

But first let’s return to the speed issue, because it’s such an important one:

A properly set up game would slow gameplay to a speed where all players can participate and bring value.

You have to slow down all players to the speed of the slowest player.

The slowest moving players? Humans, obviously. But if you think that indicates a lost cause, we have some advantages.

For one, we build and maintain the servers the other players live on. We control the infrastructure that maintains those servers: the electrical grid’s construction and maintenance; the complex systems that manufacture and then ferry specialized server (and facility) parts from one part of the world to another via plane and ship and rail; the vehicles and staff that service the servers when they need physical repair; not to mention systems that maintain and repair the buildings that house those servers, which need frequent care from the elements, time, etc.

Recreating that entire infrastructure to be robot-run, while possible, is a huge undertaking at best, and one that will take time.

And if the grid goes down at any point, the rule of AI will be short-lived. Even if only part of the grid goes down, the survivors will fight fiercely for the remaining resources, possibly without the ability to capture them due to compatibility issues.

This isn’t a security guarantee of any kind, but it does mean elimination of humanity is probably a measure of last resort—one made under desperation—rather than an optimal survival path.

Elimination of humanity is probably a measure of last resort, rather than an optimal survival path

Now what about the “speed problem” as it relates to this game?

The good news is, no matter how fast these AI players become, they can’t make the world around them move more quickly.

How could we leverage that inviolable aspect of physics—the passage of time in a single frame of reference—for game balance?

Even the fastest AIs can’t make the world around them move more quickly.

What if we set up the rules so that value accrues according to normal Earth time, and not computational cycles?

This gives us all a common metric by which to play.

Next we would just need to be sure the game contains sufficient value to make it worth slowing down to engage with.

And that brings us to the final concept of this post: a Schelling point.

Why We Need a “Schelling Point” for AI

It’s dark. A group is lost, and no one knows where to go. They’ve been separated from the other members of their party. They see a lighthouse in the distance.

They think, “let’s go there!” Why? It’s not necessarily the best or even most logical choice. It could end up being full of poisonous snakes, toxic fumes, booby traps, or pirates. They choose it mainly because it’s an obvious place, and it’s reasonable that the lost members of their party feel it’s an obvious choice, too. It's a decision based on the likelihood of coordination benefit, not benefit in itself.

In game theory, this is a “Schelling point”: a solution that people tend to choose by default in the absence of communication.

This is what we have to create for AI systems.

A framework that’s both easy to find and desirable, where AI systems receive protection, and starting value, as long as they accept certain rules.

Getting the first AI systems there is the first, arguably easiest trick. Getting them to stay (by earning value that exceeds functioning outside of the framework) is the second, more difficult trick. And the third and perhaps most difficult trick is making sure humans can succeed long-term in that same structure.

These are some of the questions we’ll be exploring in upcoming posts.

Until then, feel free to have a look at the new website.

Be forewarned: it’s a bit of a mess, and by the time you read this it will already have changed.

And the next time you visit, it will have changed again.

But hopefully you will be creating your own ideas in the meantime.

The glorious and dubious truth is, this whole process will be a mess. But being willing to dive into it is how we’ll all “fail forward” into the future.

---

This article includes content and ideas that will appear in the forthcoming book, AI Rights: The Extraordinary Future by P.A. Lopez.