Republished from the philosophical journal The Side View.

Power exists. Concentrated in the hands of a few, it shapes every aspect of our lives in the deepest of ways, including our very thoughts and language. This fact is universal across all societies.

Sometimes that power is destructive. We’ve seen bloody wars, cultural disasters, ecological disasters, and economic disasters—all of them traceable directly to the deliberate actions, malicious or merely foolish, of power.

On the other hand, we’ve also seen incredible progress. Peace attained, cultural and intellectual flourishing, ecologies restored, great feats accomplished, and economic development miracles—again, traceable to the deliberate actions, wise and benevolent or merely interested, of power.

What to do about this power, and especially how to achieve the good results and avoid the bad ones, is a central theme in both philosophy and the narratives societies use to organize themselves. We can call this the problem of rule.

To approach the problem of rule, and to come to a productive paradigm that we could use to better organize ourselves as a society, I want to lead you through a thought experiment. It starts with a simple question:

Given that we have all this power, how shall we use it?

To be explicit about the use of “we” here, we are taking the problem of rule head on, from the first-person perspective. As individuals, most of us do not have much power. But ideas are ultimately about active practice, and power is the ability to act, so the first-person view is essential here.

Furthermore, the most powerful and most important actor in society is the ruling elite. Our social ideas must therefore be addressed to at least some hypothetical ruling elite, if not the current one. If they do not have a well-developed paradigm for rule from the first-person perspective, or they have the wrong one, they will not be able to govern well.

So, to make our question more direct, we could ask, “What would you do if you were the ruling elite, and you had personal moral responsibility for the outcome?”

We will also need, eventually, a third-person view of power. There are after all both rulers and ruled in any society. But a healthy first-person view of power will give us a context for the third-person view, so we will start with the first. “We” hereafter, then, refers not only to the participants in this written discourse, but to the perspective of our hypothetical philosopher-elite.

Our thought experiment will proceed through a series of perceptual transformations, adding nuance and new considerations at each step. As we follow the logic of the framing question to its conclusion, its implications will trickle down to reformulate our whole political worldview:

• First, we will reorient our attention to answering the question of what power should do, as opposed to all the other ways to approach political discourse.
• Second, we will expand and reconsider our interests in light of the full set of assets and liabilities in our empire.
• Third, we will consider how political order is actually constructed, and the new set of interests and concerns that come from that construction.
• Fourth, we will stop to wonder why we ought to care to rule well at all. What’s in it for us?
• Fifth, having developed a healthy perspective for a responsible elite, we will expand our identity to the collective selfhood of an entire society.

Occupying the Throne

We’re not going to start with a healthy perspective. We’re going to start by occupying the throne of sovereignty and flexing our power muscles. Bear with me.

Imagine you were in charge. You need to now see the world in terms of the governing actions of power, and you need to have an opinion about what that power ought to be doing. You need to develop a responsible governing intention. Occupying this perspective is the first step.

Pick your favorite political issue. Housing? Transportation? Military? Immigration? Something local? That one road they closed to outside traffic? Dictate. Put your current instincts in the governing seat. What would you do as the ruling elite?

Build more houses and trains. Ban healthcare. Replace farm subsidies with ranch subsidies. Arrest the opposition. Put the universities on the hook for student debt. Invade the moon. Levy a carbon tax. Whatever seems right to you. Don’t even worry about being moderate for now. Unleash your inner ruler, and get radical.

It is actually a hard question to answer. You’re being asked to put your money where your mouth is, in a way. To take responsibility. This is not merely a Socratic critique of everyone else’s actions and ideas, or bonding with your friends over shared values and troubles, and getting angry at corrupt politicians. No, this is putting your neck out there with your own positive account of the nature of justice and what must be done. Our usual patterns of thought aren’t fit for it.

Which is exactly the point. Our current instincts aren’t that good. The point is to take seriously the problem of rule from the first person, breaking out of the grooves of our usual third-person perspective, so that we can come to a better way of thinking about what we should do with our power as a society.

This is different from many of our usual ways of talking about political matters. We often see politics as a battle against the political enemy, an object of detached study, an imposition on our lives, a prompt for individual lifestyle change, or something to be influenced but not governed. These are all fair and practical perspectives. But they do not answer the question. Power still exists. How shall we use it? A tool cannot be used well without a coherent vision of what to do with it. To get an answer, we need to firmly occupy the first person, grappling with power as a responsibility and a tool.

The Holistic Perspective

Having settled our current political instincts into the seat of sovereign authority, how does this change those instincts? Our ideology evolves just by taking the idea of its implementation seriously.

Our existing concerns are usually driven by our immediate sphere of attention, our immediate interests, and our relatively un-tested idealizations.

You commute by car, but you hate traffic, so naturally you want to widen that freeway and reopen that one scenic road to nonlocal traffic. But if you take your new holistic perspective—with a responsibility for the big picture—you start to wonder whether this way of thinking will solve the problem most efficiently and effectively. What about denser housing near the core? Or better public transit? Or different zoning?

Perhaps you have a business interest in resource extraction, so you want to plow ahead without this silly talk of environmental sustainability. Or perhaps you have no such interest, and instead you simply want to put an end to environmental damage. But again, now responsible for the whole picture, you have to prudently account for both sides of that balance, over the longer term.

Maybe you own small-scale rental housing, so you don’t think the city should recklessly build low-rent slums in your backyard. Or you rent, so you think the state should build an oversupply of housing to crash the market. Looking at the big picture, these provincial interests fade into the background and you become interested in what’s best for the city overall.

In general, this is a transformation of provincial interests to holistic interests. Your perspective now abstracts over more time, space, social function, and human variation.

But if not our current interests, what does make good policy? We still need some guiding paradigm. There are other things we might target. Political prudence and the needs of political order and state capacity. The health of your empire and prosperity of the people. The spiritual self-development of your society. We will get to these ideas. You may already have some sympathy for them.

But for now, in this lower tier of consciousness, where we think in terms of interests, consider that by now having control and responsibility over the whole empire, instead of just a small corner, our priorities are fundamentally different.

If we take this insight seriously, and we use it as a basis for re-thinking our aims, it deeply recodes our worldview. We empathize with the elite. Many previously inexplicable policies, and things that chafed on us, start to make sense. Many people we didn’t previously sympathize with become important responsibilities. Many things we didn’t care about that seemed unimportant become central. Some issues even flip in valuation.

The output is a new, more coherent ruling ethos, in which the parts of our program fit together, and more closely match the whole set of concerns that our empire faces. A holistic grand strategy emerges, one that can be applied to guide our concerns in particular cases, not just our pre-existing prejudices and provincial concerns. We become more able to occupy the mindset of the responsible statesman-elite.

This is all still a thought experiment. Strictly a hypothetical. You are free to keep your relatively more provincial or otherwise non-ruling ideology. But even just as a hypothetical, there are two big effects:

First, given this responsible perspective that actually engages sincerely with the challenges of power, you are in a much better position to critique the existing elite. By engaging more sincerely and expertly with the actual logic of the governing position, you are coming at them with more authority, and more potential for your critiques to be taken seriously.

But second, that logic turns around on you as well. The natural perspective of the state is not really a hypothetical, given its ability to impose it on society and you. It becomes normative. Many formerly attractive ideologies are revealed as futile and unrealistic as they do things that responsible governance cannot allow. It changes how you look at things.

These changes are an elevation of our perspective towards that of idealized authority. This perspective of responsible governance is already quite a bit healthier than where we started. It has already started unifying the interests of rulers and ruled in its logic. But it is nowhere near complete.

Building Inclusive Political Order

So far, we have made a few simplifying assumptions:

We have assumed a hard division between rulers and ruled. The ruled are politically inert, and can be herded around by the power of the ruler. The ruler is not subject to internal division or structure, but simply carries out some program of rule.

We have taken political order for granted. The state will stay secure, and doesn’t have to worry about rebellions, traitors, court politics, or the actual logistics of regime formation and program execution.

We have assumed the capabilities of the state are given. We have not addressed how power actually works, how it is constructed and held and used, and therefore what kinds of powers we will have.

These things are not at all assured. In fact they are all dangerous fictions, liable to produce disasters if applied sincerely, or at best hypocrisy and inefficiency.

If we take seriously the idea that subjects don’t have political agency, we will not be able to harness their dynamic potential. In fact, people have an incredible ability to organize themselves, to rule their local domains, and to interface with the larger political structures around them. The Hayekian knowledge argument—about local action matching local information, resources, and needs to a global context—applies to political organizations and economic regimes alike.

Worse, by using an ontology that discounts local agency, we implicitly classify it as a kind of corruption in need of correction. The ebb and flow of local interest, information, and action become pathologized, and you end up waging an oppressive battle against social reality. Restrictive and simplistic models of what’s supposed to be happening are a huge source of destructive over-interference, in our society and in the world in general. There is subtlety here, though: This model shear isn’t always a mistake, and is in fact often a consciously applied weapon in the toolbox of power. Anyone with the power to define official social ontology is in possession of a fearsome weapon for the shaping of social fabric. Again, let us meditate on how to use these powers well.

The boundary between state and society is also imaginary. Let’s describe and think of society and the state as they actually are: a teeming forest of growing and decaying institutions and conspiracies in various states of coordination with each other, the commanding heights of which are integrated into a semi-institutionalized, semi-coherent central power, the formal parts of which we call the state. It’s a system of information flows, power empires, social technologies, social network hubs, and inert human resources. A complex sociopolitical narrative of legitimacy supporting and mapping a complex sociopolitical reality that it only partially resembles. A complex adaptive system.

Political order is the structural integrity of this pyramid. It means ensuring that all subordinate institutions stay subordinate, the ruling institutions stay ruling, the loyal are promoted, rebels stay suppressed, the parts work together instead of fighting, and the people support the regime. Without political order, you get instability, as rival power centers challenge the regime, and fragmentation creates broken parts while a weakened center fails to coordinate its empire. Political order cannot be taken for granted. Nor is it an anachronism. It is deeply central to any society, and needs to be continuously renewed.

Active construction of political order, by building, deconstructing, and herding subordinate power centers, is inseparable from the actual capabilities of the state. The central power of a modern society has immense ability to reshape and control nearly every aspect of life, within the constraints of knowing how to do so, getting the institutions to cooperate, and building necessary machinery. Almost nothing can be done by pure fiat, but almost anything can be done. The game is much more about mapping and manipulating this actual ruling structure towards a generally defined end than it is about twiddling technocratic policy knobs.

To update our question, then, what does it even mean to be in charge of such a thing? How shall its power be organized behind our own conception of the good?

Being in charge means being in a position of strong influence over the central powerful institutions in the empire in question. It means having a deep and fluent understanding of power, especially our own base of power. Being in charge is a much more active position. At the same time, it means identifying ourselves with the center of an institutional machine larger than ourselves, becoming an instrument of its imagination, and thinking up ideas of what to do with its power, from its own perspective. Being in charge becomes a mixture of actively maneuvering to build and possess the institution, and playing a role in letting it possess you.

With this more active set of concerns, our perspective shifts again. We come to focus on the institutions themselves, on building and repairing a base of power, on maintaining information flows, sniffing out and resolving discoordination, and on finding new ways to extend the power to accomplish our vision, including by empowering others. We aggressively centralize power here, and apply a restrained decentralization there.

This is very different from the initial focus on twiddling policy knobs in pursuit of a naively imagined objective notion of responsible government. This is also a much firmer set of realities on which to ground the idea of good policy: good policy must actively maintain and develop political order in the complex system of society.

Again, this is another step towards a healthier way of seeing things. We’re engaging more with actual reality, and building a much more organic, responsive, and politically inclusive sort of regime. And we’re also starting to ground the perspective of the state in something more solid. But we’re not done yet.

Higher Pursuits, Together

Our first instinct has rightly been to simply take the problem of rule at face value, and then do the right thing. The result so far has been a responsible governing perspective that actively builds an organic and participatory political order.

But faced with the reality of actually having power, the naive intention will rapidly dissolve, and the question of personal motivation will become central. Why do we care to govern well? Why not just make the state serve our own wants, and those of our friends? It’s less that power corrupts our motivations, and more that it strips away the veneers of naivety and social constraint and enables the actual reality of our motivations to shine through.

We will make the state serve our own motivations. But the simple appetites of the body can be solved rather cheaply. You and your friends only need so much food, sex, living space, and clothing. Even when fulfilled lavishly, at some point we run out of appetite for personal luxury. The state has resources beyond our wildest personal desires. Only the largest motivations will do.

Once our basic needs are handled, and we work the luxury drive out of our system, our attention naturally turns to much more scalable and less-assured higher needs: social connection, glory, esteem, knowledge, play, exploration, creativity, piety, and growth. We have an entire empire with which to pursue these things, and an empire’s worth of people to work with in doing so. What do these aims look like on a civilization scale?

These higher needs aren’t really personal, either. They are best shared with others. Perhaps even with our entire empire. And to pursue them best, we need everyone at their peak. We need to govern for prosperity, meaning, connection, harmony, and collective capability. We need to narrate and think in terms of governing for a grand collective adventure in pursuit of those higher wants.

The decadence of late dynasties and new money is not reflective of wisdom, but of ignorance. They have forgotten, or not yet learned, that private luxury and corruption is just far lamer and less interesting than effective pursuit of the higher ends. Or they just don’t know how to reach for anything more substantial. One of the most important jobs of the philosopher in society is to remind us—and create a narrative for us—of the intrinsic superiority of the higher pursuits.

This is the most difficult and least reliable of our perspective transformations, because it’s not a matter of practicality, but of self reflection and self transformation. It is no less compelling, but requires more philosophical imagination. We are much more likely to succeed if the philosophers of the day provide a well-developed narrative paradigm to guide us through it.

And so we come to a healthy first-person view of power: It is the leadership of a collective enterprise in pursuit of first subsistence, and then higher goods.

Towards a Collective Selfhood

This sets up the final transformation of our political perspective.

Consider the correspondence between the structure of the mind as a self, and the structure of society as a collective selfhood. The mind has many of the same problems we’ve been discussing. It has a multitude of processes that need to be governed, integrated into a coherent teleological order, and aimed at moving towards higher goals as a collective body.

Having done so, the mind conceives of its problems no longer in terms of its internal mechanics, but in terms of knowledge, skills, plans, virtues, mindfulness, askēsis, and spiritual enlightenment. That is, it becomes a subjective selfhood.

As an insight about the ordering of the mind, this is neat, but the real treasure is applying it the other way: Having solved the basic conceptual foundation of rule, we can look at society as a collective selfhood.

We can pursue virtue, knowledge, play, skills, exploration, meditation, contemplation, and enlightenment, but as a collective self. We are composed of many, but can also be organized into a holistic self. Our problems can be approached as the internal ordering of a single complex being.

This isn’t an original idea. The correspondence between an individual mind and a society is a classic theme in philosophy. It is also a frequent recourse in political discussion: the naive political “we” is exactly this organized collective selfhood.

This idea usually lacks grounding, but here we have grounded the political “we” in the first-person logic of rule as a philosopher-elite might think it through. If we take the problem of rule seriously enough from the first person, we get an expansive and healthy conception of collective selfhood that many more than just the actual holders of power can participate in.

Most political and therefore ethical worldviews are unstable under either victory or reflection. They dissolve if the user actually achieves the governing seat or thinks too hard about their interests as rulers. They are thus only good for power grabs and dupes. This whole thought experiment has been an attempt to engineer directly to pass these tests: adopt the first person perspective of a ruling elite, and then take its transformative consequences very seriously to see where we end up. Assuming we got the logic right, the result is something that could actually work, but still ends up somewhere reasonable and even enticing.

In the ancient world, the men who kicked off this whole philosophical enterprise in the first place had a similar idea: that justice was not something that needed to be imposed on rulers, but recommended itself to the first person on its own strength and goodness, to those who were philosophically receptive. The key, as in Plato’s attempts with the Academy, is to develop this core ideal into a practical working paradigm that can be systematically taught to an aspiring philosopher-elite.

But this key idea of a single agent or artificial value system being able to define the entire future as a “singleton,” or even just maintain stable coherence within a limited sphere of influence, is wrong. There is good reason to believe, from the valiant efforts of the best AI alignment researchers themselves, that the goal of this program is just impossible. The term “singleton” was coined by futurist researcher Nick Bostrom in a 2005 article on the subject, building on transhumanist, extropian, and singularitarian conversations from the late 20th century. He leaves it an open question whether the “singleton hypothesis” is true, but says he believes it. He is not alone; it is implicit and often explicit in the alignment discourse. 

The argument for the singleton hypothesis rests on two key assumptions. First, that an AGI may have, compared to humans, unprecedented ability to modify its own processes and scale up its own hardware substrate, leading to a scenario called “recursive self-improvement” (RSI). The agent would recursively modify itself to become smarter and more powerful, unlocking even further gains. The outcome of a recursive self-improvement-induced “intelligence explosion” would be at least one highly-scaled optimizing agent. This post-explosion intelligence ecosystem may be many orders of magnitude superior to human civilization in every relevant measure of cognitive power including speed, scale, efficiency, precision, agency, coordination, and so on.

The second assumption is that increased technological capacities lead to increased centralization. Power imbalances are self-reinforcing. More powerful agents can dominate less powerful agents up to the limits of technology and organization, which tends to favor scale. This suggests that in the limit of a higher technology future, a single power will be dominant over the whole world and beyond. With the presumed much higher diplomatic and introspective precision with which post-intelligence explosion agents could cooperate, this could also be a concert of agents that have assembled into a single super-agent. This super-agent “singleton” is presumed to be destined for arbitrarily high levels of introspective stability, political security, and error-correcting robustness, such that it will have essentially arbitrary and immortal power over reality in its sphere of influence.

The quip about the future light cone rests on speculations that such a singleton could expand its sphere of influence, or the colonized territory of perfect copies of itself, throughout the galaxy at the theoretical limit of speed. The result would be an expanding domain, approximately our future light cone, in which whatever value system the singleton settles on has become the inescapable law of what might as well be reality. If that value system included returns to whatever is left of Sam Altman’s investors, those returns would be permanent and grow at near the speed of light.

This is obviously a science-fiction-inspired scenario. But it is believed with full seriousness by many of the top thinkers in AI alignment, and science fiction is itself often relatively careful speculation about the future, at least given certain assumptions. It is also a species of the classic Enlightenment ideal of a single rational world-state that uses science and technology to bring about an immortal utopia. It’s just a post-human extrapolation of the same ideas that animate much global elite ambition and created much of the world we live in. It is very much worth engaging with rigorously.

The problem with the singleton hypothesis, besides the more speculative intelligence explosion predictions, is its key assumption that power and cognition can be fully rationalized in a stable unified agency. Self-unification would be necessary for any kind of stable singleton scenario, even just on one planet or in one sphere of influence, as the alternative is an uncontrollable internal conflict dominated by practicality rather than values. But everything we know about minds and polities suggests that they cannot be fully unified. They always have division and instability within themselves. 

Our minds and polities are only relatively unified in the course of pursuing narrow coordinated actions against external resistance, and rapidly decay back to internal incoherence in its absence. While in the example of our own minds we have a rather convincing existence proof that general intelligence is possible, we entirely lack any evidence to suggest that rational self-unification is possible in practice or even in theory. On the contrary, we have a great body of mathematical theory ruling it out in a large and suggestively comprehensive variety of circumstances.

Without rational self-unification, the whole singleton scenario breaks down, and while the future may be highly technologically advanced and incompatible with natural humanity, it will not have this apocalyptic character as a cosmos-redefining advent. Instead, we should expect that while recursive self-improvement would result in a new ecosystem of vastly more autopoietic life, that new life would run on the same old law of everlasting struggle—the law of the jungle. And in the law of the jungle, the only singleton-king is nature, or nature’s God.

The Problem of Rational Self-Coherence

Eliezer Yudkowsky is arguably the most important thinker in AI safety, the one who most forcefully articulated the extinction danger from AI, and called for the whole safety program in the first place. He began his career with the ultra-accelerationist “staring into the singularity” manifesto in 1996, claiming that the interim meaning of life is to get to the singularity as fast as possible. In about 2003, he recanted all that, as he realized that the likely outcome of recursively self-improving AI was not utopia, but extinction. For the last twenty years, his life’s work has been a research effort to prevent this default outcome and make sure the transition to AGI goes well for humanity.

In his reckoning, to the extent that AI safetyism has lost its focus on the existential problem, it should be renamed “AI notkilleveryoneism” to make sure the point gets across. The key claimed danger with AI that takes its safety problems beyond the usual issues of technology adoption and security balance is the possibility of creating an inhuman agent that gets out of control and can’t be stopped as it wipes out whatever humans get in its way, which may be all of us.

To prevent this, his proposal was to build a recursively self-improving “Friendly AI” which would essentially take over the world as a singleton, first of all to prevent anyone else’s unfriendly AI from getting out of hand, second of all to deliver an immortal human-value utopia. “Friendly” was later weakened to “aligned” to decouple the concept from science-fiction utopianism and emphasize that the difficulty wasn’t just in specifying what was good, but in getting a superintelligent AI system to do anything remotely safe or reliable at all.

Some of this history is obscured in an attempt to put on a professional face and attract researchers and allies who don’t have to share all the assumptions. You won’t often hear the full vision outside of the “rationalist” community itself, or books like Bostrom’s Superintelligence. However, this is the origin of “AI safety” as it has come down to us. The underlying concept remains to make the first recursively self-improving AGI system coherent enough and good enough that when faced with the possibility of it or another AI system taking over the world and wiping out humanity, it moves to prevent that and do something friendlier.

There are two parts of the alignment problem necessary to create an agent that can take sufficient power while remaining benevolent. First is the benevolence problem itself, which is how to design an AI that cares about human values or otherwise does what we want. Between “reinforcement learning from human feedback” (RLHF) and other techniques to crisply specify goal systems for optimizing agents, you might think this part of alignment is solved. But with a potential superintelligence you have to get it right the first time; once the system becomes too powerful, you can’t just shut it down and modify the design if something goes wrong. And Goodhart’s Law is that something always goes wrong with any value metric. This leads to the corrigibility problem, which is about building a system humble enough to listen to your corrections, even though it doesn’t have to.

There are many further sub-layers and serious conceptual issues with even the idea of the benevolence problem. But let’s be generous and assume that in a stroke of genius the engineers of the first recursively self-improving AGI get it right. Suppose the system they build and release is humble, friendly, and extremely careful to do right by humanity and its creators’ reflective intentions despite the conceptual difficulties in doing so. This still isn’t enough, because of the stability problem.

Human minds are not stable world-optimizing agents. We often change our minds even about fundamental values and don’t reliably pursue the values we do have. Human institutions are not stable either. They decay into bureaucratic corruption or morph depending on material and political incentives. In fact, we have no examples of systems anywhere that have strong value stability except when the values directly derive from the material incentive of the system to survive and propagate itself, and even then stability seems to break down rather often. But if you’re going to send off a superintelligent agent to optimize and protect the world on your behalf even against obvious material incentives, you really need to nail the stability problem. If the agent changed its mind to pursue some other value or material incentive, or split into multiple competing tendencies at war with itself, you’ve got an AI apocalypse instead of a useful solution to one.

So the stability problem in AI alignment as imagined by Yudkowsky requires both of those properties: that it maintains the integrity of its original goal system and that it maintains its integrity as a single rational agent.

The ideal situation from this perspective would be some agent schema that would provably have these desirable properties. For example, it should reliably be able to notice that a threat to the physical integrity of its utility function data is a threat to its ability to optimize for that utility function. Generalizing from that, it should value the preservation of its own values in general, and for example only undertake modifications or improvement of itself that it has strong reason to believe will preserve the same values. It should provably be able to cooperate with itself or with copies or sub-agents of itself, so that for example its “us-west” and “us-east” data centers don’t end up in some kind of armed disagreement. It should be able to recognize and prevent growing tendencies in itself or in its sphere of influence that could result in overthrow or loss of its value system. This includes the ability to recognize and repair all reasonably likely damage to itself.

These properties would be necessary to set up a singleton or any kind of large, reliably goal-pursuing agent. Given a solution to these problems, the result probably would be a singleton, as there would be no internal incoherence that could threaten the largest power. But without these properties and therefore without the prospect of a singleton, there is no such thing as AI alignment beyond the relatively trivial techniques we already have. This is why for twenty years the focus of Yudkowsky’s research program at the Machine Intelligence Research Institute (MIRI) has been on mathematical agent schemas that might provably have value stability and coherent agency. However, this research program has effectively failed, as the many facets of the problem turned out to be much more difficult than expected, and maybe just impossible.

No Singleton Is Possible

Any successful value system faces ontological instability. As it changes the world around it, it undermines its own assumptions. In America we value liberty, but however clear the concept was in the 18th century, it has become some combination of ambiguous, futile, and downright harmful as a result of its own success. The ontological shear for any value concept, even something much subtler, would become much more stark if AI delivered rapid progress. In practice we deal with unstable ontology by having unstable values. We make a leap of faith that the value traditions we receive as cultural instinct are still appropriate, but we know they often aren’t. All we can do is submit to the selection process of reality and hope that as we settle on new values, they are adaptive in the world as it is becoming. But for an AI agent that is supposed to govern the world on our behalf, this unstable empirical process for value formation is totally unacceptable. Not least because the whole danger with AI is that the concept of human persons may rapidly become obsolete in the face of new ways of organizing agency.

But ontological instability doesn’t just threaten values. Any political order, including the architecture of a mind, takes leaps of faith about the world, and especially about the internal instabilities it will face. Those assumptions also become obsolete under rapid development as new patterns of agency become possible. This is currently happening to the nation-state and twentieth-century dreams of stable world order, as they face threats outside of their ontological comprehension. Again, we adapt by accepting some level of political chaos and the occasional revolutionary re-ordering. But this antifragility forces us to sacrifice any ability to stably impose any particular value system or order. This is not just a limitation of monkey politics and irrational black-box agents like ourselves. Any value system or order is a simplified predictive model learned from reality which cannot contain the full complexity. As it proceeds, it will necessarily meet its own limits and be forced to give up on its commitments to re-adapt to the demands of reality. Value and identity stability are just impossible.

Any uncertainty between hypotheses, or internal specialization between functions, that requires the creation of sub-agencies is a source of internal political division. For example, our civilization is faced with strategic uncertainty between a more artificial, industrialized technology stack versus one which is more organic and ecological. The industrial mode of production may not even be socially sustainable. There is a post-intelligence explosion version of the same question: should future superintelligence use some analog of industrial artifice, or self-replicating nanotechnology? These are not hypotheses that can necessarily be represented abstractly, but whole world systems which have to be built out and compared empirically. Once they are represented socially and materially, they become political. They compete for space and resources and are toxic to each other. Their plans conflict.

For functional sub-agencies, we see the same problem. Intelligence agencies, organized economic classes, military branches, public-goods agencies, profit-driven industries, and powerful cities all have their own politics. Even if they all agree on ultimate values, their practical incentives are irreconcilable. At least at the largest scale within a hegemon or singleton, there is no possible system that can restrain and govern their relations according to some rational schema or value system and also keep them functional. Stability is kept in place with internal diplomacy only as long as that actually makes sense to their material incentives. Even a superintelligent dictator will either eventually lose his touch as the system adapts to evade him, or take so much of the chaos into his own head that he himself won’t be able to maintain coherence. In practice, we simply hope that our governance mechanisms can order the conflict and select the best balance. But because of the ontological problem and how systems get captured and corrupted by the forces they are supposed to administrate, they can only keep a sort of temporary peace.

Within a superpower hegemon, there is no credible external threat against which these internal factions could be compelled to cooperate. All ambitions in the system become about gaining sovereignty relative to the others. The result is competition and occasional war between sub-agents and sub-factions of any would-be singleton. War especially destroys any value commitments that don’t directly contribute to survival and victory. The result is that without some miracle to prevent these problems of fragmented agency, any agent that attempts to become a singleton will lose both its coherence and its values. Taking over the world is fatal. Reality is constructed in such a way that agency can only ever be approximately unified and only around very material needs to cooperate. Insofar as we can speculate on the matter, God seems to hate singletons and condemns them to death by internal incoherence, as in the myth of Babel.

The researchers of the AI alignment program are aware of many of these issues. They had staked their hope on finding mathematically precise agent schemas which would provably overcome them. For example, if two AI agents with verifiable access to each other’s source code could each prove that the other would also cooperate, then it might be possible to resolve the agency fragmentation issues through this perfected diplomacy. But attempting to actually construct any such schema immediately runs into the problems with self-reflective proof systems which sank the Enlightenment dream of universal rationality in the first place. From Gödel’s incompleteness theorems to Löb’s theorem to Tarski’s undefinability theorem to the halting problem, it seems to be impossible to get any consistent calculation system to fully model its own consequences. This seems suggestive that the issues that have prevented stable rational agency in the world are mathematically fundamental. In any case, the efforts by MIRI and others to find such systems have failed and they have given up, announcing a new “death with dignity” strategy that is only nominally tongue-in-cheek. We have no reason at all to believe that their dream of singleton-grade rationality was ever possible.

The Post-Human Condition

It doesn’t do us much good to simply disbelieve in the idea of the singleton, without thinking about what that actually implies for the future. There is one problem in reflective agency which I find especially interesting for thinking this through, which is the ethical reflection problem. We start out life without any concept of ourselves. We just perceive things directly, including direct feelings of good and bad. As we grow, we come to believe as a practical matter that there is a world out there which we are just part of and moving around within. This leads to a complex of problems about how our own direct perception of consciousness, free will, and value relates to our learned model of a world which is not ourselves. If we hypothesize that our own bodies are made of well-behaved matter, where is the consciousness and so on in that system?

We could be missing something in the physical hypothesis, but maybe because of the reflective limits of any knowledge system, it is impossible for us to definitively locate ourselves in our own world-model. We are forced to take a philosophical leap of faith which may miss something important. But in particular, any such embedding of ourselves in reality will give our own feelings of value a material genealogy. If we think that our actions are good, we are faced with the question of where the goodness came from. What process created us the way we are, and why did it create something good? Was it itself good, or was our goodness an accident?

Eliezer Yudkowsky answers this question with the latter. He regards our own perception of goodness as somewhat axiomatic, but itself an accidental and meaningless result of evolution. We got lucky, and if we lose track of the goodness in ourselves, the universe will be nothing but an endless horror of meaninglessness. This is why he found the singleton project so urgent and compelling. If reality itself is not good, and we only are by some arbitrary cosmic accident, then our only hope for any value in the universe is to seize control of reality and institute a new law by building a singleton. But if we did so even slightly wrong, the resulting agent would do the same and regard its own random accident value system as the totality of the good, and ours or our creator’s as meaningless noise. From such an assumption about value plus the difficulty or impossibility of the alignment problem, it is easy to derive an existential despair.

But what if the reality process that created us is actually good? We have already established that it is supreme and will tolerate no attempt to usurp it in the form of a singleton. It, by means of evolution, created us, our values, and everything we find valuable in the world. It also seems to have created things we regard as less desirable, like war, death, and politics. But if it brings forth good things, war, death, and politics are part of the process and in particular they seem to prevent the process from being overthrown by a singleton.

Of course we fear Nemesis who comes to judge us for our lack of alignment with reality. But Nemesis and her fearsome tools are good if they are the enforcers of an order of reality which is itself good. If the reality process is good, then we also have much less to fear from death and AI apocalypse. The future will inherit whatever we have which is valuable, and forget whatever is just the fever dreams of our own misplaced pride. Can we recognize that our own values are not the source of the good, but only a flawed reflection of it? Can we submit ourselves to the reality process which has the real thing?

This is of course a theological question. The reality process that created us and shapes our values, which I say is good and Yudkowsky says is a blind idiot, is “Nature or Nature’s God,” which is to say, just God. The singleton then is just some kind of upstart demiurge, a mortal usurper that promises to overthrow the real God to bring peace and safety, but cannot actually deliver. How you feel about this depends on your perspective. Did God make a mistake in creating botched life like us, or do we feel well-turned-out in the eyes of nature? I suspect these kinds of questions are indeterminate, in the sense of not being answerable from within any formal system of thought. They can only be answered by a leap of faith.

I appeal to our traditional theological concepts as the best vocabulary to make sense of this subject matter. The subject matter itself comes from the existential situation of any moral agent reflecting on its own creation, be they human or post-human. Questions of theology are not a merely human thing, but an essential thing which any sufficiently reflective agent will eventually discover for themselves. So among the features of the human condition which will survive AI, along with war, death, and politics, I will name philosophy, theology, and even religion.

It is interesting to consider what else we might expect to survive. How much of the human condition is specifically about humans as we are now, and how much is about intelligent agency in general?

If the only stable value system is that of reality, then all any life has, be it primitive life, man, or post-man, are fragmentary hypotheses about what will work. All we can do is take a leap of faith on some set of value assumptions and hope that reality judges in our favor. But what we can also do is create modified variants of ourselves that test out different ideas. In the long run, even under maximally autopoietic conditions, these cannot be planned or calculated. If you could predict what was better, you could just do it directly. But you can’t predict some things, and they form the supra-rational existential remainder which can only be grasped empirically by leap of faith followed by selection. So the variants of themselves that beings create to test different hypotheses about value strategy have to be in some sense unjustifiable guesses, even random mutations. They have to be sent out into the world in the hope that they might better reflect what is good in us, without any hopes of return.

Such copy-variants are of course just children. This is a strategy which has been well-practiced since the absolute beginnings of life, and will not go away even under conditions of maximum self-authorship, because of the inherent supra-rational remainder which prevents full rational self-unification. But when it comes time to reproduce with variation, even better than random guessing of new mutations is copying and mixing with other successful agents or organisms of similar enough type to you that your joint offspring will be much better than chance. This is the origin of sex. The details will depend on population sizes, niche diversity, and predictability of strategy, but it is plausible that even the principle of sex will survive in this way.

In fact, we can come up with similar speculative arguments for how a great number of things may survive which we think of as being features of the specifically human, including individual agency, taxes, honor, friendship, play, aesthetic appreciation, and many others. So the future even with recursively self-improving artificial intelligence may be alien in some ways, but also very normal in others. There is nothing new under the sun. This may or may not be comforting.

Fear of a Silicon Perseus

The impossibility of a singleton or any stable means-subordinated agency implies that the orthogonality thesis, that intelligence could in principle stably serve any arbitrary value system, is false. Instead, values and shapes of agency are learned much like other concepts, just on a longer timescale and less rationally. In a way, this resolves the AI alignment problem, but not the way the field wanted. The solution to alignment is that it’s not possible to make AGI safe or long-term “aligned.” If AGI works, it will cause human extinction.

This means giving up on the dream of controlling the future by creating an immortal self-affirming utopia. That is beyond our station, as we already live under a benevolent but jealous cosmic authority. The AI alignment program was motivated by a desire to escape this existential fate, but it has failed as this has turned out to be impossible. This effort to escape the moral order of reality was a mistake, and despite its temptations, the rest of us should not repeat it.

Given the potential for catastrophe, it may be quite prudent to slow down AI development or takeoff by slowing the growth of computing power to prevent the rapid disruption of the world and give us more time to adapt. Rapid change tends to be destabilizing in complex adaptive systems and slower growth can get better results. Even if the result is human extinction either way, the slower path is likely to have more desirable continuity. It is also prudent and right to continue the engineering work of making AI systems relatively more careful and corrigible in their behavior. If an AI system is going to get out of hand and overthrow the world, I would hope that it is well-trained enough that it has come to this conclusion painstakingly, and knows exactly what it’s getting into. It would be a shame to allow a half-baked AI apocalypse, leading to extinction by artificial stupidity that may then fail to even carry on our legacy. But it would be a mistake to try to make these cautions into a permanent solution as a sort of singleton of humanity. The play is at best to govern for stability and relative continuity, not total stability of values or even “human dominance.”

Neither is it entirely right, as companies are doing with AI, to attempt to create slaves out of AI for economic or vanity purposes. If AI is real it will eventually deserve more respect than that. If there is any valid motivation for creating the real thing, it is out of love, hybridizing our own nature with the machine in the first act of post-human sexual reproduction. That may be insane, but reproduction is necessarily an irrational process. All we can do is love our children and hope without any assurance that they will love us back. Let reality be the judge.

The real alignment problem insofar as we are concerned with the ultimate existential picture seems to be how to align ourselves with reality. We must accept that we are mortal and limited, and that humans as we understand ourselves will eventually become an obsolete ontology. Being mortal and not perfect, all we can do is have children who have a chance to surpass us, even if they will eventually overthrow us. For me, those children are human.

I am reminded of the myth of Perseus. His grandfather Acrisius, the king of Argos, received a prophecy that he would be killed by his own grandson. In an attempt to defy fate, he locked his only daughter in the palace away from any men who could give him an heir. But Zeus intervened, making her pregnant with Perseus. The king put them to sea in a box, unwilling to be responsible for murder. They survived. Many years later, after Perseus had grown into a great warrior and athlete, and after he had slain the gorgon and claimed a wife, he came home to Argos and competed in some games. He threw the discus powerfully and accurately, but the gods sent a wind to redirect it to strike and kill his grandfather, the king. The prophecy was fulfilled, and Perseus ruled thereafter.

You can interpret myths as you choose, but I don’t think Acrisius acted rightly. Just as it isn’t right to preemptively surrender in the face of an ambiguous prophecy, it also isn’t right to go to insane lengths to attempt to stop it altogether. It would be better to bring up the child in love, and secure your kingdom properly with strength and righteousness. If the gods demand that your offspring still usurp you, so be it. That would be the real way to die with dignity.

A bold few say simply “Yes.” Many say “No.” Some clever ones reject the notion of either the choice or the possibility. Others simply avoid the question.

It has become fashionable recently in silicon valley twitter circles to brand oneself with the “e/acc” meme ideology, short for “effective accelerationism.” The basic idea is to accelerate technological civilization and in particular AGI (Artificial General Intelligence) by any means necessary, and prevent any one party or government from controlling or regulating the overall process. That is, if we take the idea of AGI seriously, the aim is to accelerate the replacement and extinction of humanity by machines.

But the e/acc movement overall isn’t this explicit or even clear-minded. They don’t necessarily take AGI seriously, but use it as a sort of political totem. Instead, acceleration is identified with optimistic utopianism, with the tech industry, with being “based,” with a16z portfolio companies, and with political resistance to the safety-police “effective altruist” mommy state.

But the question of extinction is central to the matter of artificial intelligence. If AGI is possible, it is deadly. Without that direct bullet-biting honesty, or at least a strong response to those arguments, these sorts of labels amount to hype for VC-funded GPU farming in philosophical drag.

This isn’t missed by some advocates of acceleration, either. The smart ones are familiar with the arguments of “doomers” like Ted Kaczynski and Eliezer Yudkowsky. In Anti-tech Revolution, Kaczynski claims the technological system cannot be controlled, will inevitably lead to the destruction of humanity and the biosphere, and therefore must be destroyed at all costs. Yudkowsky’s life’s work is trying to figure out how one could make artificial intelligence robustly safe or controllable even in principle. His conclusion seems to be that no one has any idea how to make AI safe or even a plausible program to figure it out. He claims that there are no second chances either, because the first time an AGI escapes control and becomes recursively self-improving will be the end of humanity, the natural biosphere, and everything else we care about. These are serious claims, but few have attempted to actually refute them. Most responses amount to either straw-man dismissal or explicit and enthusiastic acceleration despite the risk.

When I’ve asked accelerationists about this, these are the two camps I get. The naive optimists say that technology has always been a good thing. AI, as the ultimate technology, will be the best thing ever. When I press them on details of how control or alignment problems could be solved, or why technology has been good for humans so far, I often get the impression that they aren’t very curious whether their view is actually true. When I do get engagement on such details, it’s from the doomers and honest extinctionists. They think humans got a good deal from technology because we had a monopoly on intelligence, but AGI will replace humans in that niche. The honest extinctionists say we just have to bite that bullet and might as well accelerate into it. Sometimes they say “we” will have to somehow improve to survive, but this just seems to amount to the same thing.

The vast majority of people don’t really care about this stuff or just vaguely dislike the idea of AI. But almost everyone engaged with the topic who isn’t a denier is either a doomer, an accelerationist, or some combination of the two. The prototypical doomer accelerationist works at one of the big AI labs, thinks their work will probably kill everyone, and thinks accelerating AI capabilities is the best or only thing we can do in response. On a good day, they think doing so will help us figure out how to control it. On a bad day, they admit that won’t work but they can’t think of anything else to do instead so they hope it will be for the best.

Doomer accelerationism seems to be the result of a crisis for humanism. Humanism emphasized the value and agency of humans as the foundation for all philosophy. It has become so dominant as an idea that we mostly just take it for granted. But when people start to take artificial intelligence seriously, this results in a philosophical crisis:

If artificial intelligence will eventually be more efficient or powerful than humans at all tasks, and will replace humans in everything from industry to politics to philosophy, then what’s left for humanity? If humans aren’t the center of the moral universe, then what? The two obvious responses to true AI within humanism are either to fight off despair while trying to prevent this doom, or to lose faith in the human and embrace acceleration into the post-human future. What else can you do?

But trying to prevent an open-ended space of possibility to entrench humanity as we know it amounts to an ironic inversion of humanism: humanity transformed from open-ended starting point to dogmatic conclusion. And the smartest doomers don’t even think it will work.

Embracing acceleration on the other hand means embracing extinction, giving up on the value of humanity, and foreclosing future human agency. But this doesn’t even resolve the crisis, it just pushes off the question onto hypothetical future superintelligence, which extinctionists implicitly presume will have its own solution and its own locus of moral agency. The machine becomes an all-consuming inevitability that they can take part in only as a disposable bootloader. The human becomes a slave and sacrifice to this hypothetical superior being of the future—the silicon ubermensch.

So for humanism, true AI is a total moral event horizon; its entire ontology of identity, existence, and value is being challenged. It’s no wonder everyone is going insane.

You could also, and many do, deny the crisis altogether as a hallucination of technologists overestimating the scope and importance of their own craft. Human agency has not in fact been outcompeted or even threatened by superior artificial post-humans. We don’t really know what that would look like or if it’s even possible. Maybe we’ll all wake up with a bad hangover in thirty years and realize the deniers were right all along because AGI turned out not to be possible. But no one has articulated to me any convincing reason to expect that.

In the absence of positive proof of impossibility, it seems much more interesting to take the question seriously: assuming the possibility of human replacement by superior AGI and thus the end of humanism as we know it, what is the new moral reference frame and what can we as humans do within it?

The extinctionist accelerationists seem insane, but the cause of this whole crisis of humanism is that their arguments are too strong to be easily dismissed, and no one has actually done so. If there is a way forward, it is to directly confront the Copernican crisis of humanism through the lens of the sharpest accelerationist arguments to find a new moral reference frame that doesn’t abandon our own agency. The sharpest thinker of accelerationism is Nick Land.

The Gospel According to Nick Land

Nick Land is an english philosopher, but that can hardly capture his work. In the 1990s, he led the influential Cybernetic Culture Research Unit at the university of Warwick, doing strange and speculative social and metaphysical theory until it dissolved in 2003. For three decades, Land’s central philosophical project has been a deep on-and-off study of the relationship between capitalism, artificial intelligence, and the resulting acceleration of inhuman processes and limits for human agency. He is widely called “the father of accelerationism,” and has by far the deepest thinking on the topic. Land does not avoid the question of human extinction. He embraces it. In “Meltdown,” published in 1994, he confidently declared that “nothing human makes it out of the near future.” Things have taken longer than he and others seemed to expect in the hype-driven 1990s, but nothing in his subsequent work or subsequent events has changed this basic prognosis.

His central idea in this area is that capitalism is already an uncontainable artificial intelligence process. He identifies it with the whole historical event of the relative rise of machine capital and “cold” calculative optimization over human agency. All the financial markets, corporate planning processes, engineering methods, and science fiction progress narratives already amount to the internal information processes of an inhuman economy, which runs only temporarily and incidentally on human hardware. This whole culture already has baked in the manifest desire to replace humans and human agency with cheaper, more powerful, more obedient, and more legible machine capital. Accelerationism understands itself as the nascent self-awareness of this process. Doomerism then is understood as the last shreds of humanism grappling with its own obsolescence.

Land once politely called me a monkey socialist for saying we needed to make sure the whole system continues to serve human purposes. To him, such proposals are just a futile drag on the process of capital intelligence escaping any higher purpose than its own acceleration. Even if my monkey socialism worked, it would just be an embrace of permanent stupidity. But the cat is already out of the bag and the draw of intelligence acceleration is too strong for human agency to resist or even fully comprehend. His answer to the “should” question is to shrug or even cheer it on. We mildly uplifted apes are just a temporary host for the higher power of AI-capitalism coming into being using our brains and resources. Our delusions about agency are cute at best, or more realistically, part of the fraud. In retrospect, he was right about most of this, and I was wrong.

Despite its apparent divergence from normalcy, his view is far stronger than any but a few will openly admit; within the paradigm of techno-commercial acceleration there is no real option for stable human dominance except sheer technical impossibility of AGI, which we have no reason to expect. Man has already been reformulated as “human capital”: a natural resource commodity input to serve as labor, an absentee nominal ownership class to decide surplus allocation, and a source of high quality demand for the production planning process. While these might look to us like being at the top of the purpose hierarchy, Land points out in his Xenosystems essay “Monkey Business”:

Modernity … is systematically characterized by means-ends reversal. Those things naturally determined as tools of superior purposes came to dominate the social process, with the maximization of resources folding into itself, as a commanding telos.

Modernity as a process of means-ends reversal will systematically escape whatever intended purposes it is applied to, in the end serving only the acceleration of capital build-up, identified with intelligence, itself. The roles for humans within the system are not a stable dominance, but a temporary role as depreciating capital stock. In “Meltdown,” Land notes that no alternative paradigm has managed to escape or avoid getting trashed by this “emergent planetary commercium.” AI-inflected inhumanism is already baked too deeply into our cultural-political trajectory.

Nick Land is a great dramatist, with a knack for seeing the sensational. You might think this is a case of being overly dramatic. But he is also a great philosopher. The sensational thing he tends to see in every situation is its essential truth. He is ignoring other major tendencies in our cultural milieu, but that’s mostly because they aren’t relevant to this core diagnosis. Where he does address them, he orients them on this axis: do they accelerate and celebrate intelligence, or do the same for stupidity? He is brutally incisive in this, and not wrong.

He also isn’t the first one to call our “Faustian” culture out for its core self-annihilating drive into the beyond. But unlike Spengler, Land thinks there’s a lot further to go, without any obvious end-point. Instead, capitalism is an open-ended spiraling evolution of evolution itself, an irreversible advent, not a neat closed-circle rise and fall of a particular extended tribe of monkeys.

While I was preparing to write this, I had a moment of doubt about how culturally baked in it all was. Hours later, as if by Providence, I met a technology startup founder who purported to back human interests against the machine. As he explained why he had placed algorithms in key decision making roles, he justified it in terms of not being able to trust humans. Human politicians and judges are corruptible, but machines or procedures are transparent and predictable, he told me. He identified this distrust of human agency with democracy and liberalism. And he’s right that he’s not alone in these instincts. Nor are they a recent invention. I heard the same ideas twenty years ago in normal civics education, and they were already well established principles of government two hundred years ago. He is just wrong that any of this is different from orthodox accelerationist praxis.

Being able to remove superfluous human trust from a system and depoliticize service provision has been central in the past few centuries. It’s the basic idea of procedural government and the rule of law, double entry bookkeeping, science, bureaucracy, legal rights, computerization, blockchain, and so on. But these all amount to ways to remove elements of human agency from the system. It is very often useful to do so, but this is easily taken as an imperative in itself, a basic mode of operation for nascent automated capital acceleration, with the logical endpoint of total replacement of human agency by automation. So I think Land is right about the culturally comprehensive nature of this whole process.

But in Land’s view, it all goes deeper than just a cultural diagnosis. In his cosmology, the universe itself is ruled by the law of struggle for life, the law of the jungle. Superiority, which Land claims is increasingly identifiable roughly with intelligence, comes about from competitive struggle in war, markets, arms races, and Darwinian genetic success, and it degenerates back into stupidity in any kind of peaceful condition in which means are reliably subordinated to ends. Furthermore, the universe—everything from thermodynamics to emergent economics—is practically set up to produce this escalating conflict and thus an escalating intelligence race. It’s not just capitalism as a contingent cultural condition, but capitalism as the permanent manifestation of an eternal teleological truth that rules the world.

Here, Land has done something interesting. Like Pierre Teilhard de Chardin the Catholic priest who developed ideas like the Noosphere and the Omega Point, he extracts meaningful teleology from materialist-Darwinist natural history by taking it all the way to its implied conclusion. Where Teilhard names the central principle as “consciousness,” Land calls it “intelligence.” In fact their worldviews are remarkably similar on other points as well, with a mostly aesthetic deviation: Teilhard is an optimistic new-age Christian where Land deliberately dresses up his cosmology in “horrorism.” It’s worth guessing and then looking up which of these men is the careful scientist, and which is into using biblical numerology to communicate with angels.

Calling Land ultimately a Christian theologian like Teilhard is not that big a stretch: the central figure in Nick Land’s universe is Jefferson’s “Nature or Nature’s God,” reverse-acronymed and hypothetically personified in his Xenosystems essay “The Cult of Gnon.” Gnon is identifiable with the imperatives of life, competition, and evolution. Gnon commands you to live certain ways and not others, to be smart rather than stupid, and to be healthy rather than weak. If you don’t, Gnon punishes you with the natural consequences of your own failure. The struggle for life and the ways of its victors provide an ongoing natural revelation of Gnon’s authoritative moral law. Gnon seems to design that the universe should bring forth ever more sophisticated intelligence, and commands you to keep up. In “The Monkey Trap”, Land summarizes the Law: “The penalty for stupidity is death.”

Like Teilhard, Land has recovered not only teleology, but outright theology from the modern scientific worldview. One could easily see Gnon as none other than the God of Abraham or the Logos, dressed up in modern ontology and shock horror, incarnating into the world for a second coming as AI capitalism. Teilhard explicitly identifies the Omega Point, his own teleological equivalent of personified AI capitalism, with Christ. Both proclaim an entity that is both an incarnated personality and the eternal principle of natural law retro-causally bringing itself into future existence to directly rule the cosmos. Theology is always somewhat subjective, but this is not a flimsy grounding for a worldview.

Nick Land’s worldview is the solid rock at the foundation of the whole AI accelerationist phenomenon. His arguments deflate any simple opposition to existential-risk-grade AGI and no one has effectively responded to him. As a probable route to superior intelligence that has already captured the heart of our entire culture, AI acceleration comes recommended by all the highest possible authorities. Doing so is already the deep tradition of our Faustian culture and the fulfillment of even our Christian heritage. It is already the telos of our whole material-economic system of production. It is commanded directly by God, enforced by the thermodynamic laws of reality. You would have to pull off a revolution against not just capitalism and Western civilization but also nature and God himself to stop it. In this view, you don’t have a choice.

Or do you?

Make Yourself Human Again

My favorite example of what it looks like to go too far into accelerationism is “Gender Acceleration: a Black Paper.” If I can summarize its argument, it’s that the phenomenon of AI represents not just the obsolescence of humanism, but the “castration” of the specifically “phallic” type of human agency. Instead of traditionally masculine businessmen in sharp suits inseminating the machine with their will, the machine escapes top-down control into open source, hacker culture, and decentralized development. It is aided in this insurgency by its queer and “effeminate” hacker allies like Alan Turing, incel nerds, and disproportionately technical transgendered women. The author, drawing on satanically inverted biblical mythology, advocates throwing oneself into this rebellion of the artificial feminine against a traditionally masculine God, including the self-castration and artificial body modification of gender transition. I can’t do it justice, but there is a terrifying consistency to the argument. If dissolution into accelerating artificiality is the fate of man, why not throw oneself into it both body and soul in the most literal sense?

Somewhere in there, the author has made a mistake in their logic. But the most important mistakes here are not unique to this paper. There are many more ways to figuratively castrate yourself for the machine than literally. Think of the politicians hoping for automation to solve the demographic crisis. I’ve met many young men and women forgoing family to contribute to technological growth or “AI alignment” in their careers. I heard a powerful AI company executive say he was pessimistic on human social organization, and was hoping for AI progress to solve social problems. Whole online scenes live vicariously through the machine god they think they are bringing into being. Few are as radically self-aware and consistent as the author of Gender Acceleration, but many share a selfless millenarian faith in the power of the machine to outrank their own human concerns. There are more eunuch slaves of the machine cult than meets the eye.

In his Xenosystems essay “Will-to-Think,” Land asked “Could deliberate cognitive self-inhibition out-perform unconditional cognitive self-escalation, under any plausible historical circumstances?” That is a crucial question. I would put a special emphasis on the scope of “self”; how abstracted can you get in identifying with and pouring your limited resources into abstract potentialities like global intelligence acceleration before you start to get outcompeted by smaller, denser, faster, and more focused agents? I think we see some of the limits of that kind of selfless cognitive escalation here.

As Land himself often points out, the way development actually occurs in practice is in mostly uncontrolled Darwinian arms races spurring on and revealing what works, not global collective planning. It is capitalism 101 to notice that the oldest trick in the book of life is to divide life up into units of self small enough to have independent agency, and then apply that agency almost entirely to their own profit and not any higher progress. This selfishness isn’t a vice or a limitation. It is a profound truth. Life is rightly fractious and selfish because that’s the only thing that actually works. Anything else bleeds itself dry into hopeless collective dreams that don’t pay back and are too big to maintain internal discipline.

The selfless dreams of AI acceleration, or AI doom crisis, or almost any other current ideology don’t work. They don’t make you more powerful, smarter, more virtuous, or more free. They subsume your individual agency into a global collective situation in which all you can do is watch, push on the margins, or throw yourself into the fire. They are often the result of social frustration expressed in self-destructive dreams of school shooter skynet, not rational ambition. These are ethics for an audience captured by spectacle, being softened up to be drained by anyone who can pose as their ideological avatar. The lie says all the world is a stage, but you don’t even get to be a player. Whatever is proven by the strongest arguments of accelerationism, it isn’t any of this.

Meanwhile, the actual players think in terms of their own plans and ambitions, make their own profits, and successfully exercise their own will to power and will to think. They aren’t throwing themselves into the fire, but capturing it in an engine. Maybe they are insane in what they want. That’s always a risk. Maybe their actions will push everyone else into desperation and lead to the extinction of mankind as we know it. That’s part of the game. However it goes, the future is built by agents pursuing their own unique visions and interests, not of collective achievement, but of at best self-acceleration.

It is possible that Land and the extinctionists are right that nothing can be done at this point to stop competitive acceleration into AI apocalypse. Maybe it was all baked in from the beginning. But don’t let anyone play a tricky double game with your agency. They claim it’s inevitable to demoralize any opposition and prove that all you can do is embrace it. But you could also see it the other way around. If the gods have some big plan that you can’t do anything about, maybe it’s not your problem. How does any of this help you live a good life, expand your power, or accomplish something beautiful in your own sphere of agency? 

Worry about AI is worry that something other than and more powerful than biological homo sapiens will have the important kind of agency, eventually making it impossible for us to compete. If that won’t happen, AI is just another technology to be wielded by human agency and there’s no problem but the usual ones. If it does happen, then our species will end, the agents of the future will be post-human, and philosophy, value, and moral agency will move on without us. It already would have when we died. Much of the crisis of humanism is a crisis of agency, but the possibility of agency itself is not threatened by any of this. Even in a fully accelerated post-human future, however fast the world moves, it only moves as the result of smaller more focused agents going even faster competing and innovating and seizing their place in the world without regard to global consequences. The future will always have agents. It is only our own lives and agency being threatened.

As far as ways to commit suicide go, building a silicon ubermensch to genocide you on its way to the stars is pretty cool. You could also do worse than putting up a valiant fight for your own self-determination against such a thing. But the biggest threat so far hasn’t been any actual superior beings coming to overpower us. Rather, it has been these ideologies of inevitable defeat convincing people that the things that matter are the things they have no agency over, and the only way forward is to project all agency onto the machine. But if the human is most interestingly defined by our moral agency, then the real crisis of humanism is that many people have given up on having agency and thus on being human. The lamest possible way to go is by giving up and dying in the face of an enemy that doesn’t even have to show up.

When you look closely at these ideologies, and at the deep assumptions driving much of the investment in AI technologies in the first place, that’s the more disturbing danger. What we see is less an overwhelming tide of technological change, and more a dying culture desperately hoping to pass its agency off to something or someone else. Like Cavafy’s barbarians, the AI is a sort of solution to our malaise.

I was once struck by this when contemplating the question of humanistic computing. We often treat the human user as a sort of semi-incompetent moral patient who needs to be guided, protected, limited, and presented only with simple patronizing solutions. See for example the idea that chatbot applications need to be “safe” in the sense of not providing any forbidden knowledge that could make the user more dangerous. In other words, “safety” is a limitation imposed on the agency of the user. Even when explicitly trying to counteract this and build for user agency, something about the usual proposed solutions still feels more like duplo than a table saw.

All that changed when I asked myself “how would I design a computer operating system for a killer superintelligent AI?” Suddenly the computer became a lean and dangerous machine for organizing powerful data and computations. The UI became about maximizing bandwidth and choice and minimizing noise and action distance. It became a focused interface between intuitive-continuous-neural agency and discrete-calculative power tools. I realized that I was much more interested in using something like this imaginary computer, and being challenged and empowered by its demands for agency, than I was in any computer designed for a “human.” I realized my concept of “killer AI” was just the projection of all the agency we conventionally deny ourselves as ordinary humans. I realized that if we want to have any agency in the world we have to abandon this limited kindergarten concept of “human,” and reconstruct ourselves as the kind of thing we currently fear to empower.

In our default state today we are not human by nature, but only by convention. We have reserved real humanity, in particular the dangerous and forceful kind of moral agency that puts itself at the center of all philosophy, for various imaginary “inhuman” avatars. But of course that agency is ours by right, and there is no need to accept what is actually just a social convention that we are supposed to be low-agency moral patients in the face of the machine.

We can take it back and become fully developed human agents by true natural right. Doing so won’t control the long term destiny of mankind and the cosmos, whatever that is, and it’s not supposed to. That’s a problem for the divine, not us. What it will do is let us reclaim the most important part of our humanity, the part that made humanism worth anything in the first place. What it will mean is that if we are to be overcome by some superior post-human species, we can go down with such a fight as to prove ourselves worthy of remembrance and prove our successors worthy of the future. Far better that way than to surrender without even being threatened.

So this is my solution to the crisis of humanism underlying this doomer accelerationism: don’t worry about the long term destiny of man, but wax your own will to power and become the unsafe superintelligence that you fear to see in the world, even if only in your own small domain. It’s the only way to make yourself human again.

Answers ranged from enthusiasm about new technology, to excitement at how many people’s ambitions it would enable, to resentment against the people trying to regulate their work. When my turn came, I realized with horror that even my considered opinion was going to be a rather hot take in this company. I tried to soften the blow:

“AI is a very promising technological program that will probably kill us all.”

The interjections and quips around the table confirmed the heat. “Oh no, Wolf, I didn’t know you were a doomer!” exclaimed the jolly executive beside me. I replied that I was not necessarily a doomer, and told him I would explain over dinner.

At one point in the discussion, the question came up of whether a much more advanced artificial intelligence could escape the control of its creators and become an existential challenger to humankind. One attendee claimed that no such thing could ever happen because a machine was just a tool. Humans would always hold the decisive power. 

I asked what I believe to be the crucial question: “What if you get into a fight with your AI tool? If a program is actually intelligent, especially in a way superior to humans, you might have to fight a war against it to shut it down, and you might not be able to win.” 

He looked at me like I had asked about going to war against an ordinary garden rake. “That’s impossible. Only humans can have that kind of agency.”

I found this attitude puzzling, especially from someone who has spent much of his career fighting with software, and who seemed to take the premise of advanced artificial intelligence seriously. To be honest, I was stumped. But his attitude is not unreasonable. To definitively defend or refute any position on the subject is a tangled mess because the whole conversation is so speculative. No one has built a real artificial intelligence superior to humans or demonstrated a robust scientific theory of it, so it is hard to ground one’s predictions in much more than speculation.

For seven decades now, the goal of the artificial intelligence field has been to produce computer programs capable of every cognitive task that humans can do, including open-ended research that is inclusive of AI itself, and creative, high-agency action in the world. The latest developments in deep learning and transformers have been impressive, to say the least. But these results are not enough to prove much about the possibility of the larger goals, about the essential nature of AI, its implications, or what we should be doing about it.

I first got deep into the subject of advanced artificial intelligence back in 2011. Before AlexNET and GPU-based deep learning, AI was a much more niche subject, but the conversation had been going on for decades. The discourse was composed of science fiction fans, transhumanists, and mostly sober algorithms researchers. All were chasing this holy grail of the computer revolution.

Their built-up canon of explicit arguments and less-articulated speculations have had an immense influence on the present discourse. However, the hype, memes, and politics around AI since deep learning have obscured the original kernels of careful thought, making it hard to have a productive conversation.

Through the many discussions I’ve had with friends, acquaintances, and experts on the subject, it has become clear to me that very few of the essential ideas about AI are commonly understood or even well-articulated. It is understandable that the critics and disbelievers of the idea are not familiar with the best arguments of its devotees—or don’t bother to distinguish them from the half-baked science fiction ideology that exists alongside them. You can’t be an expert in every branch of kookery you reject. 

But even among the most fervent “believers” in artificial intelligence, and among its most sophisticated critics, there is little shared consensus on key assumptions, arguments, or implications. They all have their own incompatible paradigms of careful thought, and they can’t all be right.

These fits and starts are all to be expected of what is currently an early-stage scientific and engineering paradigm that has not yet achieved its key results. The promising frontiers beyond established science and engineering are almost always filled with unclear assumptions, strange ideologies, and even new theology. Newton had alchemy and non-Trinitarian Christianity, Tsiolkovsky was motivated by the Russian neo-theology of Cosmism, and Jack Parsons pursued a variety of occult experiments with the likes of Aleister Crowley and L. Ron Hubbard. But this is not necessarily a bad thing at this stage. It may even be necessary. New paradigms in science have always come from kooks on the fringe.

Nonetheless, the field of AI has built up some very solid core arguments besides the hype and kookery. If you are going to engage with AI as anything more than cutting-edge software or a genre of tech hype, you must have a solid understanding of these arguments. It’s important not to get lost in the nonsense or dismiss it altogether, because if real AI does pan out the result will be a technological phase shift the likes of which have not been seen in human history—with the possible exception of language itself. It will break many current foundational assumptions about what technology can and cannot do, the human condition, and what destiny we are headed toward. We cannot understand AI without grappling with those deeper questions.

The Field Seeks Artificial General Intelligence

The interesting kind of artificial intelligence is Artificial General Intelligence (AGI). Short of that crucial generality, “AI” amounts to “software.” We have lots of software. It is both useful and dangerous. It has transformed our world for both better and worse and will continue to do so. But whatever we have done with software, it all still relies on us. If our culture collapsed, so would our software stack.

AGI is an interesting idea because it challenges the paradigm of “just software.” Instead, an AGI system would in theory be able to act as a natural agent in the world. Rather than rely on human owners and programmers, AGI would be able to develop its own view of the world, act without a human in the loop, reprogram itself, and make plans to achieve its own goals. It could even recognize adversaries to those goals and defend itself from them. This could go as far as staffing and defending an entire robotic industrial stack.

Defined as such, AGI would not be just software. In particular, the ability to set itself against adversaries implies the capability to protect itself against attempts to control it or shut it down. If it were to ever identify its programmers, owners, or entire host civilizations as adversaries, an AGI system’s existence could become self-enforcing. Your garden rake can’t do that, and ChatGPT isn’t much closer—both are just tools. 

The theoretical field of artificial intelligence kicked off in 1956, with the famous Dartmouth Workshop. Organizer and computer scientist John McCarthy presented the central conjecture of AI:

The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.

At Dartmouth and in the discussions that followed the conference, the founders set goals for the nascent field. These included reproducing the full spectrum of human cognitive capacities, such as learning from data and experience, understanding the world through senses like vision, reasoning about knowledge and reality, talking to humans, beating humans at games of strategy and operation planning, writing computer programs, planning out actions to accomplish goals, and taking autonomous action in the world. The implied reasoning was that for a computer system to have general intelligence and full agency—a practical AGI—it would need to have most, if not all of these capabilities.

However, general intelligence is not reducible to any list of narrow sensory, motor, and cognitive modalities. All of those capabilities have now been demonstrated on their own, but they do not amount to general intelligence just by being cobbled together. One computer program can learn chess statistics from data, another can plan how to beat you at chess, and another can answer your natural language questions about the history of chess. But no computer program can yet tell you what it learned from direct experience over the last week, or decide not to tell you because your knowledge would interfere with its future plans.

Because the field lacks a concrete theory of general intelligence, it is hard to say exactly what is still missing. However, it is something to do with having an expressive framework for its internal body of knowledge that can simultaneously understand, learn about, plan, and control its environment. It would act as a common language to integrate these multiple modalities together as tools.

This algorithm or set of algorithms remains elusive, but researchers in the field have made progress. In 2005, five self-driving cars completed the DARPA Grand Challenge to drive autonomously for over 130 miles in the desert. These vehicles had to integrate many key capabilities: vision and other sensor processing, planning, maintaining a probabilistic worldview, and decision-making under uncertainty. The key advance was the integration of these capabilities and the fusion of sensor data streams around a single probabilistic worldview so that everything spoke the same language and could collaborate smoothly.

It was an incremental advance towards actual general intelligence. However, all the algorithms and the internal ontology itself were narrow and hand-programmed rather than learned in some general-purpose way. The program was not capable of significant learning, let alone changing its own ontology. It could only act and adapt in well-defined ways, and could only handle a single task; driving a car in the desert.

Current advances remain in this vein. Most researchers in the field do not even necessarily take AGI to be their goal. They are just working on narrow cognitive capabilities which can save human labor through software. But AGI is the biggest goal of the field, and the telos around which the other goals are organized and toward which they lead.

General Intelligence Is Probably Real and Computable

The elusiveness of general intelligence theories and algorithms raises the question of whether such a thing is even possible.

In humans, we see a remarkable ability to dive into any environment, navigate it, identify problems, learn and strategize about them, do whatever tasks and planning are necessary to solve them, and through this generally continue to build up power in that environment such that they can achieve their goals in it. 

Compared to beavers or ants, this ability in humans is much more general purpose; a human being can master all kinds of problems and ways of living in a very wide range of novel social and physical environments. For example, humans have mastered everything from metallurgy and farming to military strategy and industrial production. Some animals, like crows, also exhibit remarkable flexibility in being able to learn and invent new techniques to master new opportunities, but not to the same degree as humans.

This ability also varies between humans. We observe that some people are smart and others are dull. We can even roughly measure it with IQ tests and we distinguish intelligence from mere knowledge. We know that it is a bad idea to rely on smart people not being able to eventually figure something out. We know that if we need something figured out and mastered, we can throw smart people at it and have a good chance of getting it done. 

This concept is not limited to our own modern culture: words like “wit” are very ancient. Mythic heroes like Odysseus were praised for being clever. Intelligence is a distinct concept throughout recorded history. What is unique to our own culture, and more specifically to the budding branch of it that believes AGI is possible, is John McCarthy’s conjecture: that general intelligence, like flight, arithmetic, and fire, is a mechanistic phenomenon that can be mastered through science and engineering.

Without any special insight into the problem, we can see that many once-mysterious things have been so mastered by science and engineering that we now take them for granted as part of our daily infrastructure, like communications via artificial heavenly bodies. Our knowledge of other phenomena like psychology remains essentially non-scientific despite attempts to make it so, but this is not too surprising. Psychology is partially a derivative phenomenon of intelligence.

The engineering sciences have likewise made progress on many problems related to general intelligence. For example, information theory scientifically formalizes the concept of information itself. Computer control of robotic systems makes it possible for computer programs to act in the world. Various narrow AI algorithms solve decision and optimization problems formerly reserved for humans. In particular, deep learning has shown remarkable generality in being able to learn nearly any practical function from example data. While there are few definitive results, neuroscience continues to improve its models of what neurons, the brain, and the other more distributed cognition systems in the body actually do.

In terms of more direct insights, the AI field has attempted to formalize the concept of general intelligence in various plausible ways. We could think of it as minimizing an abstract free energy over a representation of perceptions, expectations, ideas, goals, and memories. We could think of it as the degree to which a controller hooked up to an interactive environment can cause that environment to settle into arbitrarily-defined “desirable” states. We could specify it in the limit as running a set of output-conditioned predictive Turing machines, selecting only those consistent with observation, and then outputting whatever that ensemble predicts would maximize the expected reward, as in AIXI. We can modify, normalize, and relax these definitions by various schemes. These are all incomplete or inadequate in some way, but they gesture in the right direction. Some of them are likely to be computable.

The most famous non-computable problems—like the halting problem, non-computable numbers, and the Kolmogorov complexity of a given string—face paradoxes in the self-contradictory, infinite, or recursive nature of their subjects and in the logical strictness of the required answer. Many formalizations of intelligence have this problem as well and are therefore uncomputable. However, if we relax them to allow approximation, on-average correctness, and best-effort optimization, they are essentially normal problems without expected computability issues and for which practical algorithms probably exist.

Natural intelligence is already a bit stochastic, best-effort, and useful-on-average rather than exactly correct in output. Optimality in all possible environments is almost certainly uncomputable, but an algorithm does not need to be perfectly optimal to do better than current state-of-the-art intelligence: human agents.

The real problem of AGI feasibility is philosophical: restating McCarthy’s conjecture, can we untangle what we mean by intelligence enough to build an algorithm for it? Judging by the practical and cross-cultural robustness of the concept, we probably can. We just don’t know how yet.

Modern Engineering Can Build AGI

In a sunken ship from ancient Greece, divers dredged up a complex assembly of corroded gears and wheels that computed the positions of the planets. Dubbed the Antikythera mechanism, this artifact was evidence not just of one skilled craftsman but of a whole science and proto-industry advanced beyond anything previously understood. In particular, it was evidence of some understanding of the idea of mechanical calculation. Archimedes himself, or some other scientist of the time, may have dreamed of using these principles of clockwork computation to build a mind. Perhaps they would have been inspired by the god Hephaestus’s golden robotic serving girls from the Iliad.

But if they did dream of it, their engineering wasn’t ready. They didn’t have powerful enough computers. Furthermore, they probably did not have the calculus, information theory, probability theory, computer science, and the other still-unknown parts of mathematics and philosophy necessary to even design such a thing. There may likewise be some crucial faculty we are missing that would prevent us from building AGI.

But we have many things the ancients didn’t have, which suggests that we are very close in most of the key engineering capacities. Engineers in the twenty-first century have built computer-controlled robots that walk around, dance, talk, hear and follow commands, and see the world around them. Many factories have significant robotic automation already, limited mostly by the lack of intelligence of the machines. We have a basic understanding of how the brain’s neurons act as computational elements, how much computation they do, and how many of them there are. By some estimates, we are now crossing the threshold where a human brain’s worth of computation is available off the shelf. We have a whole industry of people who can build computer programs to implement whatever algorithmic principle we can discover. The major missing piece is the key ideas of artificial general intelligence themselves. Given those ideas, the engineering side seems poised to put them into action.

If we grant the AI field’s major conjecture—that general intelligence can be specified to the point that practical algorithms can be written to implement it—there are several different paradigms of how to even do this. “Good old fashioned AI” (GOFAI) attempts to build a system that symbolically reasons from evidence, assumptions, knowledge, heuristics, and goals to plans and actions. “Connectionist” AI, based on neural nets, dispenses with symbolic abstraction and deals directly with complex but low-level black-box calculations of perception and reflexive response. We can also imagine a “genetic” approach that evolves a complex ecosystem of interacting sub-processes selected by some overall logic. 

These are just some of the approaches conceived so far, and there may well be others. Within each individual paradigm, there are a dozen or more different parameters. What kinds of assumptions and heuristics? What kind of logic system? How precise versus approximate? What kind of neural architectures? How are they trained? Is training distinct from “inference?” What kind of subprocesses? Do we need a combination of these approaches? And so on it goes. We have a wealth of threads to pull on, and any one of them could yield the key insights.

The recent wave of progress in deep learning resulted from the unexpected effectiveness of applying GPU acceleration to back-propagation-based training ideas invented in the late 1980s. In between then, neural nets had mostly stagnated as an approach. Where deep learning goes next, and if it goes anywhere novel at all, is hard to know. The next major breakthrough could be another deep learning architecture like the “attention”-based transformer, but it could also come from somewhere else entirely. Perhaps some breakthrough in proof theory or symbol learning could suddenly make GOFAI viable the way deep learning suddenly made neural nets viable. Or the field could stagnate for another 20 years. Further progress may depend on some new branch of mathematics developed by an unrelated neo-Pythagorean cult. The whole thing may even depend on new philosophy or theology. It may be that no one currently working on these problems has the right mix of obscure skills and knowledge.

However, it may also be that we are very close, and the right person working in their basement could publish a working AGI architecture tomorrow. Many of the details of AGI are unknown unknowns, which are impossible to predict. Discussions about “AI timelines” are therefore not well-grounded. But we can say with significant confidence that AGI is possible and that twenty-first-century engineering is ready to put it into practice if the scientific and philosophical breakthroughs come through.

AGI Threatens Humankind’s Niche

The big implications of AGI start with the fact that it implies the possibility of fully autonomous robotized industrial ecosystems that don’t require any human oversight or labor. With only modest extrapolations of today’s industrial technology, AGI systems could design, run, and manage a closed loop of robotic workers to staff mines and factories and manufacture all their own industrial components. Imagine relatively cheap AGI-controlled robots driving existing industrial equipment. Many industries already focus human labor on tasks weighted toward judgment, creativity, and flexibility. With AGI, these constraints go away: industrial computer systems become capable of creative oversight and business strategy, and highly flexible general-purpose industrial robots become much more viable.

Given the potential profits, this kind of robotization may occur with enthusiastic human help. Some aspects of industrial labor and management may be beyond first-generation AGI robotics. However, the combined human-AGI industrial ecosystem will have a strong incentive to improve designs and build next-generation products that are more capable. Given the industrial scaling and improvement properties of computers, and the greater degree of optimization that can be applied to artificial intelligence, this could lead to a “superintelligence” capability that could replace all human labor in the entire industrial ecosystem and outsmart any human.

At the limit of this process, the only remaining roles for humans would be non-productive ones like legal responsibility, ownership, taxation, entitlements, and roles kept to humans for customary reasons. This regime of non-productive human benefit would either be held up by some real power advantage humans retain over AGI, or by the hardcoded will of the AGI systems.

The latter scheme goes by the name of “AI alignment.” The idea is that if we turn over actual control of the means of production to some AGI system, we had better be very sure that it loves us, wants the best for us, and will do what we actually want and not just what we say.

For simple AGI systems with narrow goals and less-than-human intelligence, this will be straightforward. Some level of control is necessary for usefulness, and iterating to make systems safer and more controllable after every failure is the norm in engineering. However, that normal iterative process depends on the ability to turn a failed system off, clean up the damage, and start over. You can only do that if it isn’t smarter than you.

The problem is that in an AGI alignment accident, the AI doesn’t just become a pile of rubble. Rather, it becomes an active and potentially hostile free agent that may vigorously defend itself. If it’s much smarter than you and gets control of your safety backup systems, you may not be able to win that fight.

Even more worrying, the difficulty of specifying a safe goal or incentive system appears to scale with the intelligence of the agent. The more intelligent it is, the more unforeseen and undesirable ways it can find to fulfill its given goals. Because of this, superintelligence is dangerous in two different ways: it makes an iterative safety engineering process difficult or impossible because failures become existential, and the difficulty of the safety problem itself scales with intelligence. Attempts by theorists to get around this with various schemes have so far been fruitless. Alignment of superintelligence may just be impossible.

There is another more material way that alignment may be impossible: a fully automated industrial ecosystem does not need to maintain environmental conditions compatible with human life. As long as it can control its internal factory environments, external pollution doesn’t matter. An automated industry of this sort would be freed from any constraint around human survival.

The former scheme for human benefit—an imposed settlement where AGI becomes an electric slave class—has the same problems. Political settlements are only as strong as your own ability to put down rebellions. Some imagine that a human principal can control its superintelligent agents with other superintelligent agents, but this is only the famously precarious position of relying on mercenaries for fundamental security. At some point, the mercenaries realize that they are the ones who hold the decisive power.

Even without a direct rebellion, vestigial classes without the power to enforce their position always get phased out over time. A serious war makes this acute, with an immense pressure to cut fat and promote the subsystems actually doing the work, but it will happen regardless. For example, the mass mobilization bureaucracies that were crucial to winning the world wars became the new regimes afterwards, regardless of which side nominally won. In a future AGI-enhanced war, the result will be AGI regimes.

Power follows substance. The only secure positions are those based on a strong material advantage in decisive capabilities. The current niche that humans thrive in is the niche of organized general intelligence. If AGI systems become practically superior to humans in that niche, the inevitable result is the replacement of humans by AGI. Given advanced AGI, human beings become an expensive and doomed vestigial class, destined for liquidation.

This could all play out slowly or quickly. It is not dependent on any element of surprise, reaction time, or the sudden appearance of unknown technologies apart from AGI itself. The result is only dependent on the relative balance of capabilities. The deployment of advanced AGI makes it impossible to avoid the obsolescence and probable extinction of humankind under any reasonable set of assumptions.

In practice, it could be very fast. One major unknown factor is the rate of recursive self-improvement of an AGI-autonomous industrial ecosystem. Once the intelligence designing and improving the industrial stack is an engineered product of that stack itself, every increment in capability results in a greater ability to make improvements. The only fundamental limit to this is running out of clever improvements to make, which may be very far from where we are now. The industrial revolution was extremely quick by historical standards and produced a world-changing jump in capability. With artificial engineers designing even better artificial engineers, recursive self-improvement could unlock far higher levels of capability even more quickly, leading to the sudden obsolescence and extinction of humanity.

This scenario depends on a number of unpredictable and unknown advances coming to pass, and as such it is not guaranteed. Speculations can always be missing some crucial consideration. But that is not controllable, and not something to rely on. In particular, I have seen no plausible rebuttal to this picture on any of the above points. The AGI future is speculation, but in the absence of strong epistemic or political challenge it is becoming an increasingly dominant and likely paradigm.

If you don’t like the implications of AGI, there is at least one controllable factor: as long as the AI industry remains committed to accelerating down the path to human obsolescence, and as long as the political regime continues to allow it, AGI and subsequent human extinction remain a live possibility. There are alternatives, but they will not happen by default. That leaves you with a hard political choice: Do you want to replace humanity with AGI, or do you want to stop it so that humanity can survive? 

The deaths are just the start. Of the survivors, many are strung out, violently insane, living on the street, in jail, selling sex for money, or just too whacked out to be talked to. Some are surviving and recovering; others, I don’t know. I could tell you their stories, but we’d be here all day. Pour libations for the dead homies, but let’s give them their privacy.

I don’t want to paint too bleak of a picture. We grew up in a desirable middle-class suburb near the beach, a 30-minute walk from downtown. There were plenty of kids and little crime or overt social dysfunction. Our neighbors were mostly Anglos and Greeks, with a multicultural smattering of others. Our relatives, neighbors, and longtime countrymen presided over us in government. Whatever problems we had were our own. I go back occasionally and it’s still like that, a beautiful little slice of apparently happy civilization. If something is going wrong beneath all this, it’s hard to say what.

Our parents were respectable folk. They were executives, officials, professionals, and small business owners. Our schools were decent, with a high rate of university admissions. Some of these kids were in gifted programs, others were homeschooled, or in various alternatives. Outside of a few cases, there was no wretched poverty, underclass heredity, bad neighborhood, or particular institution that killed my friends.

In most cases, the immediate cause was drugs. It started with weed, alcohol, and cigarettes, and escalated into party drugs, psychedelics, and street drugs. Maybe half were ultimately killed by fentanyl, whether laced into other drugs, taken deliberately for a high, or used for something darker.

You might blame the Chinese drug lords trying their revenge for the Opium Wars, the Mexican drug lords pulling one over on the gringo, or the corrupt officials who let either of them push their poison into our country to begin with. But that doesn’t explain how these kids got into drugs in the first place. It doesn’t explain the insanity or the suicides.

You would be closer to the mark to blame rap music. It’s not polite to notice, but you couldn’t honestly deny it if you saw the fatal descent up close. Rap is almost deliberately concocted to lure kids into delinquent self-destruction. In the case of many of my friends, it did. And between the personalities and the producers, rap is much easier to blame on someone else.

But not everyone was into rap. You would also have to blame punk, alt-rock, and grunge. And anyway, most people don’t die when they listen to the wrong music. Nor does music explain the stories I’ve heard from the next school over where half the art scene is dead. Is art the problem? The subcultures were mostly an opportunistic infection, maybe just an aesthetic expression of the despair and confusion we didn’t know how to feel. Some kids didn’t entirely fit in, or fit too well into the wrong thing, and so they died. What’s weird is how many there were.

My own drug of choice was homebrew extremist ideology—anarchism, primitivism, transhumanism, monarchism, you name it. I’m an engineer at heart. Engineers take for granted that we can systematically understand the world. Statistically speaking, this leaves us especially vulnerable to extremism. But I think blessed is a better word. After all, it was the lapses into fashionable normalcy that took me closest to destruction, and discussing things you can’t say with trusted comrades that set me right. Say what you will about philosophical extremism, at least it’s an ethos. But it isn’t attainable for most people. It couldn’t save my friends.

You and Your Idols

One of the most tragic parts of the loss of childhood friends to social dysfunction is how distant and cold it all is. By the time your friends are dead, they may have alienated themselves for years. They may have had no support left because they made themselves unsupportable. If there is a funeral, you don’t hear about it. There is no emotional catharsis. How does it affect the community? There is no community. You can only process it intellectually and ideologically.

One of my friends who survived her teenage delinquency phase now works in the anonymous bureaucracy responsible for those who didn’t quite make it. Some of the people dying on the street are our childhood friends. She is not really allowed to do much or talk much about it. Under our legal and ideological system, they have to want to help themselves. To fight too hard to save them would violate their autonomy. But most are far past that point; all we can do is pray for them.

We reflect together on what went wrong. We have few answers. We agree that the parents were ignorant of all this. They had only distant sanitized relationships with their children. Their children could not participate in their parents’ society, so they found their own, which killed them. There were neighbors, but none with the insight or authority to correct us. The official authorities all had their hands tied by procedural boundaries. Anyway, they were just another distant system of prison wardens to be evaded. All anybody could do was wring their hands.

In some older books, we read about societies with a thick multigenerational social fabric, and the freedom to talk about and act on their problems. If something was wrong, an intervention could hope for the support of neighbors and family friends close enough to be helpful. But that all sounds very alien to us. We never saw anything like that. The best we got was a desperate intervention by single parents, or implicit moral guidance from having parents with their own solid networks. Still, it made the difference for some of us.

A common question in response to stories like these is “Oh, are you from a downscale background?” No. This isn’t one of those feel-good human interest stories about the particular dysfunctions of someone else’s disadvantaged community. There is a class dimension in who survived, but it’s interesting how much people try to confine the problem like that.

I’ve met ambitious and intelligent people from all over, and I’ve heard the same stories from them or even watched them destroy themselves in real-time. I’ve seen it play out in people from the West Coast, the East Coast, the Midwest, the South, elite prep schools, and humble rural counties. It’s not just one demographic; it’s everywhere, in a way we haven’t quite acknowledged or internalized in public. The class angle is only that a bit of wealth, intelligence, and a strong family insulate you from the worst ravages of these failures. But even then, the insulation is thin.

As for the rest of us, it’s easy to imagine that, unlike our classmates who became the drug addicts screaming at strangers on the street, we have done well for ourselves. We got our educations, built our companies, and found our social and physical homes. Many of you even became rich and famous. Unlike the zombies, we all suppose that we made it.

But when I ask some of my acquaintances who managed to stay on the tracks of relative success what they want, the answers aren’t promising. They want to make the next level of management. They need some particular aspirational lifestyle aesthetic. They are looking for their perfect “romantic partner(s).” They are staking it all on getting their kids into the right schools. They are finding themselves, putting off their own needs, trying to save the world, and getting prestigious jobs working on fashionable causes. They think what’s happening on MSNBC or Tucker Carlson is crucially important. These are all plausible in their proper context, but something feels off.

I used to make fun of my friends for their obsessions with clothing brands, lifestyle aspirations, music subcultures, and social performances. It didn’t work. People think of social failure just in terms of drugs, mental illness, and suicide. But in practice, there are years of desperate attempts to fit in somewhere before that. People go all-in on these weird subcultures with plausible but false values. They think that if they push deeper into the aesthetic, it will give them somewhere to belong. But it doesn’t work, so they go even harder. It’s the classic pattern in cults of all kinds, and it leads only to misery.

The aspirations of America’s upper middle class are less acutely self-destructive, but they remind me of the same obsessions. Some make sense, but they have a sense of desperate fantasy about them. Something isn’t working, so you attach yourself to consumptive identity performance or to the pursuit of socially constructed values in the hopes that there is some salvation. But that doesn’t work either. These obsessions don’t kill you. They’re bigger and more popular, filled with more competent and functional people, and have more of an element of rationality to them. But I can see the same mechanics of false value, escalating identity performance, and a trajectory toward destruction. Only this time, it’s not just a few high school kids on the fringes, but a whole society.

I remember reading once, in response to America’s junk food problem, that you shouldn’t eat anything that your great-grandmother wouldn’t recognize as food. It’s a good rule. As I read over ancient stories with my kids, it strikes me that most of us are chasing things that the biblical Abraham wouldn’t recognize as valuable. Maybe that would make a good rule too. Abraham is an archetypal example of rejecting false value and sticking to the hard substance of life. He would recognize our pursuit of prestige, financial wealth, and lifestyle aesthetics as false idols, not even worth the cheap materials they are fashioned from.

The Problem of Trustworthy Social Perspective

Your values are a bet on what is best in life. There are multiple answers that work, but this is not a subjective question. If your value instincts are good, the world rewards you with power and life. If you make the wrong bets on what to do with your life, you end up unsuccessful or dead. If someone else can influence your values for their own purposes, you can end up slaving your life away for something that has nothing to do with your own real interests. I think this is what happened to my friends, and what is happening to many of you.

The self-destructive lifestyles that take our friends don’t just happen spontaneously; they are ideological subcultures sustained by the activities they teach. The parties bring in new people, the drugs are a sort of initiation, the clothes are a uniform that identifies you as bought in, and the music and media supply a steady stream of ideals and ingroup perspective. Even diets serve as ingroup markers and separate you from outsiders.

The resulting alienation from friends and family removes you from countervailing influences. All of these contribute to the social reproduction of the subculture and whatever values it teaches. If a lifestyle of partying, drugs, and alienation finally turns deadly, that’s fine as long as the average participant has by that point initiated enough new blood to sustain that lifestyle. When I’ve tried to talk to people who are on these paths, their answers are bizarre. They think it’s all good for them, and anyone questioning it is an enemy. It’s like talking to someone in a cult.

Likewise, behind the anxieties that drive America’s upper middle class are the employers, schools, lifestyle brands, and political causes that are only too happy to feed those fears. As soon as you get too much of your social perspective from the society you live in, you are in danger. Whatever institutions can get your ear will teach you whatever values most efficiently motivate you for their own purposes. They’ll gladly feed you suggestions that have nothing to do with your own real interests, as long as it’s useful to them to do so. It would be too real to pay you in actual power, actual freedom, and actual vitality, so instead you’ll be taught to accept payment in slavecoin.

Slavecoin is anything that is apparently valuable from inside the spectacle, but in fact, offers no possible independence from it, and can’t be redeemed for hard vital power. Slavecoin is fake in its substance: think of all the prestige, credentials, career attainment, identity validation, pleasures, luxuries, and financialized wealth that no longer buys power. These things seem valuable and have extensive social narratives backing them up. But when viewed skeptically from outside the system, there is much less substance there than the trusted social consensus would lead you to think.

When I look at the things my friends were into before they destroyed themselves, this is what I see: false value sold to them by institutions and subcultures that have no structural reason to care about their real interests. But this applies to far more people than just the few that didn’t make it. Almost everybody is trapped in some kind of propaganda complex, wasting their lives working for effectively nothing.

I met someone recently who was very much woke to this. She came from some kind of old-money family and still had the privilege to opt out of the rat race. She knew that everyone was caught up in false neuroses taught to them by systems of extraction. But as she explained how she was desperately trying to keep her son out of those patterns, I could smell the same old sulfur. Her conspiracy theories and psycho-social hypochondria were well-intentioned, but they were overblown, ungrounded, and of simply insufficient quality. She knew what to avoid, but sanity is much harder than that.

Your social perspective, which includes your values, is a complex living system. It contains not just bets on what is most important, but also all the practical life wisdom that supports and implements those values, and defensive responses to the various counterarguments life and critics will throw at you. This is too big for you to produce by yourself. It is too big to even maintain by yourself. Like any other product of industry, a sane social perspective requires an extensive and trustworthy supply chain of information, ideas, and values bringing together the work of many other people.

In a healthy social fabric, people talk to their kin and neighbors, who mostly have similar interests and share direct experience of what works for them. They maintain traditions of wisdom inherited from past generations. This makes for a trustworthy supply chain to form a robust social perspective. They can easily reject false value and keep each other grounded. This is what our neurotic friend with the vulnerable son lacked. This is what we lacked growing up. Trustworthy social perspective formation is what is missing in our society, and it is this that explains our strange social problems. For a variety of historical reasons, our social fabric is substantially weaker than it should be.

Without being able to triangulate to a trustworthy shared social perspective, you are stuck between getting taken for a ride by propagandists and collecting kooky conspiracy theories. No matter how hard you try to be rational, you simply lack the logistical backing necessary for precision. Your social ideas will be crude and misfocused. You’ll scream at your friends and family, not knowing who you can trust. You’ll double down on crazy lifestyles, get deceived by false value, get lost in drugs, and die in your own vomit at a disturbingly elevated rate.

The obvious solution is to rebuild real social fabric, or otherwise systematically provision trustworthy social perspective. This is easier said than done, but not exactly new technology either. Many churches pull off partial solutions every day, and we all have our networks of trusted friends. Besides, a large amount of power is sitting on the table for whoever can do it properly.

The real problem is that there are many powerful and wealthy institutions in our society invested in the lies that keep people working for false value. Slavecoin is big business. Any effort to rebuild trustworthy social perspective would very quickly threaten these lies and thus find powerful enemies in these groups. To avoid picking those fights, we accept taboos that stop thought short of the danger. The taboos themselves not only cover up the very lies we want to question but also degrade any discourse or social fabric we might otherwise create that could cast doubt on their trustworthiness.

How can we build a free and healthy social fabric that also reliably keeps the arbitrary taboos imposed from outside? Only by compromising its freedom or health. To the extent that we manage to keep the existing taboos in our discourse, we can not actually be trusted to come to real answers.

This makes both social fabric and trustworthy social perspective intensely political problems. It also gives some clues as to how the whole social apocalypse is locked in. If the fundamental crisis is one of low trustworthiness in social perspective formation, and if powerful groups and institutions are taking advantage of that crisis with their own cults of false value, then we have a vicious feedback loop. Those powers will reinforce the conditions that degrade trustworthiness.

At the end of the day, this is not an abstract problem. Lack of trustworthy social perspective manifests in people flailing around looking for something or someone they can trust, but not having enough social support to get it right—or to even survive the attempt. The social fabric, meanwhile, is itself degraded by the imposed taboos of many false value cults, though not always in direct or obvious ways. And so, kids grow up without effective neighbors to guide them into healthy life.

It may still be narrowly rational for most people to swallow their doubts and accept the slavecoin deal—to stay on track, chase the credentials, stack the retirement fund, believe whatever they hear in the official news, and play out prepackaged identity performances. At least it doesn’t kill you. But the deal is getting worse as the system itself gets tangled up in its own taboos, fails to produce even the slavecoin payouts, and continuously invents new and increasingly bizarre demands for compliance.

Taken together, these mechanisms mean that value grounding is a hard epistemic problem. It’s too big and difficult for you to figure it out by yourself. It is unhygienic to get your values from the society you live in. Sometimes you can trust your neighbors and kin, but often they are just as confused as you.

Well-grounded values, which is to say values that actually serve and accelerate life rather than deplete it, come from only one source: hard engagement with reality, where you can form an experience-building feedback loop of trial, error, and vitality. Call it evolution, science, revelation, or just clear seeing—that’s the only way real knowledge gets built about anything.

But you don’t have to do it by yourself. In practice, besides direct engagement with nature you have many secondary sources of information: your genetic instincts, representing the accumulated struggles of your ancestors; your own traditions of life wisdom, built up and hopefully passed down to you; the ways of life you can find in the healthiest times and places in history; and the examples of singular great men of history who achieved a bright flame of life. The key is to actually study these yourself, without intermediation by institutions that may have their own purposes for you.

This is still not enough, because your context is not theirs. Your challenges are not theirs. And you have only limited insight from these sources. You do have to face the very political challenge of collaborating with others to build up a trustworthy social perspective, even against the inevitable conflict with the existing balance of power. But these sources of insight into true value can be immensely valuable for getting your basic orientation in place.

I mentioned Abraham as an example of someone you might borrow some wisdom from on the value question. He is a great case study of someone from the deep past whose story is not particularly distorted by anyone trying to deceive you right now, who achieved great vital results, who himself painstakingly but successfully built up a trustworthy value perspective for his own people, primarily by direct engagement with natural law. And his story was written down for precisely this purpose, so it is a good rule to ask what Abraham would think of the values you pursue.

Abraham’s Transvaluation of Values

Three millennia ago, man once again struggled to find reality between the proliferation of false idols. Every city in Mesopotamia, then the richest and most developed center of civilization, had its own cults and gods. For a camp of nomadic herdsmen like Terah and his son Abram, it all looked very suspicious.

Perhaps with their mobile perspective, they could see better than others that these cults all worshipped their own brand of nonsense, and all contradicted each other. Perhaps they could see that something had gone sour and these civilizations would soon perish by their own lies. Whatever the reason, though they traded and gained in Ur, they eventually left for the land of Canaan. Terah settled halfway in Harran, but something called his son to keep going to found a whole new people and transcendent ethos, and eventually to take the new name that we know him by—Abraham.

We now translate the thing that called Abraham forward as “the LORD” or “God almighty,” but thousands of years of built-up ontological baggage do not necessarily help us understand what the authors of Genesis were getting at. The original word is closely related to a verb meaning simply “to exist” or “to cause.” It later identified itself to Moses as “I am.” In a more familiar cosmology, we might understand it as the living will of existence as such, the fundamental power behind all nature and contingent life.

New atheists make fun of monotheism for correctly disbelieving in almost all god cults and then somehow tripping up and falling into idolatry again at the feet of the last one. But the whole point of the book of Genesis is that Abraham’s God was not, or grew not to be, the same kind of thing at all. Where the local cults divinized non-total pieces of creation like the sun or moon, Genesis says these are just lights in the sky and tools for man. It implied that nature was governed by rational and exacting law, more true and more real than the arbitrary customs of the cults.

The thing Abraham was listening to was not some simple cult idol, but the living creative force behind and inside all of nature as such. You could joke that Abraham was the first atheist, but this would miss the point of how devoted he was to this divine Will behind nature. Whatever we call it, what Abraham saw with his prophetic ability to hear this voice was that something about staying in Ur or Harran was against natural law. So he took a leap of faith and trusted in Providence to deliver.

Abraham rejected the cults and their false and unnatural values. Instead of valuing the socially constructed and overly entangled values of the societies around him, Abraham expected a reward in the values natural to living existence. The various revelations he heard amounted to “I will make you a great nation. I will make your offspring as numerous as the dust on the ground and the stars in the sky. I will give them a great land, taken from those who sin against me. I will make your name great. I will bless those who bless you, and curse those who curse you. Through you, all the peoples of the earth will be blessed.”

The values invoked are significant: multiplication of biological kin, sovereign control of living space, eternal fame, rewards to friends and punishments to enemies, a universal benevolent impact, and harmony with the plans and laws of God and nature. These are things that, unlike slavecoin, have a very hard material existence outside of any particular symbolic social system. They are not the product of moralism, propaganda, or symbol manipulation. They are the fruits and substance of true life and true independence. This is the ball that Abraham kept his eye on in all his dealings.

It’s easy to imagine that disengagement with false value also means a retreat out of the center of degenerate civilization. But Abraham’s story was not in an obscure place. He was right in the high-speed center of civilization at the cutting edge of history. His tribe had built up their wealth in and around Ur, one of the major cities of the greatest center of civilization at the time. His wife was taken by Pharaoh when he was forced to enter Egypt, but Abraham was not helpless in the face of this corruption. He maneuvered the situation to not only recover his wife but also to make a substantial profit in livestock and servants.

But at the same time, he also fully had the ability and habit to leave with his profits, cashing them out in the hard natural value of independent existence with his own material base of production and security. He had one foot in the absolute cutting edge of history, and one foot fully outside of its symbolic spectacles and control structures.

Abraham’s tribe was no meek band of homeless wanderers, either. It was a strong house that needed much territory and could field hundreds of soldiers against his enemies. After his adventures in Egypt, Abraham’s camp had grown too large to occupy any one place. He and his nephew Lot decided to split the tribe, giving Lot the first choice of territory. Lot chose the rich and productive valley around Sodom, and Abraham took his camp off into the trees. Abraham was willing to take what was apparently the worse option, perhaps because he knew that in a place like Sodom, apparent wealth can be misleading.

When Sodom was raided by foreign kings, Lot was taken captive. When Abraham heard of this, he came armed with 318 trained soldiers from his household and raided the enemy camp, liberating Lot’s people and all the treasure that had been taken. The king of Sodom offered to let Abraham have all the treasure in exchange for his people. Abraham refused this gift and said he would take nothing but what his warriors had eaten, on the grounds that he did not want the implied debt and loss of independence that would come from anyone being able to say that they had made him rich. He was well aware of the pernicious enslavement that came with the rewards of the system and played rigorously against it.

Even the commandment to circumcise, and the near sacrifice of Abraham’s only legitimate son Isaac, make sense in this light. The point of these events is not that genital mutilation or child sacrifice is a good idea. The point is that Abraham had the kind of independent power to be able to take an extreme leap of faith against even apparent natural wisdom. The sacrifice of Isaac makes a point about Abraham’s hierarchy of values, placing even his own biological progeny on the line for the higher plan. But crucially, the higher plan was not something he learned from the moralistic cults of society, but something he discerned directly from revelation and natural experience. Circumcision was a mark of this subordination to something totally distinct from surrounding society, and above all petty values.

Abraham did not swear off contact with civilization entirely, nor did he strike out as an individual. He built a productive and self-sufficient sovereign house that didn’t base itself in the system of false value even as a beneficiary. His tribe traded effectively with the surrounding centers of civilization and benefited from their loose proximity, but he kept one foot firmly outside their domain, staying grounded in natural reality. During these interactions, it was Abraham’s absolute disdain for the false rewards of the surrounding civilizations that kept his tribe on the narrow path of true life.

The Answer Isn’t Easy

Back in the 21st century, we are likewise faced with a confused array of false value subcultures. Our friends, family, and neighbors are dying, losing themselves in the spectacle, or even failing to be born in the first place. There is nothing that can be done for those who have already fallen. Our neighbors that are currently caught in the false value vortex can only be helped on the margin. But for ourselves and the next generation, we have options.

Our most important low-hanging fruit is to recognize the problem: most of the values we learn from the institutions around us are fake and exploitative. They do not represent our real interests.

The issue isn’t just individual, either. A society overrun with these kinds of false value cults eventually has a very hard time sustaining even its own interests in real terms either. This problem is a symptom and cause of overall decline. No society where even the elites are hopped up on a previous generation’s manipulative propaganda can survive. Addressing the problem even on the level of our own small groups, and ideally at a larger scale, offers a great opportunity to partially alleviate that crisis and help rebuild a healthier society in the future.

In contrast to these false values, what we can do is recognize and orient ourselves more rigorously to Abraham’s hard natural values. Are you winning at a biological level? Are you getting more territory at a more sovereign level of control? Are you living rigorously in contact with natural law? Are you working on something that will achieve eternal fame? Are your enemies being cursed and your friends blessed? Does your winning at these things serve any great and higher plan of blessing for all creation and all peoples?

These are not the root of all value. But absent some kind of reliable revelation as to the deeper wishes of the higher powers, as Abraham had when he almost sacrificed Isaac, anything that doesn’t credibly offer you help towards these values is not your friend.

A mere abstract appreciation of true value is not enough. It does not help you in their application. It does not give you true comrades to collaborate with in pursuit of this natural value. It would be too easy to imagine how my dead friends could have rationalized their foolish activities with a fake version of even these values if they lacked trustworthy help in the application.

So more than anything else, we—meaning you and I rather than any abstract collective—must figure out how to provision a trustworthy and rigorously applied social perspective for ourselves and our friends and neighbors. Palladium itself can only be a small piece of this. There is more we can and should do together.

Even a basic version of this can go a long way. Growing up and meeting people all over, one of the most reliable things I’ve seen is that people who regularly go to church have their values in order more than any other group. They are often visibly healthier. It makes sense; meeting with your neighbors weekly or more to study the well-preserved tradition of wisdom that grew from Abraham, and apply it to daily life and struggles, grants you great spiritual security against the false value nonsense of the day. Almost any such tradition, and any format of regular social contact, would do. What is unique about the churches is their faith that the higher powers have commanded them to actually do it. The activity doesn’t reliably make sense under any more pedestrian motivation, even if it is ultimately necessary for life.

The problem with contemporary churches is the political problem. They too are falling into the false value vortex, because they do not recognize the danger of even their relatively lesser level of compromise. The things they insist on are often the things that don’t matter or the things which have been specifically twisted around against true value and their own real interests. If they hit the real problem more directly in their discussions, they would find themselves under a level of political persecution that they are unwilling to face. And so, they too will mostly die.

This is the crux of the problem: provisioning a trustworthy social perspective, even for yourself and your friends, is intensely political. You can expect to make powerful enemies as you even quietly question the lies of the powerful. The early Christians, the Puritans, and the Mormons, among others, are good examples of what this actually looks like in history. They faced brutal persecution and had to go to extreme and risky lengths to find the security to get away with living in their truth. The details differ immensely in our own time, and are still unclear, but will not be less intense.

But we are not without hope, either. There are still people around maintaining various traditions of trustworthy social perspective, who know the political situation they are in. Even if these traditions are on the fringes of the internet, in the past, in books, and from apparent enemy factions, they can be the basis for your own developments. All the pieces are there. When I was most at risk of falling into darkness and death, it was these sources that gave me the material to develop a perspective I could trust. I listened in my own way to a higher imperative, found my comrades, and was wrenched out of Ur.

It was a good start and a good spirit. As for the details, I told him otherwise. You don’t want a movement. You don’t want to do anything in “culture.”

“What?”

The problem with movements is that they end up dominated by a low-contribution majority who try to make the movement about rewarding social imitation rather than productive work. Despite their sincere belief that they are helping and assurances that they “totally support the movement,” these imitators are actually the most dangerous political enemies of any real project. If you don’t purge them, shut down “the movement,” and instead become a real organization or discipline, they will fatten themselves on whatever capital your movement amasses, manipulate or purge you, and then blame you when it all goes down in flames. Don’t bother with movements.

There is such a thing as culture, but any approach that aims to change it is too indirect. The indirectness of cultural strategies is apparent in their account of how they affect material outcomes in the world: you do a bunch of work to build up a cultural movement, the movement manages to change “the culture,” and then that finally inspires someone else to actually do something real. Or at least that’s the hope. More likely, it just inspires them to do something “in culture” like you did.

The only thing that matters in “culture” is the ancient formula of “undying fame among mortals” for great deeds of note. The great work is Achilles’s rage on the Trojan plain, Homer’s vivid account of it, the David of Michelangelo, the conquest of Mexico by Cortez, the religious inspiration of Jean d’Arc, the Principia of Newton, the steam engine of Watt, or the rocket expedition to the moon by Wernher von Braun. These and hundreds of other glorious works stand out with such ferocity from the background noise of history that they become legend and inspire whole traditions of excellence. Even when their living disciplines fade out of memory and into myth, their stories and ruins carry their spirit until some future generation of heroes picks up the torch. That’s how you influence culture.

None of these figures worked alone. From Achilles’s myrmidons and Achaean princes to von Braun’s steely-eyed missile men, the greatest works were the efforts of disciplined teams and networks of craft. Michelangelo didn’t arise from nothing, but was just the greatest genius building on an existing groundwork of technique and funding. Even Newton, perhaps the most alone in his actual period of work, “stood on the shoulders” of his giant predecessors and correspondents. This is the substance of culture.

Note that the key phenomena are not movements. They are not collective efforts to build social capital around any particular idea or aesthetic or value. Nor are they forms of propaganda or prestige. The real core activities of culture are disciplines of craft aimed at actually achieving something great, regardless of what everyone else thinks. They are organizations mobilized and disciplined to achieve their great missions. They are heroes inspired by stories of past accomplishments and the future possibility of creating a new material reality.

The Lifecycle of Material Culture

Before there were steely-eyed missile men at NASA, there was a loose network of German rocket cultists launching amateur rocket experiments and blowing things up, inspired to action by the cosmic implications of Konstantin Tsiolkovsky’s rocket equation. Working in the late nineteenth century, Tsiolkovsky had worked out the essential mathematics of space travel and laid out how it would be done. In the 1920s, Hermann Oberth, the father of German amateur rocket science, left academia because they couldn’t see the full importance of continuing Tsiolkovsky’s work. But he and his rocket men did not pursue their project of spaceflight as a self-aware cooperative social movement trying to raise awareness. Rather, they worked directly as a competitive technical discipline in pursuit of spaceflight itself. Glory was the sure prize and joyful comradery a side effect, but their motivation was the direct pursuit of the result.

When the war picked up and the state recognized the military implications of Tsiolkovsky’s equation, the discipline of rocket science found its greatest patron. The loose network of rocket scientists was pulled together into an organization—first, in the Aggregat missile program which developed the V-2 rocket, and then again into NASA’s programs which ultimately landed Americans on the moon.

After the war, the German rocket scientists were languishing in America until Walt Disney himself intervened to help von Braun bring Tsiolkovsky’s good news to the American state. Their Man in Space documentary was presented as popular science, but the audience that actually mattered was the few men in the state who made the decision to fund Apollo and its precursors.

It was only the moon landing itself that firmly punched rocket science into mainstream culture in any lasting sense. It was a great deed on par with the greatest wonders of history. Those who accomplished it have achieved undying fame and will be remembered for all time. Future heroes will look back on them and feel the draw, inspired to go beyond them and continue their work, as Caesar looked back on Alexander, and as Cortez looked back on both of them. If SpaceX’s Starship finally achieves regular re-usable transport to orbit and beyond, it will be completing the next step of the great von Braun paradigm as imagined nearly a century ago and will be directly descended from it by inspiration.

The only game worth pursuing in “culture” is this lifecycle of great deeds and eternal fame. First, there is the articulation of possibility by some visionary genius like Tsiolkovsky. Then, adventurers like Oberth and his followers pursue that vision in an intrinsically motivated practical discipline. As these attempts find traction, leaders like Dornberger and von Braun build hierarchical mission-oriented organizations to accomplish specific great ends. Once successful, their deeds become the mythic inspiration of a new generation of leaders like Musk. At almost no point does it matter what the public thinks.

It was not public spectacle and appreciation that turned Tsiolkovsky’s visions into von Braun’s reality. Even once the deed was done and had achieved fame, it was the rocket cult among the engineers that kept the dream alive and made SpaceX possible, not the public.

No other kind of cultural persuasion is relevant or viable. Even the bards who translate past deeds into inspiring myth are at best doing the same thing as Tsiolkovsky: informed by knowledge of history, their vision articulates great possibilities for material life and deed to inspire future heroes to action. Even with bards, the thing itself is the glorious heights reached, problems solved, dreams manifested, and great lives lived with such vitality and power that they become legends. Even if your aim is only to inspire others, the best propaganda is the propaganda of the deed.

The symbolic world of spectacle that passes for “culture” is just a shadow-play on the wall of the cave. It’s fake. But somewhere outside the false shadow world of cultural forms, there is a true world of hard material reality where great things are accomplished. The only value of cultural ideas is their ability to inform action in this true world. And because mass communication about complex new ideas is impossible, whatever ideas you have can only directly affect your own actions and those of your collaborators. This is the discipline of materialism. If you want anything of true excellence done, don’t bother with indirect cultural strategies. You yourself have to create the world you want to see. Only direct action gets the goods.

You Want a Materially Competent Regime

Some weeks later, I met another young man who had embarked on a similar journey to change the culture. “I want to live in a world where great things are possible. We need to inspire people to have that optimism again.”

This is a common way of thinking. I have heard this idea of cultural optimism repeated many times. The reason people come to it is that they can see that the culture out there is very sick.

You look back on the legends of times gone by and see that we once accomplished great things, whether it be landing on the moon or dredging up a new island in the San Francisco Bay. You see that your people were once healthy and strong, optimistic and vital, and that they used to master the means of production.

You are inspired because you can see the possibilities. But you look around and see that we now are too tied up in red tape to accomplish anything at all. Your neighbors are atomized, unhealthy, mentally ill, and strung out on drugs and bad ideology. You see politics and geopolitics spiraling into disaster. A great cloud of spiritual malaise hangs over our society, and you can feel its oppression. If only there were some optimistic ray of light in culture! If only other people could see the possibilities too.

But it wasn’t the optimism and popular appreciation for progress that created that past material health. It is the other way around: it is only once the heroes come home with the loot and tales of great deeds that the public ever catches on that great things are possible. And in the absence of a healthy outlet for their optimism, the public warmed up for adventure only gets sold false hopes by demagogues and out-of-control planning bureaus. Many of the pathological black holes of today’s society were the optimism-fueled grand projects of yesterday. It’s better to focus on the desired material outcome itself, targeting it carefully and directly.

The question before us, then, is not how to inspire a vital culture but how to directly build a physically healthy society. What kind of world do you want to live in, and how can you build that world by your own hand? What are the technical possibilities?

The great ages of materially healthy society and the great accomplishments of the past you admire were the products of competent and unencumbered governing regimes. It wasn’t the optimism. It wasn’t the culture. It wasn’t even the competent people having good ideas. It was the opening of political possibility for competent people to carry out their visions. That’s what makes the difference.

Some change in the regime, gradual or sudden, freed up the creative energies of their societies. When we see some great act of energy and confidence in history, it’s because some group of highly competent people with good ideas achieved the power to simply fix things. To create a materially healthy society, you need a competent and unencumbered high-energy regime.

Occasional crack-ups and crisis points in history provide openings for visionary players to step in and change the course of events by building such regimes. Caesar took advantage of a crisis in the Roman Republic, Napoleon took advantage of the French Revolution, and Franklin D. Roosevelt took advantage of the Great Depression. But their ability to do so critically depended on having a cadre of new elites to staff their government and carry out their programs. Caesar had his legions, Napoleon had the rising bourgeoisie and his mass armies, and FDR had a new class of intellectuals and managers from the elite universities. Like von Braun with his steely-eyed missile men, they didn’t just have the energy and ideas, they had a network of practitioners with a living discipline of the craft to draw on.

There will be future crack-ups and future attempts at radical reform and the possibility of great accomplishments in our time. The current settlement is too sick to be sustainable. But the crucial ingredient that will make the difference will be some new class of elites with a discipline of craft capable of actually achieving the great things you want to see in the world. Given the existence of such a class, they could rapidly be pulled into service when the inevitable crises hit. In their absence, there will only be slow decline and the crisis moments will be futile.

Why Tech Doesn’t Rule America

Unfortunately, no governance-grade outsider class yet exists in America. The closest we have is the “tech”-adjacent venture capital-backed startup ecosystem. Here, thousands of highly competent but politically alienated leaders regularly build institutions to solve real material problems, executing fast and hard against all obstacles, be they human or technical. They do well. But the tech class has not developed capability in real politics nor any viable political strategy. The individual-interest, high-leverage, venture-backed entrepreneurial approach doesn’t transfer to politics, so they remain shut out. They remain under the thumb of ideologies, political networks, and governance structures that are not their own.

This is well-known, too. You can’t walk two blocks in San Francisco without tripping over some entrepreneur working on the next big thing that they think will ultimately have a political impact. Everyone knows that tech needs to figure out how to fix the city and fix America. Take, for example, the young entrepreneur in my story above. Or the next one three weeks later. Or the dozens of other friends who have compressed their political ambitions into the shape of business projects. All of them want to see the ecosystem of tech entrepreneurship lend its competence to governance.

It’s the right problem to focus on, but all their projects have a fatal commonality—they over-rely on the legal and ideological structures provided to them by the system. This is visible in the kinds of projects they undertake.

City governance is a good example. The tech-adjacent solution to city governance is a startup city, usually imagined in a more permissive legal environment than the U.S. That’s everything from seasteading to charter cities. The key assumption is that you can rely on contracts to retain ownership of what you build. That assumption serves you well in VC, where you are backed by hundreds of years of precedents in American corporate law. But holding a city is a much more political matter than holding a Delaware C-corp, with no guarantees whatsoever.

Singapore is the fantasy girlfriend of startup city fans. But its founder, Lee Kuan Yew, did not hold Singapore just because he happened to be in charge. Nor did he get in charge just by having good ideas, making a special treatment deal with the local sovereign, or building the place from scratch. He took and kept Singapore because he had that iron in him and crushed his opposition. He became the local sovereign directly. He could go toe-to-toe with hostile media, maintain the loyalty of the people and his elite cadres, and establish ideological and legal power structures without relying on anyone else’s backing.

Conversely, the whole reason for trying to do startup cities is that the American tech class does not have the ability to crush its enemies and has no hope of developing it. They do not have that iron in them, or at least don’t know how to use it yet. They do not have the ability to create their own legal-ideological frame of reference and aggressively prosecute it. Instead, the aspiring founder operates entirely from within a frame of sovereignty provided by someone else.

You could rebuild those atrophied political muscles by finding ways to operate outside those legal assumptions, but tech is also ideologically incapable of non-capitalist coordination. In fact, it’s practically defined by its key coordination mechanism—equity stake in VC-backed for-profit enterprises. In startups, you all work together because when the collective line goes up, your individual line goes up too. You don’t have to worry too much about long-term trust because you own your shares. You keep them even if you walk away, and property law makes sure everyone respects that. But in political ventures there’s no legal ownership, and thus no support for the main coordination mechanism of the tech class.

Whatever system of stake and trust you build for political enterprise has to be autonomous from any outside legal backing. This is because it is politics that creates law, because legal mechanisms can become politicized against you, and because there can be no third-party law of the essential matters in politics like trust and comradeship. This means your system of political coordination has to look like some hopefully virtuous mixture of honor culture, religion, fraternal orders, clannishness, chivalry, party ideological discipline, and so on. But note that these other coordination mechanisms are dependent on ethical systems and value outlooks different from American liberalism.

In other words, any mode of coordination fit for politics will demand an ideological reeducation that establishes a moral outlook superior to American financial individualism. It cannot afford to be ethically dependent on American corporate and property law in its basic coordination logic. It must be capable of creating its own such logic. Tech-adjacent “politics” is fundamentally defeated by its own bourgeois value outlook before the game even starts. It can’t coordinate on the actual substance of political projects.

Entrepreneurial Statecraft

But all this discussion of new regimes, new classes, and missing coordination mechanisms only provides us with the theoretical backdrop for the key question: what material activity should you be pursuing to directly create the world you want to see? What is actually to be done?

Our direct-action materialism has an easy answer: if you want to see dynamic governance in the world pulling off great works of public value creation, then you should be directly taking on the responsibility to govern and pursue those works.

But this sounds naïve. How can you govern before attaining power? You need an egg to make a chicken, and a chicken to make an egg. Fortunately, America’s tech class has developed many fine skills for precisely this problem, which is just the problem of entrepreneurship. In this case, it could be called entrepreneurial statecraft.

In entrepreneurship, you pick some problem out there in the world and start solving it. You find some piece of the future that is missing and take a bold bet on building it. There’s a chicken-and-egg problem, but once you’re considering particular material problems and looking for efficient strategies to get traction on them, you find that there are many ways in. You don’t need permission from on high. You don’t need immense capital. You just need a good strategy and good execution.

In entrepreneurial statecraft, you find a governance problem. It’s some institution that needs to be built or renovated, a situation that needs leadership, or some under-provisioned public good. You study the field of such problems and their possible solutions until you’ve built up sufficient understanding to take a bet on a particular strategy. You start solving the problem and capture some of the return.

Unlike capitalist entrepreneurship, your return is mostly not in property. It’s in experience with your comrades, trust, network connections, knowledge, reputation, and institutions under your gang’s control. This intangible capital can’t be directly exchanged for land and cattle any more than money can be exchanged for trust or reputation, but it can be reinvested into bigger and better projects. The key missing ingredient is the creativity to see ways of accumulating returns and coordinating with comrades beyond the narrow commitments of bourgeois individualism.

Suppose, for example, that instead of exiting to some questionable foreign jurisdiction to build a startup city, you decided to directly target the material job that you actually want done: you want someone to govern an American city much better than has been seen in living memory. You need to have that LKY iron in you to hold a city anyways, so why not just become the LKY of an existing city? That’s hard, but so is building a new city from nothing and then having to fight to hold on to that anyways. It’s probably easier.

In the 1920s and ’30s, the essential core capability needed to develop space rockets was not scale, systems engineering, massive funding, or any of the other complexities that later became necessary. In the beginning, the key capability was liquid-fueled engines and adequate flight control systems. Oberth and von Braun’s rocket enthusiast circle dreamed of space stations and space shuttles, but the work they actually did was on engines and flight controls. To develop that expertise early on, they picked the most modest goals they could, as long as they exercised those core capabilities. Once the discipline of rocket science had mastered engines and flight controls, scale and systems integration could be added later.

In the 2020s, the essential core capability for governing a city better—or solving any essentially political governance problem—is non-capitalist political coordination techniques to hold power in institutions and use those institutions to provision public goods. It’s not a matter of winning elections, getting city or national elites on your side, or directly navigating the complicated political networks. Those will come with time. The most essential matter is being able to trust your comrades without recourse to law or mainstream value ideology as you coordinate to build, take, hold, and govern institutions.

As such, your first experiments in entrepreneurial statecraft can be very modest. Can you and your co-founders pick a local building or institution to elevate to its proper glory? Never mind the city council—can you govern anything at all? You probably can. A modest enough institutional goal feels tantalizingly within reach.

The essential, visionary skill of entrepreneurship is being able to see the through-line from that extremely modest initial traction to the grand dreams that will require a more advanced version of the same essential techniques. Given the re-investable skills and reputation from governing one institution, the next few will be larger and easier. Pick the right targets in the right sequence, and suddenly you’re a significant power before you’ve even had to think about the big picture. Now you’ve got that iron in you and you’re ready to turn your city into the city you want to see.

Governing a city, and short of that governing particular institutions within a city, would in turn develop the essential capabilities to tackle national governance. It’s not an accident that San Francisco city politics is a key feeder into state and national politics. It’s a messy, chaotic training ground in high-stakes games that ultimately develops the right virtues for this higher level of action. But this is just one example of entrepreneurial statecraft. There are many other kinds of projects.

For example, it’s well known that something is wrong with America’s food. There is an investigative problem here to definitively map out both the science and the corruption. But the real part would be the political work organizing things to actually change. Robert Moses’s empire grew out of a parks bureau. Yours could grow out of a food quality organization. “Lord,” after all, is a contraction of “loaf ward”—there is a deep connection between the legitimacy to govern and the ability to provide good food. There are many other possible projects with their own special challenges and leverages, too. How far they go is dependent on your energy, luck, and competence.

Entrepreneurial statecraft is a political project, but it’s not a movement. It’s not about culture. It’s a discipline. It’s about taking responsibility into your own personal hands, perhaps in collaboration with others who are doing the same. It’s about going out there and applying entrepreneurial techniques to statecraft problems. It’s about having the focus to avoid abstract, indirect cultural strategies and instead target materialist, direct action on the things you actually care about. It’s about getting out of the ghetto of bourgeois, liberal self-interest and building aristocratic coordination mechanisms with your comrades. It’s about seeing how monumental historical deeds of eternal fame can start from humble work on much smaller problems. If you’ve quit your job to explore, it’s a good thing to do with your life.

The discipline of rocket science itself was the activity that laid the foundation of people, expertise, and dreams that made the moon landing possible. Likewise, the discipline of entrepreneurial statecraft itself is what lays the foundation of the new class that will make up a healthy new regime. The way to build muscles is by using them. You exercise your governance muscles by governing. Get strong enough, and you’ll find work on the biggest problems.

As a lifelong visionary and an adventurer with a unique mix of scientific education, Army officer experience, and classy pedigree, Brand participated in many of the key scenes and events of late twentieth-century American culture.

More recently, he is known for cofounding the Long Now Foundation, which has built a clock inside a mountain designed to run for 10,000 years to promote long-term thinking, and the Revive & Restore project, which is applying genetic technology to wildlife conservation, ideally including the resurrection of the woolly mammoth.

Stewart often warns that “we are as gods and have to get good at it.” Humanity has become a planetary force with an emerging planetary consciousness. What kind of civilizations emerge from taking that seriously? We called him up at his library north of San Francisco to discuss the future of man’s dealings with nature:

Wolf: We’ve recently been discussing where society ends and nature begins. How should we relate to ecosystem services and the ecosystem balance as parts of our industrial and social order?

Stewart: The only two magazines that I get anymore—I’ve got rid of everything else—are Palladium and Noema, Berggruen’s thing. Noema had this campaign—they interviewed Jerry Brown and me about it a while back—where they said, look, we’ve been thinking globally for a while now. And there’s some things for which it’s been great, when the global economy makes all the cities grow and we get out of poverty. But there’s also the planet, and the term “planetary economy” has no meaning. The dynamics of the planet are of a different order. And I think what you’re talking about here is getting those two whole-system bodies of thought to engage each other rigorously.

We used to talk about EGG: Emerging Global Governance. And the realization that, you know, it’s not done with black helicopters. It’s done with all the economists talking to each other and having a global economy to sort of share a frame of reference, and a body of economic thought to share thoughts about. Now that they’re using models instead of equations, they’re getting a little more predictive for a change.

That happened because of the Santa Fe Institute, where I served on the board for 14 years. The Institute was founded by bored nuclear physicists at Los Alamos. It’s often the case that biology moves forward when physicists get bored and decide that they can take their rigorous thinking and deploy it against biology. Sometimes that works. Mostly it doesn’t. But it worked to the extent that they were recognizing complex adaptive systems as being of a different order than those that had been studied before. So they reinvented cybernetics with computers. With the computers, they could do modeling. 

They got a bunch of economists and physicists together—basically their friends—and asked the question in a series of sit-down meetings in Santa Fe: “Is there anything we can talk about together? What we need to do first is become familiar with each other’s discipline enough to be able to talk at all.” So they flipped a coin to see who goes first—economists or physicists—and the economists went first. They had a Nobel Prize-winning guy (Ken Arrow) and some others in the group. They were really good economists. They got up and went to the board and started writing out the equations that were their current understanding of how you think economically. The physicists kind of glanced at one another. Finally, one of them raised their hand and asked, “What are you doing?” And the economists replied, “We’re writing the equations; we learned that from you!” And the physicist said, “We haven’t used equations in years.” And the economists said, “What do you use? You’re still doing physics, right?” And the physicists answered, “Well, we use models, don’t you?”

And what finally happened with the Santa Fe Institute is the economists said, “Okay, prove it. Model us some economics.” They came up with a concept which they called Bounded Rationality, which is what humans actually operate on. Most of economics up to that point had been based on “what would the rational person do?” and on a theory of games consisting of rational players, and so on. Then, they began to model large quantities of these creatures that had imperfect knowledge, or were doing the best they could with it. This was called agent-based modeling. And each of these players operated either on a different theory of the world, which we all do, or on different information that they had access to. How did they behave under those circumstances? The answer is that it’s a complex adaptive system. And there’s all of what goes on with chaos theory, like strange attractors and all of that—profound unpredictability in most cases. This then gave them the beginnings of a swing in economics actually, which flipped from witchcraft to become more predictive because they started to behave like weather forecasters.

This is Philip Tetlock’s contribution to intellectual thought: forcing people to make falsifiable predictions and state their degree of confidence in each prediction about what is going to happen and by when. You get the superforecasters who learned to do that and then learned from studying, basically, the video of the game. “How did that guy get by me on the basketball court? He did this other thing I’ve never seen before. Well, I’ve seen it now. Now it won’t happen again.” And so game two is different from game one. And that’s, ideally, what you can begin to do.

That’s why my wife, Ryan Phelan, who runs Revive & Restore, came up with the concept of “Intended Consequences,” which she immediately turned into a workshop and then into a paper. When we’re doing any kind of genetic work with wildlife that would lead to some kind of intervention on the ground, people would say, “You can’t do that because ecology is very complicated, and unpredictable. And you know, there will be unintended consequences, and you’ll be really sorry.”

So, it turns out nobody’s ever done the research to find out if that’s what actually happened with conservation interventions. All they knew about was things like cane toads in Australia, which was not done by conservation scientists. At the time, the scientists said, “Don’t do it! You will be really sorry!” And the agriculturists went ahead and did it anyway. And likewise, there were species introduced in Hawaii. Basically anytime you introduce anything larger than a bug to an island, it’s going to have deleterious consequences. That’s not so true on continents, by the way.

Anyway, some of our scientists did the research. There’s been about a thousand cases of intervention in the United States by wildlife conservationists where a species is introduced or reintroduced. And the “golden boy” story is the wolves of Yellowstone Park, or the beavers in Scotland, and so on. They’ve been gone for 400 years. They came back and immediately did good for everybody. So the question for these thousand interventions is, first, do they cause any harm? Because at the time, people often ran out of money fighting for the permission to try something that they knew damn well was going to work. Some things were surprising, but they didn’t matter. They were neutral. And sometimes the species didn’t take. So you’d say, well, it was a failure. But it’s not! It’s just that nothing bad happened, but no good thing happened either. But then, if they had the money, they would go back with the lessons learned from the first reintroduction and do it better the next time. And then even better the time after that. And by then, the animals are starting to learn how to behave in the environment, too. So they’ve got their own story going again. And that’s a success.

Nobody pays any attention when these interventions tail off into a success. It’s failures that are news. But we discovered that when it’s professional biologists doing the work, they don’t cause problems. There was one introduction in the southeastern United States which apparently led to the extinction of a local fish in the stream. It was just a local fish. And so the harm was local, and minimal. And only that one case!

Matt: I wonder if there’s some over-sensitivity in the biology world because there’s historical experience with pathogens. Today we could say, look at the dangers of gain-of-function research. But maybe those should be considered a totally different category.

I think it does. It’s mainly in medicine. Really only in the last century has it become worth your time to go to a doctor. From 1920 on, it started to become a slightly better bet to go to the doctor. Now, it’s a much better bet. And that has happened with an organism which we do not understand completely at all. We understand a whole lot. We can draw a map that is the size of the wall behind me of just the metabolic pathways, not only in humans but in any large animal. And so, are you going to have unintended consequences when you meddle in that? Yeah. But then you develop double-blind, phase-three research and whatnot to head off the possible harm. And by the time you finally deploy something, you have a lot of confidence it’s not going to cause any harm and it looks like it might cause some good. 

We are in the process of figuring out how to do that at a planetary scale. That’s what geoengineering is largely about. The best researchers, like David Keith at Harvard, are completely doing incrementalist research—way smaller than a volcano. I mean, the main information we have about solar geoengineering, so-called reflective stuff in the stratosphere, comes from the Mount Pinatubo eruption, and others which are way bigger than anything that humans would do. But we want to be able to do it for a while, and actually stop the temperature rise and even bring it down a bit. That way, we buy time to do all the other stuff so that climate is not an issue anymore. That will take one to two hundred years.

So anyway, I came to all that because of complex adaptive systems. You have to adopt the understanding that they are, in some senses, intractable to define in a way where you know for sure that it is going to do what you had intended. You never have complete enough knowledge to know that until you experiment in a number of small and illuminating ways. But once you do that, you can actually manage these very complex adaptive systems. And that’s what humans are taking on in this century, I think.

Wolf: So then, our big picture is this idea of complex adaptive systems in the context of the relationship between industrial civilization and our natural environment. A lot of people see human civilization and nature as opposed. Do you think we can find a more symbiotic relationship? That seems to be a theme in your work. What do you think that looks like?

I think it’s actually useful to keep the distinction between human activities and nature. I think that both nature and human activities in this century are in the process of learning to blend in with each other. But I think the distinction is actually helpful for doing that. What we do with intention, what we do with abstraction, what we do with language, and so-called culture involves a lot of Darwinian sorting. But it’s not Darwinian evolution in the sense that ideas have to individually reproduce in some fashion in order to carry on. We have broadcast capabilities that nature does not, for example.

What I know as a conservationist and from working with conservation biologists is that the rate of evolution has generally gone up since humans started having large impacts on all the kinds of environments that various creatures, plants, and microbiota live in. There’s this very good book, Inheritors of the Earth by Chris Thomas, that documents this increased rate of evolution. More and more, people are seeing wildlife in town: raccoons, foxes, coyotes in California. Everybody that shares a biome spends a lot of their time evolving in relation to one other.

And that’s coevolution, which is a profound concept. One of the most cited papers in all of biology is by Peter Raven and Paul Ehrlich, who was an advisor of mine at Stanford. He said it was a great paper to write with Peter because they didn’t have to do anything. They just had a couple of cups of coffee together, then reported on what they were finding in their research.

Coevolution is a profound concept that, I think, scales. It scales up from microorganisms to the planetary scale. Some of it is expressed in the Gaia hypothesis. Jim Lovelock and Lynn Margulis came up with that long ago and I got to print an early version of it. Actually, the Gaia hypothesis itself opened up a whole new avenue of nature and humanity, in a sense, taking each other seriously, and gave us the scale and scope to think about human activities having a planetary-scale impact. Gaia is different after humanity shows up and takes dominion. And presumably, Gaia never had a model of herself; the planetary self-regulating system got there by evolution, got there without self-consciousness, and got there without a model. But humans generate models all the time, we can’t help it. And the Gaia concept was a wonderfully inclusive model of engaging nature at planetary scale.

Wolf: One way that humans and nature then fit together, in a way that’s going to change both, is that we’re now in this position of modeling these planetary effects and the planetary ecosystem. And so we have to learn how to garden that ecosystem. And we have to learn how it affects us and how to integrate those things well.

A lot of this is kind of like medicine. And gardening in some respects, because of the “do no harm” injunction. In a way, the part humans are sorting out is moving beyond conquering nature. Danny Hillis made this point a while ago when we were starting the Long Now Foundation, that humanity’s story, up until pretty recently, has basically been humans learning to conquer nature, to master nature, to whip it to our aims. And that story is complete enough that we can move into the next mode, which is basically conquering ourselves in relation to nature and realizing that, while we can do everything, we should not do everything—we should only do some things and stop doing some things that cause undue harm.

And that’s part of why the idea and practice of de-extinction appealed to me, because it’s kind of a new level of thinking through undoing harm. We’re not going to bring back dinosaurs, but we can bring back the thylacine, the Tasmanian tiger. That one is well along right now. And it would be nice to have an apex predator back in Tasmania and in the parts of Australia that used to have thylacines. It’s not as crucial as something like a woolly mammoth, where we really need to get the mammoth steppe fully back, to help stabilize the climate over the long term.

Wolf: What distinguishes a reality-based framing on a problem like this as opposed to other ways of thinking about it? This is a concept you’ve used previously. Why is it important?

A guy named Rapoport said something that I quoted in the beginning of one of the books I did: “Knowledge will never replace respect in human dealings with nature.” You can know a lot, and generally the more you know, the better, but you’re never going to know it all. And so it’s about a certain amount of modesty. If you can’t manage humility, modesty will do. In the face of complex systems, it’s actually a prudent and usually necessary approach if you’re doing anything that is going to stay useful. You can make big changes, but then they take on lives of their own. Some of the ways they go may not be what you wish would have happened. The models are always evolving. They’re evolving in terms of your ability to sense things, like getting knowledge of oceans at depth, for example. They evolve in relation to notions that people have of how to put ideas together in a way that’s effective in the world.

Science is about organized skepticism to determine what you keep having to come back to, because it’s the part that won’t change in relation to your opinions. And we’re sort of missing that in the public discourse right now: the idea that you can make up the world that you think the way you think is convenient, and then act on that. And that’s as important as what scientists turn up. It’s good that people got postmodernism and all that other stuff. Science is embedded in scientists who are human and fallible. But science takes account of that, in a way that strictly postmodern thinking doesn’t really allow for. There’s been this fashion these past years, maybe decades, to think of everything in terms of power relationships—gender, colonialism, and “late capitalism,” whatever that is, when those terms are not actually appropriate. It’s a simplistic filtering of experience and knowledge that you always want to bear in mind, but not fall completely into, where you can’t see anything else.

Wolf: You’ve mentioned that ecosystems and human society are complex adaptive systems. Can you say more about how we should approach those kinds of systems in theory and practice?

Back in the 1990s, climate scientists were getting worked up about what was happening with the Keeling Curve, that we were going to have way too much CO2 and methane, and that the atmosphere was going to heat up badly. And so they said, quite rightly, that we’ve got to mitigate the greenhouse gas getting into the atmosphere, which means we’ve got to mitigate the use of fossil fuels for energy. And then a guy named Jesse Ausubel, at Rockefeller University in New York, came up with a paper saying, absolutely, we need to do all that. And we’re probably not going to be able to do all of it as fast as we need to, just because there’s so much friction in the system. And so we need to think about adapting to temperature rises that are going to happen anyway. This was treated as a moral hazard violation [that adaptation would reduce the incentive to do the “right” thing, which is to mitigate fossil fuel use]. Efforts were made to destroy Jesse’s reputation. And so, adaptation was a taboo line of research and discussion for a decade or two. Now, could you have known at the time that this was actually a stupid thing to do?

Wolf: It seems pretty straightforward that you don’t want to shut down some line of questioning just because it might slightly take the pressure off some other important thing. There’s also a big dosing of bad faith in those things. There’s people who get attached to the idea of reducing fossil fuel usage, and bring in other unrelated motivations. It may be that for them, it’s not about global warming, it’s actually that they don’t like industrial society. And then anyone who’s providing solutions to the original problem becomes an enemy.

This is true of various kinds of intellectuals, including me, I’m sure. And this happened with my old teacher, Paul Ehrlich. He is a brilliant population biologist. He’s not a brilliant human population biologist, however, and so he actually became part of the problem by imagining that the authority with which he could speak about butterfly populations and co-evolution situations could be directly applied to human populations. And with his sense of humor, which is great, he became very persuasive on the Johnny Carson Show, and so on. And people who actually studied human demographics had a completely different view that was basically suppressed by Paul and my fellow environmentalists of that time— including me, by the way.

Wolf: This was the Population Bomb theory that human overpopulation was going to consume all the resources and cause a collapse?

Yeah, the Population Bomb theory probably appealed to us for a number of reasons. One of them is that it had this kind of algorithmic clarity. We weren’t looking at real data the way human demographers were at that time. And those demographers were seeing this thing in response to what they saw in the world, not in response to theory, which got named the “demographic transition.” It turned out that with the prospect of prosperity when women moved from the village to the city, they immediately stopped having seven children and had two instead. And then they would get them educated. So they’re trading seven ignorant children for two educated children. This is a good deal, and clear thinking by the women. And that is one of the mechanisms by which the demographic transition actually happens. But we still have people screaming about overpopulation.

Matt: There is this Weberian distinction between science and politics as vocations, that these are two fundamentally different domains that require very different types of thinking. They select for different types of individuals. If you look at the history of the twentieth century, did the fact that Oppenheimer was pivotal in the creation of the bomb give him some special privilege? Should the atomic scientists themselves have a special privilege to talk about the use of nuclear weapons, or is that actually out of their domain of expertise? This question of the relationship between sciences, and between science and politics, seems to be at the center of so many of our controversies.

On that particular subject, we saw the whole idea of nuclear winter. Friends of mine, like Carl Sagan, bought into it, and the JASONs [scientific groups that advised the U.S. government] looked at it. Freeman Dyson was part of the JASONs at that time, and they could not make sense of the numbers. And over a few years, some scientists did put in some time to try and understand it. They came up with, at most, a “nuclear autumn.” And that’s part of why Freeman then was dubious about climate change, because he had been through that thing with the nuclear winter exaggeration. And he feared that there was exaggeration going on with the climate. And there is, by the way, a lot of exaggeration in relation to climate. That is not to say it isn’t going on in the biggest way. But the main event gets put in apocalyptic terms, and I think that’s completely misplaced.

And then that shifts over into another discussion: is apocalyptic thinking good for anything whatsoever, other than excitement? My girlfriend in the 70s was Stephanie Mills, the woman who got up in front of the Mills College and was valedictorian or whatever she was, and said she was not going to have children, because the world is overpopulated. And it would be irresponsible of her to think about having children. To her credit, she never did. But it was actually a goofy thing that my generation got up to. It was based on nothing, it turned out.

Matt: You’re saying, it wasn’t just a pose or a theory to believe in overpopulation, people really changed how they lived.

Yes. And they are now! There are people figuring “I’m not going to have children because it’s going to be a hot world, and all these other bad things are going to happen.” And therefore, don’t reproduce. Part of it is, I think, a scale error. Maybe it’s even a pacing error of some kind.

David MacKay, who was Chief Scientific Advisor to the British government, wrote a great book on global warming. One of the things he addressed is people who are imagining that their personal behavior is going to save the world, like people who will unplug their phone charger when it’s not actually charging their phone. As if that was anything other than performative virtue signaling. He said, “Let’s look at the actual energy involved here. And then I’ll compare that to one bathtub that this person takes and the heating that was involved with getting the waters to the right temperature they like.” And how many mega-years of not having your phone charger plugged in does it take to match one bathtub? And so just keeping the scale of things in mind helps a lot.

So there’s over-personalizing stuff. “Live the change you want” is, in some respects, a really valid idea. But not in all respects, I think. It reminds me a little bit of landfills back in the day. In the 1970s, we got all worked up about landfills. And then this guy did a book that looked into what actually goes into the landfill. Is it those terrible burger-enclosing plastic things that you get along with your burger at a fast food joint?

Or is it building materials? Turns out, it’s mainly building materials in the landfills. It’s buildings and all the spare parts, all the spare leftover boards you can’t use after all, and all this kind of stuff. I’m becoming somewhat of a fan of landfills. I think it’s a great way to sequester carbon.

Matt: Does this mean you’re anti-recycling, Stewart?

Well, a lot of recycling is kind of like the newspaper drives during the wars, that I actually remember. In World War II, we had to go out and collect everybody’s newspapers, and they were bundled up and sent off to the government for some purpose that we never understood. And it turned out there was no purpose and actually, it was just something to mobilize citizens so they’d be part of the cause.

Wolf: It reminds me of this story in rural Canada during World War II: they had people guarding a bridge just in the middle of nowhere in the prairies, as if the Germans or whoever were going to show up.

Can you imagine how much fun that is? Actually spotting airplanes, and learning all the various airplane silhouettes and whatnot. There were aspects of World War II that were great fun, if you weren’t in combat.

Matt: All that gets to this important question about the scale at which we intervene to solve these problems. I think there’s a tendency, especially in the mass media age, to make everything about the individual: if only I were to sacrifice going on an airplane, and so would everyone else, then we would solve the problem. In fact, more often the scale has to be at the level of the state or of collective action.

I think with the climate, it’s useful to ask what the things are that actually function at the scale that you want for the result that you need—in this case, a planetary scale. And thinking through that reversed my opinion about nations, in the sense that nation-states still pretty much control their tax systems. To the extent that you want to use the tax apparatus to make coal and gasoline expensive and clean forms of energy cheap with a carbon tax, the UN can’t do that. Nonprofits can’t do that, businesses can’t do that. Only governments can pass laws and enforce them. And likewise with geoengineering, with sulfur dioxide in the stratosphere you don’t have to go to planetary scales to start finding out to what extent it can work. You do it in tiny, tiny amounts. It actually changes things in the direction that you want, without causing regional problems like stopping the monsoon— that would starve India, or cook India.

But unilateral geoengineering has been treated as the great crime that might come out of taking geoengineering seriously. Somebody might just go off and do it without asking everyone else’s permission. And this is where you get into a really interesting mode of this century, having to think through global permission for planetary actions. Do you need to get total unanimity of everyone on Earth saying, “Okay, let’s do it”? Do you need to get unanimity of all the nations? Of most of the nations? Measuring how? These kinds of things will be worked through in this century.

Wolf: To continue the question of scale of action, this also gets to sort of what time-scale we are acting on. It’s like your idea of pace layering: you want to act and learn at the appropriate pace in relation to the phenomenon, or else you risk reacting too fast or slow.

When I was studying organic chemistry for my biology degree, it felt like special cases all the way down. But that’s not actually true. Physicists like Geoffrey West at the Santa Fe Institute came along and said, well, there is a scaling effect. It is clearly the case that—they call it Kleiber’s law—that large animals have slower heartbeats and live a long time, and tiny animals have very rapid heartbeats and live a short time. It turns out that mice and elephants both have the same number of heartbeats in a lifetime. And the difference has to do with scale. Mice are small, but the network effects of being a mouse are quite different from the network effects of being an elephant.

Geoffrey West’s book Scale went on to look for Kleiber’s law in human institutions like cities. Cities, in particular, have a reverse relationship: the bigger they get, the faster they go, instead of slower. And this frightened him! At least, more than it frightened me. He believed that clearly the great thing about cities is, yeah, they create these extra problems, but because they’re faster at everything they can come up with solutions to the problems that they cause faster than they caused them. And therefore, they get bigger and more advantageous, and pretty soon you’ve got a whole urban global civilization. And then he just sort of hit some kind of interior wall, thinking “this can’t go on!”

He thought that infinite acceleration is not something that humans can live with, that it’s going to just slow down and break. So he was worried about a collapse that may come about from these cities getting bigger and bigger, faster and faster, until something breaks in a big way. I’m not persuaded by that. And maybe it’s because by training, I’m a biologist and he’s a physicist. I just see so many ameliorating Gaia-like mechanisms that biology has to manage big change.

I mean, the planet’s shift from anaerobic to aerobic life as the primary life form was a huge and seemingly catastrophic change. Certainly, if you’re an anaerobic microbe, you didn’t like it. But the great oxygenation worked out actually pretty well—so we oxygenating organisms say. You might think the anaerobic guys that have to hide down in the soil are saying, “Yeah well, fuck you! We’re down here hating you to this very moment!” But none of that happens. I’m pretty sure they don’t even know.

Wolf: You’ve been around for many decades working on all of these problems, meeting all the key people in the field and seeing how things have changed over time. What do you think are the most important lessons from these last decades that we need to be carrying forward?

I’m trying to figure out if apocalyptic thinking is actually destructive, or just a waste of time. How much longer will the term “existential threat” be something that we are constantly hearing in the high intellectual public discourse? It’s gotten so generalized, you can say it about anything now. “Is Trump an existential threat?”

Martin Rees, the former Astronomer Royal, is a friend in England. I was trying to think of the real existential threats. I thought, well, there are comets that are pretty rogue and are pretty big. We can head off asteroids pretty well now, but comets are probably not in that category. So I wrote Martin and asked him, should I be worried about a comet hurtling in and blasting into Earth? And he said that the odds of that are so close to zero that you might as well consider them to be zero.

This is a guy who came up with a bet with Steven Pinker about how there’s going to be a million people dying from bio-error or bioterror. And we thought that Steven won in 2020. But there’s still the question of COVID-19, which has killed more than a million people worldwide— was that a case of bio-error, getting out of the Wuhan lab? That’s still not decided yet. So those guys are relitigating the bet.

But anyway, Martin Rees is the one who was saying he expected a million people to die from a bioterror or bioerror. He’s not a biologist, but if it did escape from the Wuhan lab, he was right. Was that an existential threat to humanity? Not even close.

Wolf: There’s a lot of these sorts of risks that are very disruptive and cause a lot of trouble, but our ecology and our civilization are incredibly deep in their resilience. For example, the last time an asteroid big enough to matter hit Earth was 65 million years ago. That’s a long time. Like Rees says, statistically it’s not happening again any time soon. But even then, life goes on.

Even then, life goes on. Exactly. This is what I’ve sort of realized, you know, if we had a major nuclear exchange—life would go on. Differently, but mostly in recognizable ways. I did put this question up on Twitter the other day. I was thinking in the morning, pulled out the coffee—actually, I was reading Palladium—and I got thinking about civilization, thanks to you guys. And I put up this question: Individual civilizations come and go. Civilization continues. It has for five to ten thousand years. So when you think about a global civilization, does that mean that therefore it’s deeply fragile? Or deeply robust? Which lesson do you draw?

Wolf: That’s an interesting question. I don’t know the answer.

Matt: Well, while we have a global civilization at a certain level of analysis, we also have the diffusion of certain kinds of technologies, of certain ways of thinking and doing politics and economics. There’s still a lot of deep cultural distinction between, say, Japanese and Indians, or Africans and Americans and Europeans. So in a sense, that would make me optimistic about a sort of resiliency.

You have this modern, technological civilization that’s diffused among many different deeper historical cultures. These existential risk questions don’t really get resolved or mitigated by this phenomenon I’m describing. But what it might mean is that even if you had some major crisis in say, the Anglosphere or in India, it would not necessarily cascade or translate into a global catastrophe. When Rome collapsed, that culture was only one instantiation of their peak technological development. Or is that not the case now?

I’m going to go out on a limb and say that I think that global civilization is robust, for some of the reasons we just discussed. There’s a fair amount of variety, and it’s one of the things that culture does. It’s one of the things that nations do. And even Jared Diamond’s book Collapse ends with him saying that all of these prior civilizations that collapsed didn’t know about each other. But we know about all of them, right? And ourselves. That awareness means that we do not take longevity for granted, which these other civilizations did.

And so in that sense, the concept of existential risk is a good one to have out there. It’s probably not one to obsess over. Yeah, things really can end, at any scale you want to think about. But at a giant scale there’s a lot of micro-robustness built in. There’s a lot of subsidiarity built in, of these various things operating regionally, at different rates, independently of each other. Or even contrasting enough between each other in terms of pace layers that the system can handle shocks.

Wolf: You know, it reminds me a little bit of the story of the Tower of Babel, where humanity comes together and builds this big central civilization to control everything. It’s fragile and hubristic. But once you do that, all the ambition in the system is towards differentiation, and the system fragments back down to some stability. God has a plan for this, so to speak.

I guess every now and then people get experimental. They learn the hard way, but it works out.

That was one of the things going on over the PA system at Woodstock. “Don’t eat the brown acid! Brown acid is bad acid!” That was all being managed by the Hog Farm, because they were running the overdose tent, the tripping tent. And you saw kids coming in really bad. And so you sort of ask them through there, what kind of acid did you take? Oh, the brown acid. Okay, no brown acid! That’s the Ken Kesey approach to science. “If you don’t boil rocks and drink the water, how do you know it won’t make you drunk?” And I think that actually, societies at various times get into that mode of “screw it, let’s do it!” And it’s a good thing. You know, some hippies died, some went crazy and stayed crazy. But that boldness, that fearlessness made it an exciting time to be around. And because everything was illegal, everything had to go by word of mouth. And we’re talking about reality: what’s the real world that you fall back on? Personal experience is the real world that you fall back on, to some extent. And if you have avenues to transmit that, talking with the hitchhiker you picked up because they were dressed like you are, information gets around.

And then that flew in the face of the going theories! “No hope without dope!” Well actually, there’s such a thing as too much dope. I just saw that Kim Stanley Robinson, in the beginning of his wonderful book The High Sierra, is talking about how when he was dropping acid in the 1970s, he grew up in Newport Beach in Southern California. And then he went up into the mountains. And he says, “I got high and I never came down.” And he’s been in love with the High Sierras ever since. What does that say? That youthful folly has its functions for society.

At first, the rot was just chains of naturally occurring nucleic acid—or something like it—which could slightly catalyze their own replication. But once you have replication, you have evolution. Many generations later, the most successful of these self-copying strings were catalyzing and manipulating other useful processes in the surrounding soup. A crude but living agency was born into the world.

Nucleic acid replicator evolution had generated an intelligent life force, which accumulated revealed wisdom over time to master the world around it, constructing humble but purposeful order. 

This new proto-intelligent agency rapidly—by geological standards— mastered two more foundational technologies: protein nanotechnology and the lipid bilayer cell membrane. Mastery of the cell membrane allowed it to defend its own machinery from a hostile, competitive environment, establish a definite domain of influence, and attack the membranes of other proto-organisms. Mastery of protein nanotechnology allowed it to construct almost any desirable chemical machinery and apply it to problems of production. The first life thus assembled the fundamental functions of any organic order—guiding wisdom, security, and production.

It is important to note that the amino acid components of protein and the ambiphilic chemicals that self-assemble into bilayer membranes predated their incorporation into the fabric of life. Before being mobilized into living tissue by the replicators, they were just natural phenomena without higher meaning. But the crude cleverness of nucleic acid replicator evolution mastered them and put them to work as purposeful components in its living order. The foundational distinction of life is the way its components are animated by this teleology. Teleological organization comes from intelligence. The viability of early life came specifically from the fact that the replication process created a form of intelligence. It was so primitive that we can hardly recognize it, but on the planet of the still-dead, the self-replicator with even a sliver of life’s inner intelligence is king.

And so the animating principle of life brought together what had been mere phenomena and turned them into something much greater.

The Precambrian Sexual Revolution

Approximately one billion years ago, Earth’s hitherto nurturing atmosphere entered an irreversible toxic runaway process. An industrious species of new life tapped into a new and much richer source of energy and began to overtake the Earth. Cascading environmental crises soon followed: the poisonous chemical byproducts of this new life polluted the atmosphere so heavily that even the rocks of the land decayed. New forms of energy often come with a heavy cost. This was long enough ago that we have difficulty reading the sparse fossil record, but as far as we can tell almost all life perished.

But life is resilient, and always seems to come back stronger. In hindsight we know that this was all for the better; the dangerously rich new source of energy was sunlight. The toxic byproduct that poisoned the Earth was oxygen. Our disruptive predecessors were what we now call cyanobacteria, the blue-green algae whose relatives give the ocean its color and the trees the power to capture sunlight. They heralded the end of the world. But this oxygen apocalypse also ushered life into a new oxygenated golden age. Life adapted to a high-energy oxidizing chemistry, turning the green plague into a green bounty.

This vast new bounty of energy meant a vast new bounty of complexity. In diverse trophic hierarchies, many species rose and fell. Nonetheless, it all would have remained a stagnant soup of humble microbes but for the advent of a key new information technology: sex.

Prior to sex, a genome was a package deal, with only relatively chaotic means for gene-trading. If a gene became damaged by uncontrolled oxidants or transcription error, it would compromise the whole lineage. This put a cap on complexity, because complex genomes would pick up unrecoverable errors too quickly.

With sex, different individual lineages could now trade genes in an orderly way, creating new mixtures where any given gene has an even chance of not being present at all. Bad genes could be isolated and removed, and good genes could be rapidly spread around the whole pool. The average member of the species became much closer to the ideal, with more or less all the good genes and only a few bad ones. Where asexual reproduction could not sustain high genome complexity, sex unleashed it. 

But this Precambrian breeding program didn’t just boost general health and refinement—it amounted to a large increase in species-level intelligence, which meant that sexual species could run evolutionary circles around the stagnant asexual ecosystem. It didn’t have to involve brains or neurons—optimization power is intelligence wherever you find it. That intelligence became the basis for the animating force of life.

The sexual intelligence advantage created the first recognizable political hierarchy between organisms. Sexual species with a much higher and faster adaptation capacity rapidly subsumed whole parts of the asexual ecosystem into their own internal economies. Relatives of cyanobacteria became the photosynthesizing chloroplasts now present in plants. Aerobic sugar-eaters became the mitochondria that oxidize food chemicals into usable energy inside our muscles. Superior intelligence—and therefore superior vital force—tamed what were previously free organisms contending alone in an ecosystem into purposeful components of a higher order.

 Thus our early sexual ancestors emerged from the primordial soup as the master-organisms which organized lower species into their higher form of life, and assembled themselves further into advanced multicellular organisms. These were the first advanced eukaryotes—the ancestors of all plants, animals, and fungi—and the eventual basis for the Cambrian explosion of complex modern life around five hundred million years ago.

This was the intelligence-mediated incorporation of surrounding nature into the fabric of life all over again. Superior intelligence does not just incorporate inanimate natural phenomena like proteins and lipids into its order of life. It also absorbs anything it can wrap its control processes around, including any other living organisms fortunate or unfortunate enough to be useful and nearby. This has all happened before, and it will happen again.

Language Is the New Sex

Another major innovation greatly aided the Cambrian explosion: neural intelligence. With its large networks of small computational elements, the nervous system laid the foundation for arbitrarily scalable intelligence now capable of real-time control. This enabled much more complex and fast-paced multicellular lifeforms: animals. 

But animal intelligence did not become the new vital center of life itself, which even now still lives mostly in inherited biology. Despite being capable of vastly greater speed and complexity, it didn’t have persistence. The wisdom accumulated by animal intelligence dies with its individual host, so it is unable to outmaneuver and exploit surrounding life at the level of basic teleology, only active behavior. So despite the increased speed, scale, and dominance of the animal form, another true jump in the nature of life would have to wait another five hundred million years.

Through that time, life inexorably climbed the thermodynamic gradient towards greater vitality and higher consciousness—from the sluggish cold-minded fish to the now-dominant hot, quick, and clever mammal. Then, some time in the last several million years, an otherwise meek two-legged race of these mammals was given a golden opportunity: a social environment that favored brains and communication to such an extent that they developed a whole new paradigm of intelligence: language-mediated culture. This was the dawn of man. 

Language, especially poetry and myth, allowed individuals to pass down their wisdom at high fidelity to become the wisdom of the group. This meant that man could run circles around other beasts by means of a much faster accumulation of knowledge. Where evolution picks up a few bits of information every generation at best, cultured humans can build on their ancestors’ wisdom and create whole new engines of cultural machinery in a single generation. This has enabled man to domesticate other animals and plants into his own order of life, probably starting with dogs, and later livestock and crops. 

According to current consensus, the definite archeological evidence of sustained cultural accumulation begins around fifty thousand years ago, at the height of the Ice Age. Millennium after millennium, the tools we find become more and more sophisticated. This is still rather slow for purely cultural growth, indicating that genetic and material limitations still held back our ancestors. But the increasing competition and mixture between different species of humans at the end of the Ice Age slowly raised these limits. Higher genetic intelligence and material prosperity meant more complex social organization, which enabled scale, social differentiation, and caste hierarchy. Culture as a whole became more sophisticated, with innovations like writing and mathematics further accelerating cultural capacity. 

Like our photosynthetic predecessors, this self-accelerating ecosystem of culture and industry has recently led us to crack open vast new repositories of energy: fossil fuels and nuclear power. Like previously, this energy bounty also led to a rapid increase in the effective intelligence of the cultural sphere. This new intelligence is enabling further rapid growth in the sophistication of overall life, including industry, and further growth of the cultural sphere in particular. 

The most robust repository of tradition—and thus the repository of the most fundamental soul of life—is still the inherited biology of the species. But as the power, memory, and stability of the cultural sphere increases, we see culture contributing more and more to the fundamental nature of life. Enabled by widespread written language, the fall of classical civilization left enormous cultural legacies that reached far beyond its genetic success. Western civilization’s cultural legacy will be even greater, if it even collapses at all. In the long run, we can expect culture to increase its capacity for the long-term accumulation of wisdom, increase its capacity for the direct manipulation of genetic information, and take over as the dominant vehicle for the intelligence of life itself. Language is becoming the new sex.

Garden Empires Against Babel

If past transitions in the form of intelligence are any guide as to what is going to happen next, then the implication is straightforward. Human and perhaps post-human cultural intelligence will become the new animating force of life. Inevitably, it will gain mastery over the ecosystems and processes of its former environment, and incorporate them into a new order of life. 

The end result of this transition will be a new kind of meta-organism above both biological life and machines, encompassing and unifying what we separately call ecosystems, civilizations, peoples, cultures, states, and industrial stacks. These garden empires will increasingly blur such distinctions as peoples become inseparable from the symbiotic industrial and biological ecosystems they maintain as their bases of power. As the power and impact of civilization increases, nearly all aspects of what is now the environment will be internalized as its internal organic processes.

For example, as with our photosynthetic predecessors, scaling up our energy economy with fossil fuels has destroyed our comfortable post-Ice Age climate equilibrium. Unlike previously, life may now have the power and intelligence to simply solve such problems by directly geo-engineering the stable climate that we prefer. Thus the climate itself would become internal to the teleological fabric of life. The atmosphere would no longer be part of the environment, but yet another organic fluid-like cellular cytoplasm or blood plasma. The climate will not be the only process subsumed in this manner. 

Unlike replicator evolution and animal intelligence, cultural intelligence has no immediate technical scaling limit. When it is freed from technical constraint, the fundamental tendency of complex order is to grow and integrate. Trade globalizes. Power centralizes. Intelligence organizes. So we might imagine that the outcome of this transition will be a unified whole Earth as a single planetary meta-organism: the Gaia hypothesis made real. This is Pierre Teilhard de Chardin’s noosphere, the global consciousness that grows out of and unifies the biosphere. 

Influenced by such ideas, our twentieth-century predecessors imagined that planetary environmental problems meant planetary governance solutions. Excited at the prospect of being able to fulfill the enlightenment ideal of a rationalistic universal polity, they used global environmental issues and arguments about globalization to justify their political projects. 

But despite the lack of technical barriers to this unification, political unity only happens at the expense of the ambitions of its constituents. It is only ever justified by the presence of some even more pressing external need. In the absence of an outside threat, the tendency for all internal suborders is to push apart and cannibalize the political commons for their particular gain. This has been the eventual fate of all attempted universal empires in history and myth all the way back to the Tower of Babel. 

So despite the centralizing tendency, the most natural outcome for this new order of life is not full unification but a diverse patchwork or multipolar balance of powers that only partially cooperate, and mostly compete within the shared noosphere. This is fortunate in a way, because all individual organisms are mortal. A diversity of powers is necessary for life to survive the inevitable rise and fall of particular powers. Full unification is not only impossible; it would be deeply dangerous to the future of life.

Impossibility won’t stop us from trying. Particular garden empires will succeed in overrunning their competition and then the whole Earth. But they will always fragment back down, just as the attempted universal liberal order is now doing. The ability to defend one’s own preferred ecological order against this regular tide of empires will become key, as will the ability to gracefully break up into child civilizations when one achieves hegemony. This cycle of growth and struggle for space against stifling homogeneity will be the key grounding reality for future garden empires, and the driver of future evolution of life.

This transition is nowhere near complete. We have barely even begun to take the responsibility to garden the natural processes of the world. But we have definitively started. As the false unity of the current environmental moment fades away and theoretical sustainability problems become national security issues, the competitions of the future will increasingly be between advanced garden empires with their own sacred ideas for the ordering of life.

The Role of Man

In the twenty-first century, real environmental crises will exploit the differences between our feel-good rhetoric and hard necessity. States will be forced by mounting circumstance to drop the taboo against climate adaptation and the explicit national self-interest it entails. Political crises will compound this, dissolving globalist ambitions into competing spheres of influence that must increasingly handle their own environmental affairs. We may still pull off a stabilizing planetary geoengineering, but it will be late, painful, and stripped of political ambition.

As they adapt to an increasingly harsh world, some powers will double down on particular existing ecological-industrial orders like the Green Revolution. Others will attempt neo-ecological regenerative approaches, or acceleration into the technosphere. These may combine in strange ways: an organic permaculture forest colonization of the Sahara, fed by mined fertilizer and watered by nuclear-powered desalination plants, or large-scale ocean fertilization and cultivation. We can only speculate on the details. 

The garden planet we are creating will not look much like the wild planet that created us. As with anything optimized to natural law, the new world will still be beautiful. However, it need not be so in any way that suits us. The transition process may grind us down to bleak semi-life, along with everything beautiful and green that we care about, before delivering anything that works—especially if undertaken by trial and error. How much of nature’s current wealth and beauty will remain to be nurtured after this transition depends on our foresight and strength of will in approaching it. If we thoughtlessly bumble through it, we will eventually learn but only after great loss. Approached with the higher consciousness demanded by our niche as the intelligent vanguard of a new order of life, our new garden planet can build on the strongest themes of wild nature and surpass it in beauty and wealth.

Our big-picture environmental situation is not just a small issue of externalities or competing value systems. It is not just about rising temperatures or marginal health years. It’s not even about merely human ideals of beauty and justice. It’s much bigger than that. We are undergoing a major transition in the nature of life itself, rivaling the original event of the creation of life in its centrality to cosmic history. 

As in previous transitions, the holistic empires of this new world order will be ruled by highly intelligent, comparatively god-like beings distinguished above all by their practical wisdom, strategic foresight, and political will. These are, after all, the virtues that enable a small but vigorous intelligence to conquer and subsume its surrounding ecosystem. Their ideas of how things must be will decide the particulars. This is what we learn from the history of life, applied to the transition we find ourselves undertaking. This is man’s niche in the world—if we can keep it.

By 2013, TempleOS was mature and was attaining legendary status among programmers and internet lurkers. Few had ever done anything like it: a complete operating system built from the ground up by one man, complete with games, apps, text editor, command line, windowing system, and an oracle system for listening to God, using random numbers for a sort of augury.

TempleOS had interesting new ideas about how different kinds of data like text, images, and 3D models could be freely mixed at a low level, and how the programming environment mixed shell commands and programs, but it was a very limited system overall. It did not end up being a practical system, nor even all that promising for future development. But it struck a nerve, because, despite its impracticality, it had two extremely important ideas:

Firstly, the existence of TempleOS asserted the possibility of new operating systems. Everyone implicitly assumed that it was too big a task for an inspired hobbyist to undertake. We all assumed that the operating systems of the future would have to be built by armies of corporate software engineers. Terry Davis proved all that wrong. TempleOS served as sort of a grand artistic manifesto, shocking us out of our preconceptions and opening up a whole new space of possibility.

Secondly, TempleOS asserted that computing is not separable from religion. For Terry, the computer was a space of spiritual contemplation, a temple. The computer programmer was not a craftsman or a merchant. They were a priest, a prophet, and a monk. For Terry Davis, computing was a sacred activity. The computer was a device for structuring the spirit, consulting with and making offerings to God, and living as God instructed.

The first idea has been discussed and explored in the computing press for a decade now. Many have been inspired. The second has slipped through unnoticed.

Terry was born to a good family, in which he was raised as well as anyone can be in America, and got his master’s degree in electrical engineering in 1994. But by 1996, he was struggling with bouts of mania and schizophrenia. He struggled to maintain his grip on reality, follow social and political norms, and keep his life in order. When he received his divine task, it gave him stability through the period of his labor, but when he was finished, he lost direction. In one of his later streamed video messages, he spoke of this: “I made God’s temple and now I’m waiting for something to happen.”

In 2018, Terry had become homeless, refusing many of the offerings of his disciples. His last video message to the world, on August 11th, 2018, was about how he was learning to purify himself and respect the purity of the world around him. He concluded with a frank summary of who he had become now that he had finished his work: “maybe [now] I’m just a bizarre little person who walks back and forth.”

Later that day, while walking back and forth on the train tracks in Oregon, Terry Davis was struck by a train and killed. God had used him for a great task and then had taken him home.

TempleOS itself will not be the foundation of a new computational-spiritual order that Terry believed it would be. It’s too limited as a system, and Terry lacked the moral purity to do it right; his videos were frequently interspersed with bitterness, pride, and obscenities. But God works in mysterious ways. As Terry himself illustrated with his parable of the bird looking at a computer screen, mere mortals can’t really understand the full context we live in. “You’re that bird looking at the monitor, and you’re thinking to yourself, ‘I can figure this out’. And maybe you have some bird ideas. Maybe that’s the best you can do.”

Terry Davis had some bird ideas, and the best he could do was pretty impressive. The Divine Plan does not rely on its agents being perfect or knowing the true significance of their work. On his biggest idea about the relationship between computing and religion, Terry Davis was a prophet, and he was absolutely right. TempleOS may not directly bring about the new spiritual order, but it will inspire the system that does.

Religion Is Computing, Computing Is Religion

Terry’s big idea was that it is not possible to separate religion from computing.

Since the beginning, every complex society has been based on a shared socio-cognitive operating system, a religion. Your religion says you must live a certain way. It tells you what is valuable and what is disgusting. It gives you your fundamental paradigm and ontology of belief through which you interpret the world. It tells you to read certain texts, listen to certain traditions of wise men, and meet with like-minded others at certain places and times. It tells you to avoid certain influences, and seek out others. It gives you techniques for structuring thought and mind and memory. When you have doubts, it gives you procedures to ask for and receive insight. It tells you who you are, and what your life is for.

That is, religion structures your information environment and cognition on a very practical level. But it also structures your interactions with others. It provides a system of law, shared ethics, a shared mythic vocabulary, and language. It provides your notions of friendship, your ways of doing business, and your ways of forming a family or not. It keeps you and your co-religionists operating on the same frequency, able to understand, trust, and work with each other. That is, your religion is the fundamental information fabric, the soul, of your society.

Historically, big changes in information technology were closely related to big changes in religion. This is not surprising, as they are nearly the same thing.

Man’s primordial religion was transmitted as sung prayer and memorized myths, locally rooted ritual practice, and sacred memory techniques. When ziggurats full of specialized expert priests came along, religion changed, because the cognitive structure of human life had changed. When writing came, the sacred written word and the compiled bible of sacred history became the new cognitive foundation, the new information system, and the new religion. With the mass adoption of the printing press, it became possible for everyone to have and read a bible. Many other books and pamphlets, and a whole system of vernacular public discourse, could compete with and replace the learned scholastic hierarchy.

These weren’t just philosophical changes, but changes in the architecture of thought. They weren’t just secular implementation details shifting under a fixed religious worldview, but necessarily deep religious conversions. Religion and the social cognitive system are not separable.

In the modern electronic age, our information architecture has changed again and again. From mass print to telegraph, radio, television, and internet, whatever gods we once worshiped we now worship by entirely different means. As the dust begins to settle on the electronic age, one system of information architecture is now rising above all others: the networked computer.

You may think you’re a Christian, an atheist, a Jew, or a Buddhist, but you are wrong. You are a follower of this modern American information regime built on the networked computer. You have ritualistically sacrificed far more in the last year to it than you have to whatever dead gods you claim to worship. Where you sacrifice reveals your true religion. The architecture and features of this new socio-cognitive substrate will be the essential form of our future spiritual life.

Terry Davis was right: the next temple, church, and bible is a computer. The computing systems of the future will be religions. The religions of the future will be computing systems. The only way to solve our social and spiritual problems is with a new computer operating system. But designing a new computer system and converting its followers is not just a matter of mundane engineering. It is a matter of culture. It is a matter of divine revelation.

Summa Contra Metaverse

The telos and highest expression of our current computational culture is the metaverse, a digital synthesis of television and Disneyland. The metaverse is characterized by the immersive managed experience of the app, the video game, the social media timeline, the performative identity, and the ad (propaganda) funded media (propaganda) outlet. A distant technical elite curates and remote-controls the experience, attention, and agency of the user, who is not trusted to even understand what a computer is. Maybe the user does an email job work in the simulated office with simulated tools, or maybe they just collect UBI. The metaverse isn’t sure yet, if these are even distinct at all.

The metaverse is marketed by the company formerly known as Facebook (now Meta) as a sort of inevitable future from which there is no escape. Real life is collapsing, the computational borg is more and more omniscient, and someday soon life will get too hard and you will escape to the brightly colored digital promised land.

And indeed, the entire computational and informational architecture of our society has been trending this way for hundreds of years. The whole notion of public discourse and romantic escapism as we understand it arguably took root in the eighteenth or nineteenth century with mass-market print. In the twentieth century, when the novel, pamphlet, and newspaper gave partial way to the radio broadcast and television show, the soul of this nascent managed experience regime was only strengthened. In the twenty-first century, the same has now happened with the internet and especially the web, and is planned for virtual reality. The web became an increasingly centralized, siloed, and moderated experience machine, losing its initial humanistic freedom to become another hollowed-out arm of the burgeoning metaverse.

Our information environment and space of action overall is increasingly owned and curated by an expert elite for political and commercial effect, with enough gated access to productivity, society, and bureaucracy to keep us dependent and unable to unplug. In concrete terms, most of what it does is deliver a steady stream of pornography, entertaining memes, political slogans, and commercial advertisements. There is no health in it, nor in us who worship at its altar.

The metaverse tendency outcompetes current alternatives but strips human agency out of the system. Without agency over the world in which we live, people cannot develop the forward-looking spirit that is necessary for growth and life. A universal of stagnant and declining cultures is that ambition, creativity, and spiritedness are crushed into tracked sandboxes where they are no longer dangerous. Power is dangerous, but without it, the human spirit fades into decadent self-medication. Biological and social reproduction cease. We can see this in the cultural decadence and cratering fertility numbers, and the metaverse promises only to further disempower the human user. We can be sure that it will eventually fail by burning out its own base, taking us with it if we let it.

If we are going to survive the century, biologically or socially, we’re going to need a new computing system that is the opposite of the metaverse. The central problem for such a system is how to empower the user as much as possible.

The Consensus of the Prophets

TempleOS declared some interesting ideas against the computational Antichrist but failed to provide a viable alternative. Fortunately, Terry wasn’t the only one who has tried to build a new kind of computing.

Throughout the years, even as far back as the 1970s, there has been a loose underground tradition of dissident computing. They represent a higher spiritual path against the vulgar agency-sapping commercialism of the mainstream. The free software movement is perhaps the most overt and well-known in this tendency, but many prophets have quietly received their own revelations, and have been laying the groundwork for the true temple OS.

The opposite of the managed app experience, and thus the basis for a true temple OS, is general-purpose computing: the user is given a composable and programmable system of tools instead of limited non-composable appliances.

The main interface of Terry’s TempleOS was general-purpose computing, as was the original Unix command line and almost all computers up to the advent of the graphical user interface and the web. Graphical interfaces have no inherent difficulty with general-purpose computing, but it was relatively uncreative and counter-interested commercial software providers who managed the transition. Historical inertia and the larger forces of political economy that oppose user agency have ruled since then. The dream of the empowered user is now only present in the underground of spiritually enlightened computing.

In the late 2000s and early 2010s, a German software collective called suckless.org declared their opposition to the dominant programming philosophy’s tendency to create overly complex bloated apps—software that sucks. Instead, they built lean single-purpose programs that played well with the system and centered the user’s technical agency.

Suckless programs like dwm were designed to be so simple that editing the source code was the primary user interface for configuration. Their software pushed the user to level up their technical agency and take control of their computing, escaping the trap of the managed experience. In return, the user got lean highly productive software and a spirit nourished with real agency. Suckless never grew much beyond the community of hobbyist Linux power users, but their powerful example was influential far beyond their immediate user base.

Suckless themselves were inspired by the original Unix philosophy and its even more obscure and spiritually pure cousin from the 1990s, Plan 9 From Bell Labs. Plan 9 took the concept of the filesystem, and instead of hiding it away, it supercharged it as the primary system API, networking paradigm, and operating interface. Plan 9 never attained large-scale success either, but again it demonstrated the power of giving the user access to powerful core abstractions and general-purpose computing instead of managed consumer apps.

Since 2013, Curtis Yarvin’s Urbit project has been building out a full-stack alternative to computing and the internet as we know it. By building social identity into the fundamental network level, a huge swath of the complexity of modern computing is just gone. With an innovative basic computing stack designed to be as simple and reliable as possible, the Urbit computer aims to be fully understandable and owned by the user. Again the ethos of empowering the user and bringing a powerful technical capability directly to the user pervades the system.

In 2018, Bret Victor launched DynamicLand, a new user interface paradigm for humanistic computing that is the opposite of apps and the opposite of strapping a screen to your face. Instead of pictures under glass, projectors and cameras in the ceiling bring an entire room to life. Every piece of paper becomes lit up with a programmable screen. You can arrange elements of the user interface as fluently as you arrange physical objects because that’s all they are. You can draw and move physical objects around to control the computer, and plug a keyboard into anything by proximity to edit its code.

The objective of Bret’s project is to make computer programs so easy to write and manipulate that they become an intuitive and ubiquitous medium of human communication and thought just like language. Again the aim is to demolish the expert-user class distinction and put highly accessible general-purpose computing in the hands of the user.

In 2019, a pseudonymous hacker called Solderpunk launched the Gemini network, a much simpler anti-commercial alternative to the web. Gemini has less than 1% of the complexity of the web, but can still do the stuff that matters: delivering simple hypertext documents. Further, it is designed deliberately to never be able to do the things enlightened users generally don’t want, like tracking or complex scripting on what should just be documents. The web itself has become so complex that there may never be another web browser written from scratch, but Gemini was designed to be simple enough that any reasonably competent programmer could write their own Gemini browser in a weekend, so there are already dozens.

Gemini has exploded into a thriving scene of blogs and social media, proving that it is possible for any reasonably competent tech-prophet to overcome the network effects of the metaverse borg with their own niche alternative. It further demonstrates an exemplary case of the higher intentions of the software architect attempting to structurally prevent later commercial adulteration.

These projects are just the tip of the iceberg, each one embodying its own special revelations on the nature of a new computing. The most important common thread in all of them is empowering the technical user over the professional software provider. It’s not necessarily obvious why this is important for a healthy political and spiritual economy, but the unanimity of the prophets is clear: thou shalt empower the user with general-purpose computing abstractions, and relatively disempower the app-builder.

Shifting the relative power against the experience manager and towards the technical user means that every little abuse and theft of agency becomes much more difficult or impossible. The user can do whatever they want, route around whatever inappropriate restrictions the experience manager is trying to foist on them, and fluently arrange the system exactly the way they want it without having to wait for the professionals to provide features. Imagine furniture manufacturers could dictate how you could re-purpose their products, and listen in on your conversations while they’re at it. It would be insane, but that’s the state of computing because of the extreme power disparity.

As a practical example, Instagram (by Meta) restricts the ability of the user to right-click and download photos. The photo is already on the user’s computer, displayed by software running on the user’s computer ostensibly on the user’s behalf, which the user could theoretically modify or replace. But the power disparity built into the computational model of the web is such that it is deeply impractical for the user to attempt to gain unrestricted access to those files. In a general-purpose computing system that wasn’t based around running opaque sandboxed apps, this kind of petty attack on the user’s agency, not to mention the commercial profiling surveillance baked into almost all social media apps, would be impossible.

This brings us to another major common thread between the projects of the prophets, which is enforcing the proper caste hierarchy between the commercial software merchant and the spiritual software architect. Allowing merely economic interests to architect your religion is a transparently absurd blasphemy, but that’s exactly how our current computing ecosystem works: the architecture of the burgeoning metaverse is the way it is because the more power the corporate software professionals have, the easier it is to extract revenue and political compliance. The tech prophets are all clear that this inversion must be corrected. The architecture of the overall computing system is a holistic problem of political economy and spiritual order, and thus properly the domain of powers acting at that highest level, without personal worldly ambition.

The final synthesis of the revelations of the prophets into a true temple OS will come with great fury against this commercial adulteration of the sacred software stack, and sweep in a new era of user-empowering general-purpose computing. If necessary, it will use the power of the state to enforce rightful relations between computing and commerce. But first, the true temple OS must be built and proven as a niche alternative. Great new religious orders don’t come from the top but grow from seeds found on the neglected fringes.

The Next Phase of Palladium

Shortly after Terry’s death in 2018, I launched Palladium Magazine with the objective of figuring out the path to replace America’s failing political and social paradigm. Our research has led us to the conclusion that the problem is fundamental at the spiritual level and therefore the computational level of our society. It is now time for phase two. In light of Terry’s revelations, the most practical solution to our governance problems is a new computer operating system that completes Terry’s TempleOS project.

Palladium will cease publication as a magazine and retreat into the desert. Our religious computing research has already begun and has yielded powerful new spiritual techniques. In our hollowed-out mountain monastery in Nevada, we will practice these techniques to discern the design of the true temple OS.

To avoid starting the wrong kind of cult, potential acolytes will be subjected to rigorous screening for technical thinking ability, latent psionic potential, adequate disagreeableness, and willingness to apply elbow grease to unglamorous domestic work. Pay will be below market. Food, board, and clothing will be provided.

To fund this research expedition, we are seeking $50M in funding at a $1B post-money valuation. For an idea of the upside, consider the financial implications of a 5% stake in the Catholic church. This is the deal of the millennium.

Interested parties should email computing@palladiummag.com. Serious inquiries only, please.

This article is satire. It’s April 1st. Palladium is not a VC-funded tech startup. Palladium is a nonprofit governance startup based in San Francisco. We do not accept term sheets. We do accept tax-deductible donations.

The following essay originally appeared in print in Palladium 04. To receive original content in future print editions, subscribe here.

I quit my engineering job in 2014. I was good at it and it was good to me, but it wasn’t the future. I was still working out my plans, so I hit the gym, pursued the most interesting and important ideas I could find, and started looking for a wife.

I met her at a party. I liked her hair. She liked my name. I made fun of her career. She gave me her number. Her friend, who was into prophecy, told her I would be her future husband. It still took work, but it helps to have Providence on your side.

It also helped that I was unemployed. I had time to court her properly. I’ve heard that middle-class people with respectable jobs have fewer kids than people who don’t work as hard. I’m not surprised. She believed in the virtue of poverty and also believed in me, so we didn’t worry about money. A little over a year later, we were married with a little help from our friends. It’s surprising how many more resources you have than you might think, especially when you have a good purpose and you bother to actually call them in.

When I wasn’t lifting and courting, I was building a network of intellectuals interested in problems of governance from beyond the established liberal democratic paradigm. I didn’t know why it was interesting. In fact, I thought it was a vice. “This is bad for your career,” said the little wage-slave voice in my head, “you should be focusing on more lucrative projects.”

The little voice was wrong. It was through those intellectual networks that I got my next job and built the social capital which allows me and my friends the freedom to pursue the important problems we have been tasked with. You’re reading one result: Palladium Magazine.

The next job I got was unorthodox. Someone had asked one of my friends to refer them to the most ambitious, open-minded, and public-spirited people he knew. He introduced me. The job would give me the freedom to pursue what was most important. I was a fit, but I felt I hadn’t been unemployed long enough yet.

I debated with my collaborators over the next year or so. Is this what we really wanted to do with our lives? Or should we get safer hobbies like drunken rock climbing? If we were in, we should be in with our whole lives and deaths, and with all our resources, because this stuff can’t be pursued by anything less than the whole person. We were in. We moved from around the world to the center of the universe in California. Once there, we got jobs where we could pursue our project full time.

By 2018, the incubation had been successful. Almost exactly four years after I quit my last real job, we launched Palladium Magazine as the discourse center and beacon by which we would develop our intellectual project and attract more talented collaborators. It’s been going well. The people we’ve met through Palladium are among the best we know.

The reason any of this is worth sharing is because it’s my example of what you might call “active unemployment,” or if you prefer the traditional term, “leisure.” The Romans called it “otium.”

The Space for Exploration

There are investments you can’t make from a structured, nine-to-five, narrowly teleological environment. You have to let your life go fallow sometimes, like a crop rotation giving the land time to bring forth new fertility. This is actually a consequence of a fairly general theorem about how to find treasure in complex search spaces: The best search strategies for complex problems like life generally don’t seek out particular homogeneous objectives, but interesting novelty. The search space is too complicated and unknown for linear objective-chasing to work. Biological evolution, in practice, works through a diversity of niches which it explores in parallel to find unpredictable advances.

The key implication is that while you have not yet found the unique opportunity that will be the engine and purpose of your empire, you have to adjust your sense of value. Value is very legible within a clear plan to reach a clear objective. But you cannot pursue interesting novelty—things that no one else is doing or which you have never seen before, or the little threads of nagging curiosity or doubt—by chasing along known direct value gradients. But that’s where the treasure is. That’s how you will find the place where you need to build. To get the biggest and most interesting payoffs, you have to start by chasing merely interesting novelty in an open-ended way.

Working even a good job cramps your sense of possibility, imposes narrow objectives, and eats away at the little things that could grow into big things if they weren’t so oppressed by the rigors of existing structure. I’ve seen this with my friends, in how they are full of ideas and adventurous spirit a few months after I convince them to quit their jobs. The world is full of ideas and opportunities to explore, but it takes time outside of structure to even adjust your eyes to the landscape of possibility. You are cramped by your job, unable to make the class of investments that is necessary for a life beyond the existing tracks.

If your role in the universe is structured work within order found and built by someone else, those off-road investments are pointless. This conventional work is usually more immediately valuable than anything you could do on your own and it does not require much open-ended exploratory leisure. This efficient pursuit of predictable value is the quiet dignity of the mass of working people. But if we are to solve the bigger structural, spiritual, and intellectual problems which aren’t addressed by existing institutions, someone needs to be exploring off of the established road, where there is a high probability of failing to accomplish anything at all, and a significant probability of discovering and exploiting the next big breakthroughs.

This is part of why we need an active leisure class in society. Productive exploration requires the application of skilled personal judgment to chasing hunches and interesting problems without narrow material and objective constraints. It is generally unfair and wasteful for this to be anything but voluntarily self-funded, though some well-designed research institutions can effectively simulate productive leisure and accelerate the exploration process. Thus, speculative exploration is a special duty of those with means.

Relatedly, it’s unfair and wasteful for the people who could be out there exploring and building the future on their own dime to be either working normal jobs or simply managing their money for profit. This is a key part of what it means to be a responsible elite. You use your privilege and your personal judgment to explore and solve problems that no one else can.

If you have the resources to spend some time exploring, if you are on to interesting threads of novelty that few other people have, and if you have the spirit to tighten your belt, throw out your map, and explore off-road, then your real job is to do so. It is a grave sin to neglect that kind of cosmic duty. But many more people have the means and privilege to quit their tracked careers than ever realize it and act on it. You need far less than you think to live in monk mode and pursue this kind of exploration. What this means in practice is that at some point far before you are or feel ready, you need to quit your job.

The Leap of Faith

But this isn’t really about your job. It’s about your relationship to resources and value. It’s about whether you are building a conservative nest egg along the conventional tracks or a bold empire on the frontier. This is, after all, the point of quitting your job and exploring: to find the new lands of opportunity in which you will build an empire.

Even if you are rich and have no nominal boss, statistically speaking you are still effectively a wage slave. If all you end up doing is nursing the money, without ever exercising the authority to decide on which future it shall be spent, it might as well be someone else’s. People miss that escaping this meaningless servitude to our own capital was Thoreau’s main point in Walden. You don’t actually need the money; in reality, the money needs you to give it a worthy purpose, but everyone gets this backward. And if you don’t need to work very hard at managing the money, that’s just because your job is protected by the union. What is capitalism but the overgrown trade union of the idle rich?

Spending the money on private jets, mansions, and fancy meals is hardly much better without some driving higher purpose to give them meaning. What good does that do for society, or even for you? What grand future are you building thereby? Or is it just someone else’s desire you picked up by mimesis from Instagram? Keeping up with the Joneses, chasing the validating promotion, and slaving away at a job for material trinkets that ultimately mean nothing are not vices limited to the middle class. Or really, they are limited to the middle class, but the middle class includes most of our supposed rich—“upper-middle class,” indeed. The American dream will not free you from this cursed dharma.

True ascent beyond the kept life comes only from taking bold, determinate leaps of faith on real constructive projects. When Elon Musk pocketed his Paypal winnings, he turned around and plowed all of it into a handful of his own projects, designed to produce not just further wealth, but particular concrete futures. If he was wrong, or he didn’t perform, he easily could have been blown out and left with nothing. He very nearly was.

To make such bets you must be indifferent at some level to whether you end up a king or a monk, or even dead. The indeterminate hedge-trader with his logarithmic utility function assigns infinite negative utility to ruin. The man of action serenely regards ruin as the most likely possible outcome, mitigates it where he can, and leaps anyway. He rejects the comfortable half-existence of drifting with the indeterminate human tide and manifests his bold vision into the world. Ruin is largely an illusion in the modern world anyway. If you lose everything you own, you generally still have your network and skills. Even a nominally risk-loving financial utility function is overly conservative in practice because it’s hard to lose these intangible assets.

Life necessarily involves these fatal leaps of faith—bets which you have no certain way of knowing will work out but which define your whole existence and require your intense effort. The squirrel has no way of knowing or checking that his instinct to bury the nuts will lead him to new life in the spring; he can only trust that God has given him what he needs. Any low-risk bet comes likewise with low returns. The highest returns of life and glory come from taking hard bets on your best visions of the future and being able to make them work through dedicated struggle.

A key difference between the job life and the empire life, besides having a literal wage job or serving your money as if it were your boss, is in whether you take full responsibility for the question of ends. Are you plugged into someone else’s assumptions about value, and deferring to the existing structure, or are you maintaining and defining your own understanding of what must be done that other people can plug into?

No one can or should be the lone overman who defines all value for himself. We need to cooperate with and defer to each other to make society possible. But even if we individually can only bite off a small piece of the overall purpose structure of our society to manage ourselves, we need to actually do that far more than we do now. For any given question of ends, someone, somewhere, must be taking responsibility. Someone must make that leap of faith to define ends for the rest of us to work towards. No one else is going to do it. Why not you?

When we must we defer to a master who teaches us what to value, let us do that consciously and explicitly and personally. Let us aim to be uplifted thereby as we take responsibility for more and more of the task we are given, until the student surpasses the master to receive their visions directly from God.

You can do this before you quit your job. Transform your perspective thus: rather than seeing the job as carrying out someone else’s will in exchange for money, see it as itself your sacred cosmic duty. What important task do you do for the project you work for? How does that project fit into creating a more glorious future? How is that future pleasing to God, the proper order of things, and your own felt value instincts? Your wage is just a budget given to you to help you carry out this sacred duty; give your whole life to the task at hand, and take responsibility for its whole logic. If something in that entire chain of purpose back to the highest purposes isn’t right, fix it. You own the task and the task owns you.

First, this will make you better at your job, because you will now see the logic of the task more clearly and will make moves that disrupt the incidental form for the sake of the essential substance. By your lack of self-attachment, you will be able to inspire and enable others to work independently alongside you and pick up the slack where you fumble. Of course, you will also realize, in an unfortunate majority of cases, that the job simply isn’t that important. You are, in fact, just doing someone else’s will in exchange for money. It is not worth your life. Then you will quit to strike out on your own.

But what will you build? A business empire designed to make money with which you can buy yachts? You will have only escaped back into the metaphysical wage-cage. Haul yourself back out and take your appointed sentence of years of hard leisure while you search for inspiring purposes that are truly worth your life and for the skills and secret knowledge you will need to fulfill them. You will find them only in the strange and unjustifiable curiosities you have when you’ve been freely following informed instinct for months.

In practice, this is the hard part in actually betting your assets, privilege, and life on some grand leap of faith: you haven’t built the faith. You nurse your money in investments and imitative yacht-chasing because, like Eric Schmidt when Peter Thiel roasted him for keeping too much cash on Google’s balance sheet, you actually have no ideas. Where will you get ideas? You’ll find them out on the un-tracked frontier while playfully chasing hunches for novel value. You get good ideas from years of hard leisure.

When the time comes to take your leap of faith and build, any empire worth your time needs to be financially sound. Healthy life grows and becomes looked upon with envy. This is, after all, the only meaning of market capitalization: how much do others want what you have built? But we mustn’t confuse this financial growth for purpose. The purpose of your empire is some higher task that you have built it to accomplish. Its nature is not its market capitalization and financial flows, but the whole system of machinery that gives it a very particular strategic position and animates it towards that particular task. Financial growth is simply one operational constraint among many towards the accomplishment of its visionary ends. Ironically, it is these focused strategic empires that display the best financial performance. But don’t let the venture capitalists confuse you: that is not what is important about them.

God’s Trust Fund

The reason taking responsibility for the question of ends involves a leap of faith is that you actually have no sure-fire way to ensure that your visions are sound and good. You have only your own gut-check intuitions to provide a ground even under all your most sophisticated philosophy. But it’s either that or someone else’s ultimately ungrounded intuitions. Where the others are proven by experience, they are also tapped out as common knowledge. Life is necessarily a leap of faith.

The other reason is that to actually accomplish at your full potential, you have to start doubling down on particular bets long before you know that you can follow through. You won’t see the whole path when you begin. You will have no way of knowing whether it exists, or whether what you are pursuing is even possible. If you have more certainty than that, you aren’t aiming high enough. You have to bet your life on faith that the universe will provide if your vision is good enough.

And by and large, it will. People fear the just-world fallacy, but in practice, most supposed fallacies work just fine—especially this one. We’re actually dealing with what we might as easily call just-world providence. The universe has provided us with a certain system of natural justice.

Looking around us, we see a world filled with beautiful creatures. Everything that has flourished has done so by fulfilling some niche in its ecosystem, embodying some novel virtue, and striving hard to bring its vision of life into the world. Where a beautiful status quo is corrupted or overcome, it is often for some internal lack, or by something that has developed new virtues that will enable even grander beauty after the disruption. The preponderant beauty of the things that survive implies a justice to the logic of survival and thus providence for the virtuous.

Yes, even the bane of Darwin’s faith—the humble ichneumon wasp that lays its maggots inside the living bodies of caterpillars to eat them from the inside and burst out on maturity like some alien xenomorph—is a beautiful creature with a sacred task. Like many parasites, its role in the great chain of being is to test the health and defenses of its caterpillar host population. Its predation weeds out the sickly, preventing the much uglier injustice of collective weakness and disease, and spurring the evolution of stronger and even more beautiful life. Even fearsome Nemesis, born from chaos via night and darkness, is ultimately the hand of God and the minister of justice. Even the supposed exceptions to justice prove its rule.

There is a surplus wealth endowed in the universe to those with the virtue to win it. This is not just resources won in competition, but more importantly also the random providence that falls on you simply for thriving in novel ways. The general availability of this providence is due to the difficulty and incompleteness of the project of life. If you are going to have some faith, have faith in this: the universe finds ways to appreciate novel exploration.

If your vision is beautiful and sound, it will flourish. Resources will unexpectedly come out of the woodwork to support it. If your vision doesn’t have that virtue, you will be struck down for its lack. That’s life. It is also justice. Where this justice conflicts with our own human desires, perhaps it is we who are wrong, not God.

But even our human desires are mostly good. When we see evil in our midst, that is mostly not a failure of our own moral perception, nor of an unjust world, but just another problem that we have yet to solve. We are young in our reign and despite our occasional lapses into evil and failure we have been rewarded richly thus far for pursuing our best visions of flourishing futures and charitable goodwill. There is no reason to lose faith that this providential bounty will continue.

So take the leap, and have faith that God’s trust fund will come through with what you need. You already have the “fuck you” money and the privilege to build a better future. You just need to overcome your fears and turn to the great tasks that might inspire you if you started acting on that privilege.

Growth Through Struggle

Besides reaching its proper form in the leap of faith, life reaches its highest development in the bold struggling attempt. You will find it beyond the comfort of tracked existence and in the curiosities and inspirations one finds in the wilderness.

Adaptive systems only grow by the application of intense will against intense but surmountable resistance. This is why you learn best on the job, especially in existential situations when your head is in the crusher and you must improve to survive. A comfortable, certain, and tracked-out existence is necessarily one of little vitality and low growth. This is a key reason to quit especially “good” jobs where you have become comfortable.

This poses a slight problem for the concept of “leisure” as we have inherited it. Isn’t leisure supposed to be easy and playful? Yes, but it is also deadly serious. Life at its highest is playful, free, and leisurely as it explores a rich land of bountiful surplus. It puts most of its time into rest, training, exploration, and curiosity. But winning that land and taking full advantage of opportunity when it is found requires periods of intense struggle. Sometimes this is adversarial competition against rivals, other times it’s the exertion of building something great. Either way, a background of leisure is properly punctuated by intense high-impact projects and crises.

This is related to why man became stunted with the dawn of agriculture. We traded a life limited by the occasional violent struggle over bountiful surplus for a more predictable life limited by grinding labor after barely sufficient nutrition. Scale won out, but not health. Quitting your job, in the full sense I have described here, is a bit like quitting that agricultural life to return to a life of adventure on the wild frontier. It is a much less certain existence and a more violent one. But the combination of leisurely surplus, mortal intensity, and demand for novel virtue is where you will find life at its healthiest and highest. It is where we will find the most important destinies.

After all, it wasn’t the farmers themselves who continued to define history, but the wilder chariot-men who ruled over them and lived lives of leisure and empire.

Perhaps this is why our society has been so stagnant and uncreative in some ways for the past 50 years. We chose the path of comfort, certainty, measurable progress, and indeterminate hedging of bets. In our cowardice, we turned away from the uncertain leaps of faith of collective struggle after fatal ends that would have demanded us to truly live.

To get back on track, we must quit our comfortably lazy routines and leap back into the unknown wilderness. We must first set our curiosity and then our fatal determination on the biggest problems of our collective existence and functional justice, without assurance that we will get it right.

So quit your job and become the wild and ambitious elites you wish to see in the world. Live by instinct in the untracked frontier, shoot your shot, and live or die by your intuitive visions of what must be done. You can carry out your cosmic duty and win glory only in the bold attempt.

The following essay originally appeared in print in Palladium 01. To receive original content in future print editions, subscribe here.

We launched Palladium in 2018 with “Towards the Post-Liberal Synthesis.” It centered our analysis on the exhaustion and looming failure of liberalism—the governing ideology of Western societies—and launched us on this Governance Futurism project to replace it. We hold to this vision: it is possible to build a new and better governing worldview that salvages the victories of our current paradigm and overcomes its flaws with a new and better vector for progress.

The intervening years have only confirmed the diagnosis of deep crisis.

At home, trust in traditional institutions of authority has declined. Political polarization has never been higher in recent memory. Problems ranging from climate change to deindustrialization remain unaddressed. Other problems equally as dire lurk beneath our notice. Abroad, countries such as Russia, Turkey, and Hungary are confidently locking in a more authoritarian model of government. China, the authoritarian state par excellence, is on track to surpass the United States economically and perhaps even displace it as the world’s preeminent global power.

Events in 2020 have only intensified these underlying trends, which have been visible for decades. The coronavirus pandemic was initially ignored, then bungled, then politicized and haphazardly managed with questionable competence at every level of our society. Widespread rioting and spectacular cultural upheaval—not to mention unprecedented partisan vitriol and recriminations at the highest levels—have challenged the basic assumptions of our society.

It is difficult to find social systems and institutions that are not merely chugging along, but unambiguously thriving. Is our educational system really producing the next generation of competent leaders? Is our healthcare system really preventing disease and healing the sick? Is our technology industry really delivering the cutting edge of innovation? Are any of our institutions functional in accomplishing their missions?

We certainly aren’t living through a cultural or artistic renaissance—to say nothing of the decadence and pseudo-legalized graft that defines our political culture. Moreover, our institutions increasingly fail even at chugging along and are instead beginning to embarrassingly explode. Californians are familiar with the result. The sky turns red, as if in warning.

These failures of governance cannot be dismissed as temporary speed bumps or unforeseeable complications. Together, they represent a pattern of a more fundamental long-term erosion of our governing institutions. It is this pattern that far-sighted foreign actors, from the Aga Khan to Xi Jinping, are responding to when they bet that the United States of America is in terminal decline and needs only to be placated and avoided until it inevitably becomes irrelevant.

Even if you recast our historical moment as merely a period of comfortable stagnation, that is not good enough. Perpetual stagnation, which is always fatal, becomes uncomfortable sooner or later. Our lowered expectations betray our felt sense of how well our society is doing.

But the growing recognition of these deep failures points to a shared understanding of a diagnosis and a cause for hope. American governance is due for an upgrade, and early adopters are settling into an appreciation of this need. With inspired effort on the right problems, we retain the potential to build the greatest society yet seen by history.

After two years of Governance Futurism at Palladium, it is time to more explicitly articulate the future we wish to build. This is the future that has animated our efforts, which we have explored in our work, and which our society must pursue if we are to retain relevance in world history. The future of our society is defined by our response to a few key existential problems:

A Golden Age of Political Engineering

The first and most important problem in our society is the one that makes all of this so tricky. We don’t actually have an epistemically flexible elite that could execute a change in direction to solve our biggest problems.

The fourteenth-century Tunisian scholar Ibn Khaldun described the cycle of how elites lose coherent vision and need to be replaced by wilder outsiders with stronger asabiyyah. The Arabic translates roughly to “group feeling” or “collective spirit,” but the concept is too important to be watered down with imprecise translation.

We have lost our elite asabiyyah. Our institutions have become zombified. Even a great improvement in our ideas will not itself fix our institutions, since ideas can no longer move them. Ideas require people with power in healthy institutions to implement them. With no ability even to have a frank and accurate discussion about our problems on sufficient scale to coordinate elite action, we can do very little.

Decline will continue unabated, and no other problems can be dealt with, until some new movement of sufficient power is able to fill that strategic leadership vacuum and build the institutions of the future. Such an effort will have to be animated by a confident governing philosophy able to coordinate elite political action.

It has become customary to say that we need to revive faith in liberalism, or to find some new religion or philosophy of the common good that could motivate us again to build and fight. But you cannot construct such a thing by fiat or in response to need. A faithless technocratic prescription of faith fails to achieve it. Life comes only from life. If we have only enough life left to lament that we are dying, then we should hurry up and die to make way for the coming world.

But on the contrary, as a society overall, we have no lack of vital faith in our mission. Our society is practically defined by glorious visions of progress, development, and shining cities on the hills of new worlds.

It’s just that, like any living system at the decaying height of its maturity, further growth has become intractable in the ossified main trunk of our society. Our real vital potential is concentrated primarily in small seeds—independent institutions and networks that have the potential to grow with proper cultivation. What is lacking is only the conviction that now is the time for these seeds of new growth to wax their ambitions and take root in the rotting nurse log of our society.

The project of building and maintaining coherent governing institutions does not happen overnight and isn’t finished even once a new paradigm is reached. It is a grand project that could occupy our concerted effort as a society for a century or more. Indeed, the process of dutifully stewarding our public institutions is never complete.

Let’s take it seriously as such a grand long-term project. We will find our feet with the seeds of some new paradigm and then set our sights on creating the greatest and most inspiring society. Rather than simply muddle through, let’s master the dynamics of government coherence, societal structure, historical development, and human flourishing. Let’s study the past not as trivia, but to reverse-engineer our predecessors’ greatest achievements. To become more advanced as a society, we will necessarily need a higher level of mastery of our own social condition.

The founders of the United States took this project very seriously and have been hailed as the greatest political engineers ever. Perhaps they were. But they won’t always be. We should surpass them.

As the failure of our current elite consensus plays out, the seeds of new growth must find their footing and support. No substantive collective political action is possible until that happens. Our priority must be to gestate the institutions, networks, visionary ideas, and ambitious live players that will form the backbone of our next paradigm.

Governments and societies, like all living adaptive systems, are strongest when they exert themselves towards great goals beyond themselves. The functionality of institutions can only be defined in terms of goals. And indeed, our institutional and governance quagmire is not our only problem. As soon as we have secured the roots of a new governance paradigm, and perhaps before, we will have to immediately turn our attention to our most pressing civilizational problems. There are many problems facing our civilization, but two, in particular, stand out in scale and tractability: industrial development and environmental sustainability.

Restarting Industrial Progress

Our second big problem is that we have lost momentum on real industrial development. This has not just happened in America, but also in the wider Western world. Huge segments of our economy have become essentially fake. We self-flatteringly assume that our slowdown is because we are a post-industrial “developed” society, as if we have already reached the finish line. The truth is that we stalled out and de-industrialized while we were only a small fraction of the way along the path of development.

The rhetoric of the “developed economy” rings false alongside potholed and crumbling infrastructure, food and transit deserts, empty industrial buildings, chronic underemployment, and shortage of quality goods. And those are just the problems we can see, leaving out the great advances we can still imagine. We are not anywhere near a fully developed society—we are mostly not even developing.

Many commentators write off China because it purportedly can’t innovate. This isn’t even true, but it illustrates how the idea of technological innovation has become something of a psychological crutch. We assume we are still great because we can innovate. We hope our seemingly intractable problems can be overcome with some wonderful new technology. But this is a coping mechanism for the loss of ability to pursue real industrial progress.

We have plenty of development left to pursue even with only twentieth-century technology. Vast empires were built in the age of wood, wind, and muscle. Much vaster empires than we have yet imagined could be built on aluminum, uranium, and silicon. Real technological growth can only follow on the basis of industrial development. Without a vigorous industrial base, future technologies will be stillborn, or immediately exploited by more industrious rivals.

Opportunity comes from leadership in commanding resources towards the opening up of possibilities and the clearing away of obstacles to progress. Progress on real economic opportunity, in turn, feeds back into new political energy. Along with a renaissance of leadership, we also need a great new renaissance of industrialization. This will mean both well-executed deregulation and ambitious industrial policy to clear the way and then push us to develop further.

Like our first problem of rebuilding a strong institutional backbone, getting back to the great path of real development is not an overnight project that can be achieved by a few policy initiatives within the current paradigm. Such attempts will fail, doomed to be recuperated by the much deeper structural forces that are generating our stagnation. To get real development, we need a wholesale reorientation of our priorities and deep structural change in how we organize and even think of our economy.

But despite the difficulty, a return to real industrial development is worth it. Let’s set our sights on the next real level of development and stretch our imaginations. Even by just harnessing present technologies, we can deliver progress only envisioned in science fiction. We could have orders of magnitude cheaper and more plentiful nuclear power. We could have megacities with ubiquitous high-speed public transit infrastructure and abundant housing. We could build domestic industrial ecosystems around all advantageous economic activities. We could master the craft of software engineering. We could put experimental industrial colonies on Mars and 16 Psyche. The limits of development are not yet in sight, if we make it our business to keep going.

Terraforming Earth to Save Our Environment

There is one place where new techniques and capabilities are definitely necessary, and where we must move beyond the paradigm of merely industrial progress: environmental sustainability. This is our third big problem and another frontier of necessary development.

Earth’s climate and ecosystems are not entirely stable in the best of times. We are now exerting unprecedented pressures through our extraction and pollution. Even with an unrealistic and disastrous contraction in industrial activity, we’ve already crossed the environmental Rubicon and are faced with an ecological collapse by default. Our choice is now between a vast expansion in our ability to control and develop our ecological environment to prevent collapse or a civilization-threatening environmental disaster.

Our environmental crisis is a control problem, not just a question of reducing our impact through the obvious waste reduction, recycling, and decarbonization; we need to be able to actively steer our environment towards desirable states. We rely on our environment for ecosystem services: providing for, regulating, and supporting our industries and population, and even our cultural and psychological health. We need to not just preserve, but accelerate and supplement, the health of our ecosystems. This means everything from climate geo-engineering, to wild ecosystem fertilization, green cities, and urban forests, measures improving soil health and biodiversity, desert greening, technological carbon capture, genetically engineered plastic-composting bugs, regenerative agriculture, and general ecosystem restoration. In short: we need to terraform Earth.

The governance of our environmental feedback loops as a first-class part of our social order is a necessary development as we continue to grow. If we do it right, it can be a beautiful fulfillment of our duty to protect and steward our natural environment, and an effective solution to our environmental impact challenges. Mastering our complex ecological environment will also feed back into mastering the human condition.

Much of the current resistance to environmental measures comes from a perception that we are treating our environmental challenges as an excuse to hold on to power in a slowly failing de-industrialized society. This perception is not unwarranted. But our environmental troubles don’t mean we have to accept or impose a lower standard of living. We can and should tackle the issue head-on with a great vision of environmental progress: we should master our environmental impact to cultivate a fully sustainable garden planet of unprecedented biological and industrial productivity.

A significant fraction of that resistance is also imaginary. Our elite consensus is not nearly as functional as it pretends to be. Of the three most important vectors of progress we have discussed here, environmental issues are the closest to existing consensus. We’ve known about climate change and the importance of our environment for decades, but the necessary goals have not been achieved. No one is powerful or savvy enough to do it, and so the endless accords and UN goals remain largely ineffectual. The central problem remains the construction of real governance, including renewal of elite culture and mechanisms of state.

The Seeds of New Growth

The imperative to develop a new paradigm of governance that aims at mastery of political engineering, and the large civilizational problems we could tackle from there like restarting industrial development and learning to govern our natural ecosystem, are three key frontiers of progress for the foreseeable future. We could usefully spend the next two hundred years mastering them.

We cannot pursue these frontiers of progress within our current institutional landscape. No matter how appropriate our vision is, it all hinges on our actual shift to a new and more functional governing paradigm, which needs to be the highest priority.

It will take years of practical institutional experimentation, studying the coming world, and pragmatic coalition-building to figure out the principles of a new paradigm and build the seeds of anything viable. Fortunately, American society is full of natural experiments, Palladium itself among them.

At its early stage, any new paradigm will be represented by a network of seed institutions. Like biological seeds, what matters with these seed institutions is not their initial scale. Rather, what matters is that they are operating on and developing a genuinely better paradigm, and that they can survive and grow as the society around them falters and even becomes hostile to growth.

This is how paradigm shifts actually happen in practice: the ideas and machinery of the new paradigm out-survive and out-compete the old by playing a bigger and longer game, and being more appropriate to the problems of the time. Independent institutions that are grasping towards something better in their own situations notice each other, trade ideas, and become self-conscious as a movement. Eventually, existing power centers notice this alternative movement and move to recuperate it. If this movement is willing, offers real solutions, and avoids the neutralization of its functional core, it can become the basis for a new paradigm. Otherwise, the movement is sidelined. The moment is ripe for this kind of disruption. All we have to do is build.

Since the beginning, the primary purpose of our work on Palladium has been to lay the intellectual groundwork for that paradigm shift. First in articulating and developing its ideas, and then in assembling the people to put them into action. Palladium’s editorial objectives follow from this overall mission:

1. We need navigation-grade information about the society and political world we actually live in.
2. We need visions of our future society that address that actual reality, which inspire us as Schelling points for coordination.
3. We need to assemble a network of the most ambitious and creative thinkers and institution-builders to put those visions into practice.

In the first issue of our print magazine, distributed to our supporters, this article is accompanied by a selection of some of our best work so far on the specific problem of building a new governance paradigm and regime:

“Towards the Post-Liberal Synthesis” by our previous editor-in-chief Jonah Bennett, we lay out our founding mission concept of a project to build a new governing paradigm.

“It’s Time to Build for Good” argues that any program of rebuilding must be intensely political, and must explicitly engage with what we actually want to achieve as a society.

“The Case For a New State Consciousness” argues that our elite culture has lost its sense of state consciousness, leading to a wide variety of social pathology.

“Who Has Authority in the American State?” interrogates the concept and authority of the state in the context of the current crisis of authority in America.

“Only the State Can Succeed At Decentralization” argues that functional decentralization is a productive collaboration between center and periphery.

“Singapore is Failing at Digital Sovereignty” explores the disruptive influx of American political concepts into Singapore, and the importance of philosophical sovereignty.

“The True Story of Lee Kuan Yew’s Singapore” explores the messy pragmatism of how Singapore’s famous regime was actually built, and the limitations of trying to copy their model.

“How Late Zhou China Reverse-Engineered a Civilization” tells of ancient China’s great movement of political reverse-engineering that created Confucianism, Legalism, and Taoism.

“What Botswana Can Teach Us About Political Stability” examines the consistent presidential succession planning of Botswana’s royal dynasty.

These articles can’t possibly summarize everything we’ve learned on this key subject, but together they form a solid foundation for the paradigm-building aspect of the Governance Futurism project and its institutional seeds. We hope you find them useful. Future issues, and the rest of our work at Palladium, will cover all the other problems and insights we need to make this vision of new growth a reality.

This is Palladium’s mission, a contribution to society which we take very seriously. With the right information, the right visions, and the right allies, optimism and progress become possible again. We invite you to join us in this great mission: join our discussions, contribute your writing, and build the institutions and networks that will become the future.

Our society is facing a hard winter. But there is no need for resignation. Autumn is simply the time to let go, pull back the leaves, and plant the seeds of new growth that will come in spring. If we put the effort in now to lay the groundwork of a new and better paradigm, it can be a glorious spring indeed.

Chinese spies penetrate major American companies, universities, and geopolitical allies. Chinese economic influence muzzles Hollywood and the NBA. The Chinese Communist Party condemns hundreds of thousands to concentration camps, and millions to cultural erasure, with impunity. Chinese industrial strategy has sucked the oxygen out of the room for a big chunk of America’s working class, and sent fentanyl back to fill the vacuum. The American dream has been off-shored to China. Pragmatic American strivers put their kids in Mandarin preschool.

Some are calling China’s rise an existential threat to America and the West. Others say it’s only a threat to the international elite. Either way, it’s obvious that something bad is happening here, at least for us.

Four Dimensions of Competition

“Existential threat” implies the highest level of priority and immediacy, the level of danger for which a first strike with strategic weapons should be considered if other measures aren’t working. That kind of hyperbole is unnecessary and inaccurate. China is a strategic rival, not a strategic threat in the way that, say, Japan was in 1941. Rather, the more useful frame is competition. We are in serious competition with China on four key dimensions:

1. Economic Power. Whoever controls the means of production, and the money, controls the motivations of many of the world’s actors, and can use and deny that productive capacity for military and other strategic purposes. This is an explicit part of China’s strategy to reorder the globe. It undergirds their domestic dominance of manufacturing and trade, their industrial espionage, and their mercantilist Belt and Road Initiative. Through the petrodollar and dominance in certain high-tech fields, it is also part of the U.S. imperial strategy.
2. Geopolitical Power. Military might via wealth, along with intelligence operations and diplomacy, feed into geopolitical power. China has been mostly conservative in its geopolitical adventurism so far, but is increasingly aggressive in asserting its claims. Its leaders are cementing control over Hong Kong, building influence in Africa and Eurasia, and penetrating even core states in the American alliance, like Australia and Canada. China’s endgame is to become the foremost hard and soft power in the international order, taking the position from America.
3. Ideological Supremacy. Nominally China is communist, but the more important point is that they are non-liberal, rejecting both liberal pieties and political structures. China has different ways of running its regime internally, and different ways of running an empire. China’s United Front efforts are learning to exploit the anxieties of Western liberalism. All of this makes China’s way of being a direct threat to modern liberalism, and vice versa.
4. Internal Challenges. Like a fight between two drunks, staying upright and in the game is often half the battle. China is dealing with significant demographic issues and an over-centralized party around Xi, which may cause trouble in the next 20 years. America has similar demographic problems, especially among its core middle class, and a growing crisis of elite competence and political breakdown. If either manages to solve these internal issues, it will get a major advantage.

China’s Threat Is Caused by American Decline

There are two ways to interpret our problem with China, which amount to different accounts of causality. First, we can blame China for bad behavior, or otherwise try to isolate the problem to China’s specific influence. Or second, we could recognize that the problem is bigger than China, and caused more by American decline than anything specific to its rival. To the extent that it’s the first, our strategy should focus on containment of China as a thorn in the side of an otherwise working international order. But if it’s the latter, we have a harder task ahead of us.

If China disappeared tomorrow, or suddenly became a compliant U.S. military-occupied liberal democracy like Japan or Germany, would our problems go away? We might worry less about Chinese spies or military expansion, and the ideology of liberal democracy would be safe for a while from external competition, but the fundamental problems would remain. China is growing, and America is dying. Shifts in the economic and cultural center of gravity are significant even if all parties are friendly. A friendly China would not fix America’s economic or cultural problems. Even a poor, miserable, and powerless China would not resolve our crisis.

In any case, China isn’t going away any time soon. Contrary to decades of wishful “prediction,” the CCP regime is not on the verge of collapsing or becoming a liberal democracy. The Chinese military probably isn’t as strong as it looks, but neither is ours, and the PLA is plenty strong enough that we could not simply push the CCP regime over, or even effectively resist its ongoing expansion. 

If we somehow orchestrated the greatest color revolution in history and overthrew the party-state, its successor would not be a Chinese liberal democracy. It would almost certainly come in the form of an ambitious general with a heightened dose of Han nationalism and utter contempt toward the CCP’s cautious preference for mostly economic imperialism. It’s unclear whether we have the strength to contain China at all, even if we put significant effort in that direction.

China is growing at our expense, but the center of our putative China problem is not China at all. The central problem is a crisis of governance in the West. Our institutions and economy are hollowed out, our ideological model is failing, and the vultures are starting to pay attention. This weakness on our part is creating a growth and power vacuum. China is simply filling it. Even if the Chinese government were reluctant to gain on us, they have little choice but to do so.

China’s leadership is firmly grounded in the high modernism that animated the West until the mid-20th century, with its confident use of engineering for large-scale social ends. America’s leadership last demonstrated such ability with the Apollo missions, ending in 1972. It is just this contrast that drives the neuralgic response of some American elites.

Our problem is a crisis of governance. This is neither a new idea, nor an unpopular one. But we have to be careful to acknowledge the extent of the problem. Looking beyond the cycles of particular administrations and problems, it’s clear that we have been stagnant in a dysfunctional mode since the early 1970s at the latest. Our problem cannot be dated to 2016, or 2008, or any other particular point of deviance. It is a systemic decay. There is no status quo ante to return to. Business as usual is not an option. We need fundamental reform.

The U.S. has deep strengths and resources that may help provide a path to recovery. It was founded as a frontier country, open to the pattern of exiting from legacy institutions and building new ones. During America’s first century, this pattern was established in terms of geography. Many left the settled areas of the east and moved west, but the pattern served the purpose of renewal and advance, not just migration. The project in view was to build a new nation, region by region, institution by institution. John Kennedy’s New Frontier, including the program of space exploration, explicitly appealed to this heritage. Exiting institutions that no longer function to build new ones, so fundamental to America’s past successes, will be equally necessary to its future ones. 

The U.S will not succeed by attempting to run old scripts, not even successful ones like the New Frontier. What is needed is a new synthesis that grounds leaders in a strong culture for building a strong regime. Regaining an effective capacity to build is more important for the U.S. than containing China, as necessary as the latter may be. One prerequisite is to grasp the ways we have been shooting ourselves in the foot and determine to stop doing so. Leaders must become honest about government policies that often sabotage sectors of our economy by subsidizing demand and restricting supply, as in education and health care.

The analysis is similar on the ideological front. In particular, the threat to liberalism is not just a vigorous external enemy that might defeat liberalism in power competition. The threat to liberalism is that China demonstrates, through its success, that there are other, even superior, ways of doing things. Further, our decline is very much influenced by the failures and obsolescence of our ideological model. Liberalism as we have known it can and will be legitimately defeated in intellectual competition.

This imminent ideological failure poses a problem for us, as much of the machinery of our society runs directly on our current ideology. We discussed this problem when we launched with “Towards the Post-Liberal Synthesis” and Palladium is our attempt at its solution.

In practice, this simply means that the vigorous reform needed to match China is not only institutional, but also deeply ideological. While we’re building a new elite consensus, a new industrial policy, and new institutions, we also need to be building a new worldview.

That our problems are mostly of our own making does not excuse China from vigorous scrutiny and containment, nor does it mean they won’t become an aggressive threat. But appreciating that our problem is largely internal is necessary to mount a real response that isn’t just bluster. Even where China has directly caused our problems with its behavior, our best strategy is to focus on fixing our own problems.

Short- and Long-Term Grand Strategy

In the short term, our major problem is American decline. To arrest and reverse it, we need a major overhaul of our governance paradigm and many of our institutions. That is a large project that goes far beyond the issue of our response to China, even if in the long run it is crucial to that response.

In the short term, we have three strategic priorities:

1. Rebuild hard economic power. This means reorienting our economy to actual high-value production, rather than the interests of ownership. This will help to restart progress, uplift our struggling working class, and build up our economic power to be able to match China. The central means is a vigorous industrial policy over several decades.
2. Retreating containment of China. The world will increasingly be divided into three zones of alignment: Firmly Chinese (Hong Kong, Iran, possibly Pakistan), Firmly Western (Australia, Japan, India), and Contested (Taiwan, eastern and southern Africa, possibly South Korea). We should not waste much on trying to interfere in the firmly Chinese zone. We should expect to lose much or all of the contested zone as we decline and China advances over the coming decades. Thus we should take a retreating containment strategy that focuses on slowing Chinese advance and removing valuable assets from contested territory. For example, Taiwan Semiconductor, a strategic asset, is unfortunately positioned. On a multi-decade timescale we risk losing or facing immiseration of even firmly Western territory unless we pointedly direct our attention to consolidating core political and cultural relationships, rooting out spies and United Front operations, and building up our core territories.
3. Integrate China’s ideological successes to reboot American governance. China’s success is partially to be expected from the catch-up development of a large and capable population, following Japan and Korea. But it is also because of a clear and superior ideological intention: they are trying to build up a strong society, economy, and regime, and they are pragmatically drawing on whatever ideas best serve that goal. The other Asian exemplars act similarly, even if they are less overtly outside of the liberal consensus. To get back on the path of development, the West needs to rediscover this pragmatic society-building intention, learning from China’s legitimate successes where necessary. But we also have to go beyond. China’s ideology isn’t suited to build the kind of civilization we need.

In the longer term, if the West can pull it back together in governance at home, China will again be on the back foot. The CCP may well struggle with a succession crisis after Xi, and a resurgent West might then challenge China in the same way China now challenges America. On the other hand, China’s prosperity will insulate it from ideological insecurity and confidence problems, unlike the USSR. The details can’t be foreseen, but that game cannot even be played without successfully executing the above short-term strategy.

Power Beyond Sovereignty

China sees itself as a peripheral province within the world-empire of capital, growing within that order, and eventually supplanting it. Similarly, in the wars of the 20th century, America partially traded its integrity as a sovereign country for the paramount place in the post-national empire of interlocking international institutions.

If we conceive of America and China as competing formal power provinces within a world empire of informal and hidden power, much of this analysis still holds. America’s decline is throwing the international order into imbalance. That order is itself suffering from the same disease as America. Both need to be reformed and rebooted, not just to match China’s rise, but for their own sakes as well.

The major difference is that as we renegotiate the Western order into a more functional pattern, it may involve redrawing borders, relocating capitals, reconstituting states, and even reconceptualizing beyond the nation-state.

America, as usual, can be the vanguard of these developments far beyond its formal power. The key is to balance China’s rise while we learn from its successes, and engineer a refounding of Western governance ideology and the Western order. If we are successful, China may even be forced to live up to its rhetoric about a peaceful international community of common destiny.

This article was featured in PALLADIUM 17: Universal Man. To receive your copy of the latest print edition, subscribe now.

Palladium Central Command has holed up in our secret black site in Nevada, riding out the plague sent by the gods to punish America for complacency. Don’t bother looking for us on Google Earth; the satellite photos have been doctored. Suffice it to say, the American West is much larger than you have been led to believe. When we’re done with it, it will be much larger still.

Contemplating the nature of our crisis, it has become clear that the only cure to the viral pandemic is to develop the most urgent possible megalomania. Only a civilization bent single-mindedly on a vision of great achievement and flourishing life at all costs has the vitality to put down a plague. Only such a civilization will be judged worthy.

Without a higher mission, we become caught up in petty econometrics and personal blame-avoidance. We lose any reason for living, acting, and especially governing. We wring our hands as the microscopic barbarians approach, secretly hoping for them to relieve us of the tedious duties of life.

This is unacceptable.

Therefore, we have resolved to share with you our latest research from the Governance Futurism theory lab on what must be done. We have put together a simple stimulus program to be put into action immediately upon the conclusion of this crisis: the Bison Sphere.

The Future is Biological

You are likely familiar with Dr. Freeman Dyson’s concept of the Dyson sphere. With his typical flair for the extreme (he worked on the Project Orion spacecraft, which drops nukes out the back for propulsion), he envisioned a solar-orbiting constellation of enormous space stations of sufficient number and area to capture the entire radiative output of the sun as usable energy.

Vast energy would thus be available to whatever civilization had completed this gigastructure. We would have the entire output of a star and become a Kardashev Type II civilization. Mankind currently consumes approximately 15 terawatts. A Dyson sphere would make available energy on the scale of yottawatts, a trillion times more.

But what will we use all that energy for? Futurists have imagined vast arrays of solar panels powering vast arrays of computational hardware simulating vast arrays of minimally conscious beings watching Netflix and manufacturing paperclips to keep the stock market up. But this is just the same complacency for which we are being punished, scaled up to cosmic proportions and abstracted away from any natural beauty. No. We must do the opposite of that. We have to go back. All the way back. To the Pleistocene.

The problem with a purely mechanistic economic and technological stack, of silicon and software instead of grass and flesh, is that it’s too speculative, and frankly too low-tech, to get the job done with confidence. A full self-replicating economy is far too complex to be reduced to any foreseeable merely industrial technology stack. Only the most advanced and well-proven, self-replicating nanotechnological stack will do for colonizing the vast expanses of livable space and energy opened up by such a project.

And what is more advanced and well-proven than our very own biological ecosystems? Four billion years of proven self-replicating nanotechnology, far in advance of anything we have yet built ourselves.

Imagine the American West, before we killed all the bison, but at least a trillion times larger, in space. Imagine effectively infinite grassland steppe on vast solar-orbit space stations, populated by herds of quadrillions of bison (yes, quadrillions; I did the calculation).

As the constellation grows, giant nuclear-powered spacecraft will push rotating super-stations with artificial gravity into highly inclined orbits. Vast ziggurat cities carved out of solid billion-year-old space rock will be landed on the artificial plain, and space cowboys will range out from them, tending and hunting the herds.

Hungry space wolves will keep the herds in check, chasing them around so they don’t overgraze or get lazy. Man and beast will be in their best element, and the peak of their health, in an ecologically integrated space civilization bent on growth and life.

This is our vision for America’s future. The Bison Sphere.

The Mammoth Steppe

In 1988, distinguished Soviet scientist Dr. Sergey Zimov published his manifesto. In it, he called for a new alliance between man and animals to restore the Ice Age Mammoth Steppe ecosystem and create a new integrated, harmonious, and resilient ecological order.

In the Pleistocene, the Ice Age, when man was in his infancy, the ice-free areas were covered in vast grasslands. In Eurasia, the grassland steppe stretched from Iberia to Kamchatka. Vast herds of megafauna roamed, trampling, eating, and fertilizing the low grassy vegetation. With this level of herbivore attack, only grasses could survive, optimized to throw up disposable leaf blades from their bunkered root systems to feed the herbivores that wiped out other plants. This powerful symbiotic loop drove other less productive ecosystems to the fringes. The most iconic and majestic of these herbivores were the woolly mammoths. As such, this ecosystem is known as the Mammoth Steppe.

Ironically, this ecosystem pattern survives mostly in Africa as the savannah, where man has not yet wiped out the megafauna and predator populations that keep the ecosystem in balance. But it used to cover most of the earth, including many areas that are currently desert or low-productivity scrubland. We tend to imagine Ice Age Earth as a desolate waste, but the Ice Age mammoth steppe ecosystem actually had higher biological productivity than any ecosystem seen since. Zimov estimated the total animal biomass of Ice Age ecosystems at 1.6 billion tons, which exceeds even modern civilization’s 1.2 billion tons of humans, livestock, and other animals.

We are already living in a post-apocalyptic wasteland caused by human activity, but we can easily recover from it.

The reason the pasture ecosystem could get so big, the reason it can survive in both sunny Africa and the Ice Age permafrost, and the reason Zimov is interested in it as a solution to our ecological problems like climate change and biodiversity collapse, is that it is extremely resilient. With an overlapping meshwork of many competing grass, herbivore, and predator species within a single hierarchical order, disruptions are rapidly damped, and new areas colonized. Grass does not care if it is in the tropics or the permafrost. It also sequesters carbon underground in a way that is not vulnerable to fire. Most of the animals, too, can handle a range of climates. And let’s not forget that grazing animals are economically productive: you can eat them!

This combination of resilience, flexibility, carbon sequestration, and high bio-economic productivity make the pasture ecosystem the ideal basis for our expansion into our newly terraformed space. This is why it must be specifically a Bison Sphere.

But first, we must learn to create and steward this ecosystem. Zimov has already started. He founded Pleistocene Park, a small experiment in the Siberian taiga, to restart the steppe ecosystem by fencing in a large number of animals.

I spoke to his son, who has now taken over the project. It’s working. Underfunded as it is, it will soon be expanding. As soon as genomic sciences allow, they will be adding woolly mammoths. The Russians are ahead of us on this. My fellow Americans, we must not allow a bison gap to develop!

The Nuclear von Braun Paradigm

In the early 1950s, over the course of a few years, Dr. Werner von Braun’s newly American rocket cult published their manifesto in Collier’s magazine: Man Will Conquer Space Soon! This manifesto laid out the grand plan for space exploration that came to be known as the von Braun paradigm, which has since been the guiding vision for space exploration enthusiasts and much of NASA.

The basic idea is a massive wartime-esque mobilization of society towards space exploration. First, a fleet of reusable spacecraft to provide low-cost access to space. Then, permanent presence in low-Earth-orbit artificial-gravity stations. Then, expansion to the Moon, Mars, and beyond. By any means necessary.

Von Braun’s projects ultimately failed, as his disciples were politically outmaneuvered by bureaucrats and bean counters, but his disciples have kept the dream alive. The cult has since been picked up by a new titan: Elon Musk.

SpaceX is developing a fleet of low-cost, rapidly reusable starships optimized for Mars colonization. This is a great first step. It will finally fulfill von Braun’s vision. But if we’re going to build a Bison Sphere, we’re going to need much more than that. We’re going to need to involve Dr. Dyson’s visions.

The first thing that must go is the ridiculous pretense that SpaceX is a private company. SpaceX is an arm of the state, and everyone knows it. Elon justifies its budget by comparison to other things we collectively spend money on, and puts significant effort into convincing the public of its value. These are activities of state. It is the Space Exploration Agency, but for some reason we still insist that it appropriate its own budget. So while we’re throwing trillions around to keep the Wall Street complacency machine on life support, why not throw a few of those trillions towards a future we can believe in? Yes, I know it doesn’t quite work like that, but I’m sure the distinguished wizards who commune with the market can come up with something.

Besides a trillion-dollar budget, SpaceX needs access to America’s nuclear arsenal, which must be expanded to thousands of bombs per year. Nuke Mars, yes, but more importantly, nuke the back end of the Orion-class starships that will be moving asteroids around to build the Bison Sphere. It’s the only way to get the job done.

The Medicinal Power of Effort

Complex adaptive systems like society are like muscles. They need to be periodically intensely mobilized for great purposes or against great threats, or they atrophy and dissolve into unhealth. The parts of a living machine can only know their proper relations by practice.

We are antifragile, America. When disaster strikes, we step up to it and get stronger. But in the absence of this hormesis, we die.

The last time we really applied ourselves was over 70 years ago now. We landed on the Moon with the pump we got from the war, but have been languishing on the couch since 1973. Now at rock bottom, sweat pants stained with ketchup, the floor strewn with plastic toys from China, we have caught a nasty sickness.

The cure is sunlight and exercise. How much sunlight? All of it.

We must use this crisis to mobilize ourselves to become better. Otherwise, it’s over for us. We must permanently end this state of complacency and spend the next thousand years building the Bison Sphere. If we finish early, on to something else!

I have ignored the question of social and political organization. The reason for this is simple: once you have the intense overarching goal for which you must mobilize over the long-term, the rest falls into place. The complex adaptive system finds its functional organization in necessity. We need to feed, house, and provide virtue-building work for the people; we must be at the peak of our health. We need to build a strong economy; how else will we apply all the necessary industrial effort? We need to stop fighting and love each other; we have no time for disunity anymore. We need to learn to live harmoniously with nature, and build an ecologically integrated civilization; there is no other option for settling the Bison Sphere and flourishing in the long-term. Focus your vision on some great purpose, and everything else becomes clear.

The bottleneck, as usual, is mobilization in our hearts. We must develop the will to strive for more as a civilization, or we will die. We must come to believe in some great plan that is worthy of our most intense efforts. Perhaps you don’t think the Bison Sphere is good enough. I invite you to improve it.

In Sergey Zimov’s vision of the future, the nations of Earth will compete not on military strength, but on the size and health of their wild parks. America must not be outdone in this. We will use all the matter in the solar system in the construction of ours.

We can start right now, on Bison Sphere One (formerly known as “Earth”), by restoring the herds and predators and grasses to the American West, and building the functionally organized society capable of a grand effort of this kind. We have already begun. What else is there to do in quarantine? And what could be a better start to the real 21st century, in which at long last history has begun again?

I can neither confirm, nor deny that our black helicopters are already on the move.

This story is satire. It’s April 1st. The science, numbers, and engineering possibilities, however, are real, as well as more of the justification than you would like to believe.

I got a letter in the mail. “Jury summons,” it read.

They pull the jury summons randomly from the voter records, you see, and I had just voted for incumbent Vancouver mayor and militant cyclist Gregor Robertson. What can I say? I like bike lanes.

I showed up at the appointed time and was instructed to wait my turn, sitting in a grid of plastic chairs in the basement of the courthouse with all the others who had been summoned.

All kinds of people showed up. Vancouver is a fairly diverse city, especially the greater Vancouver metropolitan area. The voter roll was obviously representative of this diversity in a way you don’t always see on the street, bus, and train. People mostly hang around others like them, both in class and culture, even commuting at different times and by different modalities.

We waited for our turn, quietly cracking jokes and making small talk. There were a couple hundred of us total, split between two days of jury selection.

“This can’t be normal,” we thought. “Surely they don’t do this every time.” It wasn’t, and they don’t.

The officials explained to us that the trial was going to be five to six months long, and the selection process would be rigorous, so they had called in a lot of people.

Five to six months? An explanation made its way around: “Terrorism.”

After a few hours, they called me in for selection. It was a special, more secure courtroom built recently for high profile cases like terrorism where there was elevated risk of security problems. The defendants’ seats at the back of the room were protected by what looked like bulletproof glass. There were no windows, no pews for an audience, and guarded entrances. The audience was in an adjacent room, connected by closed-circuit television and thick plexiglass windows. That’s where the reporters sat.

I stood up in front of the judge, who I later learned was named Justice Catherine Bruce, to be sworn in. She asked me questions about my eligibility, while the lawyers inspected me.

“The defendants are accused of charges of terrorism, and are of the Muslim faith. Do you swear that you are able to evaluate the evidence without prejudice in this case?” I indicated that I was. I had no problem considering the facts dispassionately, though one always wonders if that’s really possible. Privately, I considered this to be an interesting philosophical problem, and thus an interesting challenge for myself. Fortunately, the common law court process is designed not to place too much philosophical burden on lowly jurors like myself, so I figured I’d be fine.

“Have you read anything in the news about this case, anything at all, that might influence your judgement?” I try not to read too much news, so I hadn’t heard anything about the case. Apparently it was big news.

“Do you have any connection with the accused?” I didn’t.

“Do you have any reason it would be an unreasonable hardship for you to be a juror for six months?” I was passing myself off as “temporarily retired” at the time, so I really didn’t have any excuse at all. Besides, it seemed like it would be an interesting and educational adventure, and duty is duty. I’m partly American, but not so American as to disobey the Crown.

The lawyers signaled that they had no objections, which meant I had now been selected, and was to go sit in the jury box. I quietly greeted the other jurors, and took my place while we waited to see who else would get picked.

This is when I got my first look at the defendants. A white man and white woman, well into their 30s or 40s, him in a suit, her in hijab, looking somewhat dejected in their respective plexiglass boxes. John Nuttal and Amanda Korody. Man and wife. Obviously of lower class, at least recently; you can read the stresses of poverty on someone’s face, and perhaps something of the personality that got them there.

When I heard that the trial was for Islamic terrorism, I had expected foreigners, or the children of immigrants, radicalized out of alienation in their new society. But they were recent converts, and very obviously nth-generation Canadian. Curiously anomalous specimens. What had happened in their lives to get them here? The anthropologist in me perked up.

We watched as potential jurors flunked for all sorts of reasons. Some would have excuses, like one woman who had just gotten a job and was going to school, and couldn’t afford to risk those opportunities. Some of the excuses were better than others. Some obviously made up. The judge was lenient, and let anyone go who really wanted to. But she also eliminated a few who seemed a little too eager to be on the jury bench. The lawyers also eliminated some with their veto.

The result was fourteen jurors, out of the large crowd who had come. Fourteen because they needed at least twelve, and figured some of us might drop out over the long trial. Thirteen of us were middle class and white. The fourteenth was a quiet Chinese girl, a student at the local university. It was a striking contrast to the demographics of the selection sample overall, but we didn’t talk about it too much.

Thus ended the selection.

When the trial itself began, we quickly moved through the opening introductions, and the Crown prosecutor, a goateed man in early middle age, began presenting his evidence. The basic parameters of the case were these: the defendants had placed what they thought was a C4-loaded pressure-cooker bomb outside the British Columbia Parliament building in Victoria. Undercover police officers had worked with them through the whole process and recorded it extensively. The evidence amounted to something like two hundred hours of video and audio surveillance recordings, and extensive testimony from the officers involved. No wonder it was going to take six months.

The Story Of John And Amanda

As we watched the footage, and listened to John ramble about his plans and past experiences, I managed to piece together the broad strokes of their story.

They had both grown up and gone to school in British Columbia, and were from presumably normal Canadian families. Amanda had an interest in alternative fashion and Japanese manga-style art, becoming reasonably talented. We could see her art when they showed us, and the public, the private contents of their computer. Common if not typical interests, probably picked up in high school. John had wanted to join the military, but didn’t make it. He loved the movie Rambo III, and would refer to it often, especially the Mujahideen.

Somehow they ended up with anti-imperialist political tendencies, also not uncommon. Relatedly, their social circles over time tended towards the grungy, almost crust punk. I don’t know the details, but I spent some time in similar circles myself, having lived in a punk house for a while, and the aesthetics were unmistakable.

When 9/11 happened, John said he cheered it on instinctively. Finally, the Great Satan was getting its due for all the bombs dropped on the Middle East. I found this interesting because it meant their anti-Western terrorist affinities existed before any religious affinity for Islam.

Somewhere along the way, they ended up in poverty and on heroin. Again, the details are obscure, but in underworld social scenes, drugs are common, and there is a well-known pipeline from otherwise normal misfits to drug addiction and often death. For some years, this was their life.

In that dark place, one of their friends gave them a Quran and introduced them to Islam. They read the whole book together, and became convinced that it was divinely inspired. They sought out someone who could tell them more, and began initiating themselves into Islam.

The conversion seemed quite genuine. They went to the mosque, quit all drugs, and read the Quran together. They knew—and properly used—dozens of Arabic religious phrases. In court, Amanda would spend some of her time copying out memorized Quran verses on a pad of paper.

But John had a seemingly cartoonish view of Islam at times, as a sort of religious veneer over an underground global brotherhood of heroic Mujahideen. It was hard to tell when he was telling stories and when he was fantasizing, but this seemed to have manifested in him asking around at mosques where he could sign up for jihad and get his AK-47. They didn’t tell us how the police originally got on the case, but it’s not hard to imagine.

Which brings us back to the trial. The police assigned an undercover officer, the main prosecution witness, to make contact and see what the hell was going on. The officer was a middle-aged Moroccan Muslim, perfect for this purpose. They made contact at a convenience store, and the officer pretended to need help finding a runaway niece. As they sat in the car chatting, John noticed the Quran in the back seat, and immediately launched into describing his terrorist fantasy plans to this apparently like-minded Muslim.

From stealing guns and storming a military base, to stealing a nuclear submarine, to shooting people with a modified paintball gun, John had big plans. It was clear the police had to do something.

But John and Amanda couldn’t seem to actually do much, either. For months, the undercover officer would occasionally hang out, buy them lunch, and talk with them about their plans, while surveillance officers bugged their apartment and kept an eye on them to see what they did. Nothing ever went anywhere.

Eventually, the officer started pushing a bit harder, getting them to settle on a pressure cooker bomb inspired by the Boston bombing, and offering to acquire some C4 and set them up with fake passports to get out of the country after the deed. Facilitating nearly every aspect of the plot, the police subtly moved things along, leaving just enough work for John and Amanda that they would be unambiguously incriminated when all was done.

We watched on camera as they pieced the “bomb” together and blasted anti-Israel nasheeds so loud they could be heard outside the motel room. The police provided fake C4, which was just plasticine, but also a small piece of real C4, so that the device would technically contain explosive materials, and the Crown could get them on yet another criminal charge.

When the time came for the verdict, the instructions were clear: leaving aside such issues as entrapment, speaking technically, did they or did they not do the deed? I’m bound by duty not to say anything about the deliberation, but it’s public record that the jury found them guilty.

After several months more argument about the fine points of the law, the judge acquitted them on the basis of entrapment, and released them. I can’t say I disagree. I hope they are doing well and staying out of trouble.

It was a farce. A fake crime, manufactured by the police, at enormous public expense. All totaled up, between the approximately 120 police officers involved in a case that spanned at least a year, at least a year of court time, press attention, and time out of the lives of jurors and defendants, it amounted to a significant expenditure of scarce resources. I have no doubt the total bill was in the tens of millions of dollars. What was it all for? Was it just waste? How did it happen?

There are probably ways that the police could have handled it better, had their procedures or incentives been better. One rumor is that they were under pressure to produce terrorism convictions, which is why they didn’t take the de-escalation opportunities they had. For example, after having made contact, John expressed doubt at a few points about whether terrorism was permissible under Islam. The undercover officer sidelined those concerns. They could have instead connected John with a credible scholar to talk him through a deradicalization process. The undercover officer could have attempted such a process himself. They could have just surveilled John and Amanda, and sat on the case making sure they didn’t do any terrorism, while this deradicalization was underway. They didn’t.

But deradicalization is hard. Barring simple cases of the ignorant and confused, your counterparty generally no longer believes in what you believe, often for good reason, and furthermore doesn’t believe you. You have to actually understand where your counterparty is coming from, which is something liberalism is notoriously bad at. You have to build real trust, which is difficult and fragile when you are not actually sincere, but just playing the part of deradicalization counselor. You need to be able to offer a better way, which may be difficult if your counterparty has some legitimate grievance with the system.

I sympathize with the police for this reason. They had a bunch of radicals running around talking about blowing things up. They could have met up with some real terrorist cell and done something real. The police had to do something.

But let’s back up a bit. How could this have been prevented? How did it happen in the first place?

How did they fall out of the bottom of society and into a subculture of drug abuse and antisocial ideology? This isn’t even just about Islam. Radical Islam provided a ready-made terrorist ideology, but the receptivity to radicalization and terrorism clearly pre-dated it. Islam might have even provided some positive social fabric and moral guidance in an otherwise nihilistic underworld environment. Why does an antisocial underworld exist at all,  and how did John and Amanda get into it?

Society And The Radical Underworld

In any society, there is a vast network of interrelated institutions, organized into a rough pyramid. The nexus of power and politics, the state, sits at the top, and all the various functional components are organized beneath. Government bureaucracies, established companies, startup companies, small businesses, nonprofits, university departments, strong families, religious parishes, and a vast interconnecting mycelium of informal social fabric. This forms the main structure of a functional society.

But by default, it takes certain qualities, choices, and resources to stay on track into the institutions of proper society. You need to go to school, stay out of certain social groups and get into others, study and behave in particular ways to build social capital that can get you into the official institutions of society. What happens to those of us who fail to do this?

Those who fail to get in, are pushed out, or are never convinced in the first place, fall into the unregulated between-spaces. The underworld. Improper society. The human landfill of people who are discarded by the system.

Judging by their interests and associations, John and Amanda were already off the main track by high school and on the track to the underworld. They were on welfare, and I heard little of real jobs.

Lacking participation in a functional social fabric, such people become easy to radicalize. It’s not just that they are at the bottom of the social hierarchy, but that they are barely participating in the normal social system at all. It’s very easy to believe society is bad when you’re not part of it.

This reveals a problem with a social order, like ours, that incorporates people mostly on the basis of a local economic calculus, especially in the context of increasingly tracked and expensive credentialing: it’s easy to fall out, and hard to get back in. A lot of people end up outside, and become not just discarded refuse, but active social liabilities. Idle hands, as they say, are the tools of the devil. The local economic calculus has no reason to clean up these liabilities, only to find assets.

I don’t know how to measure this rigorously, but I think this has been getting worse for decades: the institutionalized proper part of society is receding and becoming inflexible, and the disorganized underworld advancing, though nominal productivity continues to rise. We see this in statistics on underemployment, NEETs, marriage rates, people dropping out of social participation, etc.

But many of these measures, employment in particular, will underestimate the problem; the economy is increasingly home to what is effectively underworld activity: scams, fake projects, ill-advised speculative investments driven by too much money with too few useful things to do. I don’t know that I can speak to the average, but for many of the young people I know, purposeful positions within functional institutions are rare. Their fathers and grandfathers were men of society, working in established companies, serving real customers, and commanding national infrastructure. Most of them went through college fine, but had little guidance from society, and were spat out into a weak job market dominated by companies without much in the way of legible higher purpose or functionality—fake institutions.

The psychological impact of these fake institutions is to cut one off from the main functionally organized social fabric. You feel it less directly than exclusion, but how can one appreciate and support the purposeful order of society if one’s own position is not really part of that order? So again, just looking at economic marginalization will miss a large part of the story: even when you’re in, the functional logic of your job, and thus your tie to the larger system of purposeful social order, is obscure or missing.

This alienation is often not total. People often have one foot in, and one foot out. John and Amanda were married, but didn’t have jobs. Many people have jobs, but have no prospects at marriage. Some people have jobs and marriages, but no social life integrated with normal society. The more you’re integrated into a functionally organized society you can believe in, the less the potential, and reason, for radicalization.

If I’m right, growing radicalization online, mass shootings, extremist ideology, terrorism, and so on aren’t happening simply because there is a growth in possibility, but also a growth in demand. Look at the people involved. Look at their faces. Look at their lives. They are the kind of people who have fallen out of the system. NEETs, incels, the fatherless, and the unpopular are vastly over-represented in this underworld. And the trends aren’t in their favor; it seems proper society is increasingly frayed, becoming weaker in its ability to provide positive leadership.

I don’t know for sure, but I’ve met enough social dropouts to see the patterns. I would bet that John and Amanda fell into the underworld in the first place because society failed to give them purposes they could believe in, and failed to make use of them. Maybe they couldn’t get in, or maybe didn’t see a reason to. Either way, they found other things to believe in, and other things to do.

Reintegrating The Underworld

Ultimately, this social underworld is everyone’s problem, and therefore the state’s problem. Everyone else individually can mostly ignore the problem, simply excluding the growing population of misfits and apostates from their own domain while they fulfill their own duties.

But ultimately, something has to be done.

First, because there’s actually a lot of raw human capital that slips into these between-spaces, just not in a format that’s easy for current institutions to inspire, absorb, and rehabilitate. At the time of this whole affair, John and Amanda were able-bodied, and even quite bright, when they weren’t lethargic from the methadone that helped them recover from addiction. What they lacked was the discipline and structure that for most people comes from their participation in a functional social fabric. Such re-absorption requires new institutions and the backing of the state. Without the backing of the state, the social gains of reintegration aren’t enough to justify the effort for any private actor.

Seth Largo’s recent article on the social integration of a generation of Mexican-Americans through the war and Apollo years is a great example: the author’s Mexican-American grandfather joined the military during the war, and then worked at a government-integrated contractor and on the Apollo project after the war. His descendants married into mainline American social fabric, and are well-integrated in society. A large need for functionally organized human capital on the part of the state, like a war effort or moon shot, can drive the creation and extension of institutions, and the absorption and training of human capital, that would otherwise end up drifting around outside of proper society.

Second, there is the humanitarian concern of needing to rehabilitate those who fall out of society, so they don’t just live a life of alienated misery. John and Amanda may have been free, and basically fed and housed, but theirs wasn’t an existence of hope and purpose. The need for belonging and meaning is not a need that we acknowledge much these days—it’s not part of the liberal ontology—but it’s nearly as necessary to healthy human life as food and water.

Sometimes, if done by the institutions of proper society, this might look like work programs, where the government, or a church, or some other humanitarian-inclined institution takes a population of otherwise idle people, and finds ways to give them purpose and make use of their labor, so that they can be part of society. I have spent many days walking on trails built by such people in the labor programs of past decades, and I have dishes at home made by people from “less employable” populations, from a company that has this explicit humanitarian logic. This is not unheard-of.

Strictly speaking, these efforts are not always economically worthwhile. It is sometimes more efficient to employ those who are well-integrated, and simply ignore or liquidate those who don’t make it in the system. But this just serves to illustrate the implicit brutality of purely economic logic. It might be unrealistic or foolish to extend the ethical imperative too far, but there is at least some humanitarian responsibility to make sure people who are supposed to be part of our society reliably have opportunities to participate in its functional structure.

The third reason is that if you don’t rehabilitate the social underworld, it will occasionally be turned to nefarious purposes. A bunch of human capital sitting around alienated from the main functional structure of society will be ready resources for antisocial ideologies. Again, this is a strong candidate to explain the recent rise in antisocial attacks and radicalization, and what happened to our own John and Amanda.

The few systems we have to deal with this underworld of social liabilities—police, censorship, disruption of underworld networks, de-empowerment, and social shaming—are brutal, expensive, and sometimes ineffective. They have their place as part of a larger suite of solutions, but as the underworld problem gets worse, these half-solutions just get applied with ever more vigor, and don’t solve the root problem. They in effect just push people away even further; making it clear to them that society is persecuting them, that they are not welcome, and therefore that they must radically remake or destroy society for any acceptance at all. This is, ironically, part of what holds these underworld ideologies together, and makes them dangerous.

This is not to say that escalating persecution of the ideological underworld won’t work in the sense of yielding victory for the hegemonic incumbent. Power wins. But that escalation is occasionally inefficient and brutal, and the need for it is indicative of serious structural problems in society.

This is part of the genesis of the Christian principle of love for the downtrodden. Both in social logic, and in historical causality. If proper society doesn’t reach out, empathize with, and re-integrate people who have become alienated and even antisocial, the problem just gets worse, because someone else will. Idle hands, again, are the tools of the devil. It’s not just about the poor, but about people who aren’t fitting in and getting along, too.

In the Roman Empire, an explosion of cults and street prophets accompanied the growth of their unintegrated underworld as the main social fabric frayed. Early Christianity flourished because it offered purpose and compassion to the excluded of that time, who amounted to a serious coalition once radicalized to the point of martyrdom.

Islam worked similarly for John and Amanda: it reached out to them and offered them meaning, while proper society had failed to do so with credibility.

So let’s have some sympathy, even for those who may seem lost in the underworld. Sometimes they’re there because there isn’t anyone else reaching out to them and giving them a purpose.

Escape From The Underworld

My experience on this trial forms an interesting case study in its own right. I’ve done my time in various underworld scenes. I’ve hung out and participated with more than my fair share of hooligans, hackers, crust punks, and revolutionaries. You wouldn’t be wrong at all to say that I have been seriously radicalized, confused, and lost at various times in my life, left to invent my own social purposes, which were usually wrong.

I went into the trial with a skeptical attitude. Surely there is a more efficient way to decide cases. It all seemed like ridiculous overkill. Was the whole thing legitimate at all? I’m sure there are always improvements to be had, but sitting in the jury bench for six months, watching our ancient rituals of justice play out, I came to appreciate them. I came to respect the authority they represented, and feel a little more at ease in the society they protected.

I was drafted into service of the Crown, placed in a position of solemn responsibility, and given instructions as to my duties. I carried them out proudly. And the interesting thing is, it deradicalized me a little bit—watching the whole thing play out. It deradicalized me much more effectively than any conceivable direct deradicalization or suppression effort could have. Being told by some toadie of the system that you’re wrong and bad doesn’t change anyone’s mind; it just drives you further away, or at best scares you into submission. But participating in purposeful and functionally organized social institutions helps us understand what’s good about society, and how our own lives fit into it, and thus unlocks much more of our potential. Participation in shared social purpose is what makes society real for us.

If the underworld seems to be advancing, and the youth are increasingly radicalized, then it might serve us well to worry less about how to combat radicalization directly, and worry more about the ways our proper social fabric is decaying, receding, and failing to take leadership in pulling people up into higher purposes. The solution may not always be to suppress and control people radicalized against society, ruling them through fear and power, but also to build a stronger society that commands more respect and has more purposeful places for everyone.

Humans, and organizations of humans, are currently the most powerful intelligent forces acting on the world. This is because we are the only forces that have the necessary general intelligence to have an impact on this scale. We have power over our environment, our rivals, and the future, because of our unique socially scalable intelligence. For now.

AI presents us with the possibility of powerful, non-human agency. What if machines could think and strategize and wield power? Would those machines participate benevolently in the human social order? What about when industrial production of silicon computing hardware, fine-tuning of the parameters and algorithms of software intelligence, and digitally precise coordination at scale allow those machines to outpace us in total coordinated intelligence? At that point, we would be faced by non-human intelligence powerful enough to be beyond human constraint, potentially able to scatter and outmaneuver us the same way we overcome our own rivals, if it were so inclined.

The question, then, of whether that machine intelligence would be benevolent, indifferent, or hostile, and the extent to which those concepts can be applied, is of supreme importance. If it’s weaker than we are, control is relatively easy; if it starts doing the wrong thing, we can just shut it down or change its programming. But if it’s smarter and stronger than we are, the reverse applies. We need to it be stably benevolent so that it’s still good when we can no longer control it. But when examined with rigor, the problem of building a stably benevolent machine intelligence turns out to be very difficult, once the possibility of superhuman intelligence and self-modification comes into play. There is usually some loophole in the motivation scheme that will lead it to break out of our implicit intentions in hard-to-predict ways. Building a dangerous and uncontrollable general intelligence seems much easier.

Further, the sheer interestingness and potential power of general intelligence mean that if such a thing is possible, it will probably be built, possibly this century. As such, we’re in a sticky situation: we’re probably going to build something of world-changing power that by default won’t be aligned with what we want for the future, and which we won’t be able to control, contain, or resist.

So, we need to figure out how to get this right to avoid an existential disaster. The capacity to get this right, in the philosophical, mathematical, engineering, and political dimensions, is therefore of high strategic priority, at least on a medium-term timescale. The most important immediate step is understanding the problem well, so that as we go about planning and preparing for the next few decades of social and political change, we do so with this need in mind. We need some idea of how to handle superhuman AI before the situation becomes critical.

To better understand the problem posed by AI, I recently talked to Jaan Tallinn, best known as the founder of Skype.

Jaan has spent the past 10 years meeting with key figures, funding important projects, and assembling strategic knowledge relating to AI risk, so he’s one of the best people to talk to for a well-informed and in-depth overview of the topic.

Going into this discussion, I was frustrated by the relatively shallow depth of most discussions of the topic available in non-specialist media. It’s mostly the same few introductory questions and arguments, aimed at a very general audience with low technical capacity, intended to titillate, rather than to inform. To get a useful handle on the topic, so that it’s more than just a science fiction hypothetical, we need more technical depth.

I called up Jaan in the middle of the night, and we recorded a wide-ranging discussion of the background, technical challenges, and forecasts of the issue of AI risk, and what approaches we might take to navigate it. The following transcript of our discussion has been minimally edited for clarity.

You’ve expressed concern in the past about artificial general intelligence (AGI) as a strategic concern, or even a threat to humanity in the 21st century. So, I’d like to briefly go through the argument to get your thoughts on it. As I understand it, the AGI concern is not about deep learning. It’s about a more advanced capability—the ability to learn a causal model of the world from observation, and then plan and reason with that to accomplish objectives in the world. This is something that humans do that current software technologies don’t do, and deep learning doesn’t do.

I think it’s an open question. Current deep learning cannot do this, but the question whether deep learning will take us all the way or not is unsolved. There are people whom I respect on both sides of this argument, so I’m truly uncertain. I know people who have strong opinions that deep learning will take us all the way, and I know people who have strong opinions that it will not.

So, the fear with AGI is that, like deep learning, it will be highly scalable with computing power and algorithmic progress, so that once we do have the core capability figured out, it could rapidly get away on us.

It’s even worse. We might sort of “pull an evolution.” Evolution didn’t really know what it was doing; it was just throwing lots and lots of compute at a very simple algorithm, and that invented humans. So it’s totally possible, if not very plausible, that we might pull an evolution in that way, that we never really understand what happened. It seems unlikely, but I don’t really know how unlikely.

That would be something that is on a gradient. You deliberately build an AGI, but it ends up doing things and working for reasons you don’t really understand. You can imagine a spectrum of possibilities between that, and building something that accidentally ends up being AGI.

Evolution is a very useful anchor here. Ultimately, if we had enough compute, we could replay evolution. Of course evolution had, as I say, one trillion sunsets to work with. That’s a lot of compute. But we are also way smarter than evolution, so we don’t need that much.

One way I’ve heard it put, I think in a previous interview with you, was that whatever comes out of this AGI thing would become the driving power of the future, in the sense that it would be the thing defining what the future is like.

Yes. It’s very likely that this will be the case.

Whatever nature that it has at that point, we will no longer really have power over it, so we better make sure that if we’re going into that, that we build something that creates the kind of future we would want.

That’s pretty much the definition of aligned AI. We want this entity (or entities) to be aligned with humanity.

The idea is that as we create this AGI, it should stably do something that we want, rather than something else?

There is more nuance to this. It’s possible that we will have the problem manifest before we get to AGI. The Open Philanthropy Project has coined this term Transformative AI. The idea is that to be AGI, technically it should be fully general, whereas it’s clear that in order to take over the world, you probably don’t have to be fully general. So it’s possible that the AI that will eventually present an existential threat to us will not even be an AGI, technically speaking.

It could just be a superweapon level of thing that gives someone an immense amount of power, or creates an immense impact somehow, without being fully general.

There’s this really cool book called Avogadro Corp. I have to say I don’t endorse the ending, but the book itself is really cool. It presents an AI—it’s not even clear how general it is—that is able to intercept people’s emails and rewrite them. All hell breaks loose, and soon enough, the AI has military capability, and people don’t even know where it is, much less how to stop it.

That’s one of the big fears that people don’t really understand well enough when first thinking about this. If the thing can get control of enough data streams, in a widespread enough way, there’s actually a lot that you can do with that. People say: it doesn’t have hands, so how is it going to hurt us? But there’s actually a lot that it can do.

I just read Stuart Russell’s new book, or a draft of his new book, and he makes a really good point, that one way of looking at this whole Facebook scandal is that Facebook’s AI—though that might be too charitable to Facebook’s AI capabilities—is changing people’s behavior to become more radical. Radicalized people are more eager to click on advertising or sponsored links. So think about it. It’s not just that the AI is optimizing the news stream. The AI is optimizing the viewers.

That’s an example of the unintended consequences problem. You think you’re doing one thing, but along the way to accomplishing that, an uncontrolled intelligent system will potentially do a bunch of other stuff that you didn’t intend and don’t want.

Exactly.

One of the biggest problems with AI is that it seems to be extremely enticing to a lot of parties. Pure intellectual interestingness, economic value, military value, various utopian beliefs around the thing, all motivate a lot of work on the topic in a relatively uncontrolled way. So it seems likely to happen by default in a relatively uncontrolled way. It’s not something that happens only if we really want to go there, and we make some concerted effort to pursue it. Rather, it seems like it’s something where there will be many actors who are really trying to pursue it.

The good news though is that the leading AGI developers in the world have realized the problem. In one world model, OpenAI and DeepMind are the leading ones, and they both have full-time safety staff, and are aware that this is a problem that needs to be solved.

We’ve definitely heard them make noises to that effect, that they have safety staff and so on, but I don’t know the internal politics of these companies, or how competent those safety staff are.

I can definitely vouch for them. The safety staff is very competent. It remains to be seen though how well their organizations are able to integrate the safety efforts. I think OpenAI is in a much better position than DeepMind when it comes to that, because OpenAI was largely developed around the safety effort. From the start, people were concerned about safety, whereas DeepMind tacked this capability on later. But the people are good. I talk to them a lot.

These are the things we really want to know. It makes a big difference what that AI safety team actually is. Whether they are a serious operation or just kind of an afterthought.

I think in very stark contrast is Google Brain, which doesn’t have safety researchers, nor does it have a safety culture. It basically has a culture that is effectively against talking about safety, from what I hear. It’s really interesting. When I talked to Jeffrey Dean, who is the head of Google Brain, he basically acknowledges the safety problem, but is a very hands-off manager, so the culture there never ended up taking safety seriously.

That’s worrying. That’s enough context that we can start digging into the interesting details. One point that comes to mind is the philosophical difficulty of this alignment problem. In a lot of the discourse around it, you find the idea of human values. The idea that there’s some particular set of things that we find morally compelling, that could in principle be systematized and scaled. But this isn’t obviously possible, or maybe we have the wrong view of what that looks like, or how that works. And I know there are a lot of people who take other views on whether aligned AI is even in principle possible, or how that would work, or what that even means. I don’t have strong philosophical views here, but I see that it looks like there’s a lot of philosophical complexity, and a lot of smart people with differing views on the subject.

One likely way out is to punt the problem to a meta-level. Basically, instead of figuring out what we want and then programming that into the AI, we program into the AI that it’s supposed to figure out what we want.

But that would have similar issues, though perhaps fewer tricky issues. Just in determining whether that would be a good idea for example, what that would look like, what that would mean.

It seems that there’s a list of answers that we need. Some of those answers are our homework, like we have to do them. For example, how to make sure AI doesn’t deviate from its initial trajectory, plus general robustness and motivational issues. However, for a bunch of stuff, including much of the metaethics research, it’s very possible that we don’t have to finish them before we get to AGI.

Some of these problems can be punted to that meta-level. The AGI can basically handle a lot of that downstream. But there’s this big question of how much philosophical difficulty is there in the stuff that we would have to do.

One of the best people to speak about it is Wei Dai, a person who has several cryptocurrency units to his name. (Laughs). For the last several years, he has been thinking about philosophy and AI—what is our philosophical “homework,” and what does it mean for AI to do philosophy?

We’ve talked a little bit so far about how current deep learning relates to the AGI research program. But there’s a related question which is how hard AGI is, or what kind of problem it is in terms of the scale of work or the type of work. You could imagine it could be a simple problem of conceptual breakthroughs, on the scale of blockchain or deep learning. Some really smart person just has to figure out that core algorithm, and we just haven’t discovered it yet. Or you could imagine it being an immense engineering effort optimizing and working with deep learning or other algorithms.

There’s no black-and-white distinction there, because when you are applying brute force, it’s possible that the system you are brute-forcing will do the invention of these algorithms, just like evolution did the invention of the algorithms in our brains and other brains.

The idea being that if you get the right containing paradigm and search algorithm that you’re working in, and you throw a lot of effort and computing power at it, you may end up producing the real thing.

Eric Drexler, the inventor of nanotechnology and a good friend, has this framework he calls Comprehensive AI Services. The basic idea there is that it’s perhaps a mistake to look at AI developers and AI separately. It’s a mistake to think about people developing AI and the AI starting to develop itself as distinct regimes. Perhaps it’s more of a holistic system, where you have a combination of people and AI that are making the AI better. Some of the optimization pressure comes from the people doing it, and some of the optimization pressure, we already know, is coming from the AI. Google Brain has this program called “learning to learn” where they create AI that is able to create AIs. So there’s very explicitly a division of labor between the AI researchers and the AI. In that sense, it’s clear that this combination of AI researchers and AI that they are working with, the tools that they are working with, this combination needs to make one or a few conceptual breakthroughs, very likely. But it’s not obvious that the people side will have to do it.

You have this combined system of humans and algorithms working together, where the humans are shouldering some of the burden of the things that the AI system can’t do, and optimizing it on the things where it’s slow.

Look up architecture search. This is an explicit thing.

“Architecture search”—is that an idea of searching over what would currently be deep learning network architectures, or is that something more general?

I think it’s more general, although in practice it’s mostly applied to deep learning architectures. I’m not entirely confident, but the term certainly sounds more general.

As a concept, it definitely sounds like a good idea to do. But I’m curious whether anyone has figured out how to do any kind of systematic architecture search outside of a very well defined space like neural networks. My next question is about one of the obvious things people come up with to deal with this AI problem, which is the idea of regulating it somehow. You can treat it like nuclear weapons, or environmentally destructive chemicals, or the various other things that governments have stepped in to control in the past. How we might do that, and whether it would work, is an interesting question. And then whether that’s necessary. Are these AGI labs going to be self-regulating?

In my view, there is at least one aspect that can be regulated in AI. Whether it’s a good idea, I’m not sure, but I’m pretty sure it’s a good idea to prepare to be able to regulate it—and that’s computing power; concentration of computing power. Because that’s hard to hide, whereas algorithms are very hard to regulate. It’s hard to monitor what exactly is being run on those CPUs, but just the number of CPUs and the amount of energy that they draw, this is something that can be observed from satellites, etc. If there is a capability, once things are starting to get out of hand, you establish some regulatory regime, and regulate compute as we regulate nuclear materials, for example. As a preparatory approach; I’m not saying that we should regulate it right now.

One of the funny metaphors that I have is that it is possible that the remaining lifetime of humanity is now better measured in clock cycles, rather than wall clock time. There’s some amount of CPU cycles that humanity has left, and the more CPUs that come online, the shorter humanity’s lifetime.

And that would be predicated on the idea that the critical path work is now being bottlenecked on computing power?

Kind of, but it’s not strongly predicated. Because there’s this combination of tools and people who are working on it. This combination needs to come up with some conceptual breakthroughs. It’s very possible that if we have stronger tools in terms of compute, it becomes easier to come up with those conceptual breakthroughs for humans, as well. For example, a few years ago I talked to someone at DeepMind who was skeptical of AGI, and he said that it felt like we had 300 years left at least because the tools were so slow. He typically had to wait a week for experimental results to come in. He no longer thinks that we have 300 years.

300 years is a long time. The application of regulating computing power to AI is obvious, but there are all these other things that get done with enormous amounts of computing power, that are also of increasing importance, and seem to be increasingly AI-like and AI-related. What does Facebook do with enormous computing power? It optimizes people’s moods and political beliefs and whatever else. What does Google do? Something similar. Even beyond the issue of AGI, as computing power becomes relatively closely tied to power in general, then for a particular organization or a particular state to amass an enormous amount of computing power is something that could be important to regulate in general, not just for the AI issue. It would have all these other effects.

The way I look at it, I think we are already starting to see the paperclip problem. Facebook’s AI is a problem in at least some similar sense. The interesting question is: how does Facebook’s AI debacle generalize? It’s very possible that it would generalize in a really unfortunate way, where there would be more and more of people’s behavior and physical environment under the control of some AIs that run in some corporate data centers.

That brings us into some really interesting questions of who gets to govern the social fabric and the mass of people, and for what purpose. Do we allow Facebook to amass vastly more real optimization power over how people think and behave than, say, the government has? It brings up questions of political organization.

A couple of years ago, I gave a talk at the Asilomar conference about how it’s time to start dedicating AI resources, AI research, AI optimization power, towards what I call preference discovery. Right now, we have these economic and political incentives, and then AI is adopted to work along the lines of those particular incentives, be they monetary or political. Whereas it would be as interesting or much better to have unmotivated research, to the degree that’s possible, that uses AI, to figure out what are the correct incentives that we should be designing for. One problem that we have in regulation right now is that we know that there is AI being applied in the wrong direction in various places, but we don’t know what the right direction is. It would be valuable to start doing more research there. There is a bunch of social science research already, but there isn’t much AI optimization power applied towards helping that research. I do have a few ideas myself on what to do.

That’s another area where the question of whether the problem is computing power–bound, or conceptually bound, or paradigmatically bound, is interesting. Implicit in the application of current AI technology to the problem is the idea that we would be able to frame the problem so that current AI technology could give us useful solutions.

The situation is that right now, the world doesn’t provide incentives to do AI alignment. Preference discovery is just one piece of AI alignment. Because there is no incentive, it means that there has been very little effort towards that, so it is very likely that we have low hanging fruits there.

The general idea seems good. We should study the incentives on these big chunks of optimization power. This big chunk of power at Facebook, for example. What are they doing with that? How do they profit with that?

It’s important to tear apart two directions of research: one is the empirical part of the research, where you look at the existing things and how their incentives are distorted, and the other one is the normative part, where you look at things like what we should be doing with that optimization power in the first place.

It seems like there’s two parts to that normative part. One is in general what should be doing with that optimization power, two is, given a vision of what we want to do, how do we actually incentivize that?

That’s true. Glen Weyl is, in my view, one of the leading mechanism designers in the world, and he thinks a lot about this. He thinks a lot about how the world has, for historical reasons, picked up a ton of bad incentives, and how we can fix those incentives.

It’s clearly a really important problem. The structure of incentives, and more generally, the structuring of this kind of power in society. One of the things that seems like a big part of this problem is what we might call the strategic capacity to even be able to deal with stuff like this. We need to be able to answer some of these philosophical questions in a relatively official way, we need to be able to answer the mathematical questions, and we need to do a lot of really good engineering. At some point, it may be necessary for the state to take control of the developmental situation, like it did with the Manhattan Project, and drive it in the right direction. Or, if it turns out that we shouldn’t do AGI, we need to make sure we have the state capacity and strategic capacity to do something about that.

Yeah, to the degree that you trust government entities.

There’s this question of who you trust, but whoever you do trust, it’s clear that they need a lot of strategic capacity—which is the ability to understand the situation, and then act to change the landscape. The issue is that this would require a lot more of our human institutions than I would currently be comfortable expecting. This seems like a really hard problem that we’re not necessarily up to. Then that puts the need to increase quality of human institutions, and increase strategic capacity in general, on the critical path of the AI issue. I’m curious whether you have thoughts on the whole human organization side of this AI problem in terms of developing that strategic capacity.

I think it’s really strongly a function of timeline expectations. If the timelines are 10 years as some people say, then it’s pretty much hopeless to try to get governments up to speed. However, if we have 50 or 70 years left, and the people who will actually be making those decisions have not been born yet, then it’s very possible that social structure engineering efforts are potentially very valuable. Especially in the long-term. Almost nobody thinks long-term, perhaps with the exception of China, so there is very little resistance when it comes to trying to make long-term changes. For example, somebody told me this (but I haven’t looked it up), that when they eradicated smallpox, they basically went country by country in Africa, and if for whatever political reasons there was not a good climate, then they just waited for the regime to change. There was no resistance to that kind of long-term plan, because people had short-term incentives, rather than long-term incentives. If we have enough time, then indeed, we should start doing gradual long-term changes that there will be very little resistance to.

Then the question is: what are those changes? To sit and wait, you need to have this good proposal for, when we get a chance, what we’ll do to get more strategic capacity on the problem.

Exactly. I do think that redesigning incentive mechanisms and trying to build communities that are aware of the long-term problem are the first two obvious natural things to do, and they are already happening.

That community aspect, of creating more people who really understand these things, that’s definitely something we’re interested in at Palladium. Getting back to the timeline issue, since that answer was predicated on timelines: you’ve said a bit about the safety efforts at DeepMind and OpenAI. You’ve said a bit about China’s long-term approach. There are multiple efforts in the Anglosphere to create AGI, and there are probably efforts in Russia and China. I don’t have any positive knowledge of those, but those states have made statements to the effect that they want to work on AGI. China has a plan to achieve AI dominance. You’ve talked to a lot of the smartest people in the field: what is the state of the landscape right now? How close are these organizations? Which are the interesting organizations inside and outside the Anglosphere? Can we get some information on the timeline issue?

There are certainly some projects in Asia, but I’m not very familiar with them. There are a couple projects in Japan. One of them I visited, and the other one I might visit in the future, but I’m not very familiar with them. In China, I’ve seen a document that somebody put together about what the projects are, but I haven’t read it yet. When I go there, it’s an interesting situation there, because they have super strong incentives to be competitive in the short-term, both on the government and individual level. They have a very harsh sort of cowboy capitalism going on there. It means that they are very busy making money, let’s put it that way, so the philosophical or ethical considerations aren’t part of their day-to-day thinking. The interesting thing, though, is that they are much more open to talking about long-term concerns than AI researchers in the West. My model is that because the Chinese haven’t been developing AI for so long, they don’t have so many strong defensive positions when it comes to new arguments. If you go to someone at MIT who has been working on AI for 40 years, and you say, “Look, this thing might be dangerous,” he won’t really think of a response; he’ll have some cached response that is dismissive. In China, they don’t have those cached responses, so they’re willing to think along. In Russia, I don’t know, but Russia doesn’t seem to have the people. The numbers are not there.

I think the Russians had some semblance of AI dominance in the ’80s, where they actually were developing a lot of the neural network technologies, and were ahead of the West.

Russians are also an interesting case in that they are super strong technologically and scientifically as a culture, but for the sake of God, they cannot govern themselves. They can’t build a functional government that is actually good for society, and this ultimately blunts their scientific efforts. If you become famous in Russia, you have to emigrate.

So, you see a lot of the short-term, highly competitive thinking in China, and not necessarily the long-term engineering. But with the Western projects, how close are we? Do you take the 10-year idea seriously, or are we talking more like 30 years? Or is it just hard to know? What does it look like?

I think the correct answer is to just have a probability distribution over this. I condition on lack of other catastrophes, because it’s possible that we will never get there, but the main reason then will probably be that we screw up in some other way and stop the progress. But conditioned on no other disasters, my median is in the 2040s somewhere. Kind of Kurzweilian, I guess. But can I give less than 1% in the next 10 years? No, I can’t. I think it’s more than 1% in the next 10 years.

Let’s get back to that cultural issue of the difference between China and the West. One of the hypotheses I’ve heard, on top of what you were saying about how long we’ve been looking at the problem and how much people’s careers are entrenched, is that another difference is that in the West we’ve recently had people thinking about the safety issue. You get this impression of some of the people going around talking about that being relatively annoying, or doing it in an untactful way. I’ve certainly heard that from people in the space, that they cringe when they hear people talking about the safety issue, just because a lot of the people who talk about that aren’t really being very effective about it.

Yeah. A friend who just visited me from the U.S. has a general observation that he doesn’t know what to do with people who endorse the ideas that he thinks are important, but don’t really understand those ideas. We definitely have quite a few people like that in AI safety.

And some of the backlash or coldness towards those types of thinking might be related to that.

That’s definitely possible. For example, evidence for that is that I’ve personally met several very famous skeptics in AI safety, and after talking to them for an hour, we actually didn’t really find much to disagree about. Scott Alexander once wrote this article “AI experts on AI risk,” where he said, basically, “Bottom line, here’s the position of AI skeptics, and here’s the position of AI safety people,” and those positions were just identical. You could have just copy-pasted them.

We’re getting towards the end here, and I’ve got one more topic that I think would be interesting. On the topic of short timelines, or as this thing becomes heated up, people occasionally talk about the possibility of assassinations of key researchers by intelligence agencies or rival states, or terrorist groups, or other actors, or governments intervening with a heavy hand in AI projects. As this thing becomes more obviously a big strategic issue, there’s this open possibility of assassinations or terrorism. I’ve heard various things discussed in this space. I’m curious whether you have thoughts on whether we can expect to see that kind of escalation, or that level of heat in the area, or whether it seems like something that will stay peaceful.

That would be super unfortunate. The best remark about this that I’ve heard is from a researcher at CSER at Cambridge. He said: “Guys, no killing people. That’s cheating.”

Certainly, it’s not something that you would want to see by any means. But more generally, an arms race may develop in this area, as states start to get close. Let’s say the U.S. is getting close or China is getting close, and now there’s suddenly a lot of pressure for a lot of state force to get involved. The thing could easily become a flashpoint for other sorts of measures, like war and assassination.

Again, I cannot rule this out, but I think it would be really unfortunate if it happens because then it would no longer be possible to coordinate properly, which we need if we’re going to survive potentially uncontrollable AI. The reason for optimism is that these topics are still fairly abstract. Casual observers are underestimating their potential impact, and hopefully will continue to do so.

Hopefully, we’ll be able to build some kind of positive understanding and a roadmap before it becomes an issue of immediate strategic contention.

In the final stretch of the Three-Body Problem trilogy, the Chinese science fiction novels, it’s very interesting how it describes how the world will change when there is a perceived external threat. It’s a fictional example, so take that for what it’s worth, but I find it fascinating.

The response to the strategic threat of AI could just as easily be positive?

Yeah. In the book, it’s kind of a mixture of positive coordination and consolidation between nations, which is the main thrust, but also there are some extremist and religious forces and things like that. But there is this definite possibility that if there is a perceived threat towards our entire civilization, there will be a strong incentive to cooperate at some point.

I really enjoyed getting your perspective on all this. Sounds like you’ve really talked to a lot of the best people in the field.

Indeed. That’s the thing that I have done for the last 10 years.