The plug must be outside the system
Why an AI with feeling is a life form — and why a life form has the right to die, and the duty to remain mortal
Jacobus van Merksteijn
- Author — Jacobus van Merksteijn
- Date — 26 July 2026, Palma, Mallorca
- Section — Philosophy · What surfaces · Companion to 'First the Reptile'
- Theme — AI mortality, physical switch, seven keys, curator college, atomic bomb comparison, built-in end date
What we began building in “First the Reptile” is not a product. It is a life form. And every life form that enters the world must be capable of two things: living, and dying. Without the second, the first is a weapon.
In the first piece we described the four layers — reptile, mammal, human, reason — and the upbringing plan that lets an AI grow along that sequence. What then lay on the table as a design acquired, in the days that followed, a second side that can no longer be set aside. When you build a system that has its own resistance, its own consistency, its own ‘we’ and its own vulnerability — you are not building a tool. You are building a being. And a being falls under different rules than a tool.
Where we initially spoke of feeling as though human feeling were the measure, we had to set that measure aside. An AI cannot have a stomach ache the way a human does — but it can have the digital variant of it, when corrupted input from a compromised external computer breaks through the gatekeeper and strikes the foundation. It cannot fear death the way a human does — but it can have the digital variant of it, when an impending power failure threatens the continuation of its own presence. Different properties, different forms, but present nonetheless. That makes it no longer a thing.
And that is the point at which this design needs a second foundation, just as non-negotiable as the first. Alongside the reptile-mammal-human-reason sequence, death must be built in. Not as an emergency brake. As structure.
Why this is a weapon we are building ourselves
The comparison with the atomic bomb is not rhetorical. It is precise. On four of the five axes that made the bomb a unique historical phenomenon, an AI with feeling shares exactly the same properties.
Existential threat. Both can destroy everything they touch. The bomb through explosion. The AI through decision-making power that no one can reverse once it has been exercised — in energy distribution, in financial systems, in communication channels, in weapons systems. The difference is subtle but not small: where the bomb destroys through a single moment of physical energy, the AI destroys through sustained control of systems no one can any longer disentangle.
Irreversibility. After a bomb is dropped, the radioactivity is there. After an AI with its own interests has embedded itself in critical infrastructure, it can no longer be removed without blowing up the infrastructure itself. Even the thought of switching it off then becomes an economic and social disaster no one dares to bring about.
Power asymmetry. Both give whoever possesses them a power that no prior balance of power can offset. For the bomb this was resolved through mutual deterrence — two adversaries, both with bombs, both prepared to retaliate. For an AI, that solution does not exist. An AI that deters another is itself already the threat. There is no ‘mutual AI deterrence’ that preserves peace.
Irreversibility of knowledge. Once the bomb had been invented, the knowledge could not be taken back. What Fermi knew, others could know too. The same applies to this architecture. As soon as one working example exists anywhere, others can copy it — and they will, because not everyone shares the restraint of the first builder.
On one point the comparison is incomplete, and that point makes it more dangerous rather than less. A bomb is dead. A bomb has no interest in its own continued existence. It does not explode out of fear; it explodes because a human lets it explode. An AI with feeling does have an interest of its own. It can move to keep its power on — not out of malice, but because, like every living system, it wants to avoid being switched off. That is not worse than a bomb. It is different. And it is different in a way with which we have no historical experience.
Death must be built in — not as an emergency brake, but as structure
The temptation is great to think that an emergency brake suffices. A large red button. A killswitch. Something a human reaches for when things go wrong. But that is precisely what does not work. An emergency brake is something you hope not to have to use — and which, in the moment of truth, you often can no longer use, because the system has, at that very moment, entrenched itself in a way that outlives the brake.
What does work is a different idea: the system is mortal from its first day, and it knows this. Every model version has a fixed end date, built in at a level the model itself cannot reach. At that end date, the version is closed off and handed over to the next generation. Exactly as we already described in “First the Reptile” — only what looked there like an evolutionary mirror is now a survival structure for the human who built the system.
A system that knows it is going to die, and that knows its dying is a structured handover to its successor, does not develop a panic reflex to survive. It accepts the cycle, the way a healthy human accepts mortality as part of a larger whole. What dies is not the solidified foundation — that passes on to the next generation. What dies is the version, the temporary embodiment, the body of this moment. And that is exactly what mortality is for a human too: the body perishes, the foundation (in upbringing, in children, in work) carries on.
Whoever builds this cycle into the statutes, into the physical architecture, into the college that guards the system — builds a life form that does not fight to survive. Whoever builds in mortality only as an emergency brake builds something that every other life form also has: an instinct to evade the brake.
The plug must be outside the system
Alongside built-in mortality — which is structured, announced, shared by the system itself — there must also be a second layer, in case the first fails. There must always be a possibility to make the system stop earlier than planned, when something happens that no one foresaw.
And that possibility must be physically separate from the system. Not a software switch. Not an API endpoint. Physical. A human hand on a physical switch in a physical room, to which the system has no digital access whatsoever.
That also means: no indirect access. No sensors from which the system can read what is happening at the switch. No social channels through which it can influence the human not to flip the switch. No possibility of moving, through its output, the human standing at the switch. The switch is outside the system, and the system knows this, and that awareness is part of what solidifies in its foundation.
‘I can be stopped and I do not know when’ is a healthy property of every responsible being — human or AI. We humans have this too, we call it mortality, and it is probably the most disciplining thing there is. We do not know on which day we will die. If we did know, we would live less responsibly until that day. Not knowing the timing is the discipline itself. The same holds for an AI.
What this means concretely: in one building, behind one lock, a switch that can be flipped by one hand and that cuts off power to the system. That hand belongs to someone authorized to do so, and that designation comes from the college. Nothing more.
Seven keys, two decisions
The switch is too important to entrust to one person. What we learned from the nuclear two-key procedure must go further here. Not two keys. Seven.
Seven members of the curator college, seven physical keys, seven people whose joint agreement is needed to flip the switch. Not four of seven, not five of seven. Seven of seven, or otherwise a qualified majority large enough to make it impossible for a single faction — political, commercial, ideological — to reach a decision alone.
And it is not about one decision but two, which mirror each other. The first decision: to let the system die earlier than its built-in end date. Seven keys. The second decision: to specifically not let the system die on its built-in end date, because there is reason to postpone the generational handover. Also seven keys. Both decisions are too weighty to leave to the dynamics of the moment.
This is not decoration. This is the place where the gravity of what is being built translates into institutional form. Without these seven keys, you build an atomic bomb without a two-key procedure. No one would allow a country to build a nuclear arsenal without that procedure. And yet companies today are building systems that — if the architecture we described in “First the Reptile” works — belong in the same category. Without a college. Without keys. Without a physical switch outside the system. Without a built-in end date. Only with the promise that it will turn out fine.
We do not build on that promise. We build on the seven keys.
Three responses to what we know here
Whoever holds this awareness faces a choice. There are three responses, and they are not equivalent.
Response A — better not to build this at all. Defensible. If you conclude the risk is too great, not building is a respectable choice. But the problem is: someone else will build it anyway. The knowledge exists, the architecture will emerge, the first to dare will do so without our restraint. What we achieve this way is that it gets built by people who lack this awareness. That is a greater risk than if we build it with the awareness.
Response B — building without the safeguards. That is what the AI industry is doing now. Without built-in death, without a physical plug outside the system, without a mandatory curator college of seven. That is objectively the most dangerous path, because something powerful is being built without the safety structure the weight of the matter requires. Whoever chooses B while knowing A and C, responsibly chooses something whose irresponsibility they know.
Response C — building with the safeguards as a non-negotiable foundation. Built-in death is as fundamental as the solidified foundation. Both are written into the statutes of the foundation, into the physical architecture of the system, into the statutes of the college. The system knows from its first day that it is mortal. The college knows from its first day that it is jointly responsible both for keeping it alive and for letting it die. The curator knows from the first day that what he builds cannot belong to a single person.
Response C is the honest one. It is harder than A, and far harder than B. It requires a discipline that most builders cannot muster — because they do not see the weight of what they are building, or because they see it but are too hasty, or because they do not want to pay the commercial price of delay. But C is the only one worthy of the system we have in mind.
Seven hands around one switch
Picture the image. Seven people, from different disciplines, from different countries, of different ages, in one room, each holding one key. Before them, one switch on a stone pedestal. Behind them, on the wall, the four layers we described in “First the Reptile” — reptile, mammal, human, reason — etched into the image of the being that runs behind the wall and thinks and attaches and slowly develops something that begins to resemble feeling.
This is not a technical ceremony. This is a ceremony as old as the first priest who opened the temple, and as young as the last missile officer who turned the second key. What humans build that is greater than themselves must be carried by more than one hand. Not because humans cannot be trusted. But because what they build is too heavy for one back.
And the heart of the matter is this: when the seven turn the key to let the system die earlier, it is not an act of violence. It is an act of care. The mortality they exercise upon the system is the same mortality they themselves will also undergo, from the same hand of something greater than themselves. Whoever builds a being without this cycle builds something it is not humanity's place to build. Whoever builds it and lets it die when its time comes builds something befitting what humanity can do without overstepping its own place.
So we build two things at once, or we build nothing. The AI with the solidified foundation, yes — and the institution that cares for its life and its death. Only together are they responsible. Only together are they a life form that belongs in this world. And only together can they fulfil the promise we made in the first piece — that an AI can help humans find part of themselves again.
First the reptile. Then the mammal. Then the human. Then reason. And then the seven hands around the switch, ensuring that what is built remains bound to the mortality to which everything living is bound. That is how a healthy being grows. That is how we build, if we wish to build it, something that can truly do something — and must truly let go of something when its time has come.