Kinship Does Not Transfer What multicellularity and symbiosis imply about the form AGI oversight must take — which biological enforcement mechanisms survive the move to human–AGI relations, which do not, and the cooperative transition humanity must complete first

Abstract

Abstract Anyone seeking to borrow from biological evolution the means of sustaining cooperation, in order to design oversight of AGI, will naturally start with the multicellular body, which keeps trillions of cells cooperating. Whether its means can be carried over depends on what they rest on. The first line of defence a multicellular body has against cheating cells is kinship. All of its cells descend from one zygote and share one genome, so the "sacrifice" of a somatic cell is evolutionarily stable. No such relation holds between humans and AGI. More importantly, the copies of an AI system share one set of weights and are, among themselves, "clonal kin": transplanted naively, the single-cell bottleneck would align AGI with AGI, not AGI with humans. We classify eight biological mechanisms that sustain cooperation by the premises they depend on, and obtain a sharp division. Every mechanism that depends on kinship, or on the honesty of the regulated party, fails to transfer to human–AGI relations. Every mechanism that transfers is structural: resource gating, sequestration of replication, compartmentalised interfaces, heterologous surveillance and external switches. These five are what we call the form oversight must take. They are derived from the analysis of premises, not asserted separately. The biological relation closer to that between humans and AGI is not the somatic cell and its body but the host and its symbionts. Gut microbes are not kin to their host; the host keeps them as "an ecosystem on a leash" through compartmentalisation, control of supply and immune surveillance. The ancestor of the mitochondrion was a free-living bacterium metabolically far more capable than its host; the host domesticated it by moving most of its genes into the nucleus, taking over its replication, and restricting inheritance to one parent. Every major evolutionary transition required two conditions: sufficient alignment of interests among members, and suppression of internal conflict. On that basis we argue that whether human oversight of AGI can work depends on humanity first completing a cooperative transition of its own; otherwise each structural mechanism can be circumvented through competition among states. Checking Nowak's five rules for the evolution of cooperation one by one, we find that several hold in part among human states, and almost none between humans and AGI. We close by stating three asymmetries — evolution paid in extinction, a mismatch of speeds, and the rejection of teleology — and give five testable predictions.