Dimensional open-ended evolution (OEE) as a hybrid dynamical phenomenon

Abstract

Classical evolutionary theory, confined to fixed-dimensional state spaces and static parameters, cannot model meta-evolution: the phenomenon in which the rules governing heredity, mutation, and selection are themselves transformed by the population they govern. We formalize self-referential evolutionary dynamics as a hybrid dynamical system.A first, definitional result gives a classification criterion: an operator confined to a fixed-dimensional genotype simplex cannot enlarge that simplex through its own action, so open-ended evolution (OEE), understood as unbounded growth in genotype-space dimension, is impossible without an explicit dimension-expanding mechanism. The criterion partitions existing formalisms, quasispecies dynamics, the Wright–Fisher process, fixed-representation genetic algorithms, and the standard replicator equation, onto the same side of a formal line, and a companion bound closes off a weaker, entropy-based notion of open-endedness.We then construct a hybrid system in which replicator flows drive populations toward the boundary of the probability simplex, where the Fisher–Rao metric degenerates in simplex coordinates; this degeneracy triggers a reset map that expands the genotype and parameter spaces through gene duplication, mutational regularization, and parameter inheritance. We show that an exact duplicate is dynamically sterile, the post-jump state is fixed under the transposition exchanging parent and paralog, so a deterministic equivariant flow cannot break the symmetry required for evolutionary divergence, and repair the construction with a sub-threshold asymmetric seed that breaks equivariance while leaving the viability graph unchanged.For epistasis matrices carrying a cyclic dominance pattern, the inheritance rule provably preserves the pattern and the boundary-attractivity criterion is established at the base case m = 3 via Hofbauer and Sigmund (1998, Theorem 7.7.2); the induction then delivers a constructive existence result on this named family, with the extension of the attractivity criterion to the asymmetric case m > 3 identified explicitly as the primary open question.Finally, we characterize a metric-free topological signature of novelty as the homology of an epistatic complex, tracked as a zigzag module along jumps and flows. A folding lemma shows duplication alone leaves this homology unchanged; only divergence generates new cycles, exhibited in a four-vertex example.