Berna R7 v3: A 1.2B Language Model with Self-Growing Cell Architecture

Abstract

We present Berna R7 v3, a 1.2B-parameter language model trained from scratch using the DNA-Kernel Plexus architecture — a novel approach in which neural capacity is allocated through dynamic, self-growing "cells" guided by a six-dimensional knowledge vector. Trained on 2.66B tokens (English, mathematics, and code) over 10,200 steps, the model achieves a WikiText-103 perplexity of 27.17 and matches GPT-2 Small (124M parameters) on standard benchmarks (HellaSwag 29.84%, ARC-Easy 47.69%, PIQA 60.50%) while using approximately 15x less training data. We document a self-limiting growth phenomenon: the model expands from 112 to 896 active cells over 900 steps, then stabilizes autonomously for 9,300+ additional steps without external intervention. A rigorous self-audit reveals that four of six components of the K-vector (K2, K4, K5, K6) are either static, globally-computed, or unimplemented, and the bounded forgetting mechanism central to the parent theory (R5) remains untested. We present these findings as both a proof of concept for cell-based language modeling and a roadmap for architectural refinement in R8.