YvyrAI: A Spanish-First Recurrent-Deliberation Language Model Architecture with Internal Verification, Conditional Repair, and Adaptive Compute

Abstract

We present YvyrAI, a Spanish-first decoder-only language model architecture centered on a recurrent module called the DeliberationBlock. A shared Transformer block is iterated within a single forward pass while a learned controller maintains a gated planning state, reads and writes a compressed latent scratchpad, re-injects the block input at every step, estimates an internal verification signal, applies a conditional repair update proportional to estimated unreliability, and decides how many iterations to spend before emitting the final hidden representation. This version reports the first capability-scale training run: a 105.86M-parameter configuration trained from scratch on 891,135,906 tokens of a purpose-built Spanish corpus, using a single consumer GPU with 8 GB of VRAM. Three results are measured. First, recurrent depth helps monotonically: on the same checkpoint, held-out loss falls from 4.2465 at one iteration to 3.8975 at seven, a 29.5% reduction in perplexity (69.86 to 49.28). Second, the benefit saturates exactly at the trained depth, with identical loss from eight to sixteen iterations, so no test-time extrapolation beyond the training budget is observed at this scale, and no degradation either. Third, the learned halting controller converges to 7.4 of 8 available iterations and stays within 0.0196 nats of the forced-depth curve. We also report a comparison against GPT-2 124M in bits per character over identical text: 1.3625 for YvyrAI against 2.2238 for GPT-2 on held-out Spanish web text, and 1.0280 for GPT-2 against 2.0078 for YvyrAI on English Wikipedia. We argue explicitly that this is language specialization, not architectural superiority. This version does NOT report the controlled comparison against a non-recurrent model of matched compute. That experiment is specified, implemented and pending; until it runs, no claim is made that recurrent deliberation outperforms a conventional Transformer at equal cost. No 1B-parameter model exists. Weights and inference code are released at https://huggingface.co/Manuel12435/YvyrAI-100m-Nano under Apache-2.0. Version 2.0 supersedes version 1.0 (June 2026).