Toward Continually Growing World Models: An OaK-Inspired Architecture for Learned Abstraction and Planning

Abstract

Learned world models increasingly empower autonomous agents to predict transitions, imagine rollouts, and plan behaviors in complex environments. However,conventional continual world modeling paradigms predominantly formulate lifelong adaptation along a single parametric axis: updating neural network weights(θt → θt+1) over a static, fixed-dimensional latent substrate and a fixed single-steptemporal granularity. When deployed in non-stationary or open-ended environments, this parameter-only adaptation is vulnerable to loss of plasticity, capacitysaturation, and compounding rollout errors over extended horizons. In this positionpaper, we argue that genuine continual world modeling requires introducing a second, structural adaptation axis (Kt → Kt+1), wherein the model’s predictive stateabstractions, temporal abstractions, and option models continually grow, evaluate,and reorganize over time. Grounding this dual-axis formulation in Richard Sutton’sOaK architecture and its FC-STOMP lifecycle (Feature Construction → SubTask→Option → Model →Planning), we formulate Structural Continual World Modeling (SCWM). Under SCWM, discovered state features dynamically expand theworld model’s predictive state representation, inducing reward-respecting subtasks,closed-loop options, and predictive option models, with retention and pruninggoverned by downstream planning utility. We formulate five falsifiable researchhypotheses and an experimental agenda to guide the development of continuallygrowing world models