Why Is Cross-Level Information Conversion Irreversible? An Information-Theoretic Account

Abstract

In hierarchical information systems a higher level appends coding coordinates to those of the level below, which is obtained from the higher level by deleting these coordinates. Established tools such as rate-distortion theory, the information bottleneck method and the data processing inequality all assume that encoder and decoder share one fixed measurable space; they describe loss that consists in choosing not to retain information within that space, but not cross-format conversion in which the lower level physically lacks the deleted coordinates. This paper characterizes the resulting gap with conditional entropy. The gap is shown to be additive along the hierarchy, independent of code rate, computation and sampling, and irreducible unless the lower-level measurable space is enlarged, removable only through a physical ascent of level. The analysis also separates this setting from sufficient statistics, which preserve a single prespecified parameter, and yields a thermodynamic lower bound on cross-level erasure. Two kinds of information loss are thereby distinguished: within one space what is missing is precision, recoverable by denser sampling or a higher code rate; across spaces what is missing is a coordinate, which no increase in sampling or code rate can restore and which requires enlarging the measurable space. No new information measure is introduced. Two applications follow, the structural blind spot of external observation and the width of the explanatory gap.