Shape Classification of Failure-Sets in Stationary Linear Filters

Abstract

A filter specification fixes a tolerance R² on the error variance and asks under which operating conditions the error exceeds it. As one parameter varies (observation precision, noise correlation time, sampling interval, process noise or a design parameter), the answer is a set, the failure set, and its shape matters: a threshold, a bounded window, two rays, or a mixture. This thesis classifies those shapes for scalar stationary linear filters of an Ornstein–Uhlenbeck signal. For any continuous variance curve with finitely many turning points, the class of the failure set at every tolerance is fixed by a finite signature: the endpoint limits, the turning-point values and two approach directions. For the white-noise relay, every stationary variance lies on one surface in two dimensionless coordinates, on which the floor crossing is the line q = 2 and the optimal filter is the lower envelope. Along precision, a continuous variance curve is reached by some continuous gain if and only if it never falls below the optimum. At any tolerance tighter than the floor every failure set therefore contains a lower ray, and every further restriction comes from the gain family: natural single-stage gains have at most one window component, and a Hill-type gain produces a mixed class, a lower ray with a window. When one observation process is a causal garbling of another, the optimal filtering error is monotone between them. Along such a sweep the optimal filter never windows, so a window, two rays or a mixed class certifies a miscalibrated filter. For additive stationary Gaussian noise the ordering is decided by the noise spectra, and within a power-normalised family of coloured noises the condition is sharp. Without the ordering the optimal filter can window, as it does along correlation time at matched variance. The results were checked, mostly under pre-registered criteria. Of 182 recorded checks, 37 failed, and each failure was diagnosed; 23 of 24 re-checks under corrected criteria passed. The class sets of sampled relays and of two-stage gains rest on numerical evidence, not proof.