From Raw Columns to Valid Signals: What Data Preparation and Feature Engineering Actually Change

Abstract

Data preparation and feature engineering are often presented as preliminary operations performed before modeling. This description understates their scientific role. Every transformation changes the representation available to the estimator and therefore changes the hypothesis that the model can express. This paper introduces the Feature Validity Map, a six-level framework connecting semantic validity, temporal validity, statistical validity, predictive validity, transport validity, and operational validity. Passing one level does not guarantee passing the next: a feature may be mathematically usable yet unavailable at deployment, statistically unstable, vulnerable to leakage, or unable to transport across time. The framework is executed on 392 vehicles from the UCI Auto MPG dataset. Model years 1970–1979, comprising 307 vehicles, form the development period, while model years 1980–1982, comprising 85 vehicles, remain a temporally separated final test. Six feature-engineering recipes are compared while holding the estimator fixed as Ridge regression with alpha=10, allowing changes in performance to be attributed primarily to representation rather than algorithm selection. All preprocessing operations, including imputation, scaling, encoding, and feature construction, are fitted exclusively within each validation fold. Recipe selection uses leave-one-year-out validation, and the final temporal test remains frozen until selection is complete. The selected pairwise-expansion recipe achieves a final test MAE of 3.79 mpg, RMSE of 4.68 mpg, and R² of 0.392, substantially outperforming the development-mean baseline, whose test MAE is 11.04 mpg. However, the selection result is not uniquely stable. Across 200 bootstrap repetitions, pairwise expansion and domain-informed transformations are selected exactly 100 times each. Delete-one-year analysis favors pairwise expansion in eight of ten cases, while a paired Wilcoxon comparison between the two finalists yields p=0.105. The pairwise representation expands the design to 29 features from 307 development observations. Ridge regularization constrains coefficient variance under this richer and correlated representation, but it does not eliminate collinearity, guarantee feature-level interpretability, or establish causal meaning. Conditioning, correlation, selection stability, and out-of-sample behavior are therefore evaluated as complementary evidence. The results support the paper’s central claim: feature engineering does not merely improve a score; it changes what the model is capable of learning. A constructed feature constitutes valid signal only when its meaning is defensible, it can be computed at the intended prediction time, it is produced without leakage, and its benefit remains sufficiently stable outside the data used to design it. The record includes a complete reproducibility package containing source code, data, fold-level metrics, temporal-test predictions, stability analyses, conditioning diagnostics, correlation results, figures, manifests, and structured outputs.