Reliability of machine learning methods for prediction of critical heat flux: an OECD/NEA benchmark study

Abstract

Abstract Critical heat flux (CHF) prediction is a primary thermal-hydraulic requirement for water-cooled reactors. Machine-learning studies on the OECD/NEA Phase-1 benchmark report high in-distribution accuracy, but reliability under distribution shift is poorly characterised. Four model families (random forest, XGBoost, multi-layer perceptron, Bayesian neural network) are evaluated under three protocols: random in-distribution testing (S1), cross-source validation on 10 unseen facilities (S2), and extrapolation above 16 MPa (S3). Reliability is assessed through accuracy, conformal coverage and seed reproducibility. XGBoost leads in distribution ( R 2 = 0.986 ± 0.001; relative RMSE 13.1 ± 0.9 %) but falls to 0.905 ± 0.044 on S2 and 0.648 ± 0.009 on S3. Conformal intervals calibrated at a nominal 90 % level fall to 71 % under cross-source validation and 63 % under high-pressure extrapolation, deficits of 19 and 27 % points that weighted conformal prediction does not recover. Neural networks show pronounced seed sensitivity: the standard deviation of MLP R 2 rises from 0.0014 to 0.196 between S1 and S3 ( F -test p < 10 −6 ), while tree-based methods remain at or below 0.009. Five-seed ensembling reduces this spread by close to an order of magnitude but restores no coverage, physics-aware features leave extrapolation accuracy unchanged while doubling seed variance, and anchoring the model on the Bowring correlation degrades extrapolation sharply (Δ R 2 = −0.336) because this hybrid inherits the validity limits of that correlation, which qualifies the in-distribution gains reported for physics-anchored ensembles. Random-split benchmark accuracy therefore overstates deployment reliability for CHF-ML systems. All splits, models and scripts are released openly.