Prediction of agricultural production based on multivariate data with feature aware machine learning models

Abstract

In the context of climate change, yield prediction in agriculture becomes extremely important for achieving food security, precision agriculture, and sustainable resource management but the productivity of crop production involves a nonlinear relationship between environmental, climatic, and soil variables and cannot be easily modelled using conventional statistical models. In this study, the authors present a feature-aware regression-based machine learning framework, which can be used for the prediction of agricultural yield based on a collection of multivariable environmental and agronomic data. It is a data set of 36,520 agricultural records collected from various agro-climatic conditions in multiple growing seasons and considering the nitrogen (N), phosphorus (P), potassium (K), rainfall, temperature, humidity, wind speed, soil quality factors, and soil pH. Three regression models were used: linear regression (LR), Random Forest regression (RF), and Gradient Boosting regression (GBR) and were optimized using RandomizedSearchCV with negative root mean square error (RMSE) as the optimization criterion. Fivefold cross-validation and confidence interval analysis were used to quantify the generalization of the models and to prevent the models from overfitting the data. In all the models the corrected Random Forest regression Model had the lowest value of RMSE (4.6) and MAE (2.7) and the highest R 2 (0.96) value (which indicates the performance of the prediction model) among all the models. The result of feature importance and explainability analysis using SHAP indicated potassium, nitrogen, rainfall, and temperature to be the key factors in the prediction of the yield. We reworked the explainability analysis and found strong coherence between feature importance layout and SHAP values and agronomic interpretation. The suggested model is a scientifically valid, interpretable and scalable solution for agricultural yield forecasting and precision farming applications. The results prove that the explainable ensemble learning approaches are useful for the data-driven farming decision support systems.