Extending the Vertically Reflected Exponentiated Hyperbolic Tangent Detection Function for Grouped Line‐Transect Data

Abstract

ABSTRACT Accurate estimation of population density in distance sampling depends critically on the choice of detection function. Conventional models such as the half‐normal (HN) and hazard‐rate (HR) functions may perform poorly when the underlying detection structure is complex, particularly for grouped distance data. We extend the vertically reflected exponentiated hyperbolic tangent (VREHT) detection function to grouped line‐transect sampling within a likelihood‐based conventional distance sampling framework. Model performance is evaluated through an extensive Monte Carlo simulation study covering a range of detection scenarios and sample sizes. The VREHT model demonstrates greater flexibility than the HN and HR models, achieving lower bias under irregular detection scenarios while maintaining competitive estimation accuracy for regular detection patterns. These results illustrate the trade‐off between flexibility and robustness that characterizes parametric detection function models. The proposed model is further evaluated using four actual datasets representing diverse ecological conditions. Across grouping schemes, the VREHT function provides model fits and density estimates comparable to those of the standard HN and HR models while remaining robust under coarse grouping. Overall, the VREHT detection function offers a flexible and reliable alternative for analyzing grouped line‐transect data and is particularly useful when the underlying detection structure is complex.