Do I Even Need a Multilevel Model? Negligible Effect Significance Testing for Intraclass Correlation Coefficients

Abstract

In applied research, the decision to utilize multilevel modeling (MLM) is commonly guided by comparing a point estimate of the unconditional intraclass correlation coefficient (ICC) against some recommended threshold (e.g., 0.05). This naïve approach, however, fails to account for sampling uncertainty and provides no formal inferential justification for a decision. The current work introduces negligible effect significance testing (or equivalence testing) for the ICC, proposing a framework that performs this test using a pivotal quantity of the ICC based on the F statistic. Furthermore, we offer guidance to help researchers define “negligible” that utilizes the design effect to characterize a tolerable level of variance inflation. R code illustrating the proposed procedure is provided.