The Accuracy of the I2 Statistic for Detecting Baseline Imbalances and Selection Bias in Randomized Controlled Trials: A Simulation Study.

Authors

Steffen Mickenautsch, Veerasamy Yengopal

Published in

Cureus. Volume 18. Issue 7. Pages e112766. Epub Jul 16, 2026.

Abstract

To assess the accuracy of the I² statistic, applied as a trial-adjusted, simulated comparator trials (SCTs) based I² test for detecting baseline imbalances and selection bias in randomized controlled trials (RCTs).
Fifty non-biased and 50 biased trials set at 40% selection bias severity were simulated in MS Excel (Microsoft Corporation, Redmond, Washington, USA). All trials were tested using statistical significance testing and the I² test. Null hypotheses were tested that the sensitivities and specificities of both tests did not statistically significantly differ and that the selection bias category scores, established by the I² test, do not statistically significantly correlate with the p-values from statistical significance testing. McNemar test and Spearman's rank correlation were used. The risk of the I² test overestimating selection bias severity was also examined.
I² test demonstrated higher sensitivity than significance testing for detecting both baseline imbalance and selection bias (p < 0.0001). Specificity for baseline imbalance detection was identical at 100% for both methods; 95% CI: 90%-100%. For selection bias, specificity was greater with significance testing than with the I² test (p = 0.0005). A significant negative, large (0.5 ≤ |r|) correlation emerged between I² scores and significance test p-values: Spearman's r = -0.61, p < 0.0001. All null hypotheses, with the exception of the test specificities for baseline imbalance detection, were rejected. The I² test caused 26% of 'true positive' trials to be over-classified by one severity category.
 The I² test was significantly more accurate for identifying baseline imbalance and selection bias detection, and equally accurate for ruling out baseline imbalance compared with statistical significance testing. However, the I² test was significantly less accurate in detecting selection bias absence. The I² test's bias category scores were largely negatively associated with p-values and tended to overestimate bias severity. Negative I² test results appear reliable for ruling out baseline imbalances and selection bias in RCTs, while positive I² test results are highly accurate to detect baseline imbalances but should be interpreted cautiously when assessing selection bias.

PMID:
42604264
Bibliographic data and abstract were imported from PubMed on 16 Aug 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.