The Influence of Study Selection Criteria on Statistical Conclusion Validity for Reading Interventions: Evidence From Non-Independent Data Structures.

Authors

Allyson L Hayward, Jeffrey A Shero, Jessica A R Logan

Published in

Journal of learning disabilities. Pages 222194261496849. Oct 26, 2026. Epub Oct 26, 2026.

Abstract

Randomized control trials (RCTs) of Tier 2 interventions for students with or at risk for reading disabilities define study eligibility using cut points on screening measures. These criteria vary widely across studies and may inadvertently bias estimates of effectiveness. We conducted a simulation study to test how different eligibility criteria influence intervention effect estimates. Across 500 replications, we compared three common approaches: a population 40th percentile threshold, a sample 25th percentile threshold, and selecting the lowest four students per classroom. Under varying effect sizes and treatment structures, we compared the true effect with estimated intervention effects provided by subsamples yielded from each criteria method. Each eligibility criteria method produced a biased estimation of intervention effect, the degree to which varied across data structures and treatment types. These findings illustrate that eligibility decisions, while necessary for Tier 2 intervention research, produce biased and unreliable estimates, raising concerns for statistical conclusion validity. Researchers should consider the consequences of eligibility criteria when interpreting and comparing RCT outcomes. Future research is needed to find a solution that is both statistically valid and practically feasible.

PMID:
42935119
Bibliographic data and abstract were imported from PubMed on 27 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.