Simulated changes in systematic review screening sensitivity and specificity associated with varying co-screening proportions.

Authors

Shaun Prentice, Ben Mundy, Caitlin Prentice, Diana Dorstyn, Nicola Massy-Westropp, Ishanka Weerasekara, Michael Bentley, Katie Fisher, Sahil Kharwadkar, Thuy Tran

Published in

JBI evidence synthesis. Jul 22, 2026. Epub Jul 22, 2026.

Abstract

The objective of this study was to examine how sensitivity and specificity of title/abstract screening decisions change as the proportion of records that are co-screened is varied.
With an accelerating volume of literature being published, the demand for systematic reviews is increasing. To maintain the feasibility of conducting reviews in a timely fashion, efficiencies must be found in the review process that do not compromise their rigor. Current guidelines recommend that all records be dual-screened. Less is known about the impact of co-screening a subset of records on screening accuracy.
Eight reviewers independently screened the titles and abstracts from a published systematic review and meta-analysis. Using their decisions, simulations were run to generate permutations of reviewer combinations in 2 formats: dual screening (ie, primary reviewer screened all records, secondary reviewer screened a portion, third reviewer mediated disagreements) and 3-way screening (ie, primary reviewer, secondary reviewer, and mediator roles were equally divided between 3 individuals). Within each permutation, further simulations were created so that the secondary reviewer screened increasing proportions of records in 5% increments. Decisions were compared against the final full-text decisions used in the published review to determine sensitivity and specificity of decisions for each permutation, at each increment. The relationship between sensitivity/specificity and percentage of co-screening was assessed using linear mixed-effects models.
All reviewers screened 3891 potentially eligible records, of which 80 were included in the final review. Sensitivity (median range: 93.3%-96.3%) and specificity (median range: 87.7%-89.6%) were stable across different proportions of co-screening, for both dual- and 3-way screening strategies. Sensitivity was higher in reviewer groupings where at least one reviewer had subject matter experience. Specificity tended to be higher when the primary reviewer had experience in conducting a systematic review.
Our results suggest that complete co-screening of title/abstract records does not appreciably improve sensitivity or specificity. Rather, even low levels of co-screening across dual- and 3-way screening can produce high accuracy. Reviewer experience in subject matter and in conducting a systematic review, however, appear to be influential. These results are based on simulations and permutations from a single review so should be used as a guide only.
Co-screening a proportion of title/abstracts in a systematic review appears to produce comparable sensitivity and specificity to co-screening all records. Proportional co-screening may therefore represent an opportunity for enhancing efficiency in the systematic review process.
https://osf.io/9feuh.

PMID:
42483874
Bibliographic data and abstract were imported from PubMed on 22 Jul 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.