An external peer review instrument for Korean Medicine clinical practice guideline drafts (KM-EPRI): development and content validation.

Authors

Hee Jung Lee, Yoon Jae Lee, Hyo-Weon Suh, Keumji Kim, Jihyun Sang, Junhyeok Yi, Seungwon Shin

Published in

Frontiers in medicine. Volume 13. Pages 1926917. Epub Sep 23, 2026.

Abstract

Korean Medicine clinical practice guidelines (KM-CPGs) undergo mandatory external peer review of their drafts during development, yet reviewers lack a standardized, validated instrument to guide this task. Widely used appraisal tools assess the quality of completed guidelines rather than support the section-level review of a draft. We aimed to develop an external peer review instrument for KM-CPG drafts (KM-EPRI) and to assess its content validity.
In this two-phase methodological study, a draft instrument was first derived by directed qualitative content analysis of the external review comments produced for previously published KM-CPGs, with each comment coded into one of four predefined review modules: the clinical questions (CQ), the introductory components (CPG-I), the development methods (CPG-M), and the recommendations (CPG-R). The content validity and acceptability of the items were then assessed through a modified Delphi study with a multidisciplinary panel of KM clinicians and evidence-based medicine methodologists. Consensus was defined a priori as an item-level content validity index (I-CVI) $\ge$ 0.80. The scale-level content validity index (S-CVI/Ave), Kendall's coefficient of concordance ( $W$ ), and the interquartile range (IQR) were reported as supporting measures.
From 3,630 external review comments, 2,073 content-related comments were consolidated and refined into a finalized instrument of 117 review items across 4 modules and 33 subdomains. All 10 invited experts participated (response rate 100%, no attrition), and every item reached the consensus threshold in a single round under a pre-defined early-stopping rule. All 117 items had an I-CVI $\ge$ 0.80, the S-CVI/Ave was 0.976, and the IQR was $\le$ 1 for every item (median 0.75). Kendall's $W$ was low (0.196), reflecting a ceiling effect from uniformly high ratings rather than disagreement among panelists.
KM-EPRI can serve as a consensus-based, structured reference for the external peer review of KM-CPG drafts during development. Its effect on review quality and inter-rater reliability remains to be evaluated in practice.

PMID:
42846017
Bibliographic data and abstract were imported from PubMed on 08 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.