Readability and Content Integrity of Discharge Documents for Common Pediatric Orthopaedic Procedures Before and After Artificial Intelligence Simplification.

Authors

Angel G A Prempeh, Rashid Saeed, Christina Hill, Lucas Kopp, Allison Tenfelde

Published in

Journal of pediatric orthopedics. Oct 19, 2026. Epub Oct 19, 2026.

Abstract

Caregivers of pediatric orthopaedic surgical patients routinely receive discharge documents written at reading levels exceeding established health literacy recommendations. This disparity creates a significant barrier to comprehension and adherence to postoperative instructions. While artificial intelligence (AI) language models offer a scalable solution for simplification of medical text, a systematic evaluation of the ability to preserve critical content in surgical discharge materials remains lacking.
Publicly available caregiver-facing discharge documents for 3 common pediatric orthopaedic conditions, supracondylar humerus fractures (CRPP), pediatric forearm fractures (ORIF/CRPP), and slipped capital femoral epiphysis (SCFE pinning), were collected. All documents underwent AI simplification using a standardized prompt with ChatGPT (version 5.2; OpenAI). Readability was assessed using 6 validated indices for both original and simplified documents. A structured content integrity review was conducted on all simplified documents by 2 independent reviewers to assess for safety-critical errors, completeness gaps, and factual accuracy, with inter-rater reliability quantified by Cohen kappa.
Fifteen documents were obtained from institutions across 9 US states. After AI simplification, mean Flesch-Kincaid Grade Level (FKGL) decreased from 8.91 (SD: 1.38) to 4.97 (SD: 2.14), a mean reduction of 3.94 grade levels (95% CI: 2.65-5.23; P<0.001; Cohen d=1.69). All 6 readability indices improved across all documents. Eleven of 15 documents (73%) achieved the target readability threshold of FKGL 6 or below, compared with zero at baseline. Content integrity review identified 35 flags across 14 of 15 documents: 12 safety-critical errors, 16 completeness gaps, and 7 accuracy concerns. Cohen kappa was 0.87 for agreement on whether a change was clinically meaningful, 0.85 for severity classification, and 0.89 for flag category. Three documents had content omissions sufficient to require clinician revision before clinical use. Greater word-count reduction was associated with a higher burden of safety-critical flags (Spearman rho 0.63; P=0.012).
While ChatGPT substantially reduced the reading complexity of pediatric orthopaedic discharge documents, it also introduced safety-critical alterations. AI-assisted simplification should be implemented only with structured clinician review. Larger word-count reductions were associated with more safety-critical alterations, so the magnitude of reduction may help prioritize which simplified documents receive the closest clinician review.
Level III.

PMID:
42895514
Bibliographic data and abstract were imported from PubMed on 19 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.