Augmenting structured diagnoses through effective use of pre-trained large language models on clinical notes.

Authors

Hanieh Razzaghi, Nhat Nguyen, Mohan Kashyap Pargi, Kaleigh Wieand, H Timothy Bunnell, L Charles Bailey

Published in

JAMIA open. Volume 9. Issue 5. Pages ooag171. Epub Sep 09, 2026.

Abstract

Clinical narrative provides a unique window into provider reasoning and attribution for automated diagnosis assignment, but large language models (LLMs) have traditionally not performed well at medical coding. We evaluate a reproducible method for automated diagnosis assignment using LLMs in clinical notes and compare with structured diagnoses.
We used GPT-OSS for prompt engineering and task segmentation to create a model that extracts ICD-10-CM diagnoses, with estimates of severity, currency, and importance, from progress notes. We assessed performance across multiple cohorts of patients aged 0-21 years. For each, 100 outpatient provider notes were selected across levels of severity, along with coded diagnoses from that visit (electronic health record [EHR]); a subset of 130 notes were subjected to clinical expert review.
Comparison showed 18.7% exact code and 33.3% ICD-10-CM category match between EHR and LLM, but semantic similarity of 0.93 at the category level. Compared to expert review, LLM precision was 0.84 and recall 0.49 for exact matches, and 0.92 and 0.62, respectively, for category-level matching. In contrast, coded diagnoses showed slightly higher precision (0.94 for both cases) and substantially lower recall (0.27 and 0.43) versus expert review. Codes not identified by the LLM were more often rated by the reviewer as lower importance or certainty.
We demonstrate a reusable approach to optimize LLMs for use in diagnosis extraction from clinical notes that can augment structured diagnoses and provide contextualizing metadata.
LLMs represent a viable and flexible approach to diagnosis code extraction from unstructured clinical notes.

PMID:
42723861
Bibliographic data and abstract were imported from PubMed on 11 Sep 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.