Hiring in life sciences? Share your open positions with our professional community. Read more Close

Advertisement

Evaluating Large Language Models for Standardizing Clinical Data Using Health Level Seven Fast Healthcare Interoperability Resources: Development and Evaluation Study.

Created on 16 Nov 2026

Authors

Álvaro Riquelme-Tornel, Pedro Luis Costa Del Amo, Catalina Martínez-Costa

Published in

JMIR medical informatics. Volume 14. Pages e100368. Nov 13, 2026. Epub Nov 13, 2026.

Abstract

One of the barriers to the wider adoption of clinical information modeling standards is the high cost associated with the time and effort required to model and map data from legacy systems, as well as the need for specialized modeling experts. Meanwhile, large language models (LLMs) are becoming powerful tools for data transformation and semantic processing.
The aim of this study was to evaluate the capability of LLMs to support the mapping of structured clinical data to Health Level Seven (HL7) Fast Healthcare Interoperability Resources (FHIR) resources and attributes.
Using data from the MIMIC-IV (Medical Information Mart for Intensive Care IV) database, a data pipeline integrating clustering techniques, retrieval-augmented resource grounding, and schema-guided prompting was implemented to support the FHIR mapping of structured clinical data. The pipeline was evaluated under 2 scenarios with different levels of complexity. In the first scenario, used as a baseline, the input data were well described and contextually rich. In the second scenario, designed as a controlled test under substantially reduced context, the original table structure and table-level descriptions were removed, while data element names and LLM-enriched descriptions were retained.
Using 65 data elements, top 2 resource identification accuracy reached 100% in the baseline scenario. The highest mean attribute-level mapping accuracy was 80.61% (SD 2.06) for GPT-4o and 69.54% (SD 5.15) for Llama 3.3 70B. In the low-context scenario, clustering-based context recovery increased top 5 resource identification accuracy to 94%. Across temperature settings, GPT-4o achieved mean attribute-level mapping accuracies ranging from 73.23% (SD 2.92) to 76.77% (SD 1.14), whereas Llama 3.3 70B ranged from 54.31% (SD 4.17) to 58.31% (SD 2.23). Error analysis showed that the main remaining failures were associated with resource-level confusion and terminal-attribute semantic ambiguity, whereas other error categories were comparatively infrequent.
LLMs can support structured clinical data mapping to HL7 FHIR, but fine-grained semantic disambiguation remains the principal limitation. In context-rich settings, the main errors occurred when selecting the appropriate terminal FHIR attribute. Under reduced context, accurate candidate-resource retrieval became increasingly important, and clustering-based context recovery substantially improved resource identification. These findings support the use of LLMs as assistive components within human-supervised interoperability workflows, with particular attention to candidate-resource selection and ambiguity-sensitive attribute mappings.

PMID:
43037145
Bibliographic data and abstract were imported from PubMed on 16 Nov 2026.

Read full publication at:
Please sign in to see all details.

Advertisement

Stats

  • Community rating n/a 0 votes
  • Reviewers' rating n/a 0 votes
  • Your rating

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

  • Recommendations n/a n/a positive of 0 vote(s)
  • Views 1
  • Comments 0

Recommended by

  • No recommendations yet.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.

Advertisement