Hiring in life sciences? Share your open positions with our professional community. Read more Close

Advertisement

Beyond accuracy: a three-stage evaluation of large language models for postpartum health consultation and nursing decision support.

Created on 04 Nov 2026

Authors

Xilin Zhao, Xinwen Zhang, Lei Liu

Published in

Frontiers in public health. Volume 14. Pages 1961647. Epub Oct 20, 2026.

Abstract

Large language models may support postpartum health consultation and nursing care plan development, but their performance across different task types remains uncertain. This study compared the performance of ChatGPT 5.5 and DeepSeek V3 in these settings.
A three-stage cross-sectional evaluation was conducted, comprising repeated querying of 30 guideline-based questions, consultations involving 120 postpartum women, and 120 de-identified retrospective clinical cases. Responses were evaluated by clinical experts using a blinded assessment process. In Stage 2, the postpartum women who submitted the questions also performed blinded evaluations from the user perspective. Paired-samples t tests or Wilcoxon signed-rank tests were used, as appropriate. Holm-Bonferroni correction was applied within outcome families defined by study stage and evaluator type, and intraclass correlation coefficients were calculated to assess agreement among clinical expert ratings.
In Stage 1, ChatGPT 5.5 received a higher correctness score than DeepSeek V3 (4.00 ± 0.00 versus 3.88 ± 0.33; Holm-adjusted p = 0.002). DeepSeek V3 received higher scores for completeness and clinical management appropriateness (both Holm-adjusted p < 0.001) and safety (Holm-adjusted p = 0.001), although its responses were more difficult to read. In Stage 2, ChatGPT 5.5 received higher patient ratings for satisfaction and usefulness and higher clinician ratings for correctness, completeness, helpfulness, and safety, whereas DeepSeek V3 received a higher clinician-rated readability score (all Holm-adjusted p < 0.001). In Stage 3, ChatGPT 5.5 received higher scores for clinical appropriateness, comprehensiveness, actionability, safety, and triage appropriateness (all Holm-adjusted p < 0.001), whereas DeepSeek V3 received a higher individualization score (Holm-adjusted p = 0.021).
LLM performance varied depending on the task, evaluation metric, and use case. ChatGPT 5.5 received higher scores across most clinically relevant dimensions in real-world postpartum consultations and nursing care plan generation, whereas DeepSeek V3 received higher scores for completeness and safety in standardized questions, clinician-rated readability in real-world consultations, and individualization in care-plan generation. LLMs may support health education and preparation of initial nursing care plan drafts, but their outputs require clinical review.

PMID:
42978698
Bibliographic data and abstract were imported from PubMed on 04 Nov 2026.

Read full publication at:
Please sign in to see all details.

Advertisement

Stats

  • Community rating n/a 0 votes
  • Reviewers' rating n/a 0 votes
  • Your rating

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

  • Recommendations n/a n/a positive of 0 vote(s)
  • Views 14
  • Comments 0

Recommended by

  • No recommendations yet.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.

Advertisement