Enhancing Cancer Outcomes Research Through Vision Language Models for Cause of Death Ascertainment.

Authors

Enshuo Hsu, Giordana De Las Pozas, Trey Kell, Kirk Roberts, Shida Jin, Kelly W Merriman

Published in

JCO clinical cancer informatics. Volume 10. Issue 4. Pages e2600037. Epub Oct 01, 2026.

Abstract

Cause of death is an important variable in cancer registries that supports downstream analyses, including cause-specific survival analysis. Unfortunately, in current practice, it is only available on government-issued death certificates, which are often received as scanned images. Current computer vision-based optical character recognition (OCR) methods struggle with handwriting, background noise, and blurriness, resulting in poor performance. A robust method for the cause of death ascertainment is urgently needed.
Latest vision language models (VLMs), such as Qwen3-VL and GPT-4.1, have shown promising OCR performance on poor-quality scanned documents in the general domain. We proposed two prompting methods: (1) the two-step approach, which prompts VLMs for a full-text OCR followed by a text-based information extraction and (2) end-to-end approach, which prompts VLMs with a JavaScript Object Notation schema for image-based information extraction.
We evaluated VLMs for the cause of death extraction from scanned government-issued death certificates in a major cancer center. Of 22,733 death records, 500 were manually annotated as ground truth. For the immediate cause of death, Qwen3-VL-30B-A3B-Thinking with the end-to-end approach achieved the best accuracy of 0.8625, significantly outperforming the Tesseract baseline (0.2396). For the underlying causes of death, the two-step method achieved the best F1 score of 0.7637, outperforming the baseline (0.2451).
The growing emphasis on precision medicine underscores the importance of accurate cause of death ascertainment for survival analyses. Our method demonstrated significant improvements over the current OCR approaches and provided a reliable alternative to reliance on the National Death Index.
Our VLM OCR computational framework is publicly available on GitHub. The open-weight VLMs are available for download on the Hugging Face repository.

PMID:
42821829
Bibliographic data and abstract were imported from PubMed on 02 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.