Open-source offline-deployable retrieval-augmented large language model for assisting pancreatic cancer staging.

Authors

Hisashi Johno, Akitomo Amakawa, Atsushi Komaba, Ryota Tozuka, Yuki Johno, Junichi Sato, Kentaro Yoshimura, Kazunori Nakamoto, Shintaro Ichikawa

Published in

Japanese journal of radiology. Jul 09, 2026. Epub Jul 09, 2026.

Abstract

Large language models (LLMs) are increasingly applied in radiology, but key challenges remain, including data leakage from cloud-based systems, false outputs, and limited reasoning transparency. This study aimed to develop an open-source, offline-deployable retrieval-augmented LLM (RA-LLM) system in which local execution prevents data leakage and retrieval-augmented generation (RAG) improves output accuracy and transparency using reliable external knowledge (REK), demonstrated in pancreatic cancer staging.
Llama-3.2 11B and Gemma-3 27B were used as local LLMs, and GPT-4o mini served as a cloud-based comparator. The Japanese pancreatic cancer guideline, written in English, served as REK. Relevant REK excerpts were retrieved to generate retrieval-augmented responses. System performance, including classification accuracy, retrieval metrics, and execution time, was evaluated on 100 simulated pancreatic cancer cases with CT findings described in English, using non-RAG LLMs as baselines. McNemar tests were applied to TNM staging and resectability classification.
RAG improved TNM staging accuracy for all LLMs (GPT-4o mini 61% → 90%, p < 0.001; Llama-3.2 11B 53% → 72%, p < 0.001; Gemma-3 27B 59% → 87%, p < 0.001) and mildly improved resectability classification for GPT-4o mini and Llama-3.2 11B (72% → 84%, p = 0.012; 58% → 73%, p = 0.006), while the improvement for Gemma-3 27B was not evident (77% → 86%, p = 0.093). Gemma-3 27B showed performance comparable to GPT-4o mini. Retrieval performance was high (context recall = 1; context precision = 0.5-1), and local models ran at speeds comparable to the cloud-based GPT-4o mini.
We developed an offline-deployable RA-LLM system for pancreatic cancer staging and publicly released its full source code. RA-LLMs outperformed baseline LLMs, and the offline-capable Gemma-3 27B performed comparably to the widely used cloud-based GPT-4o mini.

PMID:
42423906
Bibliographic data and abstract were imported from PubMed on 09 Jul 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.