Orchestrator multi-agent clinical decision support system for secondary headache diagnosis in primary care.

Authors

Xizhi Wu, Hang Zhang, Nelly Estefanie Garduno-Rapp, Justin F Rousseau, Mounika Thakkallapally, Yuelyu Ji, Shyam Visweswaran, Yifan Peng, Yanshan Wang

Published in

Journal of the American Medical Informatics Association : JAMIA. Jun 27, 2026. Epub Jun 27, 2026.

Abstract

Secondary headaches require urgent recognition due to potentially devastating consequences if untreated. Despite established clinical "red flag" criteria, identifying patients needing immediate evaluation remains challenging in primary care. This study developed and evaluated a large language model (LLM)-based multi-agent clinical decision support system for interpretable secondary headache diagnosis.
We first established 7 clinically relevant secondary headache red flag domains through manual review and synthesis of clinical guidelines. Based on these domains, we designed an LLM-based system using an orchestrator-specialist multi-agent architecture that decomposes diagnostic reasoning into 7 guideline-aligned agents corresponding to key red flag features. Each agent generates structured, evidence-grounded reasoning, coordinated by a central orchestrator. The system was evaluated on 90 expert-validated secondary headache cases and compared with a single-LLM baseline under 2 prompting strategies: question-based prompting (QPrompt) and guideline-based prompting (GPrompt). Five open-source LLMs (Qwen-8b, Qwen-14b, Qwen-30b, GPT-OSS-20b, and Llama-3.1-8b) were tested.
The orchestrated multi-agent system with GPrompt achieved the highest red flag classification performance across models, measured by F1 score. Performance gains were consistent and more pronounced in smaller LLMs, suggesting that structured reasoning improves efficiency and accuracy beyond prompt engineering alone. The framework also produced transparent and guideline-aligned intermediate reasoning.
Decomposing clinical reasoning into specialized agents enhances interpretability and diagnostic reliability compared with monolithic LLM approaches. Multi-agent orchestration provides a clinically aligned framework for explainable decision support.
An orchestrator-specialist multi-agent LLM framework improves secondary headache diagnosis accuracy and transparency, supporting the development of explainable AI systems for time-constrained clinical decision-making in primary care.

PMID:
42364078
Bibliographic data and abstract were imported from PubMed on 27 Jun 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.