Automated Multi-Label Adverse drug Reaction Extraction from Malaysia's National Pharmacovigilance Narratives.

Authors

Sze Gee Lim, Maizatul Akmar Ismail

Published in

Journal of medical systems. Volume 50. Issue 1. Jul 31, 2026. Epub Jul 31, 2026.

Abstract

Extracting multiple adverse drug reaction (ADR) terms from unstructured narratives remains challenging, particularly under severe label imbalance that limits the detection of rare ADRs. This study aimed to develop and evaluate a multi-label natural language processing framework for automated ADR extraction within Malaysia's national pharmacovigilance reporting system. We evaluated classical machine learning and transformer-based models within a common framework, incorporating domain-specific preprocessing and imbalance-aware optimization strategies to improve the detection of rare ADRs. The framework was applied to 28,980 real-world ADR narratives annotated with 80 Medical Dictionary for Regulatory Activities (MedDRA) Preferred Terms (PTs) from the Skin and Subcutaneous Tissue Disorders System Organ Class (SOC). Performance was evaluated using micro-F1, macro-F1, precision, recall, and label coverage using a predefined 80/20 train-test evaluation strategy, complemented by pharmacist pilot review. Transformer-based models achieved the strongest performance, with the augmented model attaining a micro-F1 of 0.88, macro-F1 of 0.55, recall of 0.88, and the highest label coverage, correctly predicting 60 of 80 PTs (75%). However, a 70/10/20 sensitivity analysis on the improved transformer model yielded lower micro-F1 and precision, highlighting the impact of further data partitioning on model performance. Improved classical models also showed substantial gains over their baseline models, particularly in micro-F1, macro-F1, recall, and label coverage, consistent with improved detection of rare ADRs. During the pilot review, pharmacists accepted the model-preselected PTs without modification in 19 of 25 narratives (76%), while additional PTs were added in the remaining cases. Although the study was limited to a single SOC and one national pharmacovigilance database, the findings demonstrate that optimization strategies, domain-specific feature enrichment, and targeted augmentation can substantially improve multi-label ADR classification under severe imbalance. These findings support further evaluation of AI-assisted pharmacovigilance with pharmacist oversight in real-world practice.

PMID:
42536121
Bibliographic data and abstract were imported from PubMed on 31 Jul 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.