Rigorous nested cross-validation and external generalization-gap analysis of a machine-learning classifier for U-251 glioblastoma cytotoxicity.

Authors

Yuki Uemichi, Shuhei Nakao, Shuji Nagano, Miyuki Mabuchi, Yuya Kawabata, Yoshihiro Uesawa, Tadashi Shimizu

Published in

PloS one. Volume 21. Issue 11. Pages e0361912. Epub Nov 03, 2026.

Abstract

Glioblastoma remains difficult to treat, and drug-repositioning strategies require efficient methods to evaluate chemical libraries. Here, we developed a machine-learning binary classifier for compound activity against U-251 glioblastoma cells using a corrected single-center dataset of 236 compounds evaluated under a standardized cell-viability assay. Compounds were labeled as active or inactive using a predefined viability criterion (<50% vs. ≥ 50%) within a single-point screening framework using 10 µM as the upper screening concentration. To reduce optimistic bias and information leakage, we used repeated nested cross-validation in which preprocessing, feature selection, and hyperparameter optimization were performed within the training folds. ExtraTrees was selected based on Matthews correlation coefficient, showing moderate internal performance (ROC-AUC, 0.76; balanced accuracy, 0.66; MCC, 0.36). Structural analyses using nearest-neighbor Tanimoto similarity and Bemis-Murcko scaffolds indicated that the training set was scaffold-diverse with limited series bias. However, when the locked model was applied to an independent external set of 53 compounds, all compounds were classified as inactive, resulting in zero sensitivity, balanced accuracy of 0.5, and MCC of 0. The external ROC-AUC based on continuous prediction scores was 0.37, indicating poor threshold-independent ranking performance. Nearest-neighbor Tanimoto similarity did not explain prediction correctness or rescue active-compound recovery. Y-randomization supported the presence of a non-random internal signal, whereas scaffold-grouped cross-validation suggested that internal ROC-AUC was not solely driven by scaffold leakage. Overall, this study provides a reusable dataset and reproducible workflow, and highlights a substantial generalization gap between internal validation and external prediction in small chemical machine-learning datasets.

PMID:
42976131
Bibliographic data and abstract were imported from PubMed on 04 Nov 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.