PathoBench: an open community-driven benchmark registry for pathogen bioinformatics tools

Authors

Dong, Y., Li, N., Chiribau, C. B., Mitchell, M., Liu, X., Perkins, A.

Abstract

The proliferation of pathogen bioinformatics pipelines has outpaced the community ability to compare them on common ground. Self-reported performance numbers, ad-hoc evaluation datasets, and inconsistent metrics make pipeline selection difficult for clinical and public-health researchers. We present PathoBench, an open web platform that addresses this gap through three coordinated mechanisms: (i) a curated registry of 26 standard benchmark datasets across 10 human pathogens, each with persistent identifiers and direct download links; (ii) pathogen-specific evaluation metrics that submissions must report, allowing direct head-to-head comparison only on the same dataset; and (iii) a credibility framework combining mandatory dataset attestation, ORCID-linked attribution, public peer comments, and administrator verification. As a case study, four published Mycobacterium tuberculosis drug-resistance pipelines were evaluated against the WHO TB mutation catalogue, demonstrating the framework discriminating power. PathoBench is open for community contributions across all ten supported pathogens.

Preprint server: bioRxiv
The authors list and abstract were imported from bioRxiv on 18 Jul 2026.

Stats

1-terrible, 9-excellent. How would you rate this preprint? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.