RADS-PDD: Reproducibility-aware dynamic similarity-based pre-training data detection against LLMs.

Authors

Xin Fan, Miyamoto Ryoto, Fan Mo, Chongxian Chen, Tsuneo Matsumoto, Fuyuko Kido, Hayato Yamana

Published in

Neural networks : the official journal of the International Neural Network Society. Volume 205. Issue Pt A. Pages 109393. Jul 16, 2026. Epub Jul 16, 2026.

Abstract

The power-law relationship between large language model (LLM) performance and the scale of pre-training corpora has driven remarkable advances, while raising serious ethical and legal concerns. Pre-training corpora often contain sensitive personal information, copyrighted content, or benchmark test data, and the opacity of pre-training corpora further exacerbates these concerns. Consequently, detecting pre-training data has become a significant research challenge. Most existing detection methods rely on intermediate outputs of LLMs (activations, token probabilities, or model loss), which are inaccessible in commercial LLMs, where only final outputs are available. Only a few early-stage attempts rely solely on final outputs by measuring lexical or semantic similarity between reproduced and original texts at the sentence or token level. Yet, these attempts overlook the dynamic nature of reproducibility: reproducibility fluctuates differently across reproduction positions, contextual spans, and expressions, between seen and unseen texts. This oversight narrows the gap of reproduction similarity, ultimately degrading detection performance. To address this limitation, we propose RADS-PDD (reproducibility-aware dynamic similarity-based pre-training data detection), which shifts black-box pre-training data detection from static similarity comparison to reproducibility-aware dynamic similarity modeling. RADS-PDD incorporates three reproducibility-aware mechanisms, positional gain weight, continual gain weight, and triplet occurrence probability, to quantify varying reproducibility by dynamic similarity, amplifying the reproduction similarity gap between seen and unseen texts. Extensive experiments across two representative datasets, multiple data domains, and diverse LLMs demonstrate that RADS-PDD consistently outperforms detection methods that rely solely on final outputs and achieves performance comparable to detection methods that require intermediate outputs.

PMID:
42480164
Bibliographic data and abstract were imported from PubMed on 22 Jul 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.