Comparison of DeepSeek AI and Human Reviewers in Structured Evaluation of Neurosurgical Manuscripts: A Pilot Study.

Authors

Kishore Balasubramanian, Daniel O'Connor, Beste Gülsuna, Visish M Srinivasan, Daniel A Donoho, Nicholas Sader, Christopher S Graffeo

Published in

Journal of neurological surgery reports. Volume 87. Issue 4. Pages e192-e201. Epub Oct 09, 2026.

Abstract

Objectives Increasing manuscript volume has strained peer review, prompting interest in artificial intelligence (AI) as a supportive tool. We evaluated DeepSeek R1 as a structured peer reviewer of neurosurgical case-based manuscripts and compared its scoring patterns with those of an experienced editor and student reviewers. Design Exploratory, cross-sectional comparative study using a convenience sample of consecutively submitted manuscripts. Setting Single skull-base neurosurgical journal during a 6-month submission window (January-June 2025). Participants Three reviewer groups independently assessed 11 de-identified, unpublished neurosurgical manuscripts: first-year medical students ( n = 154; each randomly assigned one manuscript after a course in biostatistics, epidemiology, and critical appraisal), the journal's Editor-in-Chief (Experienced Reviewer), and DeepSeek R1. Main Outcome Measures Each manuscript was rated on a 3-point Likert scale (1 = low, 2 = intermediate, 3 = high) across five domains: Study Objective, Novelty, Study Design, Thoroughness, and Clarity. Results Significant between-group differences were found in four of five domains: Study Objective ( H = 8.31, p = 0.02), Novelty ( H = 9.74, p = 0.008), Thoroughness ( H = 10.49, p = 0.005), and Clarity ( H = 8.13, p = 0.02); Study Design did not differ ( H = 1.45, p = 0.483). DeepSeek scored significantly higher than the Experienced Reviewer for Study Objective ( p = 0.015) and higher than both human groups for Novelty. Conclusions DeepSeek showed consistency in structured domains but systematically inflated subjective ratings, particularly Novelty. A human-in-the-loop model integrating AI efficiency with expert judgment may optimize peer review.

PMID:
42859550
Bibliographic data and abstract were imported from PubMed on 11 Oct 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.