Generating interesting scientific ideas using knowledge graphs and LLMs: Evaluations with 100 research group leaders.

Authors

Xuemei Gu, Mario Krenn

Published in

Proceedings of the National Academy of Sciences of the United States of America. Volume 123. Issue 46. Pages e2611349123. Nov 17, 2026. Epub Nov 09, 2026.

Abstract

The rapid growth of scientific literature makes it increasingly challenging for researchers to identify novel and impactful ideas, especially across disciplines. Modern AI systems offer new opportunities for scientific ideation, but how compelling are AI-generated ideas, and how can their quality be improved? Here, we introduce SCIMUSE, which generates personalized research ideas using a knowledge graph of 58 million papers and a large language model (LLM). A central focus of this work is to understand how interesting these ideas are. Therefore, we conducted a large-scale evaluation in which more than 100 research group leaders-spanning the natural sciences to the humanities-rated over 4,400 personalized ideas according to their level of interest. Overall, expert ratings were modest (mean 2.40 on a 5-point scale, most common rating 1), while 24.9% of ideas were rated 4 or 5. We find that supplying concept pairs selected using the knowledge graph does not improve expert-rated interest over a titles-only GPT baseline. High-citation-predicted pairs even showed a weak tendency (1.94σ) toward lower interest than random pairs. Nevertheless, graph features can be used to control properties of ideas, and, using this unique evaluation dataset, we show that idea interest can be predicted with both a supervised neural network based on graph features and a zero-shot ranking approach based on an LLM. Our work provides an AI methodology for generating scientific ideas and a large-scale interdisciplinary expert evaluation, paving the way to study and improve difficult-to-measure metrics such as expert-perceived scientific interestingness.

PMID:
43002667
Bibliographic data and abstract were imported from PubMed on 10 Nov 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.