Hiring in life sciences? Share your open positions with our professional community. Read more Close

Advertisement

Recommendations for Incorporating Large Language Models Into Psychological Research: A Commentary on Austin et al. (2025).

Created on 19 Nov 2026

Authors

Wendy C Higgins, Beth Clarke, Malte Elson, Jamie Cummins

Published in

Assessment. Pages 10731911261503860. Nov 19, 2026. Epub Nov 19, 2026.

Abstract

Large language models (LLMs) have clear potential to contribute to psychological research methods. However, as with any new research method, LLM-based methods require rigorous evaluation. Towards this end, Austin and colleagues evaluated the ability of two LLMs (GPT-4o and Gemini 2.0 Flash) to facilitate scale validation by identifying relevant criterion measures and generating validity hypotheses for assessing convergent and discriminant validity. We argue that their study has several key limitations that limit the conclusions that can be drawn about the proposed method's suitability for use in validation research. We situate our critique within three recommendations for incorporating LLMs into research: (a) avoid essentialisation and overgeneralisation about 'AI', (b) use application programming interfaces when evaluating LLMs in research contexts and (c) evaluate LLM-based methods using clearly specified criteria. Finally, we consider whether adopting the proposed method might reinforce or amplify existing limitations in the psychological literature, namely, the proliferation of overlapping constructs and inadequately validated measures.

PMID:
43053736
Bibliographic data and abstract were imported from PubMed on 19 Nov 2026.

Read full publication at:
Please sign in to see all details.

Advertisement

Stats

  • Community rating n/a 0 votes
  • Reviewers' rating n/a 0 votes
  • Your rating

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

  • Recommendations n/a n/a positive of 0 vote(s)
  • Views 0
  • Comments 0

Recommended by

  • No recommendations yet.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.

Advertisement