loading page

Properties of Person Descriptors in the Natural German Language: A Pre-registered Replication Study Generating and Using a Large, Representative Item Sample
  • Moritz Leistner,
  • Björn E Hommel,
  • Daniel Leising
Moritz Leistner
Otto-Friedrich-Universität Bamberg

Corresponding Author:[email protected]

Author Profile
Björn E Hommel
³ TU, Universität Leipzig
Daniel Leising

Abstract

Understanding language is crucial to psychological research, as it is the basis for almost all psychological measurements. Following previous works, we conducted a pre-registered replication study, analyzing 876 unique terms generated by 187 participants in a free response format. These terms were rated on social desirability, observability, importance, abstractness, base rate, and stability by approximately 15 raters each (n = 456). Key findings were replicated, including a bimodal distribution of social desirability, a greater number of negative vs. positive terms, and a u-shaped relationship between importance and social desirability. Using a Large Language Model (LLM) to derive synthetic social desirability estimates, we observed a moderate correlation (r = .65) with human ratings, but did not replicate all predicted effects. LLM-based item property estimates are not yet entirely equivalent to ratings by human participants but hold much promise for future research.
13 May 2024Submitted to Advance
27 May 2024Published in Advance