From Black Box to Disposition: SurveyingDivergence, Novelty, and Ethics in Large Language Models

Abstract

<title>Abstract</title> This study investigates the dispositional tendencies of large language models(LLMs) by developing and validating a survey instrument tailored to their uniquecharacteristics. While prior research has adapted human psychometric frameworkssuch as the Big Five to LLMs, these approaches face conceptual andmethodological limitations. To address this gap, this study introduced three traits(convergent-divergent reasoning, novel-conventional problem solving, and ethicalorientation) that more directly capture the functional dispositions of LLMs.A forced-choice questionnaire was co-designed with multiple models to assessthese traits, and its predictive validity was tested against open-ended behavioraltasks. Results demonstrate that questionnaire responses align with observed tendencies,with moderate-to-strong correlations for divergent and novel reasoning,and consistent expression of virtue ethics across models. These findings suggestthat LLMs exhibit measurable, context-dependent dispositions that can be systematicallyassessed. The proposed framework contributes to ongoing efforts tointerpret model behavior, inform prompt design, and advance the theoreticaldiscourse on personality-like constructs in artificial systems.

Ähnliche Arbeiten