Detecting AI-Generated Responses in Personality Assessments

Abstract

Personality assessments are commonly used in organizational settings due to their ability to predict workplace outcomes (Barrick & Mount, 1991) and the reduced adverse impact associated with them during hiring processes (Hough et al., 2001). However, concerns are growing about the potential misuse of AI technologies like ChatGPT to manipulate responses in personality assessments (Budhwar et al., 2023). Despite limited research in this area, studies have found that ChatGPT can and does manipulate its responses on personality assessments (Bodroza et al., 2023; Phillips & Robie, 2024). To safeguard the integrity of personality assessments against ChatGPT misuse, this research focuses on developing methods to detect ChatGPT-generated responses. Two studies were conducted: Study 1 compared ChatGPT responses to job applicant responses at the subscale level; Study 2 developed a tool to detect ChatGPT-generated responses. Results showed that ChatGPT outperformed job applicants in inflating scores on all three personality measures, regardless of whether they measure bright or dark personality traits. In addition, the developed detection tool showed a Type I error rate of 0, highlighting its accuracy and potential for practical application.

Ähnliche Arbeiten