Myth, reality, or in between: Unveiling potential geographical biases of ChatGPT

Abstract

Introduction. This research examines how geographically biased training data influence the nature of content in ChatGPT responses and to assess the potential occurrence of various geographically biased responses from the users' perspective. Method. ChatGPT was tested with geographically oriented prompts on ninety-eight countries. The responses were analysed for opinions, facts, and neutral directive sentences, as well as their qualitative and quantitative characteristics. A user survey was conducted on identifying potential geographical biases in ChatGPT responses. Analysis. The Wilcoxon signed-rank test and the Permutation test were employed, in addition to descriptive analysis. R programming language within RStudio were utilised for the data analysis. Results. Central and Western European countries exhibited more opinion and fact sentences in their responses, respectively. Qualitative responses had greater meaning consistency than quantitative ones. Sentence type depended on qualitative or quantitative nature, not prompt geography. ChatGPT 3.5 was the most used version, with no reports of geographically offensive, racially biased, or religiously biased responses. Views on geographical bias varied by region, though certain trends emerged. Conclusion. ChatGPT generates responses of similar lengths irrespective to regions. Qualitative responses are generally more consistent or reliable in terms of their meanings. Most users do not perceive geographical biases, though concerns arise in East Asia and South America.

Ähnliche Arbeiten