CAN ARTIFICIAL INTELLIGENCE LANGUAGE MODELS PERFORM FACT-CHECKING?

Abstract

The aim of this study is to examine whether artificial intelligence (AI) language models can be effectively used as fact-checking tools and contribute to verification processes. The research was designed using a qualitative method with a case study approach, and thematic analysis was conducted through document analysis. ChatGPT and Gemini were selected using a similar sampling strategy. A total of 100 claims were examited 50 verified by Teyit.org concerning the Turkish agenda, and 50 verified by PolitiFact regarding the American and European agenda. Both AI models were asked to evaluate these claims based on the Truth-o-Meter rating scale. According to the findings, ChatGPT correctly identified 27 out of 50 claims related to Turkey, resulting in an accuracy rate of 52%, while Gemini correctly verified 38 claims, reaching 76%. For claims related to the Western agenda, ChatGPT again correctly answered 27 out of 50 (54%), while Gemini achieved 66% accuracy. Overall, Gemini demonstrated higher accuracy than ChatGPT across both regional contexts. These results indicate that although AI models can contribute to fact-checking efforts, their current performance is not yet reliable enough for standalone use in news verification. However, it is anticipated that with ongoing machine learning advancements and integration of real-time data, AI models will significantly improve. In the near future, such models could become dependable tools in the fact-checking ecosystem.

Ähnliche Arbeiten