Cautions in using ChatGPT for qualitative thematic analysis: the case of social support in an internet forum for patients

Abstract

Purpose Although ChatGPT was not originally designed for qualitative data analysis, it has recently garnered interest and discussion in this area due to its potential advantages. The use of ChatGPT in qualitative thematic analysis remains understudied. On these premises, the aim of this study was to use ChatGPT to re-analyze a textual dataset that was previously analyzed by human researchers using qualitative thematic analysis, allowing for a comparison between the findings that were obtained with the two approaches (human researchers vs ChatGPT). Design/methodology/approach One hundred and eighteen posts authored by patients in an online forum were analyzed by ChatGPT using specific prompts to identify the main types of social support requests and offers. These findings were critically reviewed and compared with those previously obtained by human researchers. Findings The results indicated that ChatGPT identified five themes consistent with the textual data under analysis, which partially overlap with the themes previously identified by human-driven analysis. Research limitations/implications The lack of explainability of the analytical process and the potential biases in the results, together with the fact that probabilistic algorithms cannot be viewed as fully “qualitative”, indicate that the role of human researchers remains crucial. The findings offer some preliminary practical implications and suggestions for future research. Originality/value The novel design of this study allowed us to identify some possible advantages and limitations of using artificial intelligence tools like ChatGPT in supporting human-driven qualitative thematic analysis.

Ähnliche Arbeiten