ChatGPT as a Source of Information in Higher Education: How are the Results Provided by Generative AI Valued?

Abstract

The release of ChatGPT-3 in November 2022, followed by its subsequent versions up to the current 4.0, and its rapid expansion and widespread use, has raised several questions of particular interest regarding its role as an information source in Higher Education. Recognising the growing generalisation of generative Artificial Intelligence (AI), this article is based on the premise that it is essential to understand how university students perceived and evaluated ChatGPT-3 a few months after its launch. The research conducted using a mixed-methods approach, analyses 4,800 university student responses to an online survey using bivariate and multiple segmentation methods. The survey asked participants to state their level of knowledge and experience with AI, as well as their evaluations of the information provided by ChatGPT. Additionally, experts conducted a thematic analysis of ChatGPT's responses using an example based on the 11-M attacks. The findings reveal significant differences in students' levels of knowledge, prior experience with ChatGPT, and their evaluations of the information it provides, linked to various sociodemographic and educational variables. The results also underline that prior experience with generative AI influences its evaluation, both as an information source and as a tool for disseminating fake news. Finally, the comparison of student evaluations with those of experts stresses the need to teach students how to effectively train ChatGPT to generate better responses and how to critically evaluate them. This approach could serve as a foundation for designing educational strategies to integrate generative AI into Higher Education.