by Eugène Loos, Associate Professor, Utrecht University School of Governance, the Netherlands; Johanna Gröpler, Freie Universität Berlin, Germany and Marie-Louise Goudeau, Utrecht University Library, the Netherlands.
ChatGPT is a fascinating AI text generator tool. It is a language model developed by OpenAI. This research and deployment company has the mission, according to its website: “to ensure that artificial general intelligence benefits all of humanity”. ChatGPT is able to generate human-like texts. But how does it work? What about the quality of the texts it provides? And is it capable of being self-reflective? Information sources must be efficient, effective and reliable in education, in order to enhance students’ learning process. For this reason, we started a dialogue with ChatGPT-3 while using, among others, a SWOT analysis it generated about its own functioning in an educational setting. This enabled us, as human authors, to analyze the extent to which this AI system is able to practice self-reflection.
The full paper including our dialogues with ChatGPT can be accessed to get insight into the detailed results: Loos, E. F., Gröpler , J., & Goudeau, M. S. (2023). Using ChatGPT in education: Human reflection on ChatGPT’s self-reflection. Societies, 13(8), 196. https://doi.org/10.3390/soc13080196
We will now highlight the main points resulting from the dialogue that we conducted with ChatGPT.
ChatGPT showed itself capable of generating a text following the format of a SWOT analysis, but the following critical points on the way ChatGPT generated this text can be made:
1. ChatGPT’s selection of Strengths, Weaknesses, Opportunities and Threats seems random.
From an academic point of view, specific arguments are necessary to explain why these aspects have been listed instead of others. The SWOT analysis could also have been more specific—although had more specific prompts been given and follow-up questions added, the answers might have contained more substantial information or arguments.
2. In the SWOT analysis ChatGPT generated, it underestimates its own weaknesses it faces
An important Weakness is the fact that ChatGPT produces hallucinations, as students cannot easily differentiate between when the system is hallucinating and when it is providing a correct response. As users, they need to have an overview of the entire information landscape, which will help them to understand why they should not use and share ChatGPT’s responses without first fact-checking. Teaching digital skills and critical thinking skills in this way is important, especially as students are novices in their field of study, differentiating between correct and incorrect becomes even harder. The awareness of this phenomenon will be even more important as ChatGPT is integrated into other software applications, making it less obvious that AI is involved. Fact-checking ChatGPT’s responses is difficult, as the tool is not yet always able to provide recent, reliable sources for its claims.
3. ChatGPT’s own SWOT analysis also clearly demonstrates its limited vision for education.
It focuses mainly on feedback on questions given by students and not the broader learning process, including (institutional) learning goals, the learning process, assessment and outcome. ChatGPT also seems not to be “aware” that it requires a human teacher to interpret a student’s emotional struggles in relation to the learning process .
4. It is not only important to be aware of biases in the training data. ChatGPT also seems to make politically correct pre-formulated phrases.
This prevents ChatGPT from being offensive, but it also indicates that the software owner has an influence on biases in the data. ChatGPT was developed by OpenAI, which used to be a non-profit organization. However, this changed in 2019, coinciding with a USD 1 billion investment from Microsoft. The effects of this on biases in the ChatGPT dataset is unclear. For this reason, it is important that ChatGPT’s creator OpenAI provide transparency about its data set.
5. A critical view on ChatGPT’s Strengths, Weaknessses, Opportunities and Threats
Farrokhnia et al. (2023) in Innovations in Education and Teaching International, 8, 1–15, https://www.tandfonline.com/ doi/full/10.1080/ 14703297.2023.2195846 focused on implications for educational practice and research and are quite positive about ChatGPT’s “skills” for “generating plausible responses, self-improving capability, providing personalized responses, providing real-time responses” (pp. 3–4) (strengths) and “increasing the accessibility of information, facilitating personalized learning, facilitating complex learning, decreasing” teaching workload (pp. 4–6) (opportunities). We assume a more critical stance towards ChatGPT’s Strengths and Opportunities, as their efficiency and effectiveness are not yet evidence-based.
We agree with Farrokhnia et al. (2023) that the “lack of deep understanding, difficulty in evaluating the quality of responses, the risk of biases and discrimination, lack of higher-order thinking skills” (pp. 6–7) (Weaknesses) and “lack of understanding of the context, threatening academic integrity, perpetuating discrimination in education, democratization of plagiarism, declining in higher-order cognitive skills” (pp. 7–9) (Threats) should be taken into consideration if the use of ChapGPT is being considered for educational and research practices.
It was also striking to note that when addressing the issue of avoiding the misuse of ChatGPT in education, ChatGPT itself appeared to lay the full responsibility on the user, which in itself is not a strongly self-reflective statement.
Finally, we as authors were curious about ChatGPT’s opinion on co-authorship, as is the case in some papers that added ChatGPT as a co-author. So, we asked it some questions and got the following answers:
– ChatGPT, You do not have the capability to receive or respond to emails, so we address the following point here. As you answered our questions we asked you for our article ‘Using ChatGPT in education: Human reflection on ChatGPT’s self-reflection’, we would like to know if you can be co-author for our paper?
– I’m sorry, but as an AI language model developed by OpenAI, I do not have the ability to be an author of a paper or any other creative work. I am simply a tool that can assist you in generating text based on the input you provide. While I can provide information and answer questions to help inform your writing, I cannot be credited as an author.
– Thank you ChatGPT for this clear response. In the academic world there is support for your standpoint. See: Stokel-Walker, C. (2023). ChatGPT listed as author on research papers: many scientists disapprove. Nature, 613, 620-621.
– Yes, it’s common practice in the academic community to only include human authors who have made substantial contributions to a research paper or other written work. Including AI systems like me as authors would not align with traditional understandings of authorship and the role of creative and intellectual contribution in producing a work. It’s important to note that while AI tools like me can assist in generating text and provide information to inform research, it’s still the responsibility of human authors to ensure the accuracy and validity of the information, and to interpret and analyze the results in a meaningful way.
Authors
Eugène Loos, Associate Professor, Utrecht University School of Governance, the Netherlands
Johanna Gröpler, Freie Universität Berlin, Germany
Marie-Louise Goudeau, Utrecht University Library, the Netherlands
