by Julia Dawitz & Emma Wiersma, University of Amsterdam, The Netherlands.
On January 16th 2023, the Programming Teaching Lab and the Teaching and Learning Centre Science of the University of Amsterdam organised a 4-hour ChatGPT workshop for lecturers. ChatGPT is a language model with chat capabilities and it seems like it can do almost anything: answer all of your questions, come up with “new” ideas, write your essay for you, and whatnot. Lecturers all over the world shudder in fear: Can we still trust the content of our students or is everything generated by AI?
However, there are also a lot of opportunities with this new technique. During the workshop, we wanted to focus on the positives: What can lecturers take away from ChatGPT? How can we use ChatGPT to our own advantage? Should we think about new ways of assessing? Can it generate new assignments, embed its use into homework? We would argue that the sky is the limit here.
What is ChatGPT?
Jelle Zuidema, associate professor in Natural Language Processing, Explainable AI and Cognitive Modelling at the Institute for Logic, Language and Computation, explained that ChatGPT didn’t come out of the blue, but there has been a continuous development of natural language processing technology in recent years. There is no explicit source where it gets all its information from, but it makes abstractions from all the available online information. However, that doesn’t mean that the information it provides is always correct. It produces a lot of content that sounds right because it has learned the grammar and different types of writing genres and can reproduce those structures. However, the content can be (and regularly is) actually nonsense. As ChatGPT itself explains it:
It is a pure language model and produces content based on what are plausible continuations of text. It is trained on user data and wants to make its users happy (e.g. by providing short and clear answers), but it’s not optimised for truthfulness. For example, it can generate sources when the user asks for references because that will make the user happy, but those references can be completely made up.
But how reliable is the content in that case? How do you distinguish confabulations from facts? This is still an active field of research. A possible solution can be to first search for factual content with classical techniques (e.g., in books or with reliable online sources) and then use ChatGPT to summarise all the information from those sources. Unfortunately, asking ChatGPT in your prompt to “be truthful” won’t solve this problem. It might lead to a more truthful answer, but then that’s purely a coincidence.
How can ChatGPT be used in education?
But if producing truthful content isn’t what ChatGPT does, how can we use it in education? Martijn Stegeman, lecturer and coordinator of the UvA minor Programming, identified two main areas for which ChatGPT can be used in education:
- Student learning aid. Student enrolments are very high, leaving lecturers with too little time to properly supervise and guide their students. How can we use ChatGPT to help students study and to make education better? For example, ChatGPT can explain Python code, write flashcard questions that students can use for self-study, and execute a roleplay with ChatGPT as the lecturer.
- Lecturer inspiration. As a lecturer, ChatGPT can save you a lot of time and give you inspiration for your teaching. For example, ChatGPT can serve as a test-student to check whether your exam questions are unambiguous, create exam questions, generate a creative programming assignment, write a study guide for a course, and give feedback on an assessment scale.
How to ask the right questions?
Remember, ChatGPT is programmed to provide answers that make the reader happy. Vice versa, that means that asking the right questions – called prompts – is essential when using it. Therefore, Martijn provided guidance for writing good prompts. The most important thing is to not phrase the prompt too broadly or vaguely, but as precisely as possible. Your prompt should include all the criteria that you are looking for in the answer. For example, if you need questions that test whether your students can apply their knowledge, asking ChatGPT for 10 questions might result in questions of a quality you cannot use. However, if you ask ChatGPT to create 10 questions that require a relational response (as described in the SOLO taxonomy) it will adhere to those requirements (see images below).
Keep in mind, if there’s a lot of content available about the topic or if it is a common genre, ChatGPT’s output becomes better (e.g. ChatGPT is really good at creating a study guide entry based on a couple of your notes).
For more tips about writing good prompts for ChatGPT, you can read this Dutch article from Neuroflash.
Take-home messages
The participants experimented with ChatGPT in small groups and discussed how they can use ChatGPT in their own field of expertise. These are their most important findings:
- Talk to your students: It’s important to ask your students if they already use ChatGPT and how they are using it.
- Ask the right questions: If you write as your prompt “Can you formulate a research question about …”, the answer is very generic and superficial. It is important to formulate the right questions and make them as concrete and precise as possible.
- Generate practice exams: It can be a useful tool for generating practice exams or questions for students. There is always a high demand for that from the students.
- Use it as a starting point: ChatGPT can be a good starting point, but not an ending point. It can be used for inspiration or tips, for example to create a sample rubrics or tips on how to formulate a research question. However, the generated content should be critically analysed, improved, expanded, and supported by references.
- Use it for a first round of feedback: When writing a research proposal or an essay, students can use ChatGPT as a first round of improvement. They can ask for feedback on their draft version. ChatGPT can filter out some small mistakes that otherwise the lecturer or a peer would have given, for example feedback about the structure or the language. This saves lecturers a lot of time and can improve the quality of the students’ work.
- Don’t use it for asking questions about scientific papers: It cannot cite or use sources. When asking ChatGPT why references don’t work, it answers the following: “I am a language model and do not have access to external sources or references. I can provide information based on the knowledge that has been input into my training data. However, it is always a good idea to verify any information provided by me with credible sources.”
- Spend some time experimenting: Experience what works and what doesn’t. You can even play games with it, or can’t you?
Editor’s note: This article was first published on the Teaching & Learning Centre’s (University of Amsterdam) blog
Authors
Julia Dawitz, Educational Advisor & Lecturer Psychobiology, Teaching & Learning Centre Science and College of Life Sciences, University of Amsterdam (UvA), The Netherlands
Emma Wiersma, Educational Innovator & Advisor Blended Learning, Teaching & Learning Centre Science, University of Amsterdam (UvA), The Netherlands
