by Angela Xia & Simon Sheridan, Xi’an Jiaotong Liverpool University, China.
Language assessments have always played a crucial role in measuring students’ knowledge and ability to use the language effectively. At English as Medium of Instruction (EMI) universities, language assessments are even more crucial as they provide valuable insights into students’ academic readiness for conducting their studies in English. However, with the rapid development of generative artificial intelligence (GenAI) technologies, traditional language assessments, like written essays, are facing new challenges in maintaining their reliability and validity. After all, GenAI tools, such as ChatGPT, have the ability to generate text that can mimic human writing without being effectively detected by traditional plagiarism detection systems. Written essays are traditionally strongly relied upon as an effective test format in English for Academic Purposes (EAP) modules, which are delivered by language centers at EMI universities. As such, language centers are now faced with an urgent need to update their assessment regimes in order to respond to the impact brought by GenAI technologies.
With more than thirty credit-bearing EAP and other language related modules delivered to thousands of students each semester, the English Language Centre at our university, Xi’an Jiaotong Liverpool University (XJTLU), has responded proactively to the challenges and has developed a comprehensive strategy to update our assessments.
Our first step was to thoroughly analyze the existing assessment components to identify vulnerability to GenAI’s impact. Based on this analysis, we identified several areas, including the aforementioned overreliance on written essays as well as the isolated assessment of speaking skills, that needed to be addressed. We then developed a multi-faceted approach to updating our assessments, while ensuring the updated assessment components still measure the learning outcomes of the modules effectively.
This approach includes the following measures:
- Introducing a “Project” assessment: unlike previous standalone, one-off essay assignments and speaking tests, the Project assessment requires students to engage in a contextualized research task that integrates various language skills. For example, students enrolled in degree programs in urban planning and architecture may be asked to work on a research paper about a local site and propose design solutions. The Speaking component of the assessment may involve students presenting their findings and solutions in a group setting and defending their ideas in front of fellow students and the teacher. Throughout the semester, students will receive feedback and guidance at various stages of the project. In short, the project component will not only assess students’ language skills but also their ability to apply those skills in a real-world context, enhancing their critical thinking and problem-solving abilities. The multi-staged and contextualized design of the assessment will hopefully mitigate the possibility of student’ reliance on GenAI technologies, with an aspect of the spoken component of the assessment actively testing whether the students are fully aware of the report they purportedly wrote.
- Designing flexible in-class tasks: considering the differences in the learning aims and scopes of our EAP modules, each module leader is also given the flexibility to incorporate some in-class assessment components that are designed to specifically target the skills and language areas relevant to their students’ needs. Such in-class assessment tasks include group discussion, simulated tutorials, and timed writing. In all cases, the tasks constructively support the overarching learning outcomes of the module while ensuring that lecturers monitor student performance in ways that cannot be supported by GenAI.
- Continuing under exam conditions: lastly, we also ensure that our assessments include tasks that assess language skills without the assistance of GenAI tools. Although students inevitably need to use GenAI tools in their future careers, it’s our strong belief that they should be tested at carefully chosen timepoints in their language learning journey to measure their individual language proficiency without the assistance of GenAI tools; indeed, this is a vital undertaking given XJTLU’s ability to guarantee proficiency levels to the UK government. End-of semester examinations permit students to demonstrate their language knowledge and skills in a technology-free setting. The final scores, which are calculated as a combination of the project, in-class tasks, and examinations, provide a comprehensive yet balanced assessment in various contexts.
Once approved by the university, these changes will be implemented at the beginning of the next academic year. We will closely monitor the impact of these updated assessments on students’ language development and make adjustments based on feedback from staff and students. Although some of these changes may seem radical, we believe that they are long overdue and the advent of GenAI technologies has provided a unique opportunity to rethink and improve our language assessments. The EAP community, as a whole, should proactively evolve its assessment strategies to ensure that they will align with the rapidly changing educational landscape and effectively prepare students for the future.
Angela Xia is the Associate Dean for Learning and Teaching and a Principal Language Lecturer at the School of Languages, Xi’an Jiaotong Liverpool University, China.
Simon Sheridan is Director of the English Language Centre at Xi’an Jiaotong Liverpool University and Chair of the Exam Board for all 17 EAP modules taken by Y1 and Y2 students at the university.
