Enhancing Medical Proficiency: exploring the benefits of communication training in Virtual Reality

by Stéphanie Vanneste, KU Leuven, Belgium.

If the pandemic has taught us anything,  it’s that a well-developed healthcare sector and well-trained healthcare professionals are of vital importance. But what does it take to be a good doctor? According to the CanMEDS framework, which has the ultimate goal of improving patient care, physicians require a set of skills to effectively meet the healthcare needs of their patients. These skills are very diverse, from rather technical to soft skills, and should be trained during different courses, but especially during simulation training.

Simulation training today does not always place sufficient emphasis on these soft skills, but rather on the technical skills of diagnosing in stress-increasing conditions of limited time. Moreover, good simulation training also very often depends on the quality of feedback from the coaches and takes up a lot of (human) resources as well. That’s why several imec research groups started a joint project called ‘Smart Collaboration Tutor’ (SCoT), as one of the experiences of the Experience Twin framework. By utilizing psychophysiological sensors along with cutting-edge data analysis and signal processing, this innovative framework utilizes XR technology to gain insights into human behaviour. As it is a multi-user setting, two students go simultaneously into the digital twin of the physical simulation room. They are confronted with the virtual patient showing particular symptoms. Due to the limited amount of time, they need to work collaboratively to figure out what condition the patient has. This is a reflection of a real-life work situation, in which every second counts to save a person’s life. So good collaboration is key! Successful collaborative problem-solving (CPS) is becoming increasingly important as a 21st century skill due to its numerous benefits over individual problem-solving. As workplaces increasingly demand people who can solve non-routine problems in concert with others, CPS has become a critical skill. If managed poorly, CPS can lead to communication problems, interpersonal conflict, and inefficiencies. Thus, the acquirement of those types of skills are necessary to engage in successful CPS. The PISA 2015 Collaborative Problem Solving assessment measures students’ CPS skills, focusing on the collaborative aspects of problem-solving such as communication, conflict management and organising a team.

There are three main advantages to using virtual reality in simulation training. First of all, the students are completely immersed in the virtual world. They don’t feel the presence of the evaluating coach anymore and feel that the life of this virtual patient is now in their hands. The second one is the dashboard and corresponding coach view. Being now in another room, the coach has the ability to check the dashboard in real time and all the actions the students should do to tackle the problem. Supplementary, the coach can change the parameters, like heart rate and saturation, to check if the students follow up on the changes and react to the alternation in the symptoms. The final advantage is the possibility to send out a nudge in case the students are on the wrong path or overlooking important information. It can be sent as a red dot or a text, directed to one of both students, that alerts the students to look at the problem from another perspective. This just-in-time information opens the opportunity for students to change their behaviour and make the necessary adjustments for more efficient CPS. The first results have shown that most students found the nudges more supportive than they were disturbing or causing stress. They also preferred the textual nudge over the red dot. Post-experiment reactions of the students on the nudges were mostly “Deliverance” or “The nudge allowed us to continue […], which was positive, but also frustrating because you couldn’t come up with it yourself”. Results have also shown that the participants – both students and coaches – found VR simulation quite flexible in use and they hope to use it more in the near future.

As technology is never the best version of itself, we determined future developments for our SCoT. Currently, we are focussing on the debriefing offering it as an automatically composed resume of the most crucial moments during the VR simulation training, e.g. the time range in which a nudge was sent and good CPS communications like dividing roles or checking up on each other. The next step will be the development of fully automated training, where nudges will appear based on captured audio or eye-tracking data. This way the students can do simulation training without the presence of a coach and train more on CPS skills. If this can be achieved we can evolve to a format in which both students don’t even have to be present in the same room to be colleagues saving a patient’s life virtually.

Editor’s note: Stéphanie Vanneste will be presenting at the Media & Learning online seminar “How medical doctors and developers collaborate to develop educational XR applications” that will take place on Thursday, 20 April.


Stéphanie Vanneste, KU Leuven, Belgium