Cinematic Rendering (CR) is a physically based volume rendering method that produces jaw-dropping photorealistic 3D images of medical imaging data, take a look at these two images as examples.
The data is retrieved from different imaging techniques such as Computed Tomography (CT), or Magnetic Resonance Imaging (MRI). The technology of CR was first described in 2016 by a research group in Siemens Healthcare GmbH. Since then it has been improved and shown its great potential for different situations in which medical images are used.
The range of possible implementations is wide. Ranging from pre- and intraoperative planning, interdisciplinary patient case discussions, patient communication and of course for medical education for students as well as for medical professionals. In a research group at Friedrich-Alexander University Erlangen-Nürnberg (FAU), consisting of members from Siemens Healthcare GmbH, the Institute for Anatomy and the Department of Surgery of the University Clinic of Erlangen, we are currently testing the technology in different settings. So far, we tested it for speed and comprehension in two studies – one focusing on surgery and one on medical education. Both studies showed significant time reduction and an improvement in comprehension for both groups.
Very recently, Cinematic Reality has been introduced. The technology combines Cinematic Rendering with Microsoft HoloLens 2. This will expand the range of applications for this technology, which is available by the radiology software Syngo.via. Using a mixed-reality headset instead of viewing the clinical images on a 2D monitor it allows a completely new insight into the human body. The possibility to interact freely with the image, exposing different types of tissue, turning the image in different directions, and thus enabling the user perspectives that have never been seen before, as you can see in the following examples.
Human anatomy is very complex due to its three-dimensional character. However, each individual’s spatial imagination ability can be very different which may lead to misinterpretation. The virtual reality experience might help to overcome this issue. Traditional teaching methods are mostly based on artificial anatomy models and textbooks. These standardised methods are undeniably very helpful and irreplaceable for medical education. Considering anatomical varieties and pathologies, one can only fully comprehend human anatomy with additional information. This applies for both beginners (e.g. students) and experts (e.g. surgeons) alike. When preparing for medical exams or for surgery, for example, the Cinematic Reality app gives a realistic 3D overview of a specific anatomical area. This could make it easier to overcome the problem of individual spatial ability.
First impressions of students are very promising. From the moment they put on the Microsoft HoloLens 2 for the first time, they were stoked – awed by the level of detail and quality of the images. Since the handling is very intuitive, they needed only a very short time to get used to interacting with the images. The same applied for our surgeons. They instantly started to discover different pathologies and to discuss strategies for a surgical approach. It seems that the additional information might help also to select the right operating room (OR) strategy and thereby increase the accuracy of the surgery. Since the manipulation of the images is touchless, even an intraoperative deployment is conceivable.
Cinematic Reality for medical education could be used for both, single-user, and multi-user scenarios. This way, one can not only study individually, but also discuss interesting findings in groups. Thanks to the high performance of the new, real-time Cinematic Rendering algorithm and its photorealistic representation, the HoloLens 2 application is extremely easy to learn. However, there are of course some challenges to overcome. The technology is still relatively expensive. Especially for a multi-user scenario, multiple HoloLens 2 glasses would be needed. A stable network connection is also a key element for a smooth operation in high quality. Additionally, symptoms like nausea and vertigo, also known as cybersickness, are a common challenge in AR and VR environments and might affect the pleasure negatively.
Up to this point Cinematic Reality is not yet approved for clinical use. However, we are convinced that this new technology has great potential to revolutionise the way medical students and medical professionals comprehend anatomy, and therefore improve their education and decision-making skills in the future.
Author
Johannes Binder, M.Sc. University Clinic of Erlangen, Department of Surgery, Friedrich-Alexander-University Erlangen-Nuremberg (FAU), Germany
