Le chirurgien Thomas Grégory (casqué), s’entraîne à la réalité augmentée (capture écran). © DR Surgeon Thomas Grégory (wearing headset), practices in augmented reality (screen capture). © DR
Live on the Internet, on December 5, Hôpitaux de Paris carried out shoulder surgery via a collaborative mixed-reality platform. This global first also raises the question of using the Cloud in hospitals.
On December 5, AP-HP (Assistance publique – Hôpitaux de Paris) broadcast live on YouTube almost an entire surgical operation in augmented reality (AR). The shoulder surgery was performed at Avicenne Hospital in Seine-Saint-Denis by the surgeon Thomas Grégory, who was wearing an AR headset, the famous HoloLens by Microsoft. During the procedure, the doctor could view holograms—3D anatomical models of the patient’s shoulder—juxtaposed with reality, and exchange comments remotely with medical colleagues in the United States, South Korea and the UK.
Augmented and collaborative surgery
We saw the surgeon move and zoom in on the image using small gestures not unlike the ones we use on our own smartphones. But here, the image was projected onto the transparent lens of the headset, while his gestures were captured by the equivalent of a built-in Kinect sensor. Grégory could count on the assistance of four doctors videoconferencing from Imperial College London, Chosun University Hospital in South Korea, Lafayette Hospital in Louisiana and a professor from Pennsylvania in the United States. They followed the operation through Grégory’s eyes, augmented reality included, demonstrating the telepresence and collaborative aspects of the set up. The rebroadcast of the operation is still available online (age confirmation required).
AP-HP made a trailer to announce the event:
During the post-operation press conference, Grégory explained that it was a “proof of concept”, and that surgeons finally had a “cockpit in front of their eyes” with all the information necessary to “secure and standardize this type of operation”, as augmented reality allows us “to see what the eye cannot see”. In this case, what is hidden by the tissues. AR replaces the surgeon’s mental image, thus eliminating subjectivity. “Sometimes, we think we had it right, [whereas] the prosthesis wasn’t ideally placed,” observes the head of orthopedic and trauma surgery at Avicenne Hospital.
It was no accident that the procedure was performed by Thomas Grégory. Nicknamed the “geek doctor”, he is also president of the Moveo foundation, which promotes the use of new technologies, such as augmented reality, in surgery. He announced that these practices would soon become widespread. But it wasn’t all French pride. While the surgeon and the Evolutis prosthesis are from France, the HoloLens headset (Microsoft) and imagery (TeraRecon, Vizua) are from the U.S. And then there was the invisible Cloud—Microsoft Azure.
Our virtual giblets in the Cloud
Why the Cloud? Broadcasting high-definition 3D images in real-time into an AR headset that you can navigate around requires a lot of calculations. Part of these calculations are done in the Cloud. This is the strategy of Microsoft, thanks to its CEO Satya Nadella, which is migrating everything toward the Cloud and providing software as service. The lightweight and portable HoloLens, which prefigures our future digital interface, needs the Cloud.
Is this to say that our surgical data will be floating around everywhere? Will this very personal data be stored and manipulated on the fly on distant servers? In France, medical associations such as the CNIL (National Data Protection Commission) are closely monitoring remote hosting (Cloud storage) in hospitals, as such hosting of data could constitute “a loss of control of how data is stored ‘somewhere’, eventually offshore and in countries with different legislation.” This is especially troubling, considering that computer attacks targeting health facilities are increasing. According to the nonprofit organization Wedi, from 2010 to 2014, hackers compromised 37 million pieces of health data worldwide. Since January 2017, a French law has developed a process of certification for digital hosts of health data, specifying that “the use of data for purposes other than providing the hosting service is strictly prohibited”.