Technische Universität München
Sensors in operating rooms: How robots are becoming surgical assistants
TECHNICAL UNIVERSITY OF MUNICH
MEDIA RELEASE
Sensors are making operating rooms smarter
How robots are becoming surgical assistants
- In an experimental operating room equipped with sensors, researchers at TUM University Hospital are testing the use of robots.
- The medium-term goal is to reduce the workload of surgeons in the operating room.
- In future, complex operating room data will also be used for artificial intelligence applications.
How can robots and humans work together as effectively as possible in the operating room of the future? Researchers from the Technical University of Munich (TUM) and TUM University Hospital investigated this question as part of the ForNeRo research project. Using a sensor-equipped system, they analyzed surgeons' movements during procedures and collected data from simulated robot-assisted operations.
Five depth cameras mounted on the ceiling of the experimental operating theater at TUM University Hospital in Munich generate a three-dimensional digital image of the room 15 times per second —a digital twin of the surgical environment. At the operating table stands Prof. Dirk Wilhelm, Head of the Chair of Medical Robotics at TUM and a surgeon and senior physician at TUM University Hospital. He is wearing a suit fitted with motion markers on the joints and head, tracked by an infrared system with ten cameras. Microphones record and spatially locate conversations within the surgical team, while additional physiological data is collected to measure stress levels among staff.
The aim of this sensor data and the digital twin is to improve surgical workflows, integrate robotic assistance systems into clinical workflows as efficiently and ergonomically as possible, and ultimately reduce the workload of medical staff. The sensor system developed by Prof. Wilhelm’s research group for minimally invasive interdisciplinary therapeutic intervention (MITI) is now being used for the first time in Germany to collect data from a real operating room environment. “In the next step, this data could help improve the use of robots in surgery,” says Prof. Wilhelm. All data collection in the operating room requires the consent of patients and all parties involved.
Testing robotic systems in routine surgical procedures
For robotic systems to assist in future operating rooms, researchers will need more than sensor data alone. The experimental operating theater therefore serves not only as a data collection platform, but also as a test environment for robot-assisted procedures on anatomical models – in other words, testing collaboration with robotic assistants.
As part of the ForNeRo research project, the researchers investigated three common minimally invasive procedures: gallbladder surgery, inguinal hernia repair, and sigmoid resection, the partial removal of the large intestine. Two robotic systems were used in each procedure. The first, Solo Assist II, held and positioned the endoscope. The second was MIRO, a modular surgical robot developed by the German Aerospace Center (DLR) that surgeons can control using a joystick and other interfaces. During a simulated procedure, the surgeons used MIRO to manipulate a miniature gripper, position a plastic mesh during hernia repair and assist with suturing.
A robot can carry out simple tasks
The simulated operations are designed to help configure robotic systems that can assist surgeons during minimally invasive procedures. To evaluate their potential, Max Bergholz from the Chair of Ergonomics at TUM records surgeons’ postures and movements in a sensor-equipped operating room while performing procedures on anatomical models. Participants are also asked to assess the physical and mental strain experienced during the different phases of the operation.
“Surgeons often report back pain caused by maintaining rigid postures for long periods,” says Bergholz. “Earlier systems also required them to operate as though looking into a mirror, forcing them to constantly adapt their spatial orientation.” His goal is to make surgical work as ergonomic and intuitive as possible.
Robotic systems eliminate much of this need for readjustment. They allow surgeons to operate with greater precision, since larger hand movements with the joystick translate into movements of only a few millimeters inside the body. Unlike established systems like the da Vinci Surgical System, the new system also allows the surgeon to remain physically closer to the patient.
The research showed that robotic assistants can already take over simple tasks in an operating room — such as holding an endoscope — without increasing surgeons’ workload. “This allows us to explore how robotic assistants can be seamlessly integrated into clinical workflows,” says Bergholz.
AI is expected to better understand surgical procedures
Looking ahead, TUM Professor Dirk Wilhelm sees potential for using the complex data from the operating theatre for artificial intelligence applications. “Data are a fundamental building block for AI systems in the operating room,” says Wilhelm. “Such systems could automatically recognize which surgical instruments are being used and identify the organs being operated on.”
The initial goal is to improve surgical workflows and planning processes. In the longer term, AI could help decide when a robotic assistant would be beneficial.
About the ForNeRo research project
As part of the “Research Consortium – Seamless and Ergonomic Integration of Robotics into Clinical Workflows” (ForNeRo), researchers from the Technical University of Munich (TUM), TUM University Hospital and the German Aerospace Center (DLR), among other partners, have been investigating how robotic assistance systems (RAS) can be integrated into clinical workflows since September 2023.
The project is funded by the Bavarian Foundation for Transformation and Research with approximately two million euros and will run until November 2026. Further information: https://fornero.ed.tum.de/
Additional information for editors:
Photos: https://mediatum.ub.tum.de/1854913
Video: https://youtu.be/iksmUNrDj_k
Scientific contact:
Prof. Dirk Wilhelm
Chair of Medical Robotics, TUM
Surgeon and Senior Consultant at the Department of Surgery
TUM University Hospital
dirk.wilhelm@tum.de
Contact at the TUM Corporate Communications Center:
Andreas Schmitz
0162-27 46 193
The Technical University of Munich (TUM) is one of the world’s leading universities in terms of research, teaching and innovation, with around 700 professorships, 52,000 students and 13,000 staff. TUM’s range of subjects includes computer sciences, engineering, natural and life sciences, medicine, economics and social sciences. As an entrepreneurial university, TUM envisages itself as a global hub of knowledge exchange, open to society. Every year, around 100 start-ups are founded at TUM, which acts as a key player in Munich’s high-tech ecosystem. The university is represented around the world by its TUM Asia campus in Singapore along with offices in Beijing, Brussels, Mumbai, San Francisco, São Paulo and Shenzhen. Nobel Prize laureates and inventors such as Rudolf Diesel, Carl von Linde and Rudolf Mößbauer have conducted research at TUM, which was awarded the title of University of Excellence in 2006, 2012, 2019 and 2026. International rankings regularly cite TUM as the best university in the European Union.