AI for augmented reality in robot-assisted surgery

Background: Three-dimensional models reconstructed from MRI and CT images are useful to guide the surgeon during crucial surgical steps. Recent research activities to improve 3D-guided robotic surgery focus on the development of augmented reality systems able to visualize 3D models in the surgical field with in vivo co-registration during robotic surgery. However, the 3D model does not follow automatically the real organ during its mobilization and traction during surgery. Thus, this non-automated augmented reality may increase the surgical time due to the need of several re-adjustments of the 3D model over the surgical field during the dissection.

Purpose: To design, develop, and validate an AI-based augmented reality system able to automatically co-register, in real time, a digital 3D-model of biological structures and the surgical field.