A Helping Hand - The Surgical Assistant

September 2020 - Present

In a group of 4, I am developing an AI-driven system to automate tool delivery to the surgeon within an operating room during knee arthroplasty. The system consists of 5 stages: tool detection, gripping, translation, orientation, and hand-off. We are collaborating with the University of Waterloo RoboHub research facility to validate our design.

A need exists to automate the repetitive task of tool delivery within the OR without increasing surgeon workload

We modified an off-the-shelf gripper to meet operating room sterilization needs
I simulated the trajectory of the Kinova Gen3 robotic manipulator using Gazebo
The program was developed in Python & ROS for object avoidance during path planning. Safety protocols were added.

We used a machine learning model to predict the tool from a camera feed. This mallet was predicted with 72% confidence; I aim to increase this with more curated training data
The model will also be able to identify a safe orientation to deliver the tool (e.g. deliver a scalpel handle-first)

Thoughtful design-of-experiments drove informed decisions. This graph represents a grip detection strategy that was rejected due to low specificity.

Google Sites

Report abuse