Benefits and Challenges of Virtual Reality-Based Teleoperation of Robotic Arms

This thesis presents an overview of virtual reality (VR)-based teleoperation and describes its benefits and several existing challenges to its implementation, as well as potential solutions to these challenges. VR-based teleoperation of robotic arms enables a user to control and maneuver the robotic system from a remote distance while immersed in a virtual environment that simulates the location site of the robot. By implementing VR-based teleoperation, we can send robotic arms operated by trained professionals into harsh and inaccessible environments, including the deep sea and outer space, to accomplish manipulation tasks that would otherwise be unsafe or impossible. Teleoperated robotic arms can also be used to remotely execute fine manipulation tasks such as surgery, for instance, to reduce contamination or to perform operations in places that lack the required medical services. In order to be able to reliably and comfortably use VR-based teleoperation, we need to focus on solving the challenges of latency and sensory loss. Since the teleoperator has a limited field of view and cannot rely on certain types of sensory information, they can feel disoriented and disconnected from the environment and robotic arm. Sensory information loss can be mitigated by simulating a wider field of view in the virtual environment, implementing additional sensors such as thermometers and gas detection sensors, and using data sonification techniques. Although it may not be possible to completely eliminate latency, the effects of latency can be reduced through the use of assistive interfaces that predict the trajectory of the robotic arm in real-time based on the teleoperator’s input movement using artificial intelligence (AI)-based predictive models. When visualized in the virtual environment, this predictive real-time feedback enables the user to immediately see the effects of their movements on the robotic arm, even though the arm’s actual motion is delayed due to latency, and thus avoid collisions and improve task performance. VR-based teleoperation can be enhanced with these proposed solutions to enable the user to complete the required manipulation task with high precision and to maneuver the robotic arm with reduced cognitive load.