The role of Robotics in Virtual & Augmented Reality by Kushal Mamillapalli

Although Augmented Reality and Virtual Reality have been commercial successes in the marketplace with headsets like oculus and vive, does a practical application exist for these systems in the world? There are several in the field of robotics, and unexpectedly, with integration of artificial intelligence.

Augmented Reality (AR) is a type of simulated reality that creates an overlay on top of a video feed. General uses range from social media filters to appliance and furniture preview filters. Virtual Reality (VR) is a type of simulated reality that does not utilize a video feed and instead shows a 3d simulation that is responsive to the user’s motion and requests. The AI discussed will have capabilities that exceed general code based algorithms: adaptivity and generalizations based on a specific video feed are utilized to engage in a command. For example, an AI could be used to identify objects in different lighting and perspectives, which normal algorithms would fail to do.

Robotics not only serve as an extension of a person in physical operations but allow for limitations to be broken. For example, one application is the decommissioning of nuclear plants. If a plant is destabilizing, and the internal conditions have become too dangerous for a human to enter, a robot with several apparatuses could enter using human control. Considering the difficulty of operation in analyzing the field and cross-applying the schematics of the plant, AR could be used to simplify and provide more information on the feed that makes the operation considerably easier for the operator. This would be done by creating an extra layer of video integrated to the feed that makes the image clearer, reducing visual noise. Using a recognition AI would also smoothen up the process: cross-applying the schematics of the plant will help identify parts of the feed, which makes the robot more simple to operate with human control.

Another example that would serve fruitful to many would be the application of AR and VR in Healthcare. Difficult operations like cardiovascular surgery and neurosurgery, require difficult training and human error can be a critical factor in determining success. However, applications of robotics and simulated realities would solve these issues and increase the realisticness of training. Firstly, human error can be solved by utilizing robotics and augmented reality to operate more precisely with vital information about the precise locations of operations. A robotic apparatus would reduce human error greatly and the amount of stress induced during the operation. Augmented reality would allow the operator to see info about the operation and AI systems could even create an overlay on the visual feed of the operated parts that indicates what needs to be operated. Secondly, training would be more accurate when VR is utilized with the robotic apparatus. Instead of a real time feed, a 3d environment that simulates the operation nearly perfectly would help trainees operate better and would further reduce mistakes in operator decision making in future operations. AI could be also used to analyze the mock procedure and provide tips on how the operator could improve surgery.