Metaverse aided Teleoperation Scene Prediction Optimization using Depth Camera Transformation based on Relative Velocities
Jonghyeok An, Changyeong Jeong, Hyun-Rok Cha, Myeong-Hwan Hwang, Seungha Yoon, Eugene Kim · 2024
The biggest problem with Vehicle Teleoperation is that its performance is greatly influenced by communication delay. In particular, considering its importance, the heterogeneity of camera information felt by remote drivers inherently involves human error that can lead to incorrect commands. Focusing on these issues, we aim to generate future Camera RGB frames considering communication delay and compare their performance. For this purpose, a teleoperation environment with communication delay was implemented on the metaverse, and future frames corresponding to the delay time were predicted. In particular, selective future frame prediction was implemented through an algorithm that clusters only surrounding vehicles from the surrounding environment and calculates the relative speed at each moment. From the results, it was confirmed that when predicting these future frames, it is better to predict by taking into account the relative speed of the surrounding environment, rather than simply predicting based on one’s own speed.