Predicting Mouse Click Position Using Long Short-Term Memory Model Trained by Joint Loss Function
Datong Wei, Chaofan Yang, Xiaolong Zhang, Xiaoru Yuan · 2021
Knowing where users might click in advance can potentially improve the efficiency of user interaction in desktop user interfaces. In this paper, we propose a machine learning approach to predict mouse click location. Our model, which is LSTM (long short-term memory)-based and trained by joint supervision, can predict the rectangular region of mouse click with feeding mouse trajectories on the fly. Experiment results show that our model can achieve a result of a predicted rectangle area of 58 × 79 pixels with 92% accuracy, and reduce prediction error when compared with other state-of-the-art prediction methods using a multi-user dataset.