Enabling Pointing Assistance in Adaptive Interfaces Using Mouse Pointing Intention Prediction
Rongrong Zhu, Zhaozhe Zhang, Zhongmin Cai · 2021 China Automation Congress (CAC) · 2021
Understanding pointing intention and providing corresponding support when the user performs pointing tasks are helpful to enhance the input efficiency of graphical user interfaces and improve the experience of user-machine interaction. Existing solutions include pointing facilitation techniques and adaptive interfaces may become impractical because they do not reliably predict mouse click intention well. This paper proposes a new method to enable pointing assistance in adaptive interfaces by predicting mouse pointing intention based on mouse movements. We model the mouse movement trajectory as a multivariate time series and adopt sequence learning models to predict the intended pointing target in real-time. The pointing areas in the adaptive interfaces will make adaptive changes when the number of prediction results in the possible targets queue reaches the set threshold. Analyses and evaluations were conducted using data from 10 subjects through our built prototype system. Experiments indicate that our method was able to predict the target within 600 milliseconds before completing the pointing task with an accuracy of over 90%. Participants felt that the adaptive system could bring convenience when performing pointing tasks without high adaptation costs.