A Ray-based Asynchronous Distributed Parallel K-SVD Method
Yongkang Li, Xiaodong Chen, Xiaohui Wan, Jin Yuan Sun, Peng Zheng, Yunchang Wang, Zebin Wu · 2024
K-SVD (K-Singular Value Decomposition)is a com-monly used dictionary learning method that progressively opti-mizes dictionaries to better represent data by iterating sparse coding and dictionary update steps. However, with the increasing data size, the traditional K -SVD method faces significant challenges in terms of computational efficiency. Therefore, this paper proposes an asynchronous distributed parallel K-SVD method based on Ray framework to cope with the computational efficiency problem in large-scale data processing. This method achieves efficient parallel computation by decomposing the sparse coding and dictionary updating steps of the K-SVD method into multiple independent tasks and utilizing the distributed scheduling capability of the Ray framework. Since the zero elements of the coefficient matrix are unevenly distributed, we introduce a stride partitioning approach when performing the decomposition. In addition, in order to avoid the method falling into local optima and failing to find global optima due to parallelism, this paper synchronizes with the global at the same time as the atomic update of the dictionary. Meanwhile, for the purpose of avoiding the overhead caused by data synchronization, a parameter server is introduced to realize asynchronous updating of dictionary and coefficient matrix to reduce the overhead caused by communication. The experimental results show that, compared with the traditional K-SVD method, the method proposed in this paper has a significant advantage in computational efficiency and is able to efficiently process hyperspectral image datasets under the premise of guaranteeing the sparse representation accuracy. Compared with the synchronous parallel K-SVD method, the proposed method has faster error convergence and does not fall into local optima.