Cascaded Q-tables with Two States Corrected the Synchronicity of Power Supplies in Synchrotrons
Yanlin Li, Shi An, Wei Zhang, Ruisheng Zhang · 2022
It is related to the synchronicity of power supplies that ions accelerated successfully in a synchrotron. The weakened synchronicity will lead to a failure of the acceleration. By improving the Q-learning algorithm, we proposed an online method, which can correct the synchronicity of power supplies automatically and quickly. Inspired by the operating of a microscope, the single q-table with multiple states replace by many cascaded q-tables which contain only two states. When selecting actions, the randomness mechanism replaces by the circular one. Due to the changes, the algorithm achieves faster learning of delay time. Experimental results show that the method obviously saves learning time. At the same time, it is superior to the accuracy of standard Q learning.