Algorithm Topological Structure Search Based on Optimization of Mathematical Operation-Level Continuous Differentiable Search Framework
Ludi Wang, Qinghai Gong, Wang Zhaolei, Liu Jiarun, Lin Ping · 2024
Considering that some small-scale algorithms do not require a model with large capacity, this paper designs a topology structure graph search framework based on differentiable search of basic mathematical elements, using gradient descent to optimize the connection relationship of the algorithm topology graph, and applies this basic operation-level search framework to the following three application scenarios: 1) equation search 2) image binary classification problem 3) reinforcement learning(RL) reward function search, all of which have achieved results. The validity and efficiency of this differentiable search method has been proven. Although the examples in all three scenarios are not very complicated, the significant decrease in losses in 2) and the significant in-crease in rewards in 3) are sufficient to demonstrate the validity of the algorithm, which provides inspiration for future applications such as reward search.