A Distributed Reinforcement Learning Scheme for Network Routing
Psychology Press eBooks · 2013
In our learning scheme, the policy is distributed throughout the network as follows: each node keeps an estimate, for every neighbor/destination pair (y, d), of how long it should take for a packet with destination d to arrive there if it is sent first to neighbor node y. When a node x is asked to route a packet, it sends it to that neighbor y which x estimates will have the lowest total delivery time l . Instead of then waiting for the packet to finally reach d before updating the policy, x simply queries fJ to find out how long it expects the given packet to take in getting to d. Since the neighboring node is presumably closer to the final destination, its estimate is considered More precisely, let Qx(fj, d) be the time that node x estimates it takes to deliver a packet P bound for node d by way of x&s;s neighbor node y, including any time that P would have to spend in node x&s;s queue.2 Upon sending P to y, x immediately gets back fj&s;s estimate for the time remaining in the trip, namely If the packet spent q units oftime in x&s;s queue, then x can revise its estimate as follows: where 1] is a "learning rate" parameter (0.7 in our experiments).