Deep Reinforcement Learning Based Method for the Rule Placement Problem in Software-Defined Networks
Manuel Jiménez-Lázaro, Javier Berrocal, Jaime Galán–Jiménez · NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium · 2022
Ternary content-addressable (TCAM) memories of Software-Defined Networking (SDN) nodes are very fast and allow parallel lookups to be performed in a very short period of time. However, they have both a high energy consumption and cost, which make their size limited. This limitation has an impact on the number of rules that can be installed in the flow tables of the SDN nodes, and an inefficient rule management can lead to a degradation of the QoS (Quality of Service) of the network. In this work, a solution based on Deep Reinforcement Learning (DRL) is proposed to tackle the rule placement problem of SDN flow tables. The main idea is to remove the rules that are intended to be less used in order to make room for new rules that are prone to be used often. In this way, the goal of increasing the number of flows that can be handled in the network, and therefore improving the network QoS is achieved. Simulation results show that the proposed solution obtains better results than its static idle timeout counterpart. In particular, DRL is able to handle 7.36% more satisfied flows on average compared to the case of setting a static idle timeout of 1 s., which indeed requires a high number of rules re-installations. This is therefore a promising result that demonstrates the good performance of the proposed DRL solution.