R2L: routing with reinforcement learning

Abstract

In a packet network, the routes taken by traffic can be determined according to predefined objectives. Assuming that the network conditions remain static and the defined objectives do not change, mathematical tools such as linear programming could be used to solve this routing problem. However, networks can be dynamic or the routing requirements may change. In that context, Reinforcement Learning (RL), which can learn to adapt in dynamic conditions and offers flexibility of behavior through the reward function, presents as a suitable tool to find good routing strategies. In this work, we train an RL agent, which we call R2L, to address the routing problem. The policy function used in R2L is a neural network and we use an evolution strategy algorithm to determine its weights and biases. We tested R2L in two different scenarios: static and dynamic networks conditions. In the first, we used a 16-node network and experimented with different reward functions, observing that R2L was able to adapt its routing behavior accordingly. Finally, in the second experiment, we used a 5-node network topology where a given link's transmission rate changed during the simulation. In this scenario, we observed that R2L was able to deliver a competitive performance, compared to heuristic benchmarks, with changing network conditions.