15 out 2020
14:00 Master's Defense Fully distance
Theme
Routing based on Reinforcement Learning for Software-Defined Networking
Student
Daniela Maria Casas Velasco
Advisor / Teacher
Nelson Luis Saldanha da Fonseca
Brief summary
In communication networks, routing determines the path followed by packets from a source node to a destination node. In traditional Internet routing protocols, decisions about which paths the packets should follow are based on a limited number of information and are derived from performing the calculation of the shortest path, which can lead to slow adaptation in view of the traffic variability and the restricted support of the Quality of Service (QoS) requirements of the applications.
Software Defined-Networking (SDN) Networks were conceived to favor the adoption of innovation in network protocols. Some solutions have shown the improvement of traditional routing protocols, taking advantage of SDN resources, such as programmability, global view, logically centralized control and decoupling of network control and packet forwarding. However, these solutions do not fully exploit knowledge about the operation of the network to perform routing intelligently. Other works explored Machine Learning (ML) techniques such as Neural Networks, Logistic Regression and K-means in conjunction with SDN. However, the acquisition of training data sets, in these works, are dependent on information available in traditional routing protocols. In addition, the distributed form of routing is assumed, which tends to generate signaling overhead.
This thesis introduces two approaches to SDN routing called RSIR and DRSIR. RSIR stands for Reinforcement Learning and Software-Defined Networking Intelligent Routing, which adds a Knowledge Plan and defines a routing algorithm based on Reinforcement Learning (RL). The RSIR algorithm considers network state metrics to produce efficient and intelligent routing that adapts to dynamic traffic changes. RSIR is based on interaction with the environment and the global view and control of the network, to proactively compute and install optimal routes on packet flow routing devices. RSIR is presented in two versions, RSIR \ textsubscript {links} and RSIR \ textsubscript {paths}, which use network status information with metrics at the level of links and paths.
Also proposed is the DRSIR, an extended version of RSIR based on Deep Reinforcement Learning (DRL), which improves performance in relation to RL-based approaches.
RSIR and DRSIR were evaluated extensively by emulation using traffic matrices (real and synthetic). The results show that our solutions surpass the routing algorithms based on Dijkstra, in relation to the metrics stretching (stretch), loss and delay of packets. In addition, the results show the effectiveness of the algorithms in relation to the flow.
Examination Board
Headlines:
| Nelson Luis Saldanha da Fonseca | IC / UNICAMP |
| Jeferson Campos Nobre | INF / UFRGS |
| Ricardo da Silva Torres | IC / UNICAMP |
Substitutes:
| Edmundo Roberto Mauro Madeira | IC / UNICAMP |
| Luís Henrique Maciel Kosmalski Costa | UFRJ |