This paper proposes DeepRMSA, a deep reinforcement learning framework for routing, modulation and spectrum assignment (RMSA) in elastic optical networks (EONs). DeepRMSA learns the correct online RMSA policies by parameterizing the policies with deep neural networks (DNNs) that can sense complex EON states. The DNNs are trained with experiences of dynamic lightpath provisioning. We first modify the asynchronous advantage actor-critic algorithm and present an episode-based training mechanism for DeepRMSA, namely, DeepRMSA-EP. DeepRMSA-EP divides the dynamic provisioning process into multiple episodes (each containing the servicing of a fixed number of lightpath requests) and performs training by the end of each episode. The optimization target of DeepRMSA-EP at each step of servicing a request is to maximize the cumulative reward within the rest of the episode. Thus, we obviate the need for estimating the rewards related to unknown future states. To overcome the instability issue in the training of DeepRMSA-EP due to the oscillations of cumulative rewards, we further propose a window-based flexible training mechanism, i.e., DeepRMSA-FLX. DeepRMSA-FLX attempts to smooth out the oscillations by defining the optimization scope at each step as a sliding window, and ensuring that the cumulative rewards always include rewards from a fixed number of requests. Evaluations with the two sample topologies show that DeepRMSA-FLX can effectively stabilize the training while achieving blocking probability reductions of more than 20.3% and 14.3%, when compared with the baselines.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical Networks


    Beteiligte:
    Chen X (Autor:in) / Li B (Autor:in) / Proietti R (Autor:in) / Lu H (Autor:in) / Zhu Z (Autor:in) / Yoo S.J.B (Autor:in) / Chen, X / Li, B / Proietti, R / Lu, H

    Erscheinungsdatum :

    2019-01-01



    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Klassifikation :

    DDC:    629



    Manycast routing, modulation level and spectrum assignment over elastic optical networks

    Luo, Xiao / Zhao, Yang / Chen, Xue et al. | British Library Online Contents | 2017


    Routing, Spectrum, and Transponder Assignment in Elastic Optical Networks

    Dallaglio, M. / Giorgetti, A. / Sambo, N. et al. | British Library Online Contents | 2015


    Routing Code and Spectrum Assignment (RCSA) in Elastic Optical Networks

    Sambo, N. / Meloni, G. / Cugini, F. et al. | British Library Online Contents | 2015



    Energy Efficient Routing and Spectrum Assignment With Regenerator Placement in Elastic Optical Networks

    Fallahpour, A. / Beyranvand, H. / Nezamalhosseini, S.A. et al. | British Library Online Contents | 2014