We consider the stochastic dual dynamic programming (SDDP) algorithm - a widely employed algorithm applied to multistage stochastic programming - and propose a variant using experience replay - a batch learning technique from reinforcement learning. To connect SDDP with reinforcement learning, we cast SDDP as a Q-learning algorithm and describe its application in both risk-neutral and risk-averse settings. We demonstrate the superiority of the algorithm over conventional SDDP by benchmarking it against PSR's SDDP software using a large-scale instance of the long-term planning problem of inter-connected hydropower plants in Colombia. We find that SDDP with batch learning is able to produce tighter optimality gaps in a shorter amount of time than conventional SDDP. We also find that batch learning improves the parallel efficiency of SDDP backward passes.


    Access

    Download


    Export, share and cite



    Title :

    Batch Learning SDDP for Long-Term Hydrothermal Planning



    Publication date :

    2023-01-01


    Remarks:

    IEEE Transactions on Power Systems, (2023)



    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    006 / 629



    Simulation improves short-term planning in large-batch manufacturing industry

    Southall, J.T. / Twamley, C.J. / Nicholls, R.E. | Tema Archive | 1986


    Funding Analysis for Long-Term Planning

    A. L. C. de Cerreno / E. G. Sander | NTIS | 2003


    Airport Demand Forecasting for Long-Term Planning

    International Transport Forum | TIBKAT | 2016


    Port Productivity Standards for Long-Term Planning

    Vandeveer, D. / American Society of Civil Engineers; Committee on Ports and Harbors / Permanent International Association of Navigation Congresses; United States Section | British Library Conference Proceedings | 1998