Highlights Dynamic pricing formulated as a partially observable Markov decision process. Deep reinforcement learning (Deep-RL) algorithms used as solution methods. Deep-RL algorithms outperform feedback control heuristic on different objectives. Policies trained using Deep-RL algorithms transfer well to new input distributions. Deep-RL algorithms suitable for multi-objective optimization using reward shaping.
Abstract This article develops a deep reinforcement learning (Deep-RL) framework for dynamic pricing on managed lanes with multiple access locations and heterogeneity in travelers’ value of time, origin, and destination. This framework relaxes assumptions in the literature by considering multiple origins and destinations, multiple access locations to the managed lane, en route diversion of travelers, partial observability of the sensor readings, and stochastic demand and observations. The problem is formulated as a partially observable Markov decision process (POMDP) and policy gradient methods are used to determine tolls as a function of real-time observations. Tolls are modeled as continuous and stochastic variables and are determined using a feedforward neural network. The method is compared against a feedback control method used for dynamic pricing. We show that Deep-RL is effective in learning toll policies for maximizing revenue, minimizing total system travel time, and other joint weighted objectives, when tested on real-world transportation networks. The Deep-RL toll policies outperform the feedback control heuristic for the revenue maximization objective by generating revenues up to 8.5% higher than the heuristic and for the objective minimizing total system travel time (TSTT) by generating TSTT up to 8.4% lower than the heuristic. We also propose reward shaping methods for the POMDP to overcome the undesired behavior of toll policies, like the jam-and-harvest behavior of revenue-maximizing policies. Additionally, we test transferability of the algorithm trained on one set of inputs for new input distributions and offer recommendations on real-time implementations of Deep-RL algorithms. The source code for our experiments is available online at https://github.com/venktesh22/ExpressLanes_Deep-RL.
Deep reinforcement learning algorithm for dynamic pricing of express lanes with multiple access locations
2020-07-07
Aufsatz (Zeitschrift)
Elektronische Ressource
Englisch
Model-Based Dynamic Pricing Algorithm for Managed Lanes
Online Contents | 2013
|Model-Based Dynamic Pricing Algorithm for Managed Lanes
Transportation Research Record | 2013
|Benefit-Cost Analysis of Variable Pricing Projects: SR-91 Express Lanes
British Library Online Contents | 2006
|Benefit-Cost Analysis of Variable Pricing Projects: SR-91 Express Lanes
Online Contents | 2006
|