We present a model-free reinforcement learning algorithm to synthesize control policies that maximize the probability of satisfying high-level control objectives given as Linear Temporal Logic (LTL) formulas. Uncertainty is considered in the workspace properties, the structure of the workspace, and the agent actions, giving rise to a Probabilistically-Labeled Markov Decision Process (PL-MDP) with unknown graph structure and stochastic behaviour, which is even more general than a fully unknown MDP. We first translate the LTL specification into a Limit Deterministic Büchi Automaton (LDBA), which is then used in an on-the-fly product with the PL-MDP. Thereafter, we define a synchronous reward function based on the acceptance condition of the LDBA. Finally, we show that the RL algorithm delivers a policy that maximizes the satisfaction probability asymptotically. We provide experimental results that showcase the efficiency of the proposed method.
Reinforcement learning for temporal logic control synthesis with probabilistic satisfaction guarantees
2019-09-13
Conference paper
Electronic Resource
English
Temporal Logic Guided Safe Model-Based Reinforcement Learning
Springer Verlag | 2023
|Public transport trajectory planning with probabilistic guarantees
Elsevier | 2020
|Systems control with generalized probabilistic fuzzy-reinforcement learning
BASE | 2011
|