Classifying and resolving rare events such as failures, conflicts and anomalies is crucial for trouble-free daily operation in many industries. However, this is very challenging in many cases due to the lack of sufficient relevant data. In the Swiss railway network, anomalies in daily train traffic lead to delays and network instabilities with long ranging effects. Today such conflicts are resolved through human interventions without precise knowledge of the future impact on network stability of the taken actions.
Our research aim is to generate automated suggestions for human decision making through evaluation of the quality of different disposition strategies. Furthermore, we want to provide real time suggestions for disposition strategies with minimal network disturbance impact. Due to the sparseness and heterogeneity of conflict events in the network, however, deep learning and statistical approaches seemed unfeasible for this task so far. Thus, automatization in this domain remained extremely difficult.
To overcome this we generated a digital twin of the Swiss railway network in form of a GPU accelerated high performance simulation. Because of the immense size and complexity of our railway network consisting of 35’000 signals, 13’000 switches and more than 20’000 train runs per day (approximately 500 simultaneous train runs), the high computational power of modern GPUs is necessary in order to simulate thousands of different scenarios in an acceptable time frame. The simulator allows us to investigate the Swiss railway network in super real-time speed which is necessary in order to be able to react to the ever-changing topology and traffic situation.
We are working on a deep reinforcement learning approach with an asynchronous actor-critic model to train an autonomous dispatching and scheduling agent. We train multiple agents in parallel and use weighted experience sharing to speed up the learning process and performance.
A further speed improvement can be obtained by implementing a direct interaction between the dispatching agent and the simulation on the DGX-1.The digital twin in form of parallel simulations allows us to prepare the autonomous agent for historically unseen events in the railway system and explore novel dispatching procedures and actions.
Analogously to the results seen with AlphaGo Zero, we hope to obtain super human
performance through our reinforcement learning (combined with local search algorithms) approach. Automated scheduling can proactively minimize destabilizing situations. Furthermore, fast and close to optimal dispatching decisions in case of disturbances can quickly restore stability. This makes stable train traffic possible even in a denser future railway network, without large infrastructure changes or developments.